This project uses movie-rating survey data to develop a Global Baseline Estimate recommendation system that predicts how users may rate unseen movies. Unlike a simple ranking based only on each movie’s average rating, the Global Baseline Estimate accounts for two systematic effects: some users tend to give higher or lower ratings than others, and some movies tend to receive higher or lower ratings overall. The article Matrix Factorization Techniques for Recommender Systems explains that a useful baseline prediction can combine the overall rating average with a user’s rating tendency and an item’s popularity.
I will begin by importing the survey data and examining its structure, rating scale, missing values, and data types. Because users are stored in rows and movies are stored in columns, I will separate the critic names from the numerical rating columns before calculating user and movie averages. This structure will make it easier to calculate summaries and produce estimates for unrated movies.
The prediction for a user and movie will follow the method illustrated in the spreadsheet:
\[ \widehat{r}_{ui} = \mu + b_u + b_i \]
where \(\mu\) is the global mean of all observed ratings, \(b_u\) is the user’s average rating relative to the global mean, and \(b_i\) is the movie’s average rating relative to the global mean. For every missing user-movie combination, I will calculate a baseline estimate using these three components. I will then rank the user’s unrated movies by predicted rating and recommend the movie with the highest estimate. Predicted values will be checked against the valid survey scale so the final results remain interpretable.
I will verify by reproducing the spreadsheet example for Param and Pitch Perfect 2. This will confirm that the R calculations follow the provided algorithm before applying the method to the full survey dataset.
The main challenge will be distinguishing genuinely missing ratings from invalid entries. Blank cells represent movies a person did not rate, while text values such as ? must be converted to missing values rather than treated as ratings. I will check whether the observed ratings fall within the expected 1-to-5 scale. Duplicate users, inconsistent movie names, and users or movies with few ratings are additional data-quality concerns that could be examined in future work.
The data are likely to be sparse because many users have rated only a subset of the movies. A user or movie with very few ratings may have an unstable average, which can make its bias estimate overly dependent on one response. I will document the number of available and missing ratings for each movie and interpret estimates based on small counts cautiously. If a user has no valid ratings, a user-specific bias cannot be calculated; in that situation, the estimate would need to rely on the global mean and movie effect only.
Finally, the baseline method captures general user and movie tendencies but does not measure similarities among users, movie genres, or individual preferences. Therefore, the resulting recommendation will serve as a transparent benchmark rather than a fully personalized collaborative-filtering system.
The data question is: For every missing user-movie rating, what rating does the Global Baseline Estimate predict? The workflow downloads the spreadsheet from a public URL, selects the six rating columns, checks the rating scale, calculates the global, user, and movie effects, and ranks each user’s unseen movies. This public URL makes the analysis reproducible without a local file path.
# Packages used to read the Excel file and display tables.
library(readxl)
library(knitr)
# Public URL for the movie-rating spreadsheet.
data_url <- paste0(
"https://raw.githubusercontent.com/chanicemcken/",
"Data-607-Assignment-3A-Global-Baseline-Estimate/main/MovieRatings.xlsx"
)
# Download the workbook to a temporary file.
temp_file <- tempfile(fileext = ".xlsx")
download.file(data_url, temp_file, mode = "wb", quiet = TRUE)
# Read only the worksheet that contains the original ratings.
movie_ratings <- read_excel(
temp_file,
sheet = "MovieRatings",
na = c("", "NA", "?")
)
# Display the first six rows to confirm that the file loaded correctly.
kable(head(movie_ratings), caption = "First six rows of the movie-rating data")| Critic | CaptainAmerica | Deadpool | Frozen | JungleBook | PitchPerfect2 | StarWarsForce |
|---|---|---|---|---|---|---|
| Burton | NA | NA | NA | 4 | NA | 4 |
| Charley | 4 | 5 | 4 | 3 | 2 | 3 |
| Dan | NA | 5 | NA | NA | NA | 5 |
| Dieudonne | 5 | 4 | NA | NA | NA | 5 |
| Matt | 4 | NA | 2 | NA | 2 | 5 |
| Mauricio | 4 | NA | 3 | 3 | 4 | NA |
The Critic column identifies users, while the remaining
columns contain ratings. Blank cells represent movies that a critic did
not rate. The code converts the rating columns to numeric values and
stops with an informative message if an observed rating is outside the
expected 1-to-5 scale.
# Save the critic names and movie names.
critic_names <- movie_ratings$Critic
movie_names <- names(movie_ratings)[-1]
# Keep only the rating columns and make sure they are numeric.
ratings <- as.data.frame(movie_ratings[-1])
ratings[] <- lapply(ratings, as.numeric)
# Check for invalid observed ratings.
all_ratings <- unlist(ratings, use.names = FALSE)
invalid_ratings <- all_ratings[
!is.na(all_ratings) & (all_ratings < 1 | all_ratings > 5)
]
if (length(invalid_ratings) > 0) {
stop("At least one observed rating is outside the 1-to-5 scale.")
}
# Summarize how much data is available for each movie.
data_summary <- data.frame(
Movie = movie_names,
Number_of_Ratings = colSums(!is.na(ratings)),
Missing_Ratings = colSums(is.na(ratings))
)
kable(data_summary, caption = "Available and missing ratings by movie")| Movie | Number_of_Ratings | Missing_Ratings | |
|---|---|---|---|
| CaptainAmerica | CaptainAmerica | 11 | 5 |
| Deadpool | Deadpool | 9 | 7 |
| Frozen | Frozen | 11 | 5 |
| JungleBook | JungleBook | 10 | 6 |
| PitchPerfect2 | PitchPerfect2 | 7 | 9 |
| StarWarsForce | StarWarsForce | 13 | 3 |
The model uses the following formula:
\[ \widehat{r}_{ui} = \mu + b_u + b_i \]
Here, \(\mu\) is the mean of all observed ratings, \(b_u\) is the difference between a user’s average and the global mean, and \(b_i\) is the difference between a movie’s average and the global mean. The user effect adjusts for critics who usually rate high or low, while the movie effect adjusts for movies that are generally rated above or below average.
# Overall mean of every observed rating.
global_mean <- mean(as.matrix(ratings), na.rm = TRUE)
# User averages and user effects.
user_average <- rowMeans(ratings, na.rm = TRUE)
user_bias <- user_average - global_mean
# If a user has no ratings, use no user-specific adjustment.
user_bias[is.nan(user_bias)] <- 0
# Movie averages and movie effects.
movie_average <- colMeans(ratings, na.rm = TRUE)
movie_bias <- movie_average - global_mean
movie_bias[is.nan(movie_bias)] <- 0
# Create small tables so the calculations can be reviewed.
movie_results <- data.frame(
Movie = movie_names,
Movie_Average = round(movie_average, 3),
Movie_Bias = round(movie_bias, 3)
)
cat("Global mean rating:", round(global_mean, 3), "\n\n")## Global mean rating: 3.934kable(movie_results, caption = "Movie averages and movie effects")| Movie | Movie_Average | Movie_Bias | |
|---|---|---|---|
| CaptainAmerica | CaptainAmerica | 4.273 | 0.338 |
| Deadpool | Deadpool | 4.444 | 0.510 |
| Frozen | Frozen | 3.727 | -0.207 |
| JungleBook | JungleBook | 3.900 | -0.034 |
| PitchPerfect2 | PitchPerfect2 | 2.714 | -1.220 |
| StarWarsForce | StarWarsForce | 4.154 | 0.219 |
user_results <- data.frame(
User = critic_names,
User_Average = round(user_average, 3),
User_Bias = round(user_bias, 3)
)
kable(
user_results,
caption = "User averages and user effects"
)| User | User_Average | User_Bias |
|---|---|---|
| Burton | 4.000 | 0.066 |
| Charley | 3.500 | -0.434 |
| Dan | 5.000 | 1.066 |
| Dieudonne | 4.667 | 0.732 |
| Matt | 3.250 | -0.684 |
| Mauricio | 3.500 | -0.434 |
| Max | 3.333 | -0.601 |
| Nathan | 4.000 | 0.066 |
| Param | 3.500 | -0.434 |
| Parshu | 3.667 | -0.268 |
| Prashanth | 4.800 | 0.866 |
| Shipra | 4.000 | 0.066 |
| Sreejaya | 4.667 | 0.732 |
| Steve | 4.000 | 0.066 |
| Vuthy | 3.600 | -0.334 |
| Xingjia | 5.000 | 1.066 |
The following nested loops visit every blank rating. For each blank cell, the code adds the global mean, that critic’s user effect, and that movie’s effect. The original estimate is retained for transparency, and a second column limits the displayed rating to the survey’s 1-to-5 scale.
# Start with an empty table for the estimates.
predictions <- data.frame(
User = character(),
Movie = character(),
Baseline_Estimate = numeric(),
Rating_Scale_Estimate = numeric(),
stringsAsFactors = FALSE
)
# Calculate an estimate only where the original rating is missing.
for (i in seq_len(nrow(ratings))) {
for (j in seq_along(movie_names)) {
if (is.na(ratings[[j]][i])) {
estimate <- global_mean + user_bias[i] + movie_bias[j]
# Limit the presentation value to the valid 1-to-5 survey scale.
scale_estimate <- min(max(estimate, 1), 5)
predictions <- rbind(
predictions,
data.frame(
User = critic_names[i],
Movie = movie_names[j],
Baseline_Estimate = estimate,
Rating_Scale_Estimate = scale_estimate
)
)
}
}
}
# Round only the copy used for display; full precision is kept for ranking.
prediction_display <- predictions
prediction_display$Baseline_Estimate <- round(
prediction_display$Baseline_Estimate, 3
)
prediction_display$Rating_Scale_Estimate <- round(
prediction_display$Rating_Scale_Estimate, 3
)
kable(prediction_display, caption = "Global baseline estimates for missing ratings")| User | Movie | Baseline_Estimate | Rating_Scale_Estimate | |
|---|---|---|---|---|
| CaptainAmerica | Burton | CaptainAmerica | 4.338 | 4.338 |
| Deadpool | Burton | Deadpool | 4.510 | 4.510 |
| Frozen | Burton | Frozen | 3.793 | 3.793 |
| PitchPerfect2 | Burton | PitchPerfect2 | 2.780 | 2.780 |
| CaptainAmerica1 | Dan | CaptainAmerica | 5.338 | 5.000 |
| Frozen1 | Dan | Frozen | 4.793 | 4.793 |
| JungleBook | Dan | JungleBook | 4.966 | 4.966 |
| PitchPerfect21 | Dan | PitchPerfect2 | 3.780 | 3.780 |
| Frozen2 | Dieudonne | Frozen | 4.460 | 4.460 |
| JungleBook1 | Dieudonne | JungleBook | 4.632 | 4.632 |
| PitchPerfect22 | Dieudonne | PitchPerfect2 | 3.447 | 3.447 |
| Deadpool1 | Matt | Deadpool | 3.760 | 3.760 |
| JungleBook2 | Matt | JungleBook | 3.216 | 3.216 |
| Deadpool2 | Mauricio | Deadpool | 4.010 | 4.010 |
| StarWarsForce | Mauricio | StarWarsForce | 3.719 | 3.719 |
| CaptainAmerica2 | Nathan | CaptainAmerica | 4.338 | 4.338 |
| Deadpool3 | Nathan | Deadpool | 4.510 | 4.510 |
| Frozen3 | Nathan | Frozen | 3.793 | 3.793 |
| JungleBook3 | Nathan | JungleBook | 3.966 | 3.966 |
| PitchPerfect23 | Nathan | PitchPerfect2 | 2.780 | 2.780 |
| JungleBook4 | Param | JungleBook | 3.466 | 3.466 |
| PitchPerfect24 | Param | PitchPerfect2 | 2.280 | 2.280 |
| PitchPerfect25 | Prashanth | PitchPerfect2 | 3.580 | 3.580 |
| CaptainAmerica3 | Shipra | CaptainAmerica | 4.338 | 4.338 |
| Deadpool4 | Shipra | Deadpool | 4.510 | 4.510 |
| PitchPerfect26 | Shipra | PitchPerfect2 | 2.780 | 2.780 |
| Deadpool5 | Steve | Deadpool | 4.510 | 4.510 |
| Frozen4 | Steve | Frozen | 3.793 | 3.793 |
| JungleBook5 | Steve | JungleBook | 3.966 | 3.966 |
| PitchPerfect27 | Steve | PitchPerfect2 | 2.780 | 2.780 |
| StarWarsForce1 | Vuthy | StarWarsForce | 3.819 | 3.819 |
| CaptainAmerica4 | Xingjia | CaptainAmerica | 5.338 | 5.000 |
| Deadpool6 | Xingjia | Deadpool | 5.510 | 5.000 |
| PitchPerfect28 | Xingjia | PitchPerfect2 | 3.780 | 3.780 |
| StarWarsForce2 | Xingjia | StarWarsForce | 5.219 | 5.000 |
The spreadsheet asks how Param would rate Pitch Perfect 2. Reproducing this example is a useful accuracy check because the expected estimate is approximately 2.28.
# Select Param's estimate for Pitch Perfect 2.
param_check <- predictions[
predictions$User == "Param" &
predictions$Movie == "PitchPerfect2",
]
kable(param_check, digits = 3,
caption = "Verification of the spreadsheet example")| User | Movie | Baseline_Estimate | Rating_Scale_Estimate | |
|---|---|---|---|---|
| PitchPerfect24 | Param | PitchPerfect2 | 2.28 | 2.28 |
# Stop if the result does not match the workbook within rounding tolerance.
stopifnot(abs(param_check$Baseline_Estimate - 2.279859) < 0.001)The calculated estimate is 2.28, which matches the spreadsheet result of approximately 2.28. This confirms that the R code follows the supplied algorithm.
For each critic with at least one missing rating, the recommended movie is the unseen movie with the highest baseline estimate. This step converts the prediction table into a direct recommendation.
# Split the prediction table by user and keep each user's largest estimate.
recommendations <- do.call(
rbind,
lapply(split(predictions, predictions$User), function(user_rows) {
user_rows[which.max(user_rows$Baseline_Estimate), ]
})
)
# Clean the row names and rename the estimate column.
rownames(recommendations) <- NULL
names(recommendations)[3] <- "Predicted_Rating"
recommendations$Predicted_Rating <- round(
recommendations$Predicted_Rating, 2
)
recommendations$Rating_Scale_Estimate <- round(
recommendations$Rating_Scale_Estimate, 2
)
kable(recommendations,
caption = "Highest-ranked unseen movie for each eligible critic")| User | Movie | Predicted_Rating | Rating_Scale_Estimate |
|---|---|---|---|
| Burton | Deadpool | 4.51 | 4.51 |
| Dan | CaptainAmerica | 5.34 | 5.00 |
| Dieudonne | JungleBook | 4.63 | 4.63 |
| Matt | Deadpool | 3.76 | 3.76 |
| Mauricio | Deadpool | 4.01 | 4.01 |
| Nathan | Deadpool | 4.51 | 4.51 |
| Param | JungleBook | 3.47 | 3.47 |
| Prashanth | PitchPerfect2 | 3.58 | 3.58 |
| Shipra | Deadpool | 4.51 | 4.51 |
| Steve | Deadpool | 4.51 | 4.51 |
| Vuthy | StarWarsForce | 3.82 | 3.82 |
| Xingjia | Deadpool | 5.51 | 5.00 |
Param’s highest-ranked unseen movie is JungleBook, with an estimated rating of 3.47. This recommendation is higher than Param’s estimated rating for Pitch Perfect 2.
The Global Baseline Estimate successfully produced estimates for all missing user-movie combinations by combining the overall mean with user and movie effects. The global mean was 3.93, and the spreadsheet validation produced the expected estimate of 2.28 for Param and Pitch Perfect 2. Based on the available choices, the model recommends JungleBook to Param.
This model is useful as a simple and transparent benchmark, but it does not learn genre preferences or similarities between critics. A future version could use a train/test split to measure prediction error, add regularization for critics or movies with few ratings, and compare the baseline results with collaborative filtering. The survey could also be updated with more critics and newer movies to make the recommendations more reliable.
LLM transcript: https://chatgpt.com/share/6ab019ac-b6b4-83e9-a729-f58d95fd4dda