This project will use the movie-rating survey data to develop a recommendation system based on the Global Baseline Estimate. The goal is to estimate how a user would rate a movie they have not yet rated and then use those estimates to recommend the most promising unseen movie. Unlike a simple ranking based only on each movie’s average rating, the Global Baseline Estimate accounts for two systematic effects: some users tend to give higher or lower ratings than others, and some movies tend to receive higher or lower ratings overall.
I will begin by importing the survey data and examining its structure, rating scale, missing values, and data types. Because the spreadsheet stores users in rows and movies in columns, I plan to reshape the ratings into a long format with one row for each observed user-movie rating. This structure will make it easier to calculate summaries and produce estimates for unrated movies.
The prediction for a user and movie will follow the method illustrated in the spreadsheet:
\[ \widehat{r}_{ui} = \mu + b_u + b_i \]
where \(\mu\) is the global mean of all observed ratings, \(b_u\) is the user’s average rating relative to the global mean, and \(b_i\) is the movie’s average rating relative to the global mean. For every missing user-movie combination, I will calculate a baseline estimate using these three components. I will then rank the user’s unrated movies by predicted rating and recommend the movie with the highest estimate. Predicted values will be checked against the valid survey scale so the final results remain interpretable.
I will verify by reproducing the spreadsheet example for Param and Pitch Perfect 2. This will confirm that the R calculations follow the provided algorithm before applying the method to the full survey dataset.
The main challenge will be distinguishing genuinely missing ratings
from invalid entries. Blank cells should represent movies a person did
not rate, while text values such as ? must be converted to
missing values rather than treated as ratings. I will also check for
ratings outside the expected scale, duplicate users, inconsistent movie
names, and rows or columns containing too few observations.
The data are likely to be sparse because many users have rated only a subset of the movies. A user or movie with very few ratings may have an unstable average, which can make its bias estimate overly dependent on one response. I will document the number of available ratings for each user and movie and interpret estimates based on small counts cautiously. If a user has no valid ratings, a user-specific bias cannot be calculated; in that situation, the estimate would need to rely on the global mean and movie effect only.
Finally, the baseline method captures general user and movie tendencies but does not measure similarities among users, movie genres, or individual preferences. Therefore, the resulting recommendation will serve as a transparent benchmark rather than a fully personalized collaborative-filtering system.