The goal of this project is to use the survey data I collected on movie ratings to develop a non-personalized recommendation using the Global Baseline Estimate method. I plan to first examine the structure and distribution of the ratings, then calculate the overall average rating and compare individual user and movie ratings against that average to determine the adjustments needed for the Global Baseline Estimate. I will also review the dataset for missing ratings, inconsistent formatting, or other issues that may need to be addressed before completing the recommendation analysis.
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(readr)
library(ggplot2)
library(tidyr)movie_ratings <-read_csv("https://github.com/jmald1987/DATA607-Movie_Ratings/raw/refs/heads/main/movie_ratings.csv"
)## Rows: 77 Columns: 4
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (4): name, title, media_type, rating
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.Before calculating the Global Baseline Estimate, I examined the structure and summary of the dataset to identify the variables, data types, and missing ratings.
glimpse(movie_ratings)## Rows: 77
## Columns: 4
## $ name <chr> "Daniel C", "Daniel C", "Daniel C", "Daniel C", "Daniel C",…
## $ title <chr> "Cape Fear", "Criminal Minds", "Dexter Original Sin", "Dext…
## $ media_type <chr> "TV Show", "TV Show", "TV Show", "TV Show", "Movie", "TV Sh…
## $ rating <chr> "NULL", "NULL", "NULL", "NULL", "4", "NULL", "NULL", "NULL"…summary(movie_ratings)## name title media_type rating
## Length:77 Length:77 Length:77 Length:77
## Class :character Class :character Class :character Class :character
## Mode :character Mode :character Mode :character Mode :characterThe dataset contains 77 observations and four variables:
name, title, media_type, and
rating. The initial examination also shows that
rating was imported as a character variable because unrated
titles are represented by the text "NULL".
Before calculating averages, the "NULL" values need to
be treated as missing values and the remaining ratings converted from
character values to numeric values.
movie_ratings <- movie_ratings %>%
mutate(
rating = na_if(rating, "NULL"),
rating = as.numeric(rating)
)
glimpse(movie_ratings)## Rows: 77
## Columns: 4
## $ name <chr> "Daniel C", "Daniel C", "Daniel C", "Daniel C", "Daniel C",…
## $ title <chr> "Cape Fear", "Criminal Minds", "Dexter Original Sin", "Dext…
## $ media_type <chr> "TV Show", "TV Show", "TV Show", "TV Show", "Movie", "TV Sh…
## $ rating <dbl> NA, NA, NA, NA, 4, NA, NA, NA, NA, NA, NA, 5, NA, 3, 3, NA,…summary(movie_ratings)## name title media_type rating
## Length:77 Length:77 Length:77 Min. :3.000
## Class :character Class :character Class :character 1st Qu.:4.000
## Mode :character Mode :character Mode :character Median :5.000
## Mean :4.385
## 3rd Qu.:5.000
## Max. :5.000
## NA's :64After cleaning, rating is numeric and unrated movies are
represented as NA. These missing ratings can now be
excluded from the average calculations while remaining available as the
ratings that may later be estimated.
The global mean represents the average of all available ratings in the dataset and serves as the starting point for the Global Baseline Estimate.
global_mean <- mean(movie_ratings$rating, na.rm = TRUE)
global_mean## [1] 4.384615Next, the average rating for each user is calculated. The user effect measures the difference between each user’s average rating and the global mean.
user_effects <- movie_ratings %>%
group_by(name) %>%
summarise(
user_average = mean(rating, na.rm = TRUE),
.groups = "drop"
) %>%
mutate(
user_effect = user_average - global_mean
)
user_effects## # A tibble: 7 × 3
## name user_average user_effect
## <chr> <dbl> <dbl>
## 1 Daniel C 4 -0.385
## 2 Juilo C 4 -0.385
## 3 Julie E 5 0.615
## 4 Krystal C 4 -0.385
## 5 Lesley R 5 0.615
## 6 Mike E 4 -0.385
## 7 Sabrina H 5 0.615A positive user effect indicates that the user tends to give ratings above the overall average, while a negative effect indicates that the user tends to give lower ratings.
The same process is used for each title. The movie effect measures the difference between a movie’s average rating and the global mean.
movie_effects <- movie_ratings %>%
group_by(title) %>%
summarise(
movie_average = mean(rating, na.rm = TRUE),
.groups = "drop"
) %>%
mutate(
movie_effect = movie_average - global_mean
)
movie_effects## # A tibble: 11 × 3
## title movie_average movie_effect
## <chr> <dbl> <dbl>
## 1 Cape Fear 4.67 0.282
## 2 Criminal Minds 5 0.615
## 3 Dexter Original Sin 3 -1.38
## 4 Dexter Ressurection 3 -1.38
## 5 Gaurdians of The Galaxy V.3 4 -0.385
## 6 Hey Duggee 5 0.615
## 7 Lioness 5 0.615
## 8 Master of The Universe 5 0.615
## 9 Mouse Trap 5 0.615
## 10 The 100 4 -0.385
## 11 Things Heard And Seen 4 -0.385A positive movie effect indicates that the title was generally rated above the global average, while a negative value indicates that it was generally rated below the global average.
The Global Baseline Estimate combines the global mean with the user and movie effects. The calculation is applied to the missing ratings to estimate how each user might rate a title they did not rate in the original survey.
baseline_estimates <- movie_ratings %>%
filter(is.na(rating)) %>%
select(name, title, media_type) %>%
left_join(user_effects, by = "name") %>%
left_join(movie_effects, by = "title") %>%
mutate(
estimated_rating = global_mean + user_effect + movie_effect
)
baseline_estimates## # A tibble: 64 × 8
## name title media_type user_average user_effect movie_average movie_effect
## <chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl>
## 1 Daniel C Cape… TV Show 4 -0.385 4.67 0.282
## 2 Daniel C Crim… TV Show 4 -0.385 5 0.615
## 3 Daniel C Dext… TV Show 4 -0.385 3 -1.38
## 4 Daniel C Dext… TV Show 4 -0.385 3 -1.38
## 5 Daniel C Hey … TV Show 4 -0.385 5 0.615
## 6 Daniel C Lion… TV Show 4 -0.385 5 0.615
## 7 Daniel C Mast… Movie 4 -0.385 5 0.615
## 8 Daniel C Mous… TV Show 4 -0.385 5 0.615
## 9 Daniel C The … TV Show 4 -0.385 4 -0.385
## 10 Daniel C Thin… Movie 4 -0.385 4 -0.385
## # ℹ 54 more rows
## # ℹ 1 more variable: estimated_rating <dbl>The estimated ratings can be sorted from highest to lowest to identify the strongest potential recommendations for each user.
recommendations <- baseline_estimates %>%
group_by(name) %>%
slice_max(
order_by = estimated_rating,
n = 1,
with_ties = FALSE
) %>%
ungroup() %>%
select(name, title, media_type, estimated_rating)
recommendations## # A tibble: 7 × 4
## name title media_type estimated_rating
## <chr> <chr> <chr> <dbl>
## 1 Daniel C Criminal Minds TV Show 4.62
## 2 Juilo C Criminal Minds TV Show 4.62
## 3 Julie E Criminal Minds TV Show 5.62
## 4 Krystal C Criminal Minds TV Show 4.62
## 5 Lesley R Criminal Minds TV Show 5.62
## 6 Mike E Criminal Minds TV Show 4.62
## 7 Sabrina H Lioness TV Show 5.62The Global Baseline Estimate uses the overall rating average along with user and title effects to estimate missing ratings. These estimates are then used to identify the highest predicted title for each user. This approach accounts for differences in how individual users rate titles as well as differences in how individual titles are rated overall.