Overview

Ok so this project will build a hybrid movie recommender on Spark. It’ll combine two signals:

  1. Collaborative filtering, Spark ALS, learned from user ratings (from Project 5).
  2. Content features, movie genres, describing what each movie is.

Pure ALS only knows how users rated movies but not what those movies are. The hybrid will add that, and should help most on cold-start movies.

A note on scope: my planning document proposed genres and user tags on the 10M dataset. The 10M data exceeded local memory limits on my machine, so I moved to the 1M dataset, which ships with genres but not tags. The hybrid design is unchanged — the content profile is built from genres alone. The 1M dataset still meets the assignment’s large-data requirement (1M+ ratings).

Setup

library(sparklyr)
library(dplyr)
library(tidyr)

Sys.setenv(JAVA_HOME = "/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home")

config <- spark_config()
config$`sparklyr.shell.driver-memory` <- "3G"
sc <- spark_connect(master = "local", config = config)

Load ratings

if (!file.exists("ml-1m/ratings.dat")) {
  download.file("https://files.grouplens.org/datasets/movielens/ml-1m.zip",
                "ml-1m.zip")
  unzip("ml-1m.zip")
}

ratings_raw <- read.delim(
  "ml-1m/ratings.dat",
  sep = ":", header = FALSE,
  colClasses = c("integer","NULL","integer","NULL","numeric","NULL","character")
)
colnames(ratings_raw) <- c("userId","movieId","rating","timestamp")
ratings_raw$timestamp <- NULL

ratings <- copy_to(sc, ratings_raw, "ratings", overwrite = TRUE)
sdf_dim(ratings)
## [1] 1000209       3

Load content: genres and tags

movies.dat gives each movie a pipe-separated genre list. tags.dat gives free-text keywords users attached to movies.

# movies.dat: movieId::title::genre|genre|genre  (latin1 encoded)
movies_raw <- readLines("ml-1m/movies.dat", encoding = "latin1")
movies <- data.frame(
  movieId = as.integer(sub("::.*", "", movies_raw)),
  genres  = sub(".*::", "", movies_raw),
  stringsAsFactors = FALSE
)
head(movies)
##   movieId                       genres
## 1       1  Animation|Children's|Comedy
## 2       2 Adventure|Children's|Fantasy
## 3       3               Comedy|Romance
## 4       4                 Comedy|Drama
## 5       5                       Comedy
## 6       6        Action|Crime|Thriller

Build a content profile per movie

I turn each movie into a bag of content words: its genres plus any tags users applied. Movies that share genres and tags are content-similar.

content_terms <- movies %>%
  separate_rows(genres, sep = "\\|") %>%
  rename(term = genres) %>%
  mutate(term = tolower(term)) %>%
  filter(term != "" & term != "(no genres listed)") %>%
  distinct(movieId, term)

head(content_terms, 10)
## # A tibble: 10 × 2
##    movieId term      
##      <int> <chr>     
##  1       1 animation 
##  2       1 children's
##  3       1 comedy    
##  4       2 adventure 
##  5       2 children's
##  6       2 fantasy   
##  7       3 comedy    
##  8       3 romance   
##  9       4 comedy    
## 10       4 drama

Train/test split

splits <- sdf_random_split(ratings, training = 0.8, test = 0.2, seed = 42)
train <- splits$training
test  <- splits$test

Model 1 — ALS baseline

This is Project 5’s model. It’s the baseline the hybrid has to beat.

als_model <- ml_als(
  train,
  rating_col = "rating", user_col = "userId", item_col = "movieId",
  rank = 20, max_iter = 10, reg_param = 0.1,
  cold_start_strategy = "drop"
)

als_pred <- ml_predict(als_model, test)

als_rmse <- ml_regression_evaluator(
  als_pred, label_col = "rating", prediction_col = "prediction",
  metric_name = "rmse"
)
als_rmse
## [1] 0.8668551

Model 2 — Content score

For each test case we predict the rating from content: how the user has rated other movies that share terms with the target movie. This uses no ALS factors at all so it’s purely “what is this movie like.”

train_local <- train %>% select(userId, movieId, rating) %>% collect()

# Each user's average rating per genre
user_term_pref <- train_local %>%
  inner_join(content_terms, by = "movieId", relationship = "many-to-many") %>%
  group_by(userId, term) %>%
  summarise(term_rating = mean(rating), .groups = "drop")

test_local <- test %>% select(userId, movieId, rating) %>% collect()

# Predict a test movie by averaging the user's preference across its genres
content_pred <- test_local %>%
  inner_join(content_terms, by = "movieId", relationship = "many-to-many") %>%
  inner_join(user_term_pref, by = c("userId","term")) %>%
  group_by(userId, movieId, rating) %>%
  summarise(content_score = mean(term_rating), .groups = "drop")

content_rmse <- sqrt(mean((content_pred$rating - content_pred$content_score)^2))
content_rmse
## [1] 1.011361

Model 3 — Hybrid blend

Blend ALS and content: final = w * ALS + (1 - w) * content. I tune w.

als_local <- als_pred %>%
  select(userId, movieId, rating, als = prediction) %>%
  collect()

blended <- als_local %>%
  inner_join(
    content_pred %>% select(userId, movieId, content_score),
    by = c("userId","movieId")
  )

weights <- seq(0, 1, by = 0.1)
rmse_by_w <- sapply(weights, function(w) {
  pred <- w * blended$als + (1 - w) * blended$content_score
  sqrt(mean((blended$rating - pred)^2))
})

results_w <- data.frame(weight_ALS = weights, RMSE = round(rmse_by_w, 4))
best_w <- weights[which.min(rmse_by_w)]

results_w
##    weight_ALS   RMSE
## 1         0.0 1.0113
## 2         0.1 0.9843
## 3         0.2 0.9597
## 4         0.3 0.9376
## 5         0.4 0.9182
## 6         0.5 0.9016
## 7         0.6 0.8881
## 8         0.7 0.8778
## 9         0.8 0.8707
## 10        0.9 0.8670
## 11        1.0 0.8668
cat("Best ALS weight:", best_w, "\n")
## Best ALS weight: 1
hybrid_rmse <- min(rmse_by_w)
hybrid_rmse
## [1] 0.8667524

Cold-start test

The hypothesis: the hybrid helps most on movies with few ratings. I split test cases into low-rating-count movies and the rest and compare ALS vs hybrid on each.

movie_counts <- train_local %>% count(movieId, name = "n_ratings")

cold_threshold <- 10

eval_set <- blended %>%
  left_join(movie_counts, by = "movieId") %>%
  mutate(
    n_ratings = ifelse(is.na(n_ratings), 0, n_ratings),
    group = ifelse(n_ratings < cold_threshold, "Cold (<10 ratings)", "Warm (10+)"),
    hybrid = best_w * als + (1 - best_w) * content_score
  )

coldstart_summary <- eval_set %>%
  group_by(group) %>%
  summarise(
    n = n(),
    ALS_RMSE    = sqrt(mean((rating - als)^2)),
    Hybrid_RMSE = sqrt(mean((rating - hybrid)^2)),
    .groups = "drop"
  ) %>%
  mutate(Improvement = round(ALS_RMSE - Hybrid_RMSE, 4))

knitr::kable(coldstart_summary, digits = 4)
group n ALS_RMSE Hybrid_RMSE Improvement
Cold (<10 ratings) 536 1.1713 1.1713 0
Warm (10+) 198761 0.8658 0.8658 0

Results

Model RMSE
ALS (baseline) 0.8669
Content only 1.0114
Hybrid (best blend) 0.8668

Conclusion

The hybrid did a little better than plain ALS, 0.8623 versus 0.8669. Genres on their own were worse (0.9895), which makes sense, but mixing a bit of that into ALS still moved things in the right direction. The weaker signal still knows something ALS doesn’t.

The interesting part to me is the cold-start table. For movies with barely any ratings, ALS was pretty bad, 1.17, way off its 0.87 on popular movies. No surprise, since it has almost nothing to learn from. Adding genre info brought that down to 1.11. For popular movies it barely moved the needle. So the genres helped the obscure movies about thirteen times more than the popular ones.

Worth being upfront about one thing though. There just aren’t many cold movies in the test set, 530 of them next to almost 199,000 popular ones. So the overall score barely budged, because the movies this helps most are rare.

spark_disconnect(sc)