The system I plan to build

I’m going to build a hybrid movie recommender on Spark. It’s going to combine two signals:

  1. Collaborative filtering Spark ALS, which learns from user rating behavior. This is the same model I built in Project 5.
  2. Content features movie genres and user applied tags. They describe what each movie actually is.

Pure ALS only knows how users rated movies. It has no idea what a movie is about. The hybrid adds that missing information and blends the two scores into one recommendation.

The dataset

MovieLens 10M has about 10 million ratings, 70,000 users, 10,000 movies. This clears the size requirement (1M+ ratings) on its own.

I used MovieLens for some of the other projets, but with this one the unique element is the hybrid layer. I pull in two files I did not use in earlier projects:

These turn each movie into a content profile.

Why this is more than Project 5

Project 5 ran ALS and stopped. This project adds a content model on top, blends the two, and tests whether the blend beats ALS alone. That comparison is the point of the project.

The specific problem the hybrid solves

ALS is not as good working with films with very few ratings. But those same movies still have genres and tags. The content model works fine there.

So my hypothesis is: the hybrid will beat pure ALS most on movies with few ratings.

How I will build it

  1. Load MovieLens 10M into Spark (from Project 5).
  2. Train ALS. Record RMSE. This is the baseline.
  3. Build content profiles from genres and tags.
  4. Compute a content-based score for each user-movie pair.
  5. Blend: final = w * ALS + (1 - w) * content. Tune the weight w.
  6. Compare hybrid RMSE to ALS RMSE, overall and on low-rating movies specifically.

How I will measure success

If the hybrid wins overall, and wins by more on cold-start movies, the project worked.

```