For the final project I am building a hybrid recommender system on the full BoardGameGeek dataset.15 million ratings from hundreds of thousands of users across 22,000 games, sampled down to approximately 5 million after filtering to active users and well rated games. The system is designed as a board game marketplace analogous to Steam, with two separate recommendation streams serving different user intents rather than one general purpose recommender trying to do everything at once. Basically two landing pages the user can go to depending on their mood at the time, or 2 separate lists that run in the background and populate for each user on the platform, something that is constantly changing to accommodate new interests and new game releases.
Source: BoardGameGeek Database from Kaggle (https://www.kaggle.com/datasets/threnjen/board-games-database-from-boardgamegeek)
Files: * bgg-15m-reviews.csv : 15
million ratings with user, game ID, rating, and comment columns *
games_detailed_info.csv : game metadata including title,
category, mechanics, complexity weight, year published, min/max
players
After filtering:
We already worked with a Top 2000 subset of this data in projects 4 and 5 so the file structure and loading approach are familiar.
The system is designed as a board game marketplace analogous to Steam game store. The core problem is the same one Steam faces, a catalog of 22000 games is too large for users to browse manually, so the recommender is the primary way users discover new things to play. Rather than one general purpose recommender, we let the users choose what they want at any given point in time using two separate streams with different goals:
Stream 1 : Discovery (Games You Will Love) Focused on surfacing games similar to what the user already likes, optimizing for relevance and retention. A user coming to this stream wants reliable recommendations they are very likely to enjoy. Evaluated on Precision, Recall, and NDCG. Recall matters here. A user who clicks into this stream wants reliable recommendations, not surprises. Getting this right keeps users on the platform and buying games.
Stream 2 : Surprise Me (Hidden Gems) This stream is about discovery. The goal is to push users toward niche, newer, or underrated games they would never find on their own. A user clicking Surprise Me is opting into being shown something unexpected. Novelty and serendipity are the priority here, with a minimum relevance floor so we are not just showing random obscure games. This stream also handles cold start for newly published games that have few ratings yet, the content based layer surfaces them based on mechanics and category similarity rather than waiting for enough ratings to accumulate. Recall is intentionally lower here, the stream is deliberately not showing mainstream games that would inflate recall, in favor of genuine discovery.
These two streams run off the same underlying ALS model but use different post-processing and ranking logic to serve different user intents.
Core: Spark ALS The backbone of both streams is ALS matrix factorization running on Spark. At millions of ratings and tens of thousands of users a pure R implementation is not viable as Ive personally experienced . The UUCF similarity matrix would be On squared in memory and SVD on the full unsampled matrix kill my aging PC. Spark ALS distributes the computation and handles this scale without needing to sample down to a minisule subset. We stil sample down here but thats just to save one time. My PC was able to handle the full 14 million, it was just taking way too long.
Content Layer for Cold Start and Stream 2 The metadata file has category, mechanics, and complexity weight for all 22000 games. We build TF-IDF vectors from category and mechanics fields and use cosine similarity to find similar games. This layer serves two purposes: - Cold start users with very few ratings get content based recommendations until enough collaborative signal accumulates - Stream 2 ranking: content features help identify games outside the users usual categories that share underlying mechanics they tend to enjoy
Stream Logic - Stream 1: ALS predicted ratings ranked by score, no penalty, standard top K - Stream 2: ALS predicted ratings with popularity penalty applied before ranking, boosting games with fewer ratings to the top, with content based fallback for cold start users
Parameters are tuned systematically with a clear anchor
ALS Rank Anchor: rank=10, consistent with SVD k=10 from projects 4 and 5. Project 5 parameter sweep showed rank had less impact than regularization. We experiment with rank = 5, 10, 20.
ALS Iterations Anchor: max_iter=10. Project 5 showed diminishing returns beyond 10 iterations. We test 5 and 10.
Regularization The most important parameter based on project 5 findings where reg_param was the clear driver. Anchor: reg_param=0.1. We test 0.1 and 0.5. With a recommender that gets new data constantly we need regularization that keeps the model flexible enough to adapt.
Popularity Penalty (Stream 2) Tuned separately from the ALS parameters since it only affects Stream 2 ranking not the underlying model. We sweep penalty values 1 through 5 and evaluate novelty vs Precision at each level to find the best tradeoff. The goal is to find the highest penalty that keeps Precision above a minimum floor of 0.15.
Cold Start Threshold The switchover point from collaborative to content based recommendations tested at different activity levels. We evaluate Precision separately for lower activity users vs warm users to confirm whether the content layer actually helps.
The evaluation goes beyond standard accuracy metrics to match what each stream is actually trying to achieve.
Accuracy metrics (Stream 1 and 2) - RMSE, MAE, NMAE on held out test set
Stream 1 : Discovery metrics - Precision: of the top 10 games recommended how many did the user actually rate highly. Primary metric for a retention focused stream - Recall: of all the games the user rated highly how many appeared in the top 10. Matters here because we want the stream to be comprehensive not just accurate - NDCG: ranking quality , a great recommendation at rank 1 counts more than one at rank 8 for a user browsing quickly
Stream 2 : Surprise Me metrics - Novelty: log inverse popularity, same definition as project 4. Higher means more obscure games being surfaced - Serendipity: games outside the users typical category profile that are still predicted to be rated highly. Measures whether Stream 2 is actually surprising users in a good way - Precision floor: Stream 2 still needs a relevance check. We report Precision for Stream 2 and aim to keep it above 0.15 even as we optimize for novelty - Recall is intentionally deprioritized for Stream 2 . Low recall is expected and acceptable when the goal is discovery not comprehensiveness
Cold start evaluation Precision@ compared between lower activity users using ALS vs content based recommendations, to show whether the TF-IDF layer actually helps when collaborative signal is limited.
What makes one recommender better than another for this use case For Stream 1 the answer is Precision@10, Recall@10, and NDCG. A stream that surfaces games users will enjoy, covers most of what they would love, and puts the best ones first is doing its job. For Stream 2 the answer is novelty and serendipity above the precision floor. RMSE alone cannot distinguish between these two goals which is exactly why recommender specific metrics matter.
Memory and runtime: The full dataset is 15 million rows. Aggressive filtering is needed to keep Spark from running out of memory in local mode, which may remove most true cold start users from the working set.
Content feature quality: The category and mechanics fields contain multiple values per game as strings. They need parsing and cleaning before TF-IDF can be applied cleanly.
Popularity penalty tuning: Finding the right penalty level for Stream 2 that balances novelty and relevance requires careful evaluation. Project 4 showed that too aggressive a penalty collapses precision entirely.
Serendipity measurement: Hard to measure offline. We define it as games outside the users top category profile that are still predicted above a rating threshold. This would just be an approximation of the real thing which would require online evaluation.
Kaminskas, M., & Bridge, D. (2016). Diversity, Serendipity, Novelty, and Coverage: A Survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems. ACM Transactions on Interactive Intelligent Systems, 7(1).
BoardGameGeek Dataset: https://www.kaggle.com/datasets/threnjen/board-games-database-from-boardgamegeek