Assignment 3A: Global Baseline Estimate — Approach & Code Base

DATA 607 · Week 3

Author

Dillon Leeper

Overview

Most recommender systems rely on personalized algorithms such as content-based filtering or item-item collaborative filtering. The Global Baseline Estimate is a useful, simpler alternative: a non-personalized recommender that predicts a rating using only the overall rating distribution, how a given item tends to be rated relative to everyone else, and how a given user tends to rate relative to everyone else. It serves as a benchmark that any personalized algorithm should be expected to beat.

This document implements a Global Baseline Estimate recommender in R using the movie ratings survey data collected for Assignment 2A — the same 7-respondent, 6-movie dataset stored in PostgreSQL and documented in week02_2a_approach.qmd.

The Algorithm

For a user $u$ and item (movie) $i$, the baseline prediction is:

\[ \text{baseline}(u, i) = \mu + b_u + b_i \]

where:

$\mu$ — the overall average across all users and all movies
$b_i$ — the item bias: how much movie $i$’s average rating deviates from $\mu$
$b_u$ — the user bias: how much user $u$’s ratings deviate from $\mu$, after accounting for the items they rated

The result is typically clipped to the valid rating scale, since $\mu + b_u + b_i$ can fall outside the original bounds.

Planned Approach

1. Organize the Data

Import the survey data and reshape it from wide format (one row per respondent, one column per movie) into long/tidy format with tidyr::pivot_longer(), so each row represents a single (user, movie, rating) observation. Missing/unrated cells will be filtered out rather than treated as zero ratings.

2. Compute the Global Mean

Calculate $\mu$ as the mean of every recorded rating in the long-format data frame, ignoring missing values.

3. Compute Item (Movie) Bias

For each movie, compute:

\[ b_i = \overline{\text{rating}_i} - \mu \]

grouping by movie and averaging its ratings, then subtracting the global mean.

4. Compute User Bias

For each user, correct for item bias before averaging: subtract both $\mu$ and the relevant movie’s $b_i$ from each of that user’s ratings, then average the residuals:

\[ b_u = \text{mean}(\text{rating}_{u,i} - \mu - b_i) \]

5. Compute Baseline Predictions

Combine $\mu$, $b_u$, and $b_i$ for every (user, movie) pair — including pairs the user has not yet rated — to produce a predicted rating, clipped to the valid scale (1–5).

6. Generate Recommendations

For each user, rank the movies they have not yet rated by predicted baseline score and surface the top-ranked titles as recommendations.

R Implementation

library(tidyverse)

Dataset

This reproduces the exact 7-respondent × 6-movie survey table from Assignment 2A (week02_2a_approach.qmd), where NA marks “haven’t seen it” / unanswered cells — never a 0 rating.

ratings_wide <- tribble(
  ~user,          ~`Everything Everywhere All at Once`, ~`The Batman`, ~Hereditary, ~Obsession, ~`The Odyssey`, ~`Mean Girls`,
  "Eduardo",       4,  3, NA, NA,  5, NA,
  "Annie",         4,  3, NA, NA, NA,  5,
  "James",         2,  2, NA, NA,  4,  3,
  "JOYEEEEEEEE",  NA, NA, NA,  5,  4, NA,
  "Chris",         5, NA,  3,  5, NA,  5,
  "Garrett",      NA,  4, NA, NA,  4,  3,
  "Robyn",         1, NA, NA, NA,  4,  4
)

# To pull live from the Assignment 2A PostgreSQL database instead
# (same connection pattern as week02_2a_approach.qmd):
# library(DBI); library(RPostgres)
# con <- dbConnect(RPostgres::Postgres(),
#   host = Sys.getenv("PGHOST", "localhost"), port = as.integer(Sys.getenv("PGPORT", "5432")),
#   dbname = Sys.getenv("PGDATABASE", "movie_ratings"), user = Sys.getenv("PGUSER", "postgres"),
#   password = Sys.getenv("PGPASSWORD"))
# ratings_long <- dbGetQuery(con, "SELECT u.name AS user, m.title AS movie, r.rating
#                             FROM ratings r JOIN users u ON r.user_id = u.user_id
#                             JOIN movies m ON r.movie_id = m.movie_id") |> filter(!is.na(rating))
# dbDisconnect(con)

ratings_wide

Reshape to Long Format

ratings_long <- ratings_wide |>
  pivot_longer(-user, names_to = "movie", values_to = "rating") |>
  filter(!is.na(rating))

ratings_long

Global Mean

mu <- mean(ratings_long$rating)
mu

[1] 3.727273

Item Bias

item_bias <- ratings_long |>
  group_by(movie) |>
  summarize(b_i = mean(rating) - mu, n_ratings = n(), .groups = "drop")

item_bias

User Bias

user_bias <- ratings_long |>
  left_join(item_bias, by = "movie") |>
  group_by(user) |>
  summarize(b_u = mean(rating - mu - b_i), n_ratings = n(), .groups = "drop")

user_bias

Baseline Predictions

Predictions for every (user, movie) pair, clipped to the 1–5 scale, joined back against the actual rating where one exists.

baseline_predictions <- expand_grid(
  user  = unique(ratings_long$user),
  movie = unique(ratings_long$movie)
) |>
  left_join(user_bias |> select(user, b_u), by = "user") |>
  left_join(item_bias |> select(movie, b_i), by = "movie") |>
  mutate(
    predicted_rating = pmin(pmax(mu + b_u + b_i, 1), 5)
  ) |>
  left_join(ratings_long, by = c("user", "movie")) |>
  rename(actual_rating = rating)

baseline_predictions

Validation

Fit Against Known Ratings

We evaluate model accuracy on observed data using Root Mean Squared Error (RMSE).

# --- 1. Filter Down to Known Ratings ---
known_ratings <- baseline_predictions |> 
  filter(!is.na(actual_rating))

# --- 2. Calculate Root Mean Squared Error (RMSE) ---
rmse <- sqrt(mean((known_ratings$predicted_rating - known_ratings$actual_rating)^2))

# Display the RMSE value
rmse

[1] 0.6169328

# --- 3. Interpretation & Scale Context ---
# On a 1 to 5 rating scale:
# - 0.0 represents perfect prediction (zero error)
# - 4.0 represents the maximum possible error
# An RMSE of ~0.62 means our predictions are off by an average of roughly 0.6 stars.

Manual Spot Check

To ensure our automated data pipeline is reliable, we perform a manual spot check on a single prediction (Eduardo’s score for Mean Girls).

By calculating this prediction directly using basic arithmetic and comparing it to the output generated through our multi-step left_join() pipeline, we verify two critical things:

Mathematical Accuracy: The join logic executes the formula ($\mu + b_u + b_i$) exactly as intended.
Data Integrity: The automated pipeline has not scrambled user profiles or movie attributes during processing.

manual_pred <- pmin(pmax(
  mu +
    user_bias$b_u[user_bias$user == "Eduardo"] +
    item_bias$b_i[item_bias$movie == "Mean Girls"],
  1), 5)

computed_pred <- baseline_predictions |>
  filter(user == "Eduardo", movie == "Mean Girls") |>
  pull(predicted_rating)

tibble(manual_pred, computed_pred, match = isTRUE(all.equal(manual_pred, computed_pred)))

Recommendations

For each respondent, the top-ranked movies they haven’t rated, by predicted baseline score.

recommendations <- baseline_predictions |>
  filter(is.na(actual_rating)) |>
  group_by(user) |>
  arrange(desc(predicted_rating), .by_group = TRUE) |>
  slice_head(n = 3) |>
  select(user, movie, predicted_rating) |>
  ungroup()

recommendations

Visualization

baseline_predictions |>
  mutate(known = !is.na(actual_rating)) |>
  ggplot(aes(x = movie, y = user, fill = predicted_rating)) +
  geom_tile(color = "white", linewidth = 0.6) +
  geom_text(aes(label = ifelse(known, as.character(actual_rating), round(predicted_rating, 1))),
            color = "white", fontface = "bold") +
  scale_fill_gradient(low = "#4C72B0", high = "#C44E52", name = "Predicted\nrating") +
  labs(
    title = "Global Baseline Estimate: Predicted vs. Actual Ratings",
    subtitle = "Bold numbers = actual rating given · faint-background numbers = predicted (recommendation candidates)",
    x = NULL, y = NULL
  ) +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 30, hjust = 1))

Conclusions

The Global Baseline Estimate guesses how someone would rate a movie they haven’t seen using just three numbers: the overall average rating, how generous or harsh that person tends to rate, and how well-liked that movie is overall. It’s a simple baseline, and any fancier recommender should be able to beat it.

Findings: On the 22 ratings we actually collected, predictions were off by about 0.6 stars on average (RMSE ≈ 0.62) on a 1–5 scale. That’s a believable guess, but it’s a low bar since we’re checking accuracy on the same ratings the model was built from. The recommendations table matches intuition pretty well too: people who rated things generously, like Chris, get more optimistic predictions across the board, while stingier raters, like Robyn, get lower ones.

Limitations: We only had 7 respondents and 6 movies, and only 22 of the 42 possible (user, movie) pairs actually got rated. That means both the user bias and movie bias numbers come from very little data. One rating from a single person, or for a single movie, can swing that bias a lot, so the biases (and the recommendations built from them) shouldn’t be trusted as much as they would with a bigger survey.

Ideas to extend or verify this work:

Collect more responses per movie and more ratings per person so the bias estimates aren’t so shaky.
Hold out a chunk of the real ratings and predict against that instead of checking accuracy on the same data the model trained on. That gives a more honest read on predictive accuracy.
Compare this baseline against something a bit smarter, like item-item collaborative filtering, to actually confirm it gets beaten. That’s the whole point of having a baseline in the first place.

AI Disclosure and Citation

Claude (Sonnet 5, accessed through Claude for Cowork) reviewed this document’s Code Base section on September 20, 2026. It found and removed a duplicated calculation in the Validation section (the manual spot-check math had been run twice by mistake) and drafted this Conclusions section based on the analysis results already in the document.

[Add a citation here for whatever AI tool(s) you used to draft the original Approach and Code Base content: model name, version, developer, and access date, matching the format below.]

Anthropic. (2026). Claude (Sonnet 5) [Large language model]. https://claude.ai/. Accessed September 20, 2026.

--- title: "Assignment 3A: Global Baseline Estimate — Approach & Code Base" subtitle: "DATA 607 · Week 3" author: "Dillon Leeper" format: html: theme: cosmo toc: true toc-location: left toc-depth: 3 code-tools: true df-print: paged smooth-scroll: true embed-resources: true css: styles.css execute: warning: false message: false editor: visual --- ## Overview Most recommender systems rely on personalized algorithms such as content-based filtering or item-item collaborative filtering. The **Global Baseline Estimate** is a useful, simpler alternative: a non-personalized recommender that predicts a rating using only the overall rating distribution, how a given item tends to be rated relative to everyone else, and how a given user tends to rate relative to everyone else. It serves as a benchmark that any personalized algorithm should be expected to beat. This document implements a Global Baseline Estimate recommender in R using the movie ratings survey data collected for Assignment 2A — the same 7-respondent, 6-movie dataset stored in PostgreSQL and documented in `week02_2a_approach.qmd`. ## The Algorithm For a user $u$ and item (movie) $i$, the baseline prediction is: $$ \text{baseline}(u, i) = \mu + b_u + b_i $$ where: - $\mu$ — the overall average across all users and all movies - $b_i$ — the item bias: how much movie $i$'s average rating deviates from $\mu$ - $b_u$ — the user bias: how much user $u$'s ratings deviate from $\mu$, after accounting for the items they rated The result is typically clipped to the valid rating scale, since $\mu + b_u + b_i$ can fall outside the original bounds. ## Planned Approach ### 1. Organize the Data Import the survey data and reshape it from wide format (one row per respondent, one column per movie) into long/tidy format with `tidyr::pivot_longer()`, so each row represents a single (user, movie, rating) observation. Missing/unrated cells will be filtered out rather than treated as zero ratings. ### 2. Compute the Global Mean Calculate $\mu$ as the mean of every recorded rating in the long-format data frame, ignoring missing values. ### 3. Compute Item (Movie) Bias For each movie, compute: $$ b_i = \overline{\text{rating}_i} - \mu $$ grouping by movie and averaging its ratings, then subtracting the global mean. ### 4. Compute User Bias For each user, correct for item bias before averaging: subtract both $\mu$ and the relevant movie's $b_i$ from each of that user's ratings, then average the residuals: $$ b_u = \text{mean}(\text{rating}_{u,i} - \mu - b_i) $$ ### 5. Compute Baseline Predictions Combine $\mu$, $b_u$, and $b_i$ for every (user, movie) pair — including pairs the user has not yet rated — to produce a predicted rating, clipped to the valid scale (1–5). ### 6. Generate Recommendations For each user, rank the movies they have not yet rated by predicted baseline score and surface the top-ranked titles as recommendations. ## R Implementation ```{r} #| label: setup library(tidyverse) ``` ### Dataset This reproduces the exact 7-respondent × 6-movie survey table from Assignment 2A (`week02_2a_approach.qmd`), where `NA` marks "haven't seen it" / unanswered cells — never a 0 rating. ```{r} #| label: build-dataset ratings_wide <- tribble( ~user, ~`Everything Everywhere All at Once`, ~`The Batman`, ~Hereditary, ~Obsession, ~`The Odyssey`, ~`Mean Girls`, "Eduardo", 4, 3, NA, NA, 5, NA, "Annie", 4, 3, NA, NA, NA, 5, "James", 2, 2, NA, NA, 4, 3, "JOYEEEEEEEE", NA, NA, NA, 5, 4, NA, "Chris", 5, NA, 3, 5, NA, 5, "Garrett", NA, 4, NA, NA, 4, 3, "Robyn", 1, NA, NA, NA, 4, 4 ) # To pull live from the Assignment 2A PostgreSQL database instead # (same connection pattern as week02_2a_approach.qmd): # library(DBI); library(RPostgres) # con <- dbConnect(RPostgres::Postgres(), # host = Sys.getenv("PGHOST", "localhost"), port = as.integer(Sys.getenv("PGPORT", "5432")), # dbname = Sys.getenv("PGDATABASE", "movie_ratings"), user = Sys.getenv("PGUSER", "postgres"), # password = Sys.getenv("PGPASSWORD")) # ratings_long <- dbGetQuery(con, "SELECT u.name AS user, m.title AS movie, r.rating # FROM ratings r JOIN users u ON r.user_id = u.user_id # JOIN movies m ON r.movie_id = m.movie_id") |> filter(!is.na(rating)) # dbDisconnect(con) ratings_wide ``` ### Reshape to Long Format ```{r} #| label: reshape-long ratings_long <- ratings_wide |> pivot_longer(-user, names_to = "movie", values_to = "rating") |> filter(!is.na(rating)) ratings_long ``` ### Global Mean ```{r} #| label: global-mean mu <- mean(ratings_long$rating) mu ``` ### Item Bias ```{r} #| label: item-bias item_bias <- ratings_long |> group_by(movie) |> summarize(b_i = mean(rating) - mu, n_ratings = n(), .groups = "drop") item_bias ``` ### User Bias ```{r} #| label: user-bias user_bias <- ratings_long |> left_join(item_bias, by = "movie") |> group_by(user) |> summarize(b_u = mean(rating - mu - b_i), n_ratings = n(), .groups = "drop") user_bias ``` ### Baseline Predictions Predictions for every (user, movie) pair, clipped to the 1–5 scale, joined back against the actual rating where one exists. ```{r} #| label: baseline-predictions baseline_predictions <- expand_grid( user = unique(ratings_long$user), movie = unique(ratings_long$movie) ) |> left_join(user_bias |> select(user, b_u), by = "user") |> left_join(item_bias |> select(movie, b_i), by = "movie") |> mutate( predicted_rating = pmin(pmax(mu + b_u + b_i, 1), 5) ) |> left_join(ratings_long, by = c("user", "movie")) |> rename(actual_rating = rating) baseline_predictions ``` ## Validation ### Fit Against Known Ratings We evaluate model accuracy on observed data using Root Mean Squared Error (RMSE). ```{r} #| label: validate-rmse # --- 1. Filter Down to Known Ratings --- known_ratings <- baseline_predictions |> filter(!is.na(actual_rating)) # --- 2. Calculate Root Mean Squared Error (RMSE) --- rmse <- sqrt(mean((known_ratings$predicted_rating - known_ratings$actual_rating)^2)) # Display the RMSE value rmse # --- 3. Interpretation & Scale Context --- # On a 1 to 5 rating scale: # - 0.0 represents perfect prediction (zero error) # - 4.0 represents the maximum possible error # An RMSE of ~0.62 means our predictions are off by an average of roughly 0.6 stars. ``` ### Manual Spot Check To ensure our automated data pipeline is reliable, we perform a manual spot check on a single prediction (Eduardo's score for *Mean Girls*). By calculating this prediction directly using basic arithmetic and comparing it to the output generated through our multi-step `left_join()` pipeline, we verify two critical things: 1. **Mathematical Accuracy:** The join logic executes the formula ($\mu + b_u + b_i$) exactly as intended. 2. **Data Integrity:** The automated pipeline has not scrambled user profiles or movie attributes during processing. ```{r} #| label: validate-manual-check manual_pred <- pmin(pmax( mu + user_bias$b_u[user_bias$user == "Eduardo"] + item_bias$b_i[item_bias$movie == "Mean Girls"], 1), 5) computed_pred <- baseline_predictions |> filter(user == "Eduardo", movie == "Mean Girls") |> pull(predicted_rating) tibble(manual_pred, computed_pred, match = isTRUE(all.equal(manual_pred, computed_pred))) ``` ## Recommendations For each respondent, the top-ranked movies they haven't rated, by predicted baseline score. ```{r} #| label: recommendations recommendations <- baseline_predictions |> filter(is.na(actual_rating)) |> group_by(user) |> arrange(desc(predicted_rating), .by_group = TRUE) |> slice_head(n = 3) |> select(user, movie, predicted_rating) |> ungroup() recommendations ``` ## Visualization ```{r} #| label: fig-baseline-heatmap #| fig-width: 8 #| fig-height: 5 baseline_predictions |> mutate(known = !is.na(actual_rating)) |> ggplot(aes(x = movie, y = user, fill = predicted_rating)) + geom_tile(color = "white", linewidth = 0.6) + geom_text(aes(label = ifelse(known, as.character(actual_rating), round(predicted_rating, 1))), color = "white", fontface = "bold") + scale_fill_gradient(low = "#4C72B0", high = "#C44E52", name = "Predicted\nrating") + labs( title = "Global Baseline Estimate: Predicted vs. Actual Ratings", subtitle = "Bold numbers = actual rating given · faint-background numbers = predicted (recommendation candidates)", x = NULL, y = NULL ) + theme_minimal() + theme(axis.text.x = element_text(angle = 30, hjust = 1)) ``` ## Conclusions The Global Baseline Estimate guesses how someone would rate a movie they haven't seen using just three numbers: the overall average rating, how generous or harsh that person tends to rate, and how well-liked that movie is overall. It's a simple baseline, and any fancier recommender should be able to beat it. **Findings:** On the 22 ratings we actually collected, predictions were off by about 0.6 stars on average (RMSE ≈ 0.62) on a 1–5 scale. That's a believable guess, but it's a low bar since we're checking accuracy on the same ratings the model was built from. The recommendations table matches intuition pretty well too: people who rated things generously, like Chris, get more optimistic predictions across the board, while stingier raters, like Robyn, get lower ones. **Limitations:** We only had 7 respondents and 6 movies, and only 22 of the 42 possible (user, movie) pairs actually got rated. That means both the user bias and movie bias numbers come from very little data. One rating from a single person, or for a single movie, can swing that bias a lot, so the biases (and the recommendations built from them) shouldn't be trusted as much as they would with a bigger survey. **Ideas to extend or verify this work:** - Collect more responses per movie and more ratings per person so the bias estimates aren't so shaky. - Hold out a chunk of the real ratings and predict against that instead of checking accuracy on the same data the model trained on. That gives a more honest read on predictive accuracy. - Compare this baseline against something a bit smarter, like item-item collaborative filtering, to actually confirm it gets beaten. That's the whole point of having a baseline in the first place. ## AI Disclosure and Citation Claude (Sonnet 5, accessed through Claude for Cowork) reviewed this document's Code Base section on September 20, 2026. It found and removed a duplicated calculation in the Validation section (the manual spot-check math had been run twice by mistake) and drafted this Conclusions section based on the analysis results already in the document. \[Add a citation here for whatever AI tool(s) you used to draft the original Approach and Code Base content: model name, version, developer, and access date, matching the format below.\] Anthropic. (2026). *Claude (Sonnet 5)* \[Large language model\]. <https://claude.ai/>. Accessed September 20, 2026.