MetacriticRatings - User vs Critic

Jesus Ramos
September 24 2015

Motivation & Business Question

  1. Video games are now deeply embedded in western culture.
  2. VG industry is now a whopping $18bn (2011).
  3. Most titles are aimed at a predominantly young male audience.
  4. This represents a barrier of entry for more 'art house' or 'conceptual' titles.
  5. Ratings from critics offer a counterweight for these titles.

Question

Do ratings from users reliably predict ratings from critics?

Methodology

  1. Gather and clean data from Metacritic website for the top-rated 100 VGs of all time for all 4th-gen platforms (PC, PS4, XBoxOne, WiiU).
  2. Explore the relationship between critics' and users' ratings via a linear model.
  3. Try to predict the rating from critics of a new game for a given platform given a user rating.

Linear Plot and Model

plot of chunk unnamed-chunk-1

                           Estimate Std. Error Pr(>|t|)
(Intercept)                    8.74       0.40     0.00
UserRating                     0.05       0.05     0.28
PlatformPS4                   -1.21       0.56     0.03
PlatformWiiU                  -3.92       0.60     0.00
PlatformXBoxOne               -0.92       0.52     0.08
UserRating:PlatformPS4         0.06       0.07     0.37
UserRating:PlatformWiiU        0.35       0.07     0.00
UserRating:PlatformXBoxOne    -0.02       0.07     0.74

According to the figures, every 1-point increment in user ratings brings a 0.28 increase in critic ratings. Though this relationship is very significant at a p-value of 0, it only explains 15.98% of the variance. However, if we control for platform we can account for 60.37% of the variance, but then the only good predictor between ratings is for WiiU.

Conclusion & Pitch

  1. Even though the prediction for platforms other than WiiU may not be so reliable, it is nonetheless interesting to observe how for more 'hardcore' platforms users give a wider range of ratings than critics, while for more 'family-friendly' platforms users and critics are more consistent.
  2. To explore this relationship, the difference in ratings for the titles, and to estimate a critic rating from a user rating (regardless of the confidence on the prediction) for a new game, we have developed a Shiny app.
  3. The Shiny app can be found here. We submit it for your consideration. Enjoy!