The article “Be Suspicious Of Online Movie Ratings, Especially Fandango’s” by Walt Hickey published at https://fivethirtyeight.com/features/fandango-movies-ratings/ outlines the disparity in ratings between the movie ticket-selling company Fandango against the ratings of aggregation sites such as Rotten Tomatoes, Metacritic, and IMDB. The article indicates that Fandango movie ratings are typically higher than the users and the critics ratings from the other aggregation web sites. The author of the article also identified a likely defect in the Fandango web site display of ratings which always rounds a movie’s rating up to the nearest half-star which is counter to normal rounding standards. According to the author’s research, due to Fandango’s rating algoritm, essentially all movies on the Fandango site are rated 3 stars or above on a 5-star scale.
I load the raw csv file from the fivethirtyeight Github repo into an R data frame and output the number of rows and columns as a sanity check against the information provided on the Github repo. The repo page does identify 147 entries in the csv dataset. I also perform the head function to visually see some of the records.
# Import RCurl to pull the csv file from the Github repo
library(RCurl)## Loading required package: bitops# Load the csv file from the repo's URL
movings_ratings_url <- getURL("https://raw.githubusercontent.com/fivethirtyeight/data/master/fandango/fandango_score_comparison.csv")
movie_ratings <-data.frame(read.csv(text=movings_ratings_url, header=T))
# Output initial dimensions of the data frame object
dim(movie_ratings)## [1] 146 22head(movie_ratings)## FILM RottenTomatoes RottenTomatoes_User
## 1 Avengers: Age of Ultron (2015) 74 86
## 2 Cinderella (2015) 85 80
## 3 Ant-Man (2015) 80 90
## 4 Do You Believe? (2015) 18 84
## 5 Hot Tub Time Machine 2 (2015) 14 28
## 6 The Water Diviner (2015) 63 62
## Metacritic Metacritic_User IMDB Fandango_Stars Fandango_Ratingvalue
## 1 66 7.1 7.8 5.0 4.5
## 2 67 7.5 7.1 5.0 4.5
## 3 64 8.1 7.8 5.0 4.5
## 4 22 4.7 5.4 5.0 4.5
## 5 29 3.4 5.1 3.5 3.0
## 6 50 6.8 7.2 4.5 4.0
## RT_norm RT_user_norm Metacritic_norm Metacritic_user_nom IMDB_norm
## 1 3.70 4.3 3.30 3.55 3.90
## 2 4.25 4.0 3.35 3.75 3.55
## 3 4.00 4.5 3.20 4.05 3.90
## 4 0.90 4.2 1.10 2.35 2.70
## 5 0.70 1.4 1.45 1.70 2.55
## 6 3.15 3.1 2.50 3.40 3.60
## RT_norm_round RT_user_norm_round Metacritic_norm_round
## 1 3.5 4.5 3.5
## 2 4.5 4.0 3.5
## 3 4.0 4.5 3.0
## 4 1.0 4.0 1.0
## 5 0.5 1.5 1.5
## 6 3.0 3.0 2.5
## Metacritic_user_norm_round IMDB_norm_round Metacritic_user_vote_count
## 1 3.5 4.0 1330
## 2 4.0 3.5 249
## 3 4.0 4.0 627
## 4 2.5 2.5 31
## 5 1.5 2.5 88
## 6 3.5 3.5 34
## IMDB_user_vote_count Fandango_votes Fandango_Difference
## 1 271107 14846 0.5
## 2 65709 12640 0.5
## 3 103660 12055 0.5
## 4 3136 1793 0.5
## 5 19560 1021 0.5
## 6 39373 397 0.5I remove several of the columns from the dataset that pertain to critics’ ratings. As the Fandango data is based on user voting, I have decided to only include user rating data from the aggregation sites to ensure a fair comparison.
# Remove the columns from Rotten Tomatoes and Metacritic for the critics reviews
# Considering Fandango only allows for user voting, only keep data from equivalent users for comparison
movie_ratings <- movie_ratings[,c('FILM', 'RottenTomatoes_User', 'Metacritic_User', 'IMDB', 'Fandango_Stars', 'Fandango_Ratingvalue', 'RT_user_norm', 'Metacritic_user_nom', 'IMDB_norm', 'RT_user_norm_round', 'Metacritic_user_norm_round', 'IMDB_norm_round', 'Metacritic_user_vote_count', 'IMDB_user_vote_count', 'Fandango_votes', 'Fandango_Difference')]
dim(movie_ratings)## [1] 146 16I rename several of the columns to remove abbreviations and shortened words to allow for better readability of the columns. The column names tend to get long with this approach, but I do believe it adds clarity to the resulting document.
library(plyr)
movie_ratings <- rename(movie_ratings ,c("FILM"="Movie","IMDB"="IMDB_User","Fandango_Ratingvalue"="Fandango_User_Rating","RT_user_norm"="RottenTomatoes_user_normalized","Metacritic_user_nom"="Metacritic_user_normalized","IMDB_norm"="IMDB_normalized", "RT_user_norm_round"="RottenTomatoes_user_normalized_rounded", "Metacritic_user_norm_round"="Metacritic_user_normalized_rounded","IMDB_norm_round"="IMDB_normalized_rounded"))To more easily compare the Fandango star ratings to the normalized user ratings of the three aggregation sites, I calculate the difference in resulting stars and then add that result as a column to the movie ratings data frame. By calculating and adding the three derived columns to the primary data frame, the data is now readily available to assist in identifying potentially meaningful differences in the star ratings. I consider the movie_ratings data frame with the derived columns as the primary dataset for any analysis to be performed.
# Calculate difference between Rotten Tomatoes normalized 5-star score with Fandango
# Add derived column to data frame
movie_ratings$RottenTomatoes_Fandango_Star_Difference <- with(movie_ratings,
Fandango_Stars - RottenTomatoes_user_normalized_rounded)
# Calculate difference between Metacritic normalized 5-star score with Fandango
# Add derived column to data frame
movie_ratings$Metacritic_Fandango_Star_Difference <- with(movie_ratings,
Fandango_Stars - Metacritic_user_normalized_rounded)
# Calculate difference between IMDB normalized 5-star score with Fandango
# Add derived column to data frame
movie_ratings$IMDB_Fandango_Star_Difference <- with(movie_ratings,
Fandango_Stars - IMDB_normalized_rounded)
dim(movie_ratings)## [1] 146 19As the article highlights a meaningful disparity in the Fandango web site ratings against those of three aggregation sites, I created subset data frames for each of the three aggregation sites in which the Fandango star rating was at least one star or greater than the normalized users rating from the aggregation site. I thought it would be interesting to then be able to compare the resulting data frames in which the discrepancy is at least a whole star difference. The following subsets allow for analysis between Fandango and each of the aggregation sites.
#
ratings_diff_greater_than_5_rotten_tomates <- subset(movie_ratings, RottenTomatoes_Fandango_Star_Difference >= 1.0)
summary(ratings_diff_greater_than_5_rotten_tomates)## Movie RottenTomatoes_User Metacritic_User
## 5 Flights Up (2015) : 1 Min. :20.00 Min. :2.4
## A Little Chaos (2015): 1 1st Qu.:38.50 1st Qu.:4.8
## Aloha (2015) : 1 Median :52.00 Median :5.8
## American Ultra (2015): 1 Mean :50.55 Mean :5.7
## Annie (2014) : 1 3rd Qu.:61.25 3rd Qu.:6.8
## Black or White (2015): 1 Max. :84.00 Max. :8.4
## (Other) :74
## IMDB_User Fandango_Stars Fandango_User_Rating
## Min. :4.000 Min. :3.000 Min. :2.700
## 1st Qu.:5.500 1st Qu.:3.500 1st Qu.:3.400
## Median :6.300 Median :4.000 Median :3.800
## Mean :6.131 Mean :3.994 Mean :3.739
## 3rd Qu.:6.600 3rd Qu.:4.500 3rd Qu.:4.100
## Max. :7.800 Max. :5.000 Max. :4.800
##
## RottenTomatoes_user_normalized Metacritic_user_normalized IMDB_normalized
## Min. :1.000 Min. :1.20 Min. :2.000
## 1st Qu.:1.925 1st Qu.:2.40 1st Qu.:2.750
## Median :2.600 Median :2.90 Median :3.150
## Mean :2.527 Mean :2.85 Mean :3.066
## 3rd Qu.:3.062 3rd Qu.:3.40 3rd Qu.:3.300
## Max. :4.200 Max. :4.20 Max. :3.900
##
## RottenTomatoes_user_normalized_rounded Metacritic_user_normalized_rounded
## Min. :1.000 Min. :1.00
## 1st Qu.:2.000 1st Qu.:2.50
## Median :2.500 Median :3.00
## Mean :2.544 Mean :2.85
## 3rd Qu.:3.000 3rd Qu.:3.50
## Max. :4.000 Max. :4.00
##
## IMDB_normalized_rounded Metacritic_user_vote_count IMDB_user_vote_count
## Min. :2.000 Min. : 5.0 Min. : 952
## 1st Qu.:3.000 1st Qu.: 40.5 1st Qu.: 9467
## Median :3.000 Median : 80.0 Median : 19594
## Mean :3.087 Mean :136.4 Mean : 33107
## 3rd Qu.:3.500 3rd Qu.:167.5 3rd Qu.: 41617
## Max. :4.000 Max. :779.0 Max. :207211
##
## Fandango_votes Fandango_Difference
## Min. : 35.0 Min. :0.000
## 1st Qu.: 818.5 1st Qu.:0.100
## Median : 2169.5 Median :0.300
## Mean : 4018.5 Mean :0.255
## 3rd Qu.: 5483.0 3rd Qu.:0.400
## Max. :34846.0 Max. :0.500
##
## RottenTomatoes_Fandango_Star_Difference
## Min. :1.00
## 1st Qu.:1.00
## Median :1.50
## Mean :1.45
## 3rd Qu.:1.50
## Max. :2.50
##
## Metacritic_Fandango_Star_Difference IMDB_Fandango_Star_Difference
## Min. :-0.500 Min. :0.0000
## 1st Qu.: 0.500 1st Qu.:0.5000
## Median : 1.000 Median :1.0000
## Mean : 1.144 Mean :0.9062
## 3rd Qu.: 1.500 3rd Qu.:1.0000
## Max. : 3.000 Max. :2.5000
## The above data frame subset for Rotten Tomatoes compared to Fandango returns 80 entries in which the Fandango star rating is 1 or more stars greater than the Rotten Tomatoes normalized user rating.
#
ratings_diff_greater_than_5_metacritic <- subset(movie_ratings, Metacritic_Fandango_Star_Difference >= 1.0)
summary(ratings_diff_greater_than_5_metacritic)## Movie RottenTomatoes_User Metacritic_User
## Aloha (2015) : 1 Min. :20.00 Min. :2.400
## American Sniper (2015) : 1 1st Qu.:46.50 1st Qu.:4.625
## Annie (2014) : 1 Median :61.50 Median :5.950
## Ant-Man (2015) : 1 Mean :61.54 Mean :5.677
## Avengers: Age of Ultron (2015): 1 3rd Qu.:81.00 3rd Qu.:6.900
## Child 44 (2015) : 1 Max. :94.00 Max. :8.200
## (Other) :68
## IMDB_User Fandango_Stars Fandango_User_Rating
## Min. :4.000 Min. :3.00 Min. :2.700
## 1st Qu.:5.825 1st Qu.:4.00 1st Qu.:3.700
## Median :6.600 Median :4.50 Median :4.100
## Mean :6.518 Mean :4.27 Mean :4.012
## 3rd Qu.:7.300 3rd Qu.:4.50 3rd Qu.:4.400
## Max. :8.400 Max. :5.00 Max. :4.800
##
## RottenTomatoes_user_normalized Metacritic_user_normalized IMDB_normalized
## Min. :1.000 Min. :1.200 Min. :2.000
## 1st Qu.:2.325 1st Qu.:2.312 1st Qu.:2.913
## Median :3.075 Median :2.975 Median :3.300
## Mean :3.077 Mean :2.839 Mean :3.259
## 3rd Qu.:4.050 3rd Qu.:3.450 3rd Qu.:3.650
## Max. :4.700 Max. :4.100 Max. :4.200
##
## RottenTomatoes_user_normalized_rounded Metacritic_user_normalized_rounded
## Min. :1.000 Min. :1.000
## 1st Qu.:2.500 1st Qu.:2.500
## Median :3.000 Median :3.000
## Mean :3.095 Mean :2.818
## 3rd Qu.:4.000 3rd Qu.:3.500
## Max. :4.500 Max. :4.000
##
## IMDB_normalized_rounded Metacritic_user_vote_count IMDB_user_vote_count
## Min. :2.000 Min. : 4.0 Min. : 243
## 1st Qu.:3.000 1st Qu.: 49.5 1st Qu.: 14204
## Median :3.500 Median : 101.5 Median : 25174
## Mean :3.284 Mean : 209.5 Mean : 53063
## 3rd Qu.:3.500 3rd Qu.: 209.8 3rd Qu.: 50401
## Max. :4.000 Max. :1330.0 Max. :334164
##
## Fandango_votes Fandango_Difference
## Min. : 38 Min. :0.0000
## 1st Qu.: 1200 1st Qu.:0.1000
## Median : 3344 Median :0.3000
## Mean : 5936 Mean :0.2581
## 3rd Qu.: 7098 3rd Qu.:0.4000
## Max. :34846 Max. :0.5000
##
## RottenTomatoes_Fandango_Star_Difference
## Min. :0.000
## 1st Qu.:0.500
## Median :1.000
## Mean :1.176
## 3rd Qu.:1.500
## Max. :2.000
##
## Metacritic_Fandango_Star_Difference IMDB_Fandango_Star_Difference
## Min. :1.000 Min. :0.5000
## 1st Qu.:1.000 1st Qu.:0.5000
## Median :1.500 Median :1.0000
## Mean :1.453 Mean :0.9865
## 3rd Qu.:1.500 3rd Qu.:1.0000
## Max. :3.000 Max. :2.5000
## The above data frame subset for Metacritic compared to Fandango returns 74 entries in which the Fandango star rating is 1 or more stars greater than the Metacritic normalized user rating.
#
ratings_diff_greater_than_5_imdb <- subset(movie_ratings, IMDB_Fandango_Star_Difference >= 1.0)
summary(ratings_diff_greater_than_5_imdb)## Movie RottenTomatoes_User Metacritic_User
## 5 Flights Up (2015) : 1 Min. :20.00 Min. :2.400
## A Little Chaos (2015) : 1 1st Qu.:47.75 1st Qu.:5.300
## American Sniper (2015) : 1 Median :63.00 Median :6.500
## Annie (2014) : 1 Mean :62.72 Mean :6.069
## Ant-Man (2015) : 1 3rd Qu.:80.25 3rd Qu.:7.125
## Avengers: Age of Ultron (2015): 1 Max. :94.00 Max. :8.800
## (Other) :58
## IMDB_User Fandango_Stars Fandango_User_Rating
## Min. :4.000 Min. :3.000 Min. :2.700
## 1st Qu.:6.100 1st Qu.:4.000 1st Qu.:3.875
## Median :6.600 Median :4.500 Median :4.200
## Mean :6.520 Mean :4.375 Mean :4.089
## 3rd Qu.:7.225 3rd Qu.:4.500 3rd Qu.:4.400
## Max. :8.400 Max. :5.000 Max. :4.800
##
## RottenTomatoes_user_normalized Metacritic_user_normalized IMDB_normalized
## Min. :1.000 Min. :1.200 Min. :2.000
## 1st Qu.:2.388 1st Qu.:2.650 1st Qu.:3.050
## Median :3.150 Median :3.250 Median :3.300
## Mean :3.136 Mean :3.034 Mean :3.260
## 3rd Qu.:4.013 3rd Qu.:3.562 3rd Qu.:3.612
## Max. :4.700 Max. :4.400 Max. :4.200
##
## RottenTomatoes_user_normalized_rounded Metacritic_user_normalized_rounded
## Min. :1.000 Min. :1.000
## 1st Qu.:2.500 1st Qu.:2.500
## Median :3.000 Median :3.500
## Mean :3.164 Mean :3.039
## 3rd Qu.:4.000 3rd Qu.:3.500
## Max. :4.500 Max. :4.500
##
## IMDB_normalized_rounded Metacritic_user_vote_count IMDB_user_vote_count
## Min. :2.000 Min. : 6.0 Min. : 883
## 1st Qu.:3.000 1st Qu.: 46.5 1st Qu.: 12202
## Median :3.500 Median : 95.5 Median : 24348
## Mean :3.219 Mean : 204.7 Mean : 50727
## 3rd Qu.:3.500 3rd Qu.: 207.2 3rd Qu.: 50089
## Max. :4.000 Max. :1330.0 Max. :334164
##
## Fandango_votes Fandango_Difference
## Min. : 55 Min. :0.0000
## 1st Qu.: 995 1st Qu.:0.2000
## Median : 3388 Median :0.3000
## Mean : 6032 Mean :0.2859
## 3rd Qu.: 7050 3rd Qu.:0.4000
## Max. :34846 Max. :0.5000
##
## RottenTomatoes_Fandango_Star_Difference
## Min. :0.500
## 1st Qu.:0.875
## Median :1.000
## Mean :1.211
## 3rd Qu.:1.500
## Max. :2.500
##
## Metacritic_Fandango_Star_Difference IMDB_Fandango_Star_Difference
## Min. :-0.500 Min. :1.000
## 1st Qu.: 1.000 1st Qu.:1.000
## Median : 1.500 Median :1.000
## Mean : 1.336 Mean :1.156
## 3rd Qu.: 1.625 3rd Qu.:1.125
## Max. : 3.000 Max. :2.500
## The above data frame subset for IMDB compared to Fandango returns 64 entries in which the Fandango star rating is 1 or more stars greater than the IMDB normalized user rating.
Based on the article analysis and subset data frames above, the results do indicate a Fandango rating system not in-line with the three movie rating aggregation sites. Not only do the results indicate higher scores by Fandango but scores quite a bit higher than the aggregation sites. For future analysis, as the article was initially published in 2015, a good exercise would be to perform the same ratings’ analysis for movies released after the article’s published date. This subsequent analysis exercise would then confirm if Fandango performed any meaningful adjustments to its movie-rating algorithm to be more in-line with aggregation sites. Also, additional research should be performed to confirm that Fandango did fix the abnormal rounding practice on Fandango’s web site.