Sentiment Analysis for Fallout 3 and Fallout 4 Steam Reviews
Author
Trevor Reule
Published
May 2, 2024
Introduction
I’ve always been a huge fan of the Fallout video game series, but I’ve often wondered if others on Steam share the same sentiment. Steam reviews are often a funny, interactive way of seeing if a game is worth playing or not. Based on the reviews taken from the Steam API for Fallout 3 and Fallout 4 reviews, I will compare them in different ways and analyze.
# First thing to do is load up the necessary libraries for sentiment analysislibrary(tidyverse)
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.1.2 ✔ readr 2.1.4
✔ forcats 1.0.0 ✔ stringr 1.5.0
✔ ggplot2 3.4.3 ✔ tibble 3.2.1
✔ lubridate 1.9.2 ✔ tidyr 1.3.0
✔ purrr 1.0.2
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(tidytext)
Warning: package 'tidytext' was built under R version 4.3.3
library(textdata)
Warning: package 'textdata' was built under R version 4.3.3
# Next is loading in the data setfallout_combined_reviews <-read_csv("fallout_combined_reviews.csv")
Rows: 137 Columns: 23
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (3): reviews.language, reviews.review, game
dbl (14): reviews.recommendationid, reviews.author$steamid, reviews.author$n...
lgl (6): reviews.voted_up, reviews.steam_purchase, reviews.received_for_fre...
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Question 1: Is there a difference in average word counts between “positive” and “negative” reviews?
To answer this question, I am first going to find the average amount of words per review. I will then find the difference of the mean length in positive reviews and the mean length in negative reviews, which will tell me for both games which types of review had longer lengths. This could be useful for finding out why some people like or dislike the games.
# Find the average amount of words per reviewaverage_words <- fallout_combined_reviews %>%mutate(word_count =str_count(reviews.review, "\\w+")) %>%group_by(game, reviews.voted_up) %>%summarise(average_word_count =mean(word_count))
`summarise()` has grouped output by 'game'. You can override using the
`.groups` argument.
# Pivot the data to wide format for visualizationaverage_words_wide <- average_words %>%pivot_wider(names_from = reviews.voted_up, values_from = average_word_count)# Calculate the difference between the average word counts for TRUE and FALSE votesaverage_words_wide <- average_words_wide %>%mutate(difference =`TRUE`-`FALSE`)# Create the plotggplot(average_words_wide, aes(x = game, y = difference, fill = game, label =round(difference, 2))) +geom_bar(stat ="identity") +geom_text(position =position_stack(vjust =0.5), color ="white") +labs(title ="Difference in Average Word Count Between Positive and Negative Reviews",x ="Game",y ="Difference in Average Word Count",fill ="Game") +theme_minimal()
On average, reviews marked as “positive” (or as a thumbs-up on steam) for Fallout 3 were on average 8.24 words longer than “negative” reviews. This is not the case at all for Fallout 4 Reviews, where positive reviews were, on average, 19.28 words less than negative reviews. I have a hypothesis for this: Fallout 3 has been around for 7 more years than Fallout 4 has, which I think equates it more to having a “nostalgia factor”. I’ve seen many steam reviewers before write a review on a game that they have put thousands of hours into, and I feel that those people are more likely to write long, encouraging reviews for the older games that they like. Fallout 3 does not get any more current support, while Fallout 4 does, which means that people are still going to be more critical of the condition of their newer game. Since Fallout 3 is more of a ‘classic’, people are going to go easier on it.
Question 2: What are the most frequently used negative words in Fallout 3 and Fallout 4 reviews?
To answer this question, I combined the Fallout 3 and Fallout 4 reviews I took using Steam’s API and filtered out the most frequently used negative words. This is done using the NRC lexicon. This will help to understand why people may not like the Fallout games.
# To use a word cloud, we must first install the package for it and load itlibrary(wordcloud)
Warning: package 'wordcloud' was built under R version 4.3.3
Loading required package: RColorBrewer
# Tokenize the reviews into wordstidy_fallout_reviews <- fallout_combined_reviews %>%unnest_tokens(word, reviews.review)# Remove stop wordstidy_fallout_reviews <- tidy_fallout_reviews %>%anti_join(stop_words)
Joining with `by = join_by(word)`
# Perform sentiment analysis using the NRC lexiconnrc_sentiment <-get_sentiments("nrc")sentiment_analysis <- tidy_fallout_reviews %>%inner_join(nrc_sentiment, by ="word")
Warning in inner_join(., nrc_sentiment, by = "word"): Detected an unexpected many-to-many relationship between `x` and `y`.
ℹ Row 1 of `x` matches multiple rows in `y`.
ℹ Row 2592 of `y` matches multiple rows in `x`.
ℹ If a many-to-many relationship is expected, set `relationship =
"many-to-many"` to silence this warning.
# Filter negative sentiment wordsnegative_words <- sentiment_analysis %>%filter(sentiment =="negative")# Group by game and count the frequency of negative wordsnegative_word_counts <- negative_words %>%count(game, word, sort =TRUE)# Filter top 5 negative words for each gametop_negative_words <- negative_word_counts %>%group_by(game) %>%top_n(5, n)# Create word cloud with different colors for Fallout 3 and Fallout 4 negative wordswordcloud(words = top_negative_words$word, freq = top_negative_words$n, scale =c(4, 0.5), colors =c("red", "blue"), random.order =FALSE, rot.per =0.2, main ="Top 50 Negative Words in Fallout 3 and Fallout 4 Reviews")
Note: Fallout 3 review words are in blue, Fallout 4 review words are in red.
The word cloud words for Fallout 4 are pretty much what I expected from Steam reviews: Some people thought the game is too buggy, too many patches in the game, or as some eloquently put it, the game is just ‘shit’. The Negative words found frequently on Fallout 3 Steam reviews have more to do with Steam itself, however. It turns out that, after reading some of these reviews, it is impossible to launch Fallout 3 through Steam unless you’re running Windows. This isn’t a slight against Fallout 3 itself, but rather reviewers angry at the fact they can’t play their beloved game properly on the platform they purchased it on. This also may explain why Fallout 3’s positive reviews are longer than their negative reviews, because people can simply state that the game is flat out “broken” and not even detail what is actually wrong with the game.
Question 3: Are reviews generally more positive or negative over time for Fallout 3 and Fallout 4?
To do this, I will again use the NRC lexicon, but this time adding a time component to see if sentiments change over time. A smoothing line will be necessary, as the time stamps found on the Steam API are not able to be changed using lubridate; however, the graph will still tell a general story of the trajectory of Fallout 3 and Fallout 4’s average sentiment over time through Steam reviews.
# Tokenize the reviews into wordstidy_fallout_reviews_time <- fallout_combined_reviews %>%unnest_tokens(word, reviews.review)# Perform sentiment analysis using the NRC lexiconnrc_sentiment <-get_sentiments("nrc")sentiment_analysis <- tidy_fallout_reviews_time %>%inner_join(nrc_sentiment, by ="word")
Warning in inner_join(., nrc_sentiment, by = "word"): Detected an unexpected many-to-many relationship between `x` and `y`.
ℹ Row 1 of `x` matches multiple rows in `y`.
ℹ Row 2592 of `y` matches multiple rows in `x`.
ℹ If a many-to-many relationship is expected, set `relationship =
"many-to-many"` to silence this warning.
# Filter sentiment scores for Fallout 3 and Fallout 4fallout_sentiments_time <- sentiment_analysis# Group by game, year, and sentiment, and calculate the average sentiment scoreaverage_sentiments <- fallout_sentiments_time %>%group_by(game, reviews.timestamp_created, sentiment) %>%summarise(average_score =mean(as.numeric(sentiment =="positive") -as.numeric(sentiment =="negative")))
`summarise()` has grouped output by 'game', 'reviews.timestamp_created'. You
can override using the `.groups` argument.
# Plot the average sentiment scores over time for Fallout 3 and Fallout 4ggplot(average_sentiments, aes(x = reviews.timestamp_created, y = average_score, color = game)) +geom_point() +geom_smooth() +labs(title ="NRC Sentiments of Fallout 3 and Fallout 4 Reviews Over Time",x ="Time Stamp (Chronological)",y ="Average Sentiment Score",color ="Game") +theme_minimal()
`geom_smooth()` using method = 'loess' and formula = 'y ~ x'
The trajectory of the average sentiment scores based on Steam reviews for Fallout 3 is, as we’ve seen prior, overall more positive than those of Fallout 4. Fallout 3, as hypothesized earlier with my “nostalgia” theory, actually has a pretty positive positive sentiment analysis more recently than not, possibly because people are just remembering the game and their love for it. This is especially true because of the Fallout television series that recently came out, so I suspect the average sentiment for Fallout 3 to rise even more. Again, the main problem Steam users have with Fallout 3 is that it just doesn’t work, so that could also skew the data. Fallout 4’s trajectory is a bit more polarizing, as people seem to be more in disagreement about that game’s greatness. This is true for the game in general, because many people actually view Fallout 4 as the weakest in the franchise. All in all, it’s good to see that overall sentiment for both games is on an upwards trajectory as of late, because they truly are some of my favorite games ever.
