Executive summary

My main question: How do people’s emotions expressed in movie reviews with the specific words they use?

When people watch movies, their emotional responses can vary widely, from joy and excitement to sadness and anger. These emotions are often reflected in the language they use when writing reviews. Understanding this connection can provide insights into what aspects of a movie evoke certain feelings and how review sentiment is linguistically constructed. This project seeks to uncover the relationship between the overall sentiment of a movie review (positive or negative) and the particular words and emotional tones embedded within the text.

In this report, I analyze the IMDb movie review dataset to explore the link between expressed sentiment and word choice. The final graphics will illustrate the overall distribution of positive and negative reviews, highlight the most frequently used words associated with each sentiment, and identify prevalent emotional words (e.g., joy, anger) that appear in positive versus negative reviews.

Data background

Explain where the data came from, what agency or company made it, how it is structured, what it shows, etc.

The dataset utilized for this analysis is IMDB Dataset.csv, a widely recognized collection of movie reviews. This dataset is a common resource for sentiment analysis tasks and is frequently found on platforms like Kaggle. While the specific original agency or company that compiled this exact version is not explicitly stated within the file, it is generally understood to be derived from the Internet Movie Database (IMDb) and other public movie review sources. Its public availability makes it a standard dataset for natural language processing and sentiment analysis research.

The dataset is structured as a CSV (Comma Separated Values) file, where each row represents an individual movie review. It primarily consists of two key columns: revies & sentiment.

review: This column contains the raw, unstructured text of the movie review, written by users. These reviews capture diverse opinions and expressions about films.

sentiment: This column provides a pre-labeled categorical sentiment for each review, indicating whether it is positive or negative. This pre-existing sentiment label is fundamental for our analysis, allowing us to categorize and compare the language used in reviews based on their overall emotional tone.

The straightforward structure of this dataset makes it highly suitable for text mining and sentiment analysis, enabling us to directly investigate the linguistic patterns that characterize positive and negative movie feedback.

Data loading, cleaning and preprocessing

Describe and show how you cleaned and reshaped the data

The first step in data preprocessing involves loading the IMDB Dataset.csv file into an R data frame. To ensure each review has a unique identifier, a review_id column is added. This column is useful for tracking individual reviews throughout the analysis.

Next, the core of the preprocessing involves transforming the unstructured review text into a “tidy” format suitable for text analysis. This is achieved using the unnest_tokens() function from the tidytext package. This function breaks down each review into individual words, placing each word on a new row and associating it with its original review_id and sentiment. During this tokenization, words are automatically converted to lowercase and most punctuation is removed, standardizing the text.

Finally, a crucial step in text cleaning is the removal of common English stop words (e.g., “the”, “a”, “is”, “of”). These words are very frequent but typically do not carry significant semantic meaning for sentiment analysis. By using anti_join() with the stop_words dataset, these irrelevant words are filtered out, allowing our analysis to focus on more emotionally or semantically relevant vocabulary.

# Load the dataset
imdb <- read_csv("IMDB Dataset.csv")
## Rows: 50000 Columns: 2
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (2): review, sentiment
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
# Add a unique review_id for later tracking, if necessary
imdb <- imdb %>%
  rowid_to_column(var = "review_id")

# Tokenize the reviews into individual words
tidy_reviews <- imdb %>%
  unnest_tokens(word, review)

# Remove stop words from the tidy reviews
tidy_reviews <- tidy_reviews %>%
  anti_join(stop_words, by = "word")

tidy_reviews
## # A tibble: 4,601,528 × 3
##    review_id sentiment word     
##        <int> <chr>     <chr>    
##  1         1 positive  reviewers
##  2         1 positive  mentioned
##  3         1 positive  watching 
##  4         1 positive  1        
##  5         1 positive  oz       
##  6         1 positive  episode  
##  7         1 positive  hooked   
##  8         1 positive  happened 
##  9         1 positive  br       
## 10         1 positive  br       
## # ℹ 4,601,518 more rows

Text data analysis

Individual analysis and figures

Anaysis and Figure 1

# Get AFINN lexicon
afinn <- get_sentiments("afinn")

# Calculate sentiment score for each review using AFINN
review_sentiment_scores <- tidy_reviews %>%
  inner_join(afinn, by = "word") %>%
  group_by(review_id, sentiment) %>%
  summarise(afinn_score = sum(value), .groups = 'drop')

# Plotting the distribution of AFINN scores, faceted by original sentiment
ggplot(review_sentiment_scores, aes(x = afinn_score, fill = sentiment)) +
  geom_histogram(binwidth = 10, color = "white") +
  facet_wrap(~ sentiment, scales = "free_y") + 
  labs(
    title = "Distribution of AFINN Sentiment Scores by Original Review Sentiment",
    x = "AFINN Sentiment Score (Sum of Word Scores)",
    y = "Number of Reviews",
    fill = "Original Sentiment"
  ) +
  theme_minimal() +
  NULL

This figure visualizes the distribution of sentiment scores, specifically using the AFINN lexicon, separately for positive and negative reviews. It was created by first joining our tokenized tidy_reviews with the AFINN lexicon to get a numerical sentiment score for each word. Then, for each unique review_id, the individual word scores were summed to get a total AFINN score for that review. Finally, these scores were grouped by the original sentiment label (positive or negative).

Histograms are an excellent choice for this figure as they display the distribution of a continuous variable (AFINN scores). By faceting the histograms by the original review sentiment, we can clearly compare the range and concentration of scores for positive reviews versus negative reviews. This allows for a detailed understanding of how numerical sentiment scores align with the given positive/negative labels, showing the spectrum of emotional intensity within each category.

Anaysis and Figure 2

# Get AFINN lexicon
afinn <- get_sentiments("afinn")

# Identify top words for positive reviews (AFINN score > 0)
positive_words_afinn <- tidy_reviews %>%
  filter(sentiment == "positive") %>%
  inner_join(afinn, by = "word") %>%
  filter(value > 0) %>% 
  count(word, sort = TRUE) %>%
  top_n(15, n) %>% 
  mutate(review_type = "Positive Reviews")

# Identify top words for negative reviews (AFINN score < 0)
negative_words_afinn <- tidy_reviews %>%
  filter(sentiment == "negative") %>%
  inner_join(afinn, by = "word") %>%
  filter(value < 0) %>%
  count(word, sort = TRUE) %>%
  top_n(15, n) %>% 
  mutate(review_type = "Negative Reviews")

# Combine for plotting
top_words_by_review_sentiment <- bind_rows(positive_words_afinn, negative_words_afinn) %>%
  mutate(word = fct_reorder(word, n))

# Plotting the top words
ggplot(top_words_by_review_sentiment, aes(x = word, y = n, fill = review_type)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~ review_type, scales = "free_y") +
  coord_flip() +
  labs(
    title = "Top 15 Most Frequent Positive and Negative Words by Review Sentiment",
    subtitle = "Words filtered by AFINN scores (positive words from positive reviews, negative words from negative reviews)",
    x = NULL,
    y = "Frequency"
  ) +
  theme_minimal() +
  NULL

This figure displays the most frequent single words associated with positive and negative sentiments. It was created by first joining our tidy_reviews data with the AFINN lexicon to get a sentiment score for each word. Then, we filtered for words that are positively scored (value > 0) within positive reviews and negatively scored (value < 0) within negative reviews. We then counted their frequencies and selected the top 15 words for each sentiment.

A faceted bar chart (facet_wrap) with flipped coordinates (coord_flip()) is chosen for this visualization. This allows for a clear, side-by-side comparison of the most prominent words in positive versus negative contexts. Faceting creates two separate panels, one for each sentiment type, preventing clutter. Flipping the coordinates ensures that the word labels are clearly readable, even for longer words. Using fct_reorder ensures that the bars within each facet are ordered by frequency, making it easy to identify the most impactful words.

Anaysis and Figure 3-1 (Positive reviews)

# Get AFINN lexicon
afinn <- get_sentiments("afinn")

# Identify top words for positive reviews (AFINN score > 0)
positive_words_for_cloud <- tidy_reviews %>%
  filter(sentiment == "positive") %>%
  inner_join(afinn, by = "word") %>%
  filter(value > 0) %>% 
  count(word, sort = TRUE) %>%
  top_n(150, n) 

# Ensure data frames have 'word' and 'freq' columns for wordcloud2
positive_df_for_wc <- data.frame(word = positive_words_for_cloud$word,
                                  freq = positive_words_for_cloud$n)

# Create the word cloud for positive words
wordcloud2(data = positive_df_for_wc,
           size = 0.8,
           minSize = 0,
           fontWeight = "normal",
           fontFamily = "Segoe UI",
           backgroundColor = "white",
           color = "darkgreen", 
           rotateRatio = 0.35, 
           shuffle = FALSE,
           ellipticity = 0.6)

Anaysis and Figure 3-2 (Negative reviews)

# Get AFINN lexicon
afinn <- get_sentiments("afinn")

# Identify top words for negative reviews (AFINN score < 0)
negative_words_for_cloud <- tidy_reviews %>%
  filter(sentiment == "negative") %>%
  inner_join(afinn, by = "word") %>%
  filter(value < 0) %>% 
  count(word, sort = TRUE) %>%
  top_n(150, n) 

# Ensure data frames have 'word' and 'freq' columns for wordcloud2
negative_df_for_wc <- data.frame(word = negative_words_for_cloud$word,
                                  freq = negative_words_for_cloud$n)

# Create the word cloud for negative words
wordcloud2(data = negative_df_for_wc,
           size = 1.0,
           minSize = 0.1,
           fontWeight = "normal",
           fontFamily = "Segoe UI",
           backgroundColor = "white",
           color = "darkred", 
           rotateRatio = 0.35,
           shuffle = FALSE,
           ellipticity = 0.6)

This figure visualizes the most frequent words from movie reviews, distinctly separated by sentiment (positive at the top, negative at the bottom), as depicted in the provided image. This helps in immediately identifying key vocabulary for each sentiment. Words are counted by their frequencies in positive and negative reviews from Analysis and Figure 2’s data.

Two separate word clouds are generated and displayed vertically. Positive words are colored green, and negative words are red, allowing for clear sentiment recognition. The size of each word indicates its frequency. This design offers a direct, intuitive, and truthful representation of the contrasting language used in positive versus negative movie reviews.

Conclusion

This report investigated how emotions in movie reviews are reflected in the words people use. My analysis revealed a strong connection between a review’s sentiment and its words. Positive reviews consistently used distinct positive words, while negative reviews employed characteristic negative terms. These findings highlight how language directly conveys emotional responses, offering valuable insights into audience perception.

There’s one more unfortunate point to mention: The wordcloud for Analysis and Figure 3-2 (Negative reviews) fails to display when converted to HTML. Despite seeking assistance from chatGPT, I was unfortunately unable to find a resolution. I will conclude this report by expressing my regret regarding this unresolved issue.