Assignment 10a

Approach

For this assignment, I will reproduce the sentiment analysis in Chapter 2 of Text Mining with R using Jane Austen’s novels. Then I will extend the analysis to restaurant reviews and compare the sentiment in written reviews with customer star ratings. I will also test an additional sentiment lexicon.

Part 1: Reproducing the Jane Austen Example

The following analysis follows Chapter 2 of Text Mining with R by Julia Silge and David Robinson. It uses the authors’ examples and adapts their code slightly for readability.

2.1 Sentiment Lexicons

The chapter introduces three lexicons: AFINN assigns numeric scores, Bing classifies words as positive or negative, and NRC identifies sentiment and emotions.

library(tidyverse)
library(tidytext)
library(janeaustenr)

# Load sentiment lexicons
afinn <- get_sentiments("afinn")
bing <- get_sentiments("bing")
nrc <- get_sentiments("nrc")

The dictionaries classify words differently, so their sentiment scores may not match exactly.

2.2 Sentiment Analysis with inner_join()

First, I convert the novels into tidy text, with one word per row and information about the original line and chapter.

# Prepare Jane Austen's novels as tidy text
tidy_books <- austen_books() %>%
  group_by(book) %>%
  mutate(
    linenumber = row_number(),
    chapter = cumsum(str_detect(text, regex("^chapter [\\divxlc]", ignore_case = TRUE)))
  ) %>%
  ungroup() %>%
  unnest_tokens(word, text)

tidy_books

Next, I use NRC to find the most frequent words associated with joy in Emma.

# Match words in Emma with the NRC joy category
nrc_joy <- get_sentiments("nrc") %>%
  filter(sentiment == "joy")

tidy_books %>%
  filter(book == "Emma") %>%
  inner_join(nrc_joy, by = "word") %>%
  count(word, sort = TRUE)

Words such as good, friend, and hope appear frequently, although a dictionary cannot always tell how a word is used in context.

I then use Bing to calculate net sentiment in consecutive sections of 80 lines across all six novels.

# Count positive and negative words in each section
jane_austen_sentiment <- tidy_books %>%
  inner_join(get_sentiments("bing"), by = "word") %>%
  count(book, index = linenumber %/% 80, sentiment) %>%
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
  mutate(sentiment = positive - negative)

jane_austen_sentiment

# Plot changes in sentiment across each novel
ggplot(jane_austen_sentiment, aes(index, sentiment, fill = book)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~ book, ncol = 2, scales = "free_x") +
  labs(x = "Section (80 lines)", y = "Net sentiment") +
  theme_minimal()

The plot shows how positive and negative language changes as each story progresses.

2.3 Comparing Three Sentiment Dictionaries

Following the chapter, I compare AFINN, Bing, and NRC on Pride and Prejudice. AFINN sums word scores, while Bing and NRC count positive words minus negative words.

# Select one novel for the comparison
pride_prejudice <- tidy_books %>%
  filter(book == "Pride & Prejudice")

# AFINN uses numerical word scores
afinn <- pride_prejudice %>%
  inner_join(get_sentiments("afinn"), by = "word") %>%
  group_by(index = linenumber %/% 80) %>%
  summarise(sentiment = sum(value), .groups = "drop") %>%
  mutate(method = "AFINN")

# Bing and NRC use positive/negative word categories
bing_and_nrc <- bind_rows(
  pride_prejudice %>%
    inner_join(get_sentiments("bing"), by = "word") %>%
    mutate(method = "Bing"),
  pride_prejudice %>%
    inner_join(
      get_sentiments("nrc") %>%
        filter(sentiment %in% c("positive", "negative")),
      by = "word"
    ) %>%
    mutate(method = "NRC")
) %>%
  count(method, index = linenumber %/% 80, sentiment) %>%
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
  mutate(sentiment = positive - negative)

# Compare sentiment trends from all three lexicons
bind_rows(afinn, bing_and_nrc) %>%
  ggplot(aes(index, sentiment, fill = method)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~ method, ncol = 1, scales = "free_y") +
  labs(x = "Section (80 lines)", y = "Sentiment score") +
  theme_minimal()

The three lexicons show broadly similar changes in sentiment, but the size of their scores differs because they classify and score words differently.

# Compare positive and negative entries in each dictionary
get_sentiments("nrc") %>%
  filter(sentiment %in% c("positive", "negative")) %>%
  count(sentiment)

get_sentiments("bing") %>%
  count(sentiment)

The dictionaries contain different proportions of positive and negative words, which can affect the results.

2.4 Most Common Positive and Negative Words

I use the Bing lexicon to identify which words contribute most to the sentiment counts.

# Count the most frequent sentiment words
bing_word_counts <- tidy_books %>%
  inner_join(get_sentiments("bing"), by = "word") %>%
  count(word, sentiment, sort = TRUE)

bing_word_counts

# Visualize the ten most frequent words in each category
bing_word_counts %>%
  group_by(sentiment) %>%
  slice_max(n, n = 10, with_ties = FALSE) %>%
  ungroup() %>%
  mutate(word = reorder(word, n)) %>%
  ggplot(aes(n, word, fill = sentiment)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~ sentiment, scales = "free_y") +
  labs(x = "Word count", y = NULL) +
  theme_minimal()

One important limitation is the word miss: Bing marks it as negative, but in Austen’s novels it is often a title used before a woman’s name. This shows why context matters in sentiment analysis.

Part 1 Summary

I reproduced the chapter’s main sentiment analyses: matching words to emotion categories, tracking sentiment throughout novels, comparing three lexicons, and identifying influential words. The results also show that lexicon-based methods can misinterpret words when context is missing.

Part 2: Yelp Restaurant Review Sentiment Analysis

Do restaurant star ratings agree with the sentiment expressed in written reviews?

I will use the Yelp Restaurant Reviews Sentiment Dataset published on Zenodo. It contains 14,351 reviews from restaurants in multiple U.S. cities, with review text and star ratings. Unlike the Jane Austen example, these texts express customers’ opinions about real experiences. I will calculate sentiment from the written reviews rather than use the dataset’s existing sentiment labels.

Dataset: https://doi.org/10.5281/zenodo.18723813

Import and Inspect the Data

Download yelp_sentiment_master_dataset.zip from Zenodo, extract it, and place the CSV file in a data folder beside this .qmd file. The code below finds the CSV automatically. The exact column names have not been independently verified, so the next chunk checks them before proceeding.

reviews_raw <- read_csv("yelp_sentiment_master_dataset.csv")

Rows: 14351 Columns: 11
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (7): text, date, name, categories, city, state, rating_review
dbl (4): Sl No, stars_review, stars_business, review_count

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

names(reviews_raw)

 [1] "Sl No"          "stars_review"   "text"           "date"          
 [5] "name"           "categories"     "city"           "state"         
 [9] "stars_business" "review_count"   "rating_review" 

Clean the Reviews

I will select the written review and star rating, remove missing values, and use a reproducible sample of up to 2,000 reviews. The code recognizes several common names for these fields and stops with a clear message if the dataset uses different ones.

# Select and clean restaurant reviews
reviews <- reviews_raw %>%
  transmute(
    review_id = row_number(),
    review_text = text,
    stars = stars_review,
    city = city,
    state = state
  ) %>%
  filter(
    !is.na(review_text),
    review_text != "",
    !is.na(stars),
    stars %in% 1:5
  ) %>%
  slice_head(n = 2000)

# Check the number of reviews by rating
reviews %>%
  count(stars)

I selected the review text, customer star ratings, and restaurant locations. I removed reviews with missing text or ratings and kept a sample of 2,000 reviews to make the analysis more manageable.

Convert Reviews to Tidy Text

As in Chapter 2, I split the review text into individual words using unnest_tokens().

# Create one row per word while retaining the review ID
review_words <- reviews %>%
  select(review_id, review_text) %>%
  unnest_tokens(word, review_text)

review_words

Calculate Sentiment with Bing

Bing labels individual words as positive or negative. I count both types of words in each review and calculate a net sentiment score: positive minus negative.

# Calculate Bing sentiment for each review
bing_scores <- review_words %>%
  inner_join(get_sentiments("bing"), by = "word") %>%
  count(review_id, sentiment) %>%
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
  mutate(bing_score = positive - negative) %>%
  select(review_id, bing_score)

reviews_scored <- reviews %>%
  left_join(bing_scores, by = "review_id") %>%
  mutate(bing_score = replace_na(bing_score, 0))

# Compare average Bing sentiment across star ratings
reviews_scored %>%
  group_by(stars) %>%
  summarise(reviews = n(), mean_bing = mean(bing_score), .groups = "drop")

The Bing sentiment scores generally increase with star ratings. One-star reviews have a negative average score (-0.83), while reviews with higher ratings contain more positive language.

Interestingly, four-star reviews have a slightly higher average sentiment score (5.56) than five-star reviews (5.37). This suggests that star ratings and sentiment scores do not always match perfectly. One possible explanation is that longer reviews contain more sentiment words, which could affect the scores.

Add a Fourth Lexicon: Loughran

The textbook already uses AFINN, Bing, and NRC. For my extension, I add Loughran, a fourth lexicon developed for financial text. I use its positive and negative categories to calculate a comparable score. Because it was not designed for restaurant reviews, it may miss important food-related language.

# Inspect Loughran categories
get_sentiments("loughran") %>% count(sentiment)

# Calculate Loughran net sentiment for each review
loughran_scores <- review_words %>%
  inner_join(
    get_sentiments("loughran") %>%
      filter(sentiment %in% c("positive", "negative")),
    by = "word"
  ) %>%
  count(review_id, sentiment) %>%
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
  mutate(loughran_score = positive - negative) %>%
  select(review_id, loughran_score)

reviews_scored <- reviews_scored %>%
  left_join(loughran_scores, by = "review_id") %>%
  mutate(loughran_score = replace_na(loughran_score, 0))

The Loughran lexicon contains six sentiment categories, including positive, negative, uncertainty, and litigious. Unlike Bing, it was originally designed for financial text. It also contains considerably more negative words than positive words, which may affect how it evaluates restaurant reviews.

Compare Sentiment with Star Ratings

I compare the average sentiment scores across the five rating groups. Since the two dictionaries have different vocabularies, their scores may differ in scale as well as direction.

# Summarize sentiment by star rating and lexicon
rating_sentiment <- reviews_scored %>%
  group_by(stars) %>%
  summarise(
    bing = mean(bing_score),
    loughran = mean(loughran_score),
    .groups = "drop"
  ) %>%
  pivot_longer(c(bing, loughran), names_to = "lexicon", values_to = "mean_score")

rating_sentiment

Both Bing and Loughran show a similar pattern: sentiment generally becomes more positive as restaurant star ratings increase. Interestingly, both lexicons give slightly higher average scores to four-star reviews than five-star reviews.

Bing produces higher positive scores overall, while Loughran identifies fewer positive words. This may be because Loughran was originally developed for financial documents rather than restaurant reviews.

# Plot the relationship between star ratings and average sentiment
rating_sentiment %>%
  ggplot(aes(x = stars, y = mean_score, color = lexicon)) +
  geom_line(linewidth = 1) +
  geom_point(size = 3) +
  scale_x_continuous(breaks = 1:5) +
  labs(
    title = "Written Review Sentiment by Star Rating",
    x = "Star rating",
    y = "Average net sentiment",
    color = "Lexicon"
  ) +
  theme_minimal()

The graph shows that average sentiment generally increases with restaurant star ratings for both lexicons. Interestingly, both Bing and Loughran reach their highest scores at four stars, followed by a slight decrease at five stars.

Bing produces higher positive scores than Loughran, although both show similar trends. This suggests that the choice of sentiment lexicon affects the scores, but both methods identify a positive relationship between written reviews and star ratings.

Reviews Where Ratings and Sentiment Disagree

A review can have a high star rating but contain more negative than positive words, or the reverse. I count these disagreements using Bing.

# Identify reviews with opposite rating and sentiment directions
reviews_scored %>%
  mutate(
    disagreement = case_when(
      stars >= 4 & bing_score < 0 ~ "High rating, negative words",
      stars <= 2 & bing_score > 0 ~ "Low rating, positive words",
      TRUE ~ "Other"
    )
  ) %>%
  count(disagreement)

I also looked for cases where star ratings and written sentiment did not match. Out of 2,000 reviews, 24 had high ratings but negative sentiment scores, while 156 had low ratings but positive sentiment scores.

Interestingly, disagreements were more common among lower-rated reviews. This could happen when customers mention positive aspects of their experience but still give a low overall rating.

Results and Comparison with Jane Austen

The analysis of 2,000 restaurant reviews showed that average sentiment generally increased with star ratings. Both Bing and Loughran followed a similar pattern, although Bing produced higher positive scores. Interestingly, both lexicons showed slightly higher sentiment for four-star reviews than five-star reviews.

I also found some differences between ratings and written sentiment. There were 24 reviews with high ratings but negative sentiment scores and 156 reviews with low ratings but positive sentiment scores. This suggests that star ratings do not always match the language used in reviews.

Compared with the Jane Austen example, which examined how sentiment changes throughout novels, my analysis focused on the relationship between written sentiment and numerical ratings. Both approaches use sentiment dictionaries, but restaurant reviews allow us to compare the results with customers’ actual ratings.

One limitation is that sentiment analysis based on individual words cannot fully understand context, sarcasm, or mixed opinions. Also, Loughran was originally developed for financial text, which may affect its accuracy for restaurant reviews.

References

Silge, J., & Robinson, D. (2017). Text Mining with R: A Tidy Approach. O’Reilly Media. Chapter 2: https://www.tidytextmining.com/sentiment

G, S. (2026). Yelp Restaurant Reviews Sentiment Dataset [Data set]. Zenodo. https://doi.org/10.5281/zenodo.18723813

Loughran, T., & McDonald, B. (2011). When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks. The Journal of Finance, 66(1), 35–65.