Assignment 10A: Sentiment Analysis with Text Mining in R

Author

Ummay Rukiya

Overview

For this assignment, I first reproduced the Jane Austen sentiment example from Chapter 2 of Text Mining with R. Then I used a different book, Alice’s Adventures in Wonderland, to see how its sentiment changes throughout the story. I used Bing on both sets of texts and compared Bing with the Loughran–McDonald dictionary on the same passages from Alice.

Step 1: Load the packages

library(dplyr)
library(tidyr)
library(stringr)
library(tidytext)
library(janeaustenr)
library(ggplot2)

options(timeout = max(120, getOption("timeout")))

Step 2: Reproduce the Jane Austen example

The next three chunks reproduce the method in Section 2.2 of Text Mining with R, by Julia Silge and David Robinson. I used the same novels, line indexing, and Bing scoring method as the original example.

First, I numbered the lines within each book. Then I split the text into words so that each row contains one word.

tidy_books <- austen_books() |>
  group_by(book) |>
  mutate(
    linenumber = row_number(),
    chapter = cumsum(str_detect(
      text, regex("^chapter [\\divxlc]", ignore_case = TRUE)
    ))
  ) |>
  ungroup() |>
  unnest_tokens(word, text)

head(tidy_books)
# A tibble: 6 × 4
  book                linenumber chapter word       
  <fct>                    <int>   <int> <chr>      
1 Sense & Sensibility          1       0 sense      
2 Sense & Sensibility          1       0 and        
3 Sense & Sensibility          1       0 sensibility
4 Sense & Sensibility          3       0 by         
5 Sense & Sensibility          3       0 jane       
6 Sense & Sensibility          3       0 austen

Next, I matched the words to the Bing dictionary. The code groups the lines into blocks of 80 and subtracts the number of negative words from the number of positive words.

bing_words <- get_sentiments("bing")

jane_austen_sentiment <- tidy_books |>
  inner_join(bing_words, by = "word", relationship = "many-to-many") |>
  count(book, index = linenumber %/% 80L, sentiment) |>
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0L) |>
  mutate(sentiment = positive - negative)
ggplot(jane_austen_sentiment, aes(index, sentiment, fill = book)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~book, ncol = 2, scales = "free_x") +
  labs(
    title = "Sentiment through Jane Austen’s novels",
    x = "Position in the book (80-line blocks)",
    y = "Positive words minus negative words"
  ) +
  theme_minimal()

A bar above zero means that Bing found more positive than negative words in that block. A bar below zero means it found more negative words. The bars move up and down within the books, so one score for an entire novel would hide some of those changes.

Step 3: Prepare a different book

I downloaded the plain text of Alice’s Adventures in Wonderland from Project Gutenberg. I started at the first chapter heading that appears by itself on a line, so the table of contents does not enter the analysis. I stopped before the end-of-ebook marker.

alice_url <- "https://www.gutenberg.org/cache/epub/11/pg11.txt"
alice_file <- tempfile(fileext = ".txt")
download.file(alice_url, alice_file, mode = "wb", quiet = TRUE)
alice_lines <- readLines(alice_file, encoding = "UTF-8", warn = FALSE)
unlink(alice_file)

start_line <- which(grepl("^CHAPTER I[.]?$", trimws(alice_lines)))[1]
end_line <- which(grepl("^\\*\\*\\* END OF THE PROJECT GUTENBERG EBOOK",
                        alice_lines))[1]

if (is.na(start_line) || is.na(end_line) || start_line >= end_line) {
  stop("The Gutenberg text layout changed. Check the chapter and end markers.")
}

alice_words <- tibble(
  text = alice_lines[seq.int(start_line, end_line - 1L)],
  linenumber = seq_len(end_line - start_line)
) |>
  mutate(index = linenumber %/% 80L) |>
  unnest_tokens(word, text)

head(alice_words)
# A tibble: 6 × 3
  linenumber index word   
       <int> <int> <chr>  
1          1     0 chapter
2          1     0 i      
3          2     0 down   
4          2     0 the    
5          2     0 rabbit 
6          2     0 hole

Step 4: Add another sentiment dictionary

The Loughran–McDonald dictionary was designed for financial writing. I used the active positive and negative entries from the University of Notre Dame’s 1993–2025 master dictionary. The two word lists are stored in the source file so the report can run without downloading the dictionary again. The other labels, such as uncertainty, are useful for other questions, but I left them out so I could use the same scoring rule for both dictionaries.

lm_words <- bind_rows(
  tibble(word = lm_positive, sentiment = "positive"),
  tibble(word = lm_negative, sentiment = "negative")
) |>
  distinct(word, sentiment)

Step 5: Score the same passages with both dictionaries

I used positive minus negative for both dictionaries. I also counted how many word occurrences each one matched. I kept blocks with no matches and assigned them zero for the net count. A zero by itself does not tell me whether the words were balanced or the dictionary simply did not recognize any of them.

score_alice <- function(dictionary, method) {
  hits <- alice_words |>
    inner_join(dictionary, by = "word", relationship = "many-to-many") |>
    group_by(index) |>
    summarise(
      positive = sum(sentiment == "positive"),
      negative = sum(sentiment == "negative"),
      .groups = "drop"
    )

  alice_words |>
    count(index, name = "total_words") |>
    left_join(hits, by = "index") |>
    mutate(
      across(c(positive, negative), ~replace_na(.x, 0L)),
      net = positive - negative,
      method = method
    )
}

alice_scores <- bind_rows(
  score_alice(bing_words, "Bing"),
  score_alice(lm_words, "Loughran–McDonald")
)

alice_coverage <- bind_rows(
  tibble(method = "Bing", matched = nrow(semi_join(alice_words, bing_words, by = "word"))),
  tibble(method = "Loughran–McDonald",
         matched = nrow(semi_join(alice_words, lm_words, by = "word")))
) |>
  mutate(
    total_words = nrow(alice_words),
    percent_matched = round(100 * matched / total_words, 2)
  )

knitr::kable(alice_coverage, caption = "Dictionary coverage in Alice (word occurrences)")
Dictionary coverage in Alice (word occurrences)
method matched total_words percent_matched
Bing 1409 26687 5.28
Loughran–McDonald 407 26687 1.53 
ggplot(alice_scores, aes(index, net, fill = method)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~method, ncol = 1) +
  labs(
    title = "Sentiment in Alice’s Adventures in Wonderland",
    subtitle = "The two panels use the same scale",
    x = "Position in the book (80-line blocks)",
    y = "Positive words minus negative words"
  ) +
  theme_minimal()

Step 6: Compare the results

I used Bing to compare Alice with Pride and Prejudice, one of the novels in the original example. Since the books have different lengths, I divided each book’s net count by its total number of words and multiplied by 1,000.

pride_words <- tidy_books |>
  filter(book == "Pride & Prejudice")

book_score <- function(words, title) {
  matches <- inner_join(words, bing_words, by = "word",
                        relationship = "many-to-many")
  tibble(
    book = title,
    total_words = nrow(words),
    positive = sum(matches$sentiment == "positive"),
    negative = sum(matches$sentiment == "negative")
  ) |>
    mutate(net_per_1000 = round(1000 * (positive - negative) / total_words, 2))
}

book_comparison <- bind_rows(
  book_score(pride_words, "Pride and Prejudice"),
  book_score(alice_words, "Alice's Adventures in Wonderland")
)
knitr::kable(book_comparison, caption = "Bing sentiment per 1,000 words")
Bing sentiment per 1,000 words
book total_words positive negative net_per_1000
Pride and Prejudice 122204 5052 3652 11.46
Alice’s Adventures in Wonderland 26687 678 731 -1.99 
passage_comparison <- alice_scores |>
  select(index, method, net) |>
  mutate(method = if_else(method == "Bing", "bing", "lm")) |>
  pivot_wider(names_from = method, values_from = net) |>
  mutate(gap = abs(bing - lm)) |>
  arrange(desc(gap), index)

knitr::kable(
  head(passage_comparison, 3),
  caption = "Alice passages with the largest difference between dictionaries"
)
Alice passages with the largest difference between dictionaries
index bing lm gap
41 -13 5 18
26 4 -10 14
4 1 -10 11 
largest_gap <- slice_head(passage_comparison, n = 1)

top_matches <- function(dictionary, name) {
  alice_words |>
    filter(index == largest_gap$index) |>
    inner_join(dictionary, by = "word", relationship = "many-to-many") |>
    count(word, sentiment, sort = TRUE) |>
    slice_head(n = 5) |>
    mutate(dictionary = name, .before = 1)
}

knitr::kable(bind_rows(
  top_matches(bing_words, "Bing"),
  top_matches(lm_words, "Loughran–McDonald")
), caption = "Most frequent matched words in the block with the largest gap")
Most frequent matched words in the block with the largest gap
dictionary word sentiment n
Bing strange negative 3
Bing bright positive 2
Bing eager positive 2
Bing mock negative 2
Bing queer negative 2
Loughran–McDonald dream positive 6
Loughran–McDonald confused negative 1
Loughran–McDonald happy positive 1
Loughran–McDonald lying negative 1
Loughran–McDonald pleasure positive 1

With Bing, the net score per 1,000 words was 11.46 for Pride and Prejudice and -1.99 for Alice’s Adventures in Wonderland. These are whole-book summaries, so I used the plots to see changes within each story.

In Alice, Bing matched 5.28% of word occurrences and Loughran–McDonald matched 1.53%. The largest difference was in block 41: Bing scored it -13, while Loughran–McDonald scored it 5. The word table shows which matches contributed to that difference. A dictionary built for financial documents can miss words that carry emotion in a novel. Both dictionaries also score words individually, so they can miss context such as “not happy.”

References