Show code
library(tidytext)
library(janeaustenr)
library(dplyr)
library(stringr)
library(tidyr)
library(ggplot2)
library(wordcloud)
library(reshape2)
library(gutenbergr)
library(readr)
library(sentimentr)
library(knitr)| Part 1 | Reproduces Chapter 2 of Text Mining with R (Silge and Robinson): sentiment analysis of Jane Austen’s novels, using the book’s code unchanged |
| Part 2 | Runs the same tests on The Great Gatsby (1925) and The Sun Also Rises (1926), adds a fourth method (sentimentr), and scores the scenes around each main character |
| Question | Does the Chapter 2 approach tell the same story on two 1920s novels as it does on Austen? |
Source: Silge, J., & Robinson, D. (2017). Text Mining with R: A Tidy Approach, Chapter 2, “Sentiment analysis with tidy data.” https://www.tidytextmining.com/sentiment.html. Original code: https://github.com/dgrtwo/tidy-text-mining.
sentimentr fixes some negations (“not too pleasant”) and breaks others (“Don’t be a damned fool”).Details and numbers are in How the Results Differ from the Original.
library(tidytext)
library(janeaustenr)
library(dplyr)
library(stringr)
library(tidyr)
library(ggplot2)
library(wordcloud)
library(reshape2)
library(gutenbergr)
library(readr)
library(sentimentr)
library(knitr)The AFINN and NRC lexicons download through the textdata package, which asks you to accept each license the first time. That prompt can’t be answered during a render, so run get_sentiments("afinn") and get_sentiments("nrc") once in the R console first. After that they load from the local cache.
Three general-purpose lexicons, each scoring single words:
| Lexicon | How it scores a word |
|---|---|
| AFINN | A number from -5 (very negative) to +5 (very positive) |
| Bing | Positive or negative |
| NRC | Positive or negative, plus eight emotions (anger, anticipation, disgust, fear, joy, sadness, surprise, trust) |
get_sentiments("afinn")# A tibble: 2,477 × 2
word value
<chr> <dbl>
1 abandon -2
2 abandoned -2
3 abandons -2
4 abducted -2
5 abduction -2
6 abductions -2
7 abhor -3
8 abhorred -3
9 abhorrent -3
10 abhors -3
# ℹ 2,467 more rows
get_sentiments("bing")# A tibble: 6,786 × 2
word sentiment
<chr> <chr>
1 2-faces negative
2 abnormal negative
3 abolish negative
4 abominable negative
5 abominably negative
6 abominate negative
7 abomination negative
8 abort negative
9 aborted negative
10 aborts negative
# ℹ 6,776 more rows
get_sentiments("nrc")# A tibble: 13,872 × 2
word sentiment
<chr> <chr>
1 abacus trust
2 abandon fear
3 abandon negative
4 abandon sadness
5 abandoned anger
6 abandoned fear
7 abandoned negative
8 abandoned sadness
9 abandonment anger
10 abandonment fear
# ℹ 13,862 more rows
Turn the Austen novels into one word per row, keeping track of the line and chapter each word came from.
tidy_books <- austen_books() %>%
group_by(book) %>%
mutate(
linenumber = row_number(),
chapter = cumsum(str_detect(text,
regex("^chapter [\\divxlc]",
ignore_case = TRUE)))) %>%
ungroup() %>%
unnest_tokens(word, text)The most common “joy” words in Emma:
nrc_joy <- get_sentiments("nrc") %>%
filter(sentiment == "joy")
tidy_books %>%
filter(book == "Emma") %>%
inner_join(nrc_joy) %>%
count(word, sort = TRUE)# A tibble: 301 × 2
word n
<chr> <int>
1 good 359
2 friend 166
3 hope 143
4 happy 125
5 love 117
6 deal 92
7 found 92
8 present 89
9 kind 82
10 happiness 76
# ℹ 291 more rows
Net sentiment (positive words minus negative words) in each 80-line section of each novel, using Bing:
jane_austen_sentiment <- tidy_books %>%
inner_join(get_sentiments("bing")) %>%
count(book, index = linenumber %/% 80, sentiment) %>%
pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
mutate(sentiment = positive - negative)
ggplot(jane_austen_sentiment, aes(index, sentiment, fill = book)) +
geom_col(show.legend = FALSE) +
facet_wrap(~book, ncol = 2, scales = "free_x")All three lexicons on Pride & Prejudice:
pride_prejudice <- tidy_books %>%
filter(book == "Pride & Prejudice")
afinn <- pride_prejudice %>%
inner_join(get_sentiments("afinn")) %>%
group_by(index = linenumber %/% 80) %>%
summarise(sentiment = sum(value)) %>%
mutate(method = "AFINN")
bing_and_nrc <- bind_rows(
pride_prejudice %>%
inner_join(get_sentiments("bing")) %>%
mutate(method = "Bing et al."),
pride_prejudice %>%
inner_join(get_sentiments("nrc") %>%
filter(sentiment %in% c("positive",
"negative"))
) %>%
mutate(method = "NRC")) %>%
count(method, index = linenumber %/% 80, sentiment) %>%
pivot_wider(names_from = sentiment,
values_from = n,
values_fill = 0) %>%
mutate(sentiment = positive - negative)
bind_rows(afinn,
bing_and_nrc) %>%
ggplot(aes(index, sentiment, fill = method)) +
geom_col(show.legend = FALSE) +
facet_wrap(~method, ncol = 1, scales = "free_y")The book’s explanation for why NRC runs higher than Bing: the two lexicons have different ratios of negative to positive words.
get_sentiments("nrc") %>%
filter(sentiment %in% c("positive", "negative")) %>%
count(sentiment)| sentiment | n |
|---|---|
| negative | 3316 |
| positive | 2308 |
get_sentiments("bing") %>%
count(sentiment)| sentiment | n |
|---|---|
| negative | 4781 |
| positive | 2005 |
bing_word_counts <- tidy_books %>%
inner_join(get_sentiments("bing")) %>%
count(word, sentiment, sort = TRUE) %>%
ungroup()
bing_word_counts# A tibble: 2,585 × 3
word sentiment n
<chr> <chr> <int>
1 miss negative 1855
2 well positive 1523
3 good positive 1380
4 great positive 981
5 like positive 725
6 better positive 639
7 enough positive 613
8 happy positive 534
9 love positive 495
10 pleasure positive 462
# ℹ 2,575 more rows
bing_word_counts %>%
group_by(sentiment) %>%
slice_max(n, n = 10) %>%
ungroup() %>%
mutate(word = reorder(word, n)) %>%
ggplot(aes(n, word, fill = sentiment)) +
geom_col(show.legend = FALSE) +
facet_wrap(~sentiment, scales = "free_y") +
labs(x = "Contribution to sentiment",
y = NULL)“miss” is scored negative, but Austen uses it as a title (“Miss Bennet”). The book’s fix is a custom stop word. Stop words are common words like “the” and “and” that get removed before counting, using anti_join(). Adding “miss” to that list means it gets removed too, so it can’t count as negative. The book builds the list here but doesn’t rerun anything with it. Part 2 applies it to Gatsby.
custom_stop_words <- bind_rows(tibble(word = c("miss"),
lexicon = c("custom")),
stop_words)
custom_stop_words# A tibble: 1,150 × 2
word lexicon
<chr> <chr>
1 miss custom
2 a SMART
3 a's SMART
4 able SMART
5 about SMART
6 above SMART
7 according SMART
8 accordingly SMART
9 across SMART
10 actually SMART
# ℹ 1,140 more rows
tidy_books %>%
anti_join(stop_words) %>%
count(word) %>%
with(wordcloud(word, n, max.words = 100))tidy_books %>%
inner_join(get_sentiments("bing")) %>%
count(word, sentiment, sort = TRUE) %>%
acast(word ~ sentiment, value.var = "n", fill = 0) %>%
comparison.cloud(colors = c("gray20", "gray80"),
max.words = 100)Splitting into sentences and chapters instead of words:
p_and_p_sentences <- tibble(text = prideprejudice) %>%
unnest_tokens(sentence, text, token = "sentences")
p_and_p_sentences$sentence[2][1] "by jane austen"
austen_chapters <- austen_books() %>%
group_by(book) %>%
unnest_tokens(chapter, text, token = "regex",
pattern = "Chapter|CHAPTER [\\dIVXLC]") %>%
ungroup()
austen_chapters %>%
group_by(book) %>%
summarise(chapters = n())| book | chapters |
|---|---|
| Sense & Sensibility | 51 |
| Pride & Prejudice | 62 |
| Mansfield Park | 49 |
| Emma | 56 |
| Northanger Abbey | 32 |
| Persuasion | 25 |
The most negative chapter in each novel, as a share of its words:
bingnegative <- get_sentiments("bing") %>%
filter(sentiment == "negative")
wordcounts <- tidy_books %>%
group_by(book, chapter) %>%
summarize(words = n())
tidy_books %>%
semi_join(bingnegative) %>%
group_by(book, chapter) %>%
summarize(negativewords = n()) %>%
left_join(wordcounts, by = c("book", "chapter")) %>%
mutate(ratio = negativewords/words) %>%
filter(chapter != 0) %>%
slice_max(ratio, n = 1) %>%
ungroup()| book | chapter | negativewords | words | ratio |
|---|---|---|---|---|
| Sense & Sensibility | 43 | 161 | 3405 | 0.0472834 |
| Pride & Prejudice | 34 | 111 | 2104 | 0.0527567 |
| Mansfield Park | 46 | 173 | 3685 | 0.0469471 |
| Emma | 15 | 151 | 3340 | 0.0452096 |
| Northanger Abbey | 21 | 149 | 2982 | 0.0499665 |
| Persuasion | 4 | 62 | 1807 | 0.0343110 |
| The Great Gatsby | The Sun Also Rises | |
|---|---|---|
| Author | F. Scott Fitzgerald | Ernest Hemingway |
| Published | 1925 | 1926 |
| Project Gutenberg ID | 64317 | 67138 |
| US public domain since | 2021 | 2022 |
Downloaded with gutenbergr. The first render saves a CSV copy, and later renders read from it, so the report doesn’t depend on the Gutenberg mirror.
book_titles <- c(`64317` = "The Great Gatsby", `67138` = "The Sun Also Rises")
books_file <- "gatsby_sun_also_rises.csv"
if (file.exists(books_file)) {
raw_books <- read_csv(books_file, col_types = "ic", na = character())
} else {
raw_books <- gutenberg_download(c(64317, 67138))
write_csv(raw_books, books_file, na = "")
}The book’s Austen chapter pattern doesn’t fit either book, so each gets its own rule:
| The Great Gatsby | The Sun Also Rises | |
|---|---|---|
| Story starts at | The second “I” (the first is in the table of contents) | The first “CHAPTER” |
| Chapter headings | A Roman numeral alone on a line | “CHAPTER” on one line, the number on the next |
| Also dropped | Title page, dedication, epigraph | Front matter, “BOOK II/III” titles, transcriber’s note after “THE END” |
Curly quotes and apostrophes are straightened so “don’t” matches the lexicons’ spelling.
new_books <- raw_books %>%
mutate(
book = factor(book_titles[as.character(gutenberg_id)], levels = book_titles),
text = str_replace_all(text, "[‘’]", "'"),
text = str_replace_all(text, "[“”]", '"'),
trimmed = str_trim(text)
) %>%
group_by(book) %>%
mutate(
start = if_else(book == "The Great Gatsby",
which(trimmed == "I")[2],
which(trimmed == "CHAPTER")[1]),
end = coalesce(which(trimmed == "THE END")[1], n() + 1L)
) %>%
filter(row_number() >= start, row_number() < end) %>%
mutate(
heading = if_else(book == "The Great Gatsby",
str_detect(trimmed, "^[IVX]+$"),
trimmed == "CHAPTER"),
chapter = cumsum(heading)
) %>%
filter(!heading,
!str_detect(trimmed, "^BOOK [IVX]+$"),
!str_detect(trimmed, "^\\d+$")) %>%
mutate(linenumber = row_number()) %>%
ungroup() %>%
select(book, linenumber, chapter, text)
tidy_new <- new_books %>%
unnest_tokens(word, text)Check against the books (9 chapters in Gatsby, 19 in The Sun Also Rises):
new_books %>%
group_by(Book = book) %>%
summarise(Chapters = n_distinct(chapter),
`First line of the story` = first(text[str_trim(text) != ""]))| Book | Chapters | First line of the story |
|---|---|---|
| The Great Gatsby | 9 | In my younger and more vulnerable years my father gave me some advice |
| The Sun Also Rises | 19 | Robert Cohn was once middleweight boxing champion of Princeton. Do not |
The book scores 80-line sections and notes the right size depends on the text. These books have shorter lines than Austen’s:
section_size <- bind_rows(
tibble(book = "Jane Austen (all 6 novels)",
words = nrow(tidy_books),
lines = nrow(austen_books())),
tidy_new %>% count(book, name = "words") %>%
left_join(new_books %>% count(book, name = "lines"), by = "book") %>%
mutate(book = as.character(book))
) %>%
mutate(words_per_line = words / lines)
section_size %>%
transmute(Book = book,
Words = format(words, big.mark = ","),
Lines = format(lines, big.mark = ","),
`Words per line` = round(words_per_line, 1))| Book | Words | Lines | Words per line |
|---|---|---|---|
| Jane Austen (all 6 novels) | 725,055 | 73,422 | 9.9 |
| The Great Gatsby | 48,636 | 6,357 | 7.7 |
| The Sun Also Rises | 68,305 | 10,110 | 6.8 |
austen_wpl <- section_size %>% filter(!book %in% book_titles) %>% pull(words_per_line)
new_wpl <- section_size %>% filter(book %in% book_titles) %>% pull(words_per_line)
new_section_size <- round(80 * austen_wpl / mean(new_wpl), -1)Takeaway: the new books use 110-line sections, so each section holds about as much text as the book’s 80-line Austen sections. Pride & Prejudice stays at 80.
The Bing net-sentiment arc from Section 2.2, for the new books:
new_sentiment <- tidy_new %>%
inner_join(get_sentiments("bing"), by = "word") %>%
count(book, index = linenumber %/% new_section_size, sentiment) %>%
pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
mutate(sentiment = positive - negative)
ggplot(new_sentiment, aes(index, sentiment, fill = book)) +
geom_col(show.legend = FALSE) +
facet_wrap(~book, ncol = 1, scales = "free_x") +
labs(x = paste0(new_section_size, "-line section"), y = "Net sentiment (positive - negative)")| Sections that score negative | |
|---|---|
| Austen (6 novels) | 11% to 26% |
| The Great Gatsby | 64% |
| The Sun Also Rises | 33% |
Takeaway: Austen’s arcs stay mostly above zero. Gatsby spends most of its length below.
All eight books, scored per 1,000 words so long and short books compare fairly:
all_tidy <- bind_rows(
tidy_books %>% mutate(book = as.character(book), author = "Austen"),
tidy_new %>% mutate(book = as.character(book),
author = if_else(book == "The Great Gatsby", "Fitzgerald", "Hemingway"))
)
bing_rates <- all_tidy %>%
left_join(get_sentiments("bing"), by = "word") %>%
group_by(author, book) %>%
summarise(words = n(),
matched = sum(!is.na(sentiment)),
positive = sum(sentiment == "positive", na.rm = TRUE),
negative = sum(sentiment == "negative", na.rm = TRUE),
.groups = "drop")
nrc_rates <- all_tidy %>%
inner_join(get_sentiments("nrc") %>% filter(sentiment %in% c("positive", "negative")),
by = "word") %>%
group_by(book) %>%
summarise(nrc_positive_share = mean(sentiment == "positive"))
tone <- bing_rates %>%
left_join(nrc_rates, by = "book") %>%
mutate(matched_per_1k = matched / words * 1000,
net_per_1k = (positive - negative) / words * 1000,
bing_positive_share = positive / matched)
tone %>%
arrange(desc(net_per_1k)) %>%
transmute(Author = author,
Book = book,
`Net sentiment per 1,000` = round(net_per_1k, 1),
`Sentiment words per 1,000` = round(matched_per_1k, 1),
`Bing: % positive` = scales::percent(bing_positive_share, accuracy = 1),
`NRC: % positive` = scales::percent(nrc_positive_share, accuracy = 1))| Author | Book | Net sentiment per 1,000 | Sentiment words per 1,000 | Bing: % positive | NRC: % positive |
|---|---|---|---|---|---|
| Austen | Persuasion | 15.2 | 67.8 | 61% | 70% |
| Austen | Emma | 14.6 | 74.3 | 60% | 68% |
| Austen | Mansfield Park | 12.0 | 72.1 | 58% | 66% |
| Austen | Pride & Prejudice | 11.5 | 71.2 | 58% | 67% |
| Austen | Sense & Sensibility | 10.5 | 71.7 | 57% | 65% |
| Austen | Northanger Abbey | 9.3 | 74.1 | 56% | 65% |
| Hemingway | The Sun Also Rises | 5.6 | 50.8 | 56% | 60% |
| Fitzgerald | The Great Gatsby | -5.9 | 57.6 | 45% | 55% |
Takeaway: the new books have less sentiment vocabulary for the lexicons to work with, and Gatsby is the only negative book.
sentimentrThe Chapter 2 lexicons score one word at a time, so “not good” counts as positive. sentimentr (Rinker), which the book mentions in Section 2.6, scores whole sentences and adjusts for nearby words:
| Word type | Example | Effect |
|---|---|---|
| Negator | “not” | Flips the score |
| Amplifier | “very” | Strengthens it |
| De-amplifier | “hardly” | Weakens it |
All four methods are scored on the same sections. For sentimentr, each section’s sentences are scored and summed.
score_sections <- function(lines, size = 80) {
lines <- lines %>% mutate(index = linenumber %/% size)
words <- lines %>% unnest_tokens(word, text)
afinn_scores <- words %>%
inner_join(get_sentiments("afinn"), by = "word") %>%
group_by(book, index) %>%
summarise(sentiment = sum(value), .groups = "drop") %>%
mutate(method = "AFINN")
bing_nrc_scores <- bind_rows(
words %>%
inner_join(get_sentiments("bing"), by = "word") %>%
mutate(method = "Bing et al."),
words %>%
inner_join(get_sentiments("nrc") %>%
filter(sentiment %in% c("positive", "negative")),
by = "word") %>%
mutate(method = "NRC")
) %>%
count(book, method, index, sentiment) %>%
pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
mutate(sentiment = positive - negative) %>%
select(book, index, method, sentiment)
sentimentr_scores <- lines %>%
group_by(book, index) %>%
summarise(text = str_squish(paste(text, collapse = " ")), .groups = "drop") %>%
get_sentences() %>%
sentiment() %>%
as_tibble() %>%
group_by(book, index) %>%
summarise(sentiment = sum(sentiment, na.rm = TRUE), .groups = "drop") %>%
mutate(method = "sentimentr")
bind_rows(afinn_scores, bing_nrc_scores, sentimentr_scores) %>%
mutate(method = factor(method,
levels = c("AFINN", "Bing et al.", "NRC", "sentimentr")))
}
pp_lines <- austen_books() %>%
filter(book == "Pride & Prejudice") %>%
mutate(linenumber = row_number(),
chapter = cumsum(str_detect(text, regex("^chapter [\\divxlc]", ignore_case = TRUE))),
book = "Pride & Prejudice")
section_scores <- bind_rows(
score_sections(pp_lines),
score_sections(new_books %>% mutate(book = as.character(book)), size = new_section_size)
) %>%
mutate(book = factor(book, levels = c("Pride & Prejudice", book_titles)))Each panel has its own scale. Compare the shape of the bars, not their height:
ggplot(section_scores, aes(index, sentiment, fill = method)) +
geom_col(show.legend = FALSE) +
facet_grid(method ~ book, scales = "free") +
labs(x = "Section", y = "Section score")The book found the lexicons share the same ups and downs. Correlation measures that: 1 = they move together, 0 = no relationship.
agreement <- section_scores %>%
pivot_wider(names_from = method, values_from = sentiment, values_fill = 0) %>%
group_by(Book = book) %>%
summarise(
`AFINN vs Bing` = cor(AFINN, `Bing et al.`),
`AFINN vs NRC` = cor(AFINN, NRC),
`Bing vs NRC` = cor(`Bing et al.`, NRC),
`sentimentr vs Bing` = cor(sentimentr, `Bing et al.`),
`sentimentr vs AFINN` = cor(sentimentr, AFINN)
)
agreement %>% mutate(across(-Book, ~ round(.x, 2)))| Book | AFINN vs Bing | AFINN vs NRC | Bing vs NRC | sentimentr vs Bing | sentimentr vs AFINN |
|---|---|---|---|---|---|
| Pride & Prejudice | 0.81 | 0.72 | 0.71 | 0.76 | 0.72 |
| The Great Gatsby | 0.77 | 0.60 | 0.63 | 0.68 | 0.59 |
| The Sun Also Rises | 0.80 | 0.70 | 0.56 | 0.68 | 0.69 |
Takeaway: average agreement between the three word lexicons is 0.75 on Pride & Prejudice, 0.67 on Gatsby and 0.69 on The Sun Also Rises. They still move together on the new books, just less closely.
Every sentence is scored twice, by Bing and by sentimentr, to see how often they point in opposite directions:
score_sentences <- function(lines) {
sentences <- lines %>%
group_by(book, chapter) %>%
summarise(text = str_squish(paste(text, collapse = " ")), .groups = "drop") %>%
get_sentences() %>%
sentiment() %>%
as_tibble() %>%
transmute(book, chapter, sentence_id = row_number(), text = as.character(text),
word_count, sentimentr = sentiment)
bing_by_sentence <- sentences %>%
select(sentence_id, text) %>%
unnest_tokens(word, text) %>%
inner_join(get_sentiments("bing"), by = "word") %>%
group_by(sentence_id) %>%
summarise(bing = sum(sentiment == "positive") - sum(sentiment == "negative"))
sentences %>%
left_join(bing_by_sentence, by = "sentence_id") %>%
mutate(bing = coalesce(bing, 0L))
}
sentence_scores <- bind_rows(
score_sentences(pp_lines),
score_sentences(new_books %>% mutate(book = as.character(book)))
) %>%
mutate(book = factor(book, levels = c("Pride & Prejudice", book_titles)))
disagreement <- sentence_scores %>%
filter(bing != 0, sentimentr != 0) %>%
group_by(book) %>%
summarise(scored = n(), opposite = mean(sign(bing) != sign(sentimentr)))
disagreement %>%
transmute(Book = book,
`Sentences both methods scored` = format(scored, big.mark = ","),
`Scored in opposite directions` = scales::percent(opposite, accuracy = 0.1))| Book | Sentences both methods scored | Scored in opposite directions |
|---|---|---|
| Pride & Prejudice | 3,346 | 20.6% |
| The Great Gatsby | 1,405 | 14.1% |
| The Sun Also Rises | 2,428 | 17.5% |
Examples with a negating word. Bing = positive minus negative words. sentimentr = the sentence score. Above 0 is positive for both.
negators <- "\\b(not|no|never|nothing|nobody|don't|didn't|isn't|wasn't|couldn't|wouldn't|can't)\\b"
examples <- sentence_scores %>%
filter(book != "Pride & Prejudice",
bing != 0, sign(bing) != sign(sentimentr),
between(word_count, 5, 22),
str_detect(str_to_lower(text), negators)) %>%
group_by(book) %>%
slice_max(abs(sentimentr), n = 4, with_ties = FALSE) %>%
ungroup()
for (b in book_titles) {
cat("\n\n**", b, "**\n\n", sep = "")
examples %>%
filter(book == b) %>%
transmute(Sentence = text, Bing = bing, sentimentr = round(sentimentr, 2)) %>%
knitr::kable() %>%
print()
}The Great Gatsby
| Sentence | Bing | sentimentr |
|---|---|---|
| “I told her she might fool me but she couldn’t fool God. | -2 | 0.49 |
| He knew that Daisy was extraordinary, but he didn’t realize just how extraordinary a “nice” girl could be. | 3 | -0.49 |
| “I just slip it on sometimes when I don’t care what I look like.” | 1 | -0.47 |
| But he didn’t despise himself and it didn’t turn out as he had imagined. | -1 | 0.45 |
The Sun Also Rises
| Sentence | Bing | sentimentr |
|---|---|---|
| “But he’s not aficionado like you.” | 1 | -1.19 |
| “But he’s not aficionado like you are.” | 1 | -1.11 |
| This is all awfully amusing, but it’s not too pleasant. | 1 | -0.87 |
| “Don’t be a damned fool.” | -2 | 0.78 |
sentimentr gets it right |
sentimentr gets it wrong |
|---|---|
| “it’s not too pleasant”: reads the “not”, scores negative | “Don’t be a damned fool”: the “don’t” flips the insult, scores positive |
Takeaway: the two methods disagree on 14% to 18% of sentences in the new books (21% in Pride & Prejudice), and neither is consistently right.
The Section 2.4 chart for each new book:
new_word_counts <- tidy_new %>%
inner_join(get_sentiments("bing"), by = "word") %>%
count(book, word, sentiment, sort = TRUE)
new_word_counts %>%
group_by(book, sentiment) %>%
slice_max(n, n = 10, with_ties = FALSE) %>%
ungroup() %>%
mutate(word = reorder_within(word, n, list(book, sentiment))) %>%
ggplot(aes(n, word, fill = sentiment)) +
geom_col(show.legend = FALSE) +
facet_wrap(book ~ sentiment, scales = "free") +
scale_y_reordered() +
labs(x = "Contribution to sentiment", y = NULL)Applying the book’s custom stop word fix from Section 2.4:
net_per_1k <- function(tidy) {
tidy %>%
inner_join(get_sentiments("bing"), by = "word") %>%
group_by(book) %>%
summarise(net = sum(sentiment == "positive") - sum(sentiment == "negative")) %>%
left_join(tidy_new %>% count(book, name = "words"), by = "book") %>%
mutate(net_per_1k = net / words * 1000)
}
miss_fix <- bind_rows(
net_per_1k(tidy_new) %>% mutate(version = "As scored"),
net_per_1k(tidy_new %>% anti_join(custom_stop_words %>% filter(lexicon == "custom"), by = "word")) %>%
mutate(version = "Without \"miss\"")
)
miss_fix %>%
select(Book = book, version, net_per_1k) %>%
mutate(net_per_1k = round(net_per_1k, 1)) %>%
pivot_wider(names_from = version, values_from = net_per_1k)| Book | As scored | Without “miss” |
|---|---|---|
| The Great Gatsby | -5.9 | -5.1 |
| The Sun Also Rises | 5.6 | 5.6 |
Takeaway: Gatsby is still negative without “miss”, so the misread word isn’t what makes it negative.
The characters use “damned” and “damn” to mean “very” (“damned good”), but Bing scores them negative. The word right after each one:
damned_bigrams <- new_books %>%
filter(book == "The Sun Also Rises") %>%
unnest_tokens(bigram, text, token = "ngrams", n = 2) %>%
filter(!is.na(bigram)) %>%
separate(bigram, c("word1", "word2"), sep = " ") %>%
filter(word1 %in% c("damned", "damn")) %>%
inner_join(get_sentiments("bing"), by = c("word2" = "word"))
damned_bigrams %>%
count(`Next word is` = sentiment, name = "Times") %>%
mutate(Share = scales::percent(Times / sum(Times), accuracy = 1))| Next word is | Times | Share |
|---|---|---|
| negative | 18 | 35% |
| positive | 34 | 65% |
damned_bigrams %>%
unite(Phrase, word1, word2, sep = " ") %>%
count(Phrase, sort = TRUE) %>%
head(8)| Phrase | n |
|---|---|
| damned good | 8 |
| damn good | 4 |
| damned nice | 4 |
| damned well | 4 |
| damn nice | 2 |
| damned fine | 2 |
| damned hard | 2 |
| damn bad | 1 |
Takeaway: the next word is positive 65% of the time, so “damned good” cancels to zero instead of counting as praise.
One cloud per book instead of the book’s split gray halves:
cloud_colors <- c("#7f1d1d", "#991b1b", "#b91c1c", "#d62f2f", "#e05252", # -5 to -1
"#888888", # 0
"#3f9e5f", "#2f8a4f", "#237a42", "#166534", "#0f4d27") # +1 to +5
sentiment_cloud <- function(tidy) {
words <- tidy %>%
anti_join(stop_words, by = "word") %>%
inner_join(get_sentiments("afinn"), by = "word") %>%
count(word, value, sort = TRUE) %>%
head(100)
set.seed(607)
wordcloud(words$word, words$n,
colors = cloud_colors[words$value + 6], ordered.colors = TRUE,
random.order = FALSE, scale = c(4, 0.6))
}sentiment_cloud(tidy_books)sentiment_cloud(tidy_new %>% filter(book == "The Great Gatsby"))sentiment_cloud(tidy_new %>% filter(book == "The Sun Also Rises"))The Section 2.6 test: the chapter with the highest share of negative words.
new_wordcounts <- tidy_new %>%
count(book, chapter, name = "words")
new_negative <- tidy_new %>%
semi_join(bingnegative, by = "word") %>%
count(book, chapter, name = "negativewords") %>%
left_join(new_wordcounts, by = c("book", "chapter")) %>%
mutate(ratio = negativewords / words) %>%
group_by(book) %>%
slice_max(ratio, n = 1) %>%
ungroup()
new_negative %>%
transmute(Book = book, Chapter = chapter,
`Negative words` = negativewords, Words = words,
`Share negative` = scales::percent(ratio, accuracy = 0.1))| Book | Chapter | Negative words | Words | Share negative |
|---|---|---|---|---|
| The Great Gatsby | 7 | 320 | 8839 | 3.6% |
| The Sun Also Rises | 12 | 128 | 3981 | 3.2% |
The top negative words in The Sun Also Rises’s pick:
tidy_new %>%
filter(book == "The Sun Also Rises",
chapter == new_negative$chapter[new_negative$book == "The Sun Also Rises"]) %>%
semi_join(bingnegative, by = "word") %>%
count(word, sort = TRUE) %>%
head(10)| word | n |
|---|---|
| hell | 10 |
| irony | 10 |
| pity | 9 |
| cold | 8 |
| falls | 5 |
| ironical | 4 |
| jam | 4 |
| steep | 4 |
| bait | 3 |
| bum | 3 |
| Book | Chapter | What happens | Right call? |
|---|---|---|---|
| The Great Gatsby | 7 | The Plaza Hotel confrontation and Myrtle’s death | Yes |
| The Sun Also Rises | 12 | The fishing trip in the mountains, one of the happiest stretches. Top “negative” words come from Bill’s “Irony and Pity” joke, plus “cold” and “falls” (waterfalls) | No |
Names aren’t in any lexicon, so this scores the sentences that mention each character with sentimentr, then compares the share that read positive vs negative.
| This measures | This doesn’t measure |
|---|---|
| The mood of the scenes a character appears in | Whether the character is good or bad |
| How the narrator describes them | Sentences that use “he” or “she” instead of the name |
| How that mood changes chapter by chapter | The narrators, Nick and Jake, who are mostly “I” |
characters <- tribble(
~book, ~character, ~pattern,
"The Great Gatsby", "Gatsby", "\\bgatsby",
"The Great Gatsby", "Daisy", "\\bdaisy",
"The Great Gatsby", "Tom", "\\btom\\b",
"The Great Gatsby", "Jordan", "\\bjordan\\b|\\bmiss baker\\b",
"The Great Gatsby", "Myrtle", "\\bmyrtle\\b|\\bmrs\\. wilson\\b",
"The Great Gatsby", "George Wilson", "(?<!mrs\\. )\\bwilson\\b",
"The Great Gatsby", "Nick (narrator)", "\\bnick\\b|\\bcarraway\\b",
"The Sun Also Rises", "Brett", "\\bbrett\\b",
"The Sun Also Rises", "Cohn", "\\bcohn\\b|\\brobert\\b",
"The Sun Also Rises", "Bill", "\\bbill\\b",
"The Sun Also Rises", "Mike", "\\bmike\\b|\\bmichael\\b",
"The Sun Also Rises", "Romero", "\\bromero\\b",
"The Sun Also Rises", "Montoya", "\\bmontoya\\b",
"The Sun Also Rises", "Jake (narrator)", "\\bjake\\b|\\bbarnes\\b"
)
new_sentences <- sentence_scores %>%
filter(book != "Pride & Prejudice") %>%
mutate(book = as.character(book), lower = str_to_lower(text))
character_sentences <- characters %>%
inner_join(new_sentences, by = "book", relationship = "many-to-many") %>%
filter(str_detect(lower, pattern))
character_summary <- character_sentences %>%
group_by(book, character) %>%
summarise(mentions = n(),
positive = mean(sentimentr > 0),
negative = mean(sentimentr < 0),
.groups = "drop") %>%
mutate(net = positive - negative)
book_baseline <- new_sentences %>%
group_by(book) %>%
summarise(net = mean(sentimentr > 0) - mean(sentimentr < 0))The bar is the share of a character’s sentences that read positive minus the share that read negative. The dashed line is the whole book’s average.
character_summary %>%
mutate(label = paste0(character, " (", mentions, ")"),
label = reorder_within(label, net, book)) %>%
ggplot(aes(net, label, fill = net > 0)) +
geom_col(show.legend = FALSE) +
geom_vline(data = book_baseline, aes(xintercept = net), linetype = "dashed") +
facet_wrap(~book, ncol = 1, scales = "free_y") +
scale_y_reordered() +
scale_fill_manual(values = c(`TRUE` = "#2f8a4f", `FALSE` = "#d62f2f")) +
scale_x_continuous(labels = scales::label_percent()) +
labs(x = "% of sentences positive minus % negative",
y = "Character (sentences that mention them)")character_summary %>%
arrange(book, net) %>%
transmute(Book = book, Character = character, Mentions = mentions,
`Read positive` = scales::percent(positive, accuracy = 1),
`Read negative` = scales::percent(negative, accuracy = 1))| Book | Character | Mentions | Read positive | Read negative |
|---|---|---|---|---|
| The Great Gatsby | Jordan | 99 | 31% | 52% |
| The Great Gatsby | Tom | 188 | 30% | 46% |
| The Great Gatsby | George Wilson | 57 | 25% | 40% |
| The Great Gatsby | Myrtle | 39 | 28% | 41% |
| The Great Gatsby | Gatsby | 257 | 34% | 32% |
| The Great Gatsby | Daisy | 185 | 40% | 35% |
| The Great Gatsby | Nick (narrator) | 33 | 39% | 12% |
| The Sun Also Rises | Mike | 281 | 23% | 22% |
| The Sun Also Rises | Cohn | 278 | 23% | 22% |
| The Sun Also Rises | Romero | 112 | 34% | 30% |
| The Sun Also Rises | Brett | 431 | 26% | 19% |
| The Sun Also Rises | Montoya | 54 | 30% | 22% |
| The Sun Also Rises | Jake (narrator) | 146 | 27% | 19% |
| The Sun Also Rises | Bill | 342 | 24% | 16% |
How the mood around the central characters changes chapter by chapter (chapters with at least 5 mentions):
central <- c("Gatsby", "Daisy", "Tom", "Brett", "Cohn", "Mike")
by_chapter <- character_sentences %>%
filter(character %in% central) %>%
group_by(book, character, chapter) %>%
summarise(mentions = n(),
net = mean(sentimentr > 0) - mean(sentimentr < 0),
.groups = "drop") %>%
filter(mentions >= 5)
line_labels <- by_chapter %>%
group_by(book, character) %>%
slice_max(chapter, n = 1)
ggplot(by_chapter, aes(chapter, net, color = character)) +
geom_hline(yintercept = 0, color = "grey60") +
geom_line() +
geom_point(aes(size = mentions)) +
geom_text(data = line_labels, aes(label = character),
hjust = -0.25, size = 3.5, show.legend = FALSE) +
facet_wrap(~book, ncol = 1, scales = "free_x") +
scale_x_continuous(breaks = function(lims) seq(1, floor(lims[2] - 1.5), by = 1),
expand = expansion(add = c(0.3, 1.6))) +
scale_y_continuous(labels = scales::label_percent()) +
scale_size_area(max_size = 4, guide = "none") +
labs(x = "Chapter", y = "% positive minus % negative") +
theme(legend.position = "none")Takeaway: Gatsby’s negativity clusters around particular characters. The Sun Also Rises spreads its tone evenly across the whole group.
The book’s comparison.cloud(), comparing the two books instead of positive and negative words. Color = which book (blue Gatsby, purple The Sun Also Rises), not sentiment. Bigger = used much more in that book.
set.seed(607)
tidy_new %>%
anti_join(stop_words, by = "word") %>%
count(word, book) %>%
acast(word ~ book, value.var = "n", fill = 0) %>%
comparison.cloud(colors = c("#2c5f8a", "#7b4fa6"),
max.words = 120, title.size = 1.4)| The Great Gatsby | The Sun Also Rises | |
|---|---|---|
| Net sentiment per 1,000 words | -5.9 | 5.6 |
| Sections that score negative | 64% | 33% |
| Sentiment words per 1,000 | 58 | 51 |
| Lexicon agreement | 0.67 | 0.69 |
| Misread word | “miss” (a title) | “damned” (emphasis) |
| Most negative chapter | 7, fits the plot | 12, a misread joke |
| Characters | Negativity clusters around a few | Everyone near the book’s average |
Takeaway: same era, very different results. The author’s style matters more than the decade.
| Question from my approach | Austen (Chapter 2) | Gatsby and The Sun Also Rises |
|---|---|---|
| Overall tone | Positive: 9.3 to 15.2 per 1,000 words | Gatsby -5.9, Sun 5.6 |
| Do the lexicons agree? | Yes: 0.75 | Less: 0.67 and 0.69 |
| Does sentence-level scoring change things? | Disagrees with Bing on 21% of sentences | 14% to 18%, and not clearly better |
| The “miss” of the corpus | “miss” | “miss” (Gatsby), “damned” (Sun) |
| Darkest chapter | Matches the plot in all six novels | Right for Gatsby, wrong for Sun |
Bottom line: the Chapter 2 method holds up on a book with dark events (Gatsby), but struggles with Hemingway’s sparse, joking dialogue. The biggest errors come from context (titles, swearing for emphasis, a running joke), which no word list can see.
Silge, J., & Robinson, D. (2017). Text mining with R: A tidy approach. O’Reilly Media. https://www.tidytextmining.com/sentiment.html
Silge, J., & Robinson, D. (2017). tidy-text-mining [Source code for the book]. GitHub. https://github.com/dgrtwo/tidy-text-mining
Rinker, T. W. (2021). sentimentr: Calculate text polarity sentiment [R package]. https://github.com/trinker/sentimentr
Nielsen, F. Å. (2011). A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. Proceedings of the ESWC2011 Workshop on Making Sense of Microposts, 93–98. (AFINN lexicon)
Hu, M., & Liu, B. (2004). Mining and summarizing customer reviews. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 168–177. (Bing lexicon)
Mohammad, S. M., & Turney, P. D. (2013). Crowdsourcing a word–emotion association lexicon. Computational Intelligence, 29(3), 436–465. (NRC lexicon)
Fitzgerald, F. S. (1925). The great Gatsby. Project Gutenberg. https://www.gutenberg.org/ebooks/64317
Hemingway, E. (1926). The sun also rises. Project Gutenberg. https://www.gutenberg.org/ebooks/67138
Claude (Anthropic) helped write and test the code in this report, including the cleaning rules for the two books, the section-size adjustment, the four-method comparison, the sentence-level comparison and the character analysis. I chose the books, the extra lexicon, the questions in my approach and the idea to analyze the characters.
Anthropic. (2026). Claude [Large language model]. https://claude.ai/. Accessed November 8, 2026.