library(dplyr) # data manipulation
library(tidyr) # reshape sentiment counts
library(tidytext) # tokenize text and retrieve lexicons
library(janeaustenr) # Jane Austen novels
library(stringr) # identify chapter headings
library(ggplot2) # plots
library(readr) # read review text
library(purrr) # apply functions consistently
library(tibble) # tidy tablesSentiment Analysis: From Jane Austen to Amazon Product Reviews
Approach
Sentiment analysis uses words and linguistic rules to estimate whether text expresses positive or negative opinions. This report first reproduces the principal Jane Austen example in Chapter 2 of Text Mining with R by Julia Silge and David Robinson (2017). It then extends the approach to short Amazon product-review sentences from the UCI Sentiment Labelled Sentences dataset. The extension compares the original Bing and AFINN lexicons with an additional lexicon, Loughran, and evaluates predicted sentiment against the supplied labels.
The book’s example examines narrative trajectories, which means changes in the balance of positive and negative words across consecutive 80 line sections of novels. Product reviews have no comparable narrative trajectory, so the extension instead measures sentiment per review. The dataset contains 500 positive and 500 negative Amazon sentences, selected from a larger collection. Thus the sample is intentionally balanced and does not represent the natural prevalence of positive reviews.
Original example and code: Silge, J., & Robinson, D. (2017). Text Mining with R: A Tidy Approach, Chapter 2, “Sentiment analysis with tidy data.” https://www.tidytextmining.com/sentiment . The reproduction below follows the book’s code structure, with explicit join keys and modern pivot_wider() syntax.
Packages
Part 1 — Reproducing Chapter 2
Prepare Jane Austen’s novels
The original analysis uses austen_books() and converts each line of each novel to a tidy table of individual words. The line number is retained so the location of each word within its novel is known.
tidy_books <- austen_books() %>%
group_by(book) %>%
mutate(
linenumber = row_number(),
chapter = cumsum(str_detect(text, regex("^chapter [\\divxlc]", ignore_case = TRUE)))
) %>%
ungroup() %>%
unnest_tokens(word, text)
tidy_books %>% count(book, name = "total_words")# A tibble: 6 × 2
book total_words
<fct> <int>
1 Sense & Sensibility 119957
2 Pride & Prejudice 122204
3 Mansfield Park 160460
4 Emma 160996
5 Northanger Abbey 77780
6 Persuasion 83658Inspect sentiment dictionaries
Bing labels words as positive or negative, AFINN scores them numerically, and NRC assigns emotion categories as well as polarity. These dictionaries are used in the original textbook example.
bing <- get_sentiments("bing")
afinn <- get_sentiments("afinn")
nrc <- get_sentiments("nrc")
bing %>% count(sentiment)# A tibble: 2 × 2
sentiment n
<chr> <int>
1 negative 4781
2 positive 2005afinn %>% summarise(minimum = min(value), maximum = max(value))# A tibble: 1 × 2
minimum maximum
<dbl> <dbl>
1 -5 5nrc %>% count(sentiment)# A tibble: 10 × 2
sentiment n
<chr> <int>
1 anger 1245
2 anticipation 837
3 disgust 1056
4 fear 1474
5 joy 687
6 negative 3316
7 positive 2308
8 sadness 1187
9 surprise 532
10 trust 1230Joy words in Emma
Following the book, I identified words assigned to the NRC category “joy” in Emma.
tidy_books %>%
filter(book == "Emma") %>%
inner_join(nrc %>% filter(sentiment == "joy"), by = "word") %>%
count(word, sort = TRUE) %>%
slice_head(n = 12)# A tibble: 12 × 2
word n
<chr> <int>
1 good 359
2 friend 166
3 hope 143
4 happy 125
5 love 117
6 deal 92
7 found 92
8 present 89
9 kind 82
10 happiness 76
11 pretty 68
12 true 66Bing sentiment throughout the novels
As in the textbook, each novel is divided into 80-line sections. Within each section, net sentiment is the number of positive Bing words minus the number of negative Bing words.
jane_austen_sentiment <- tidy_books %>%
inner_join(bing, by = "word") %>%
count(book, index = linenumber %/% 80, sentiment) %>%
pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
mutate(sentiment = positive - negative)
jane_austen_sentiment %>% slice_head(n = 8)# A tibble: 8 × 5
book index negative positive sentiment
<fct> <dbl> <int> <int> <int>
1 Sense & Sensibility 0 16 32 16
2 Sense & Sensibility 1 19 53 34
3 Sense & Sensibility 2 12 31 19
4 Sense & Sensibility 3 15 31 16
5 Sense & Sensibility 4 16 34 18
6 Sense & Sensibility 5 16 51 35
7 Sense & Sensibility 6 24 40 16
8 Sense & Sensibility 7 23 51 28ggplot(jane_austen_sentiment, aes(index, sentiment, fill = book)) +
geom_col(show.legend = FALSE) +
facet_wrap(~book, ncol = 2, scales = "free_x") +
labs(title = "Sentiment throughout Jane Austen's novels",
subtitle = "Bing positive minus negative words per 80-line section",
x = "80-line section", y = "Net sentiment") +
theme_minimal()
The plot shows how the relative frequency of positive and negative words changes within each story. The bars are not review ratings or probabilities; they are raw differences in word counts.
Compare AFINN, Bing, and NRC in Pride & Prejudice
The textbook also compares three lexicons within one novel. AFINN sums numerical word scores, whereas Bing and NRC calculate positive minus negative word counts. Because these scales differ, the plots use independent vertical axes.
pride_prejudice <- tidy_books %>% filter(book == "Pride & Prejudice")
pp_afinn <- pride_prejudice %>%
inner_join(afinn, by = "word") %>%
group_by(index = linenumber %/% 80) %>%
summarise(sentiment = sum(value), .groups = "drop") %>%
mutate(method = "AFINN")
pp_other <- bind_rows(
pride_prejudice %>%
inner_join(bing, by = "word") %>%
mutate(method = "Bing"),
pride_prejudice %>%
inner_join(nrc %>% filter(sentiment %in% c("positive", "negative")), by = "word") %>%
mutate(method = "NRC")
) %>%
count(method, index = linenumber %/% 80, sentiment) %>%
pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
mutate(sentiment = positive - negative)
pp_comparison <- bind_rows(pp_afinn, pp_other)ggplot(pp_comparison, aes(index, sentiment, fill = method)) +
geom_col(show.legend = FALSE) +
facet_wrap(~method, ncol = 1, scales = "free_y") +
labs(title = "Sentiment in Pride & Prejudice by lexicon",
x = "80-line section", y = "Net sentiment score") +
theme_minimal()
Words driving sentiment in the novels
austen_word_counts <- tidy_books %>%
inner_join(bing, by = "word") %>%
count(word, sentiment, sort = TRUE)
#| fig-width: 9
#| fig-height: 6
austen_word_counts %>%
group_by(sentiment) %>%
slice_max(n, n = 10, with_ties = FALSE) %>%
ungroup() %>%
ggplot(aes(x = reorder(word, n), y = n, fill = sentiment)) +
geom_col(show.legend = FALSE) +
facet_wrap(~sentiment, scales = "free_y") +
coord_flip() +
labs(title = "Common sentiment-bearing words in Jane Austen",
x = NULL, y = "Occurrences") +
theme_minimal()
Part 2 — Extension: Amazon product reviews
Source and acquisition
The UCI dataset Sentiment Labelled Sentences (Kotzias et al., 2015) provides three separate collections of sentences from Amazon, Yelp, and IMDb. Only amazon_cells_labelled.txtwas used, which has 1,000 labeled sentences. Each line contains a review sentence, a tab, and a binary label (0 = negative; 1 = positive). The download is cached locally to make repeated renders more convenient; the original UCI URL remains documented for reproducibility.
zip_url <- "https://archive.ics.uci.edu/static/public/331/sentiment+labelled+sentences.zip"
zip_path <- "sentiment_labelled_sentences.zip"
if (!file.exists(zip_path)) {
download.file(zip_url, zip_path, mode = "wb", quiet = TRUE)
}
amazon_file <- "sentiment labelled sentences/amazon_cells_labelled.txt"
if (!amazon_file %in% unzip(zip_path, list = TRUE)$Name) {
stop("Amazon file was not found inside the downloaded UCI ZIP archive.")
}
amazon_lines <- readLines(unz(zip_path, amazon_file), warn = FALSE, encoding = "UTF-8")
reviews <- tibble(raw = amazon_lines) %>%
separate_wider_delim(raw, delim = "\t", names = c("review", "label")) %>%
mutate(
review_id = row_number(),
label = as.integer(label),
actual = if_else(label == 1L, "positive", "negative")
) %>%
select(review_id, review, label, actual)
stopifnot(nrow(reviews) == 1000L, all(reviews$label %in% c(0L, 1L)))
reviews %>% count(actual)# A tibble: 2 × 2
actual n
<chr> <int>
1 negative 500
2 positive 500reviews %>% slice_head(n = 6)# A tibble: 6 × 4
review_id review label actual
<int> <chr> <int> <chr>
1 1 So there is no way for me to plug it in here in the US… 0 negat…
2 2 Good case, Excellent value. 1 posit…
3 3 Great for the jawbone. 1 posit…
4 4 Tied to charger for conversations lasting more than 45… 0 negat…
5 5 The mic is great. 1 posit…
6 6 I have to jiggle the plug to get it to line up right t… 0 negat…Tokenize reviews
unnest_tokens() converts reviews to individual lowercase words for the Bing, AFINN, and Loughran dictionaries. The review identifier was kept so each word can be traced back to its original sentence.
review_words <- reviews %>%
select(review_id, review) %>%
unnest_tokens(word, review)
review_words %>% count(word, sort = TRUE) %>% slice_head(n = 12)# A tibble: 12 × 2
word n
<chr> <int>
1 the 515
2 i 318
3 and 311
4 it 283
5 is 243
6 a 218
7 this 207
8 to 197
9 phone 162
10 my 143
11 for 121
12 of 120Additional lexicon: Loughran
The Loughran-McDonald lexicon was developed for financial language and is available through tidytext::get_sentiments("loughran"). It is intentionally a different domain from consumer product reviews.Only its positive and negative categories were used to compare directly with Bing; other categories are not treated as positive or negative. This comparison helps illustrate why a dictionary’s domain matters.
loughran <- get_sentiments("loughran") %>%
filter(sentiment %in% c("positive", "negative"))
loughran %>% count(sentiment)# A tibble: 2 × 2
sentiment n
<chr> <int>
1 negative 2355
2 positive 354Score each review using three lexicons
Bing and Loughran use positive minus negative word counts. AFINN sums signed word scores. A zero score means no net polarity, which may reflect no matched words or cancellation of positive and negative words. Such reviews were labeled neutral and count them as incorrect in binary accuracy calculations rather than silently removing them.
score_polarity <- function(tokens, lexicon, method) {
tokens %>%
inner_join(lexicon, by = "word") %>%
mutate(word_score = if_else(sentiment == "positive", 1, -1)) %>%
group_by(review_id) %>%
summarise(score = sum(word_score), .groups = "drop") %>%
right_join(reviews %>% select(review_id), by = "review_id") %>%
mutate(score = replace_na(score, 0), method = method)
}
bing_scores <- score_polarity(review_words, bing, "Bing")
loughran_scores <- score_polarity(review_words, loughran, "Loughran")
afinn_scores <- review_words %>%
inner_join(afinn, by = "word") %>%
group_by(review_id) %>%
summarise(score = sum(value), .groups = "drop") %>%
right_join(reviews %>% select(review_id), by = "review_id") %>%
mutate(score = replace_na(score, 0), method = "AFINN")
all_scores <- bind_rows(bing_scores, afinn_scores, loughran_scores) %>%
left_join(reviews %>% select(review_id, actual, review), by = "review_id") %>%
mutate(predicted = case_when(
score > 0 ~ "positive",
score < 0 ~ "negative",
TRUE ~ "neutral"
))
all_scores %>% count(method, predicted)# A tibble: 9 × 3
method predicted n
<chr> <chr> <int>
1 AFINN negative 230
2 AFINN neutral 292
3 AFINN positive 478
4 Bing negative 265
5 Bing neutral 215
6 Bing positive 520
7 Loughran negative 176
8 Loughran neutral 519
9 Loughran positive 305Visualize the score distributions
The scores are on different scales, so I chose to display the three methods in separate panels with independent x-axes, grouped by the known positive and negative labels.
ggplot(all_scores, aes(x = score, fill = actual)) +
geom_histogram(binwidth = 1, position = "identity", alpha = 0.55) +
facet_wrap(~method, scales = "free_x", ncol = 1) +
labs(title = "Review sentiment scores by lexicon and actual label",
x = "Lexicon score (different scales)", y = "Review count", fill = "Actual label") +
theme_minimal()
Compare predictions with known labels
Accuracy is the proportion of all 1,000 reviews classified correctly. Coverage is the proportion receiving a non-neutral prediction. I also calculated accuracy among classified reviews to distinguish low dictionary coverage from wrong polarity.
metrics <- all_scores %>%
group_by(method) %>%
summarise(
total = n(),
correct = sum(predicted == actual),
neutral = sum(predicted == "neutral"),
coverage = mean(predicted != "neutral"),
accuracy = mean(predicted == actual),
accuracy_classified = if (sum(predicted != "neutral") > 0)
mean(predicted[predicted != "neutral"] == actual[predicted != "neutral"])
else NA_real_,
.groups = "drop"
)
metrics# A tibble: 3 × 7
method total correct neutral coverage accuracy accuracy_classified
<chr> <int> <int> <int> <dbl> <dbl> <dbl>
1 AFINN 1000 589 292 0.708 0.589 0.832
2 Bing 1000 672 215 0.785 0.672 0.856
3 Loughran 1000 427 519 0.481 0.427 0.888metrics %>%
ggplot(aes(x = reorder(method, accuracy), y = accuracy, fill = method)) +
geom_col(show.legend = FALSE, width = 0.6) +
geom_text(aes(label = scales::percent(accuracy, accuracy = 0.1)), hjust = -0.15) +
coord_flip() +
scale_y_continuous(limits = c(0, 1), labels = scales::percent) +
labs(title = "Accuracy on 1,000 labeled Amazon review sentences",
subtitle = "Neutral predictions count as incorrect", x = NULL, y = "Accuracy") +
theme_minimal()
confusion <- all_scores %>%
count(method, actual, predicted) %>%
complete(method, actual, predicted = c("negative", "neutral", "positive"), fill = list(n = 0))
confusion# A tibble: 18 × 4
method actual predicted n
<chr> <chr> <chr> <int>
1 AFINN negative negative 212
2 AFINN negative neutral 187
3 AFINN negative positive 101
4 AFINN positive negative 18
5 AFINN positive neutral 105
6 AFINN positive positive 377
7 Bing negative negative 251
8 Bing negative neutral 150
9 Bing negative positive 99
10 Bing positive negative 14
11 Bing positive neutral 65
12 Bing positive positive 421
13 Loughran negative negative 162
14 Loughran negative neutral 298
15 Loughran negative positive 40
16 Loughran positive negative 14
17 Loughran positive neutral 221
18 Loughran positive positive 265Examples where dictionaries disagree
disagreements <- all_scores %>%
select(review_id, method, predicted) %>%
pivot_wider(names_from = method, values_from = predicted) %>%
filter(Bing != AFINN | Bing != Loughran | AFINN != Loughran) %>%
left_join(reviews, by = "review_id") %>%
select(review_id, review, actual, Bing, AFINN, Loughran)
disagreements %>% slice_head(n = 12)# A tibble: 12 × 6
review_id review actual Bing AFINN Loughran
<int> <chr> <chr> <chr> <chr> <chr>
1 6 I have to jiggle the plug to get it to… negat… posi… neut… neutral
2 7 If you have several dozen or several h… negat… posi… posi… neutral
3 10 What a waste of money and time!. negat… nega… nega… neutral
4 13 If the two were seperated by a mere 5+… negat… nega… neut… negative
5 16 Highly recommend for any one who has a… posit… posi… posi… neutral
6 17 I advise EVERYONE DO NOT BE FOOLED! negat… nega… neut… neutral
7 20 It clicks into place in a way that mak… negat… posi… neut… neutral
8 22 I bought this to use with my Kindle Fi… posit… posi… posi… neutral
9 28 I didn't think that the instructions p… negat… posi… posi… neutral
10 33 It is unusable in a moving car at free… negat… nega… neut… negative
11 34 I have two more years left in this con… negat… nega… nega… neutral
12 35 Car charger as well as AC charger are … posit… posi… posi… neutral Comparison and Discussion
The original analysis uses long-form fiction and evaluates sentiment within a narrative: 80-line bins reveal changes in positive and negative language as the story progresses. In contrast, this extension treats each product-review sentence as an independent observation and compares predictions with existing labels. These tasks cannot be compared using the same numerical scale: the Austen plots measure raw word-count differences across sections, while the review analysis reports per-sentence predictions and classification accuracy.
best <- metrics %>% arrange(desc(accuracy), desc(coverage)) %>% slice(1)
cat(sprintf("In the Amazon dataset, **%s** had the highest measured accuracy (%.1f%%), with %.1f%% coverage. ",
best$method, 100 * best$accuracy, 100 * best$coverage))In the Amazon dataset, Bing had the highest measured accuracy (67.2%), with 78.5% coverage.
cat("The ranking is specific to this balanced set of short Amazon sentences and should not be generalized to all customer reviews.\n\n")The ranking is specific to this balanced set of short Amazon sentences and should not be generalized to all customer reviews.
for (i in seq_len(nrow(metrics))) {
x <- metrics[i, ]
cat(sprintf("- **%s:** %.1f%% accuracy, %.1f%% coverage, and %d neutral predictions.\n",
x$method, 100 * x$accuracy, 100 * x$coverage, x$neutral))
}- AFINN: 58.9% accuracy, 70.8% coverage, and 292 neutral predictions.
- Bing: 67.2% accuracy, 78.5% coverage, and 215 neutral predictions.
- Loughran: 42.7% accuracy, 48.1% coverage, and 519 neutral predictions.
The differences are expected because each lexicon has a different vocabulary and scoring method. Bing counts positive and negative terms equally; AFINN incorporates word intensity; Loughran was developed for financial documents and may omit everyday product-review vocabulary. None of these simple word-level approaches reliably interprets sarcasm, negation, or context. For example, “not good” can be scored incorrectly if “good” is treated as positive without considering “not.” The gold-standard labels are also sentence-level annotations, not necessarily labels for full customer reviews.
Conclusion
Reproducing the Jane Austen example demonstrates how tidy text data and dictionary joins can uncover sentiment patterns across a literary narrative. Extending the analysis to Amazon reviews demonstrates a different use case: evaluating sentiment classification against human-provided labels. The additional Loughran lexicon provides a domain-specific contrast and shows that sentiment results depend on the words and scoring rules in each dictionary. The comparison illustrates why lexicon selection and coverage matter as much as the resulting score.
References
Silge, J., & Robinson, D. (2017). Text Mining with R: A Tidy Approach. O’Reilly Media. Chapter 2: https://www.tidytextmining.com/sentiment
Kotzias, D., Denil, M., de Freitas, N., & Smyth, P. (2015). From group to individual labels using deep features. Proceedings of KDD 2015. Dataset: Sentiment Labelled Sentences, UCI Machine Learning Repository. https://doi.org/10.24432/C57604
Loughran, T., & McDonald, B. (2011). When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks. The Journal of Finance, 66(1), 35–65. https://doi.org/10.1111/j.1540-6261.2010.01625.x
Hutto, C. J., & Gilbert, E. (2014). VADER: A parsimonious rule-based model for sentiment analysis of social media text. Proceedings of ICWSM. (Background alternative; not implemented in this report.)