Assignment 10A: Sentiment Analysis

Introduction

Hello, it is now week 10 and we’re tasked with Sentiment analysis. We see sentiment analysis come up in chapter 2 of our book “Text Mining with R”. Our task for this assignment is to reproduce and extend the example provided in the chapter.

Planned Workflow

I plan to obtain the primary example code from the chapter, cite it, and create a quarto document that can run the code. Then I’ll have to extend it, which I plan to do by choosing a different text corpus, which will be using gutenbergr, and I’ll choose one sentiment lexicon which will be syuzhet.

Anticipated Challenges

A challenge I can foresee dealing with will be successfully extending the example with my two unique choices of gutenbergr and syuzhet. Successfully loading and using them as a text corpus and lexicon will be the main event for this assignment.

Source: Silge, J., & Robinson, D. (2017). Text Mining with R: A Tidy Approach. O’Reilly Media. Chapter 2.

library(janeaustenr)
library(stringr)
library(dplyr)

Attaching package: 'dplyr'

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

library(tidyr)
library(ggplot2)
library(tidytext)

tidy_books <- austen_books() %>%
  group_by(book) %>%
  mutate(
    linenumber = row_number(),
    chapter = cumsum(str_detect(text,
                                regex("^chapter [\\divxlc]",
                                      ignore_case = TRUE)))) %>%
  ungroup() %>%
  unnest_tokens(word, text)

nrc_joy <- get_sentiments("nrc") %>%
  filter(sentiment == "joy")
         tidy_books %>%
           filter(book == "Emma") %>%
           inner_join(nrc_joy) %>%
           count(word, sort = TRUE)

Joining with `by = join_by(word)`

# A tibble: 301 × 2
   word          n
   <chr>     <int>
 1 good        359
 2 friend      166
 3 hope        143
 4 happy       125
 5 love        117
 6 deal         92
 7 found        92
 8 present      89
 9 kind         82
10 happiness    76
# ℹ 291 more rows

jane_austen_sentiment <- tidy_books %>%
  inner_join(get_sentiments("bing")) %>%
  count(book, index = linenumber %/% 80, sentiment) %>%
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
  mutate(sentiment = positive - negative)

Joining with `by = join_by(word)`

Warning in inner_join(., get_sentiments("bing")): Detected an unexpected many-to-many relationship between `x` and `y`.
ℹ Row 435434 of `x` matches multiple rows in `y`.
ℹ Row 5051 of `y` matches multiple rows in `x`.
ℹ If a many-to-many relationship is expected, set `relationship =
  "many-to-many"` to silence this warning.

library(ggplot2)

ggplot(jane_austen_sentiment, aes(index, sentiment, fill = book)) + 
  geom_col(show.legend = FALSE) +
  facet_wrap(~book, ncol = 2, scales = "free_x")

Extend the Analysis

library(syuzhet)
library(gutenbergr)

frankenstein_raw <- gutenberg_download(84)

Using mirror https://aleph.pglaf.org.

tidy_frankenstein <- frankenstein_raw %>%
  unnest_tokens(word, text) %>%
  anti_join(stop_words)

Joining with `by = join_by(word)`

frankenstein_sentiment <- get_sentiment(tidy_frankenstein$word, method = "syuzhet")

tidy_frankenstein$sentiment_score <- frankenstein_sentiment

frankenstein_plot_data <- tidy_frankenstein %>%
  mutate(index = row_number() %/% 100) %>%
  group_by(index) %>%
  summarise(sentiment = sum(sentiment_score))

ggplot(frankenstein_plot_data, aes(index, sentiment)) +
  geom_col(fill = "darkred") +
  theme_minimal() +
  labs(title = "Sentiment trajectory of Frankenstein",
       subtitle = "Syuzhet Lexicon",
       x = "Narrative Progress (100-word chunks)",
       y = "Total Sentiment Score")

Conclusion

In conclusion, using the syuzhet lexicon for Frankenstein helped show the emotional weight of the amount of negative words vs. positive words on the plot. Per 100 word chunks, we can see that as the story progresses there’s a drastic drop in the total sentiment score. This drop can signal that in the story, there are many moments that are written that can be interpreted as dark and scary in nature. The plot as a whole shows that this story is primarily very dark with a few light moments throughout the story.

This is different in comparison to the example from chapter 2 I reproduced. Chapter 2 showed that the NRC and bing lexicons are not weighted but rather a sum per 80 words. A bad word is a negative like -1 and a positive word gives a +1. So per 80 it’s doing a sum count of the positive and negative words and plots that.