10A Sentiment Analysis

Task

Assignment: Sentiment Analysis with Text Mining with R In Chapter 2 of Text Mining with R, the authors introduce sentiment analysis. Your task is to reproduce and extend that example.

Step 1 — Reproduce the Base Example Obtain the primary example code from Chapter 2.

Create a Quarto (.qmd) file that successfully runs this code.

Include a proper citation to the book and the original example code.

Step 2 — Extend the Analysis Extend the original example in two ways:

Use a different text corpus of your choosing (e.g., news articles, book reviews, speeches, social media, etc.).

Add at least one additional sentiment lexicon, which may come from:

Another R package, or

A lexicon you locate through external research.

You should clearly explain how the results differ from the original example.

Step 1: Reproduce the base example

library(tidytext)
library(janeaustenr)
library(dplyr)

Attaching package: 'dplyr'

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

library(stringr)


get_sentiments("afinn")

# A tibble: 2,477 × 2
   word       value
   <chr>      <dbl>
 1 abandon       -2
 2 abandoned     -2
 3 abandons      -2
 4 abducted      -2
 5 abduction     -2
 6 abductions    -2
 7 abhor         -3
 8 abhorred      -3
 9 abhorrent     -3
10 abhors        -3
# ℹ 2,467 more rows

get_sentiments("bing")

# A tibble: 6,786 × 2
   word        sentiment
   <chr>       <chr>    
 1 2-faces     negative 
 2 abnormal    negative 
 3 abolish     negative 
 4 abominable  negative 
 5 abominably  negative 
 6 abominate   negative 
 7 abomination negative 
 8 abort       negative 
 9 aborted     negative 
10 aborts      negative 
# ℹ 6,776 more rows

get_sentiments("nrc")

# A tibble: 13,872 × 2
   word        sentiment
   <chr>       <chr>    
 1 abacus      trust    
 2 abandon     fear     
 3 abandon     negative 
 4 abandon     sadness  
 5 abandoned   anger    
 6 abandoned   fear     
 7 abandoned   negative 
 8 abandoned   sadness  
 9 abandonment anger    
10 abandonment fear     
# ℹ 13,862 more rows

tidy_books <- austen_books() %>%
  group_by(book) %>%
  mutate(
    linenumber = row_number(),
    chapter = cumsum(str_detect(text, 
                                regex("^chapter [\\divxlc]", 
                                      ignore_case = TRUE)))) %>%
  ungroup() %>%
  unnest_tokens(word, text)

nrc_joy <- get_sentiments("nrc") %>% 
  filter(sentiment == "joy")

tidy_books %>%
  filter(book == "Emma") %>%
  inner_join(nrc_joy) %>%
  count(word, sort = TRUE)

Joining with `by = join_by(word)`

# A tibble: 301 × 2
   word          n
   <chr>     <int>
 1 good        359
 2 friend      166
 3 hope        143
 4 happy       125
 5 love        117
 6 deal         92
 7 found        92
 8 present      89
 9 kind         82
10 happiness    76
# ℹ 291 more rows

library(tidyr)

jane_austen_sentiment <- tidy_books %>%
  inner_join(get_sentiments("bing")) %>%
  count(book, index = linenumber %/% 80, sentiment) %>%
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>% 
  mutate(sentiment = positive - negative)

Joining with `by = join_by(word)`

Warning in inner_join(., get_sentiments("bing")): Detected an unexpected many-to-many relationship between `x` and `y`.
ℹ Row 435434 of `x` matches multiple rows in `y`.
ℹ Row 5051 of `y` matches multiple rows in `x`.
ℹ If a many-to-many relationship is expected, set `relationship =
  "many-to-many"` to silence this warning.

library(ggplot2)
ggplot(jane_austen_sentiment, aes(index, sentiment, fill = book)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~book, ncol = 2, scales = "free_x")

Step 2: Extend the Analysis

2.1 Social Media Dataset

For the extended analysis, I will use the Social Media Sentiments Analysis Dataset from Kaggle,https://www.kaggle.com/datasets/kashishparmar02/social-media-sentiments-analysis-dataset. Unlike the original example, which analyzes Jane Austen’s novels, this dataset contains user-generated social media posts from different platforms, including Instagram, Facebook, and Twitter.

I will apply the Bing, AFINN, and NRC sentiment lexicons to the social media text and compare the results with the original Jane Austen analysis.

2.2 Additional Sentiment Method: VADER

In addition to the three original sentiment lexicons, I will use VADER (Valence Aware Dictionary and sEntiment Reasoner). VADER is a lexicon-based and rule-based sentiment analysis tool designed specifically for social media text. It considers features such as capitalization, punctuation, emoticons, and the intensity of words.

Anticipated challenges

Since this is my first time working with sentiment analysis, I may face challenges when applying different sentiment lexicons to analyze the social media data. Understanding how each lexicon classifies words and interprets sentiment may also be challenging.