Exploring the contextual impact of negation words on sentiment analysis

0. INTRODUCTION

As a follow up study about sentiment analysis on the Next Generation Science Standards (NGSS) and Common Core State Standards (CCSS), the purpose of this analysis is to investigate how bigrams in sentiment analysis can provide context that should be considered when words are combined with negation words (e.g., not, without, no, and never). Previous analyses of sentiments toward the NGSS and CCSS showed that positive sentiments outweighed negative ones. However, these analyses did not consider how sentiment words could have the opposite meaning when preceded by negation words. Therefore, this analysis using bigrams aims to explore: 1) how many words in the tweets of NGSS and CSSS datasets are combined with negation words, and 2) how much those combinations could impact overall sentiment analysis.

---

1. PREPARE - set up

library(dplyr)
library(readr)
library(tidyr)
library(rtweet)
library(writexl)
library(readxl)
library(tidytext)
library(textdata)
library(ggplot2)
library(textdata)
library(scales)

---

2. WRANGLE

2a. Importing Data and Setting up the Dataset

# Importing data 
ngss_tweets <- read_xlsx("/cloud/project/dataset/ngss_tweets.xlsx")
ccss_tweets <- read_xlsx("/cloud/project/dataset/csss_tweets.xlsx")

# Setting up the dataset 
ngss_text <-
  ngss_tweets %>%
  filter(lang == "en") %>%
  select(screen_name, created_at, text) %>%
  mutate(standards = "ngss") %>%
  relocate(standards)

ccss_text <-
  ccss_tweets %>%
  filter(lang == "en") %>%
  select(screen_name, created_at, text) %>%
  mutate(standards = "ccss") %>%
  relocate(standards)

tweets <- bind_rows(ngss_text, ccss_text)

2b. Tokenizing text into bigrams

tweets_bigrams <- tweets %>%
  unnest_tokens(bigram, text, token = "ngrams", n = 2)

tweets_bigrams %>%
  count(bigram, sort = TRUE)

## # A tibble: 25,435 × 2
##    bigram          n
##    <chr>       <int>
##  1 common core  1098
##  2 https t       623
##  3 t co          623
##  4 core math     251
##  5 the common    117
##  6 of the        102
##  7 in the         90
##  8 this is        72
##  9 core is        59
## 10 to the         59
## # ℹ 25,425 more rows

bigrams_separated <- tweets_bigrams %>%
  separate(bigram, c("word1", "word2"), sep = " ")

Now I will count the number of bigrams that are combined with negation words:

bigrams_separated %>%
  filter(word1 == "not") %>%
  count(word1, word2, sort = TRUE)

## # A tibble: 103 × 3
##    word1 word2        n
##    <chr> <chr>    <int>
##  1 not   common       9
##  2 not   a            7
##  3 not   sure         7
##  4 not   in           6
##  5 not   the          6
##  6 not   just         4
##  7 not   as           3
##  8 not   on           3
##  9 not   teaching     3
## 10 not   that         3
## # ℹ 93 more rows

bigrams_separated %>%
  filter(word1 == "no") %>%
  count(word1, word2, sort = TRUE)

## # A tibble: 48 × 3
##    word1 word2              n
##    <chr> <chr>          <int>
##  1 no    one                8
##  2 no    child              7
##  3 no    sense              5
##  4 no    common             4
##  5 no    way                4
##  6 no    idea               3
##  7 no    longer             3
##  8 no    more               3
##  9 no    right              3
## 10 no    indoctrination     2
## # ℹ 38 more rows

bigrams_separated %>%
  filter(word1 == "never") %>%
  count(word1, word2, sort = TRUE)

## # A tibble: 22 × 3
##    word1 word2       n
##    <chr> <chr>   <int>
##  1 never mention     2
##  2 never balance     1
##  3 never be          1
##  4 never been        1
##  5 never brought     1
##  6 never forgive     1
##  7 never get         1
##  8 never hated       1
##  9 never have        1
## 10 never heard       1
## # ℹ 12 more rows

bigrams_separated %>%
  filter(word1 == "without") %>%
  count(word1, word2, sort = TRUE)

## # A tibble: 8 × 3
##   word1   word2        n
##   <chr>   <chr>    <int>
## 1 without being        1
## 2 without blurring     1
## 3 without early        1
## 4 without god          1
## 5 without grades       1
## 6 without him          1
## 7 without the          1
## 8 without your         1

Results show that the number of words that combined with “not” and “no” is greater than the number of words that combined with “never” and “without”.

2c. Sentiment Analysis

afinn <- get_sentiments("afinn")

afinn

## # A tibble: 2,477 × 2
##    word       value
##    <chr>      <dbl>
##  1 abandon       -2
##  2 abandoned     -2
##  3 abandons      -2
##  4 abducted      -2
##  5 abduction     -2
##  6 abductions    -2
##  7 abhor         -3
##  8 abhorred      -3
##  9 abhorrent     -3
## 10 abhors        -3
## # ℹ 2,467 more rows

not_words <- bigrams_separated %>%
  filter(word1 == "not") %>%
  inner_join(afinn, by = c(word2 = "word")) %>%
  count(word2, value, sort = TRUE)

not_words

## # A tibble: 12 × 3
##    word2     value     n
##    <chr>     <dbl> <int>
##  1 like          2     2
##  2 bad          -3     1
##  3 creative      2     1
##  4 free          1     1
##  5 hard         -1     1
##  6 helpful       2     1
##  7 important     2     1
##  8 liked         2     1
##  9 limited      -1     1
## 10 smart         1     1
## 11 support       2     1
## 12 true          2     1

negation_words <- c("not", "no", "never", "without")

negated_words <- bigrams_separated %>%
  filter(word1 %in% negation_words) %>%
  inner_join(afinn, by = c(word2 = "word")) %>%
  count(word1, word2, value, sort = TRUE)

3. VISUALIZATION

negated_words %>%
  mutate(contribution = n * value) %>%
  arrange(desc(abs(contribution))) %>%
  group_by(word1) %>%  
  mutate(word2 = reorder(word2, contribution)) %>%
  ggplot(aes(x = contribution, y = word2, fill = contribution > 0)) +
  geom_col(show.legend = FALSE) +
  labs(x = "Sentiment value * number of occurrences",
       y = "Words preceded by negation term") + 
  facet_wrap(~word1, scales = "free_y", ncol = 1) +
  theme_minimal() +
  theme(axis.text.y = element_text(size = 8))  

The length of the bars represents the influence of a word, calculated as the product of the word’s sentiment score and its frequency of occurrence. This length notably longer in the “no” and “not” graph, indicating that the sentiment conveyed by those words might be substantially altered by the negation. Additionally, the direction of sentiment, shown by the color of the bar, demonstrates that observing whether words following negation words tend to have positive or negative sentiment scores can help understand the overall shift in sentiment caused by the negation.

4. Conclusion

Most negation words except for “without”, show a considerable magnitude of influence that researchers conducting sentiment analysis should consider. This implies that when researchers analyze the sentiments toward NGSS and CCSS, the context of sentiment words should be carefully considered.