Sentiment analysis on school reform: Focusing on California State

Questions

1. What is the public sentiment of California expressed toward the NGSS?
2. What is the public sentiment of California expressed toward the CCSS?
3. To what extent does a difference in sentiment appear in California between NGSS and CCSS?

Methods

** Data Source: Twitter data from February 2021 related to NGSS and CCSS Data processing: Tokenized and tided text Data analysis**: Word counts & Sentiment analysis

Findings

library(dplyr)
library(readr)
library(tidyr)
library(rtweet)
library(writexl)
library(readxl)
library(tidytext)
library(textdata)
library(ggplot2)
library(textdata)
library(scales)

ngss_tweets <- read_xlsx("data/ngss_tweets.xlsx")
ccss_tweets <- read_xlsx("data/csss_tweets.xlsx")

ngss_text <-
  ngss_tweets %>%
  filter(lang == "en") %>%
  select(screen_name, created_at, text, location) %>%
  mutate(standards = "ngss") %>%
  relocate(standards)

tweets <- bind_rows(ngss_text, ccss_text)

#head(tweets)
#tail(tweets)
library(dplyr)
filtered_tweets <- filter(tweets, grepl("CA|California", location))

tweet_tokens <- 
  filtered_tweets %>%
  unnest_tokens(output = word, 
                input = text, 
                token = "words")

tidy_tweets <-
  tweet_tokens %>%
  anti_join(stop_words, by = "word")

tidy_tweets <-
  tweet_tokens %>%
  anti_join(stop_words, by = "word") %>%
  filter(!word == "amp")

afinn <- get_sentiments("afinn")

afinn

## # A tibble: 2,477 × 2
##    word       value
##    <chr>      <dbl>
##  1 abandon       -2
##  2 abandoned     -2
##  3 abandons      -2
##  4 abducted      -2
##  5 abduction     -2
##  6 abductions    -2
##  7 abhor         -3
##  8 abhorred      -3
##  9 abhorrent     -3
## 10 abhors        -3
## # ℹ 2,467 more rows

bing <- get_sentiments("bing")

bing

## # A tibble: 6,786 × 2
##    word        sentiment
##    <chr>       <chr>    
##  1 2-faces     negative 
##  2 abnormal    negative 
##  3 abolish     negative 
##  4 abominable  negative 
##  5 abominably  negative 
##  6 abominate   negative 
##  7 abomination negative 
##  8 abort       negative 
##  9 aborted     negative 
## 10 aborts      negative 
## # ℹ 6,776 more rows

nrc <- get_sentiments("nrc")

nrc

## # A tibble: 13,872 × 2
##    word        sentiment
##    <chr>       <chr>    
##  1 abacus      trust    
##  2 abandon     fear     
##  3 abandon     negative 
##  4 abandon     sadness  
##  5 abandoned   anger    
##  6 abandoned   fear     
##  7 abandoned   negative 
##  8 abandoned   sadness  
##  9 abandonment anger    
## 10 abandonment fear     
## # ℹ 13,862 more rows

loughran <- get_sentiments("loughran")

loughran

## # A tibble: 4,150 × 2
##    word         sentiment
##    <chr>        <chr>    
##  1 abandon      negative 
##  2 abandoned    negative 
##  3 abandoning   negative 
##  4 abandonment  negative 
##  5 abandonments negative 
##  6 abandons     negative 
##  7 abdicated    negative 
##  8 abdicates    negative 
##  9 abdicating   negative 
## 10 abdication   negative 
## # ℹ 4,140 more rows

sentiment_afinn <- inner_join(tidy_tweets, afinn, by = "word")

sentiment_afinn

## # A tibble: 106 × 6
##    standards screen_name created_at          location                word  value
##    <chr>     <chr>       <dttm>              <chr>                   <chr> <dbl>
##  1 ngss      rjmama      2021-02-26 17:56:00 Ohlone homeland (Bay A… awes…     4
##  2 ngss      STEM4Real   2021-02-26 01:17:20 California, USA         join      1
##  3 ngss      STEM4Real   2021-02-26 01:17:20 California, USA         comm…     1
##  4 ngss      STEM4Real   2021-02-26 01:17:20 California, USA         just…     2
##  5 ngss      STEM4Real   2021-02-26 15:50:26 California, USA         raci…    -3
##  6 ngss      STEM4Real   2021-02-26 15:50:26 California, USA         anti     -1
##  7 ngss      STEM4Real   2021-02-26 15:50:26 California, USA         raci…    -3
##  8 ngss      STEM4Real   2021-02-26 15:50:26 California, USA         share     1
##  9 ngss      STEM4Real   2021-02-26 15:50:26 California, USA         resp…     2
## 10 ngss      STEM4Real   2021-02-23 15:56:57 California, USA         comm…     1
## # ℹ 96 more rows

sentiment_bing <- inner_join(tidy_tweets, bing, by = "word")

sentiment_bing

## # A tibble: 111 × 6
##    standards screen_name created_at          location            word  sentiment
##    <chr>     <chr>       <dttm>              <chr>               <chr> <chr>    
##  1 ngss      rjmama      2021-02-26 17:56:00 Ohlone homeland (B… awes… positive 
##  2 ngss      STEM4Real   2021-02-26 15:50:26 California, USA     raci… negative 
##  3 ngss      STEM4Real   2021-02-26 15:50:26 California, USA     raci… negative 
##  4 ngss      STEM4Real   2021-02-26 15:50:26 California, USA     resp… positive 
##  5 ngss      SRVUSD_NGSS 2021-02-24 03:51:28 California, USA     love  positive 
##  6 ngss      SRVUSD_NGSS 2021-02-23 00:32:10 California, USA     amaz… positive 
##  7 ngss      SRVUSD_NGSS 2021-02-23 00:32:10 California, USA     insp… positive 
##  8 ngss      SRVUSD_NGSS 2021-02-23 00:32:10 California, USA     impr… positive 
##  9 ngss      SRVUSD_NGSS 2021-02-23 20:59:28 California, USA     love  positive 
## 10 ngss      SRVUSD_NGSS 2021-02-20 17:47:41 California, USA     cong… positive 
## # ℹ 101 more rows

## Warning in inner_join(tidy_tweets, nrc, by = "word"): Detected an unexpected many-to-many relationship between `x` and `y`.
## ℹ Row 21 of `x` matches multiple rows in `y`.
## ℹ Row 8556 of `y` matches multiple rows in `x`.
## ℹ If a many-to-many relationship is expected, set `relationship =
##   "many-to-many"` to silence this warning.

## Warning in inner_join(tidy_tweets, loughran, by = "word"): Detected an unexpected many-to-many relationship between `x` and `y`.
## ℹ Row 921 of `x` matches multiple rows in `y`.
## ℹ Row 3919 of `y` matches multiple rows in `x`.
## ℹ If a many-to-many relationship is expected, set `relationship =
##   "many-to-many"` to silence this warning.

summary_bing <- count(sentiment_bing, sentiment, sort = TRUE)

summary_bing

## # A tibble: 2 × 2
##   sentiment     n
##   <chr>     <int>
## 1 positive     63
## 2 negative     48

Since our main goal is to compare positive and negative sentiment between CCSS and NGSS, let’s use the group_by function again to get sentiment summaries for NGSS and CCSS separately:

summary_bing <- sentiment_bing %>% 
  group_by(standards) %>% 
  count(sentiment) 

summary_bing

## # A tibble: 4 × 3
## # Groups:   standards [2]
##   standards sentiment     n
##   <chr>     <chr>     <int>
## 1 ccss      negative     43
## 2 ccss      positive     36
## 3 ngss      negative      5
## 4 ngss      positive     27

summary_bing <- sentiment_bing %>% 
  group_by(standards) %>% 
  count(sentiment, sort = TRUE) %>% 
  spread(sentiment, n) 

summary_bing

## # A tibble: 2 × 3
## # Groups:   standards [2]
##   standards negative positive
##   <chr>        <int>    <int>
## 1 ccss            43       36
## 2 ngss             5       27

summary_bing <- sentiment_bing %>% 
  group_by(standards) %>% 
  count(sentiment, sort = TRUE) %>% 
  spread(sentiment, n) %>%
  mutate(sentiment = positive - negative) %>%
  mutate(lexicon = "bing") %>%
  relocate(lexicon)

summary_bing

## # A tibble: 2 × 5
## # Groups:   standards [2]
##   lexicon standards negative positive sentiment
##   <chr>   <chr>        <int>    <int>     <int>
## 1 bing    ccss            43       36        -7
## 2 bing    ngss             5       27        22

head(sentiment_afinn)

## # A tibble: 6 × 6
##   standards screen_name created_at          location                 word  value
##   <chr>     <chr>       <dttm>              <chr>                    <chr> <dbl>
## 1 ngss      rjmama      2021-02-26 17:56:00 Ohlone homeland (Bay Ar… awes…     4
## 2 ngss      STEM4Real   2021-02-26 01:17:20 California, USA          join      1
## 3 ngss      STEM4Real   2021-02-26 01:17:20 California, USA          comm…     1
## 4 ngss      STEM4Real   2021-02-26 01:17:20 California, USA          just…     2
## 5 ngss      STEM4Real   2021-02-26 15:50:26 California, USA          raci…    -3
## 6 ngss      STEM4Real   2021-02-26 15:50:26 California, USA          anti     -1

summary_afinn <- sentiment_afinn %>% 
  group_by(standards) %>% 
  summarise(sentiment = sum(value)) %>% 
  mutate(lexicon = "AFINN") %>%
  relocate(lexicon)

summary_afinn

## # A tibble: 2 × 3
##   lexicon standards sentiment
##   <chr>   <chr>         <dbl>
## 1 AFINN   ccss            -15
## 2 AFINN   ngss             53

nrc

## # A tibble: 13,872 × 2
##    word        sentiment
##    <chr>       <chr>    
##  1 abacus      trust    
##  2 abandon     fear     
##  3 abandon     negative 
##  4 abandon     sadness  
##  5 abandoned   anger    
##  6 abandoned   fear     
##  7 abandoned   negative 
##  8 abandoned   sadness  
##  9 abandonment anger    
## 10 abandonment fear     
## # ℹ 13,862 more rows

## # A tibble: 2 × 5
## # Groups:   standards [2]
##   standards method negative positive sentiment
##   <chr>     <chr>     <int>    <int>     <dbl>
## 1 ccss      nrc          34      118      3.47
## 2 ngss      nrc           4       67     16.8

## # A tibble: 2 × 5
## # Groups:   standards [2]
##   standards method   negative positive sentiment
##   <chr>     <chr>       <int>    <int>     <dbl>
## 1 ccss      loughran       17        6     0.353
## 2 ngss      loughran        3        4     1.33

ngss_text <-
  ngss_tweets %>%
  filter(lang == "en") %>%
  select(status_id, text, location) %>%
  mutate(standards = "ngss") %>%
  relocate(standards)

ccss_text <-
  ccss_tweets %>%
  filter(lang == "en") %>%
  select(status_id, text, location) %>%
  mutate(standards = "ccss") %>%
  relocate(standards)

tweets <- bind_rows(ngss_text, ccss_text)

library(dplyr)
filtered_tweets <- filter(tweets, grepl("CA|California", location))

filtered_tweets

## # A tibble: 87 × 4
##    standards status_id           text                                   location
##    <chr>     <chr>               <chr>                                  <chr>   
##  1 ngss      1365359977238433793 "@SRVUSD_NGSS @Meliseymo So Awesome! … Ohlone …
##  2 ngss      1365108654618210306 "@staciekunihisa @WilsonDolphins1 @NY… Califor…
##  3 ngss      1365328375737327620 "Our Clubhouse talk was called \"raci… Califor…
##  4 ngss      1364242855066148867 "What is The STEM4Real Network? \n\nC… Califor…
##  5 ngss      1364422668129234948 "@miss_schilling_ @NGSS_tweeps Unit m… Califor…
##  6 ngss      1364010124663169027 "Amazing panel of educators on this e… Califor…
##  7 ngss      1365325780608184322 "Thank you @Meliseymo for continuing … Califor…
##  8 ngss      1363869840222523396 "So many great suggestions from Edu-T… Califor…
##  9 ngss      1364318984628150273 "I love my job!  I partnered a studen… Califor…
## 10 ngss      1363183556617379841 "@sraser88 @NCWITAIC Congratulations,… Califor…
## # ℹ 77 more rows

sentiment_afinn <- filtered_tweets %>%
  unnest_tokens(output = word, 
                input = text, 
                token = "words")  %>% 
  anti_join(stop_words, by = "word") %>%
  filter(!word == "amp") %>%
  inner_join(afinn, by = "word")

sentiment_afinn

## # A tibble: 106 × 5
##    standards status_id           location                       word       value
##    <chr>     <chr>               <chr>                          <chr>      <dbl>
##  1 ngss      1365359977238433793 Ohlone homeland (Bay Area, CA) awesome        4
##  2 ngss      1365108654618210306 California, USA                join           1
##  3 ngss      1365108654618210306 California, USA                committed      1
##  4 ngss      1365108654618210306 California, USA                justice        2
##  5 ngss      1365328375737327620 California, USA                racist        -3
##  6 ngss      1365328375737327620 California, USA                anti          -1
##  7 ngss      1365328375737327620 California, USA                racism        -3
##  8 ngss      1365328375737327620 California, USA                share          1
##  9 ngss      1365328375737327620 California, USA                responsive     2
## 10 ngss      1364242855066148867 California, USA                committed      1
## # ℹ 96 more rows

afinn_score <- sentiment_afinn %>% 
  group_by(standards, status_id) %>% 
  summarise(value = sum(value))

afinn_score

## # A tibble: 56 × 3
## # Groups:   standards [2]
##    standards status_id           value
##    <chr>     <chr>               <dbl>
##  1 ccss      1363341600168894467     4
##  2 ccss      1363380595825414144    -2
##  3 ccss      1363664916234727427     0
##  4 ccss      1363677592004354048    -7
##  5 ccss      1363700630569713666    -3
##  6 ccss      1363723464138313728    -3
##  7 ccss      1363743035679404032     4
##  8 ccss      1363977333573439489    -6
##  9 ccss      1364021137370279938    -5
## 10 ccss      1364227817949437965    -4
## # ℹ 46 more rows

afinn_sentiment <- afinn_score %>%
  filter(value != 0) %>%
  mutate(sentiment = if_else(value < 0, "negative", "positive"))

afinn_sentiment

## # A tibble: 53 × 4
## # Groups:   standards [2]
##    standards status_id           value sentiment
##    <chr>     <chr>               <dbl> <chr>    
##  1 ccss      1363341600168894467     4 positive 
##  2 ccss      1363380595825414144    -2 negative 
##  3 ccss      1363677592004354048    -7 negative 
##  4 ccss      1363700630569713666    -3 negative 
##  5 ccss      1363723464138313728    -3 negative 
##  6 ccss      1363743035679404032     4 positive 
##  7 ccss      1363977333573439489    -6 negative 
##  8 ccss      1364021137370279938    -5 negative 
##  9 ccss      1364227817949437965    -4 negative 
## 10 ccss      1364230058647318530    -8 negative 
## # ℹ 43 more rows

afinn_ratio <- afinn_sentiment %>% 
  group_by(standards) %>% 
  count(sentiment) %>% 
  spread(sentiment, n) %>%
  mutate(ratio = negative/positive)

afinn_ratio

## # A tibble: 2 × 4
## # Groups:   standards [2]
##   standards negative positive ratio
##   <chr>        <int>    <int> <dbl>
## 1 ccss            16       16 1    
## 2 ngss             4       17 0.235

afinn_counts <- afinn_sentiment %>%
  group_by(standards) %>% 
  count(sentiment) %>%
  filter(standards == "ngss")

afinn_counts %>%
ggplot(aes(x="", y=n, fill=sentiment)) +
  geom_bar(width = .6, stat = "identity") +
  labs(title = "Next Gen Science Standards",
       subtitle = "Proportion of Positive & Negative Tweets") +
  coord_polar(theta = "y") +
  theme_void()

NGSS vs CCSS

summary_afinn2 <- sentiment_afinn %>% 
  group_by(standards) %>% 
  filter(value != 0) %>%
  mutate(sentiment = if_else(value < 0, "negative", "positive")) %>% 
  count(sentiment, sort = TRUE) %>% 
  mutate(method = "AFINN")

summary_bing2 <- sentiment_bing %>% 
  group_by(standards) %>% 
  count(sentiment, sort = TRUE) %>% 
  mutate(method = "bing")

summary_nrc2 <- sentiment_nrc %>% 
  filter(sentiment %in% c("positive", "negative")) %>%
  group_by(standards) %>% 
  count(sentiment, sort = TRUE) %>% 
  mutate(method = "nrc") 

summary_loughran2 <- sentiment_loughran %>% 
  filter(sentiment %in% c("positive", "negative")) %>%
  group_by(standards) %>% 
  count(sentiment, sort = TRUE) %>% 
  mutate(method = "loughran") 

summary_sentiment <- bind_rows(summary_afinn2,
                               summary_bing2,
                               summary_nrc2,
                               summary_loughran2) %>%
  arrange(method, standards) %>%
  relocate(method)

summary_sentiment

## # A tibble: 16 × 4
## # Groups:   standards [2]
##    method   standards sentiment     n
##    <chr>    <chr>     <chr>     <int>
##  1 AFINN    ccss      negative     37
##  2 AFINN    ccss      positive     31
##  3 AFINN    ngss      positive     32
##  4 AFINN    ngss      negative      6
##  5 bing     ccss      negative     43
##  6 bing     ccss      positive     36
##  7 bing     ngss      positive     27
##  8 bing     ngss      negative      5
##  9 loughran ccss      negative     17
## 10 loughran ccss      positive      6
## 11 loughran ngss      positive      4
## 12 loughran ngss      negative      3
## 13 nrc      ccss      positive    118
## 14 nrc      ccss      negative     34
## 15 nrc      ngss      positive     67
## 16 nrc      ngss      negative      4

total_counts <- summary_sentiment %>%
  group_by(method, standards) %>%
  summarise(total = sum(n))

## `summarise()` has grouped output by 'method'. You can override using the
## `.groups` argument.

sentiment_counts <- left_join(summary_sentiment, total_counts)

## Joining with `by = join_by(method, standards)`

sentiment_counts

## # A tibble: 16 × 5
## # Groups:   standards [2]
##    method   standards sentiment     n total
##    <chr>    <chr>     <chr>     <int> <int>
##  1 AFINN    ccss      negative     37    68
##  2 AFINN    ccss      positive     31    68
##  3 AFINN    ngss      positive     32    38
##  4 AFINN    ngss      negative      6    38
##  5 bing     ccss      negative     43    79
##  6 bing     ccss      positive     36    79
##  7 bing     ngss      positive     27    32
##  8 bing     ngss      negative      5    32
##  9 loughran ccss      negative     17    23
## 10 loughran ccss      positive      6    23
## 11 loughran ngss      positive      4     7
## 12 loughran ngss      negative      3     7
## 13 nrc      ccss      positive    118   152
## 14 nrc      ccss      negative     34   152
## 15 nrc      ngss      positive     67    71
## 16 nrc      ngss      negative      4    71

sentiment_percents <- sentiment_counts %>%
  mutate(percent = n/total * 100)

sentiment_percents

## # A tibble: 16 × 6
## # Groups:   standards [2]
##    method   standards sentiment     n total percent
##    <chr>    <chr>     <chr>     <int> <int>   <dbl>
##  1 AFINN    ccss      negative     37    68   54.4 
##  2 AFINN    ccss      positive     31    68   45.6 
##  3 AFINN    ngss      positive     32    38   84.2 
##  4 AFINN    ngss      negative      6    38   15.8 
##  5 bing     ccss      negative     43    79   54.4 
##  6 bing     ccss      positive     36    79   45.6 
##  7 bing     ngss      positive     27    32   84.4 
##  8 bing     ngss      negative      5    32   15.6 
##  9 loughran ccss      negative     17    23   73.9 
## 10 loughran ccss      positive      6    23   26.1 
## 11 loughran ngss      positive      4     7   57.1 
## 12 loughran ngss      negative      3     7   42.9 
## 13 nrc      ccss      positive    118   152   77.6 
## 14 nrc      ccss      negative     34   152   22.4 
## 15 nrc      ngss      positive     67    71   94.4 
## 16 nrc      ngss      negative      4    71    5.63

sentiment_percents %>%
  ggplot(aes(x = standards, y = percent, fill=sentiment)) +
  geom_bar(width = .8, stat = "identity") +
  facet_wrap(~method, ncol = 1) +
  coord_flip() +
  labs(title = "Public Sentiment on Twitter", 
       subtitle = "The Common Core & Next Gen Science Standards",
       x = "", 
       y = "Percentage of Words")

## Conclusion Overall, sentiment regarding educational reform was more positive than negative. However, there is a difference in the extent of negative and positive sentiment for each NGSS and CCSS. In the case of CCSS, Californians expressed more negative sentiment than positive sentiment. In contrast, they exhibited more positive sentiment than negative regarding NGSS.

Discussion

Even though educational reforms share a similar context, responses to them can vary. If negative sentiment is stronger than positive, such policies may not be successful.Given that the United States has various rules and policy implementations depending on states, it is meaningful to examine the sentiments California has. Studying the effects of CCSS should involve detailed research how stakeholders actually perceive and implement such policies in educational settings.