APratt tweets Assignment

This is an R Markdown Notebook.

Assignment:

1. Unnest the words of the tweets, remove stop words and weird web “words”, and create a table and a word cloud of the top words.
   
2. Conduct a sentiment analysis using bing, remove any errors like trump = positive, and create a graph of the words that contribute most to each sentiment.
   
3. Do the same as above but with the nrc sentiment lexicon.
   
4. Unnest the tweets as bigrams, remove stop words and errors, and create a table and word cloud of the most common bigrams.
   
5. Using the bigrams, look for the most common words that follow two different words. You may choose trump and pelosi, or choose your own.

usatoday_tweets <- get_timeline("usatoday", n =5000)

usatoday_words <- usatoday_tweets %>% 
  unnest_tokens(word, text) %>% 
  select(screen_name, word)

usatoday_words %>% 
  anti_join(get_stopwords()) %>% 
  filter(!word == "https") %>% 
  filter(!word == "t.co") %>% 
  count(word, sort = T)

Joining, by = "word"

God, this is going to be sad. Okay. Top words are corona-related. No surprise.

usatoday_words %>% 
  anti_join(get_stopwords()) %>% 
  filter(!word == "https") %>% 
  filter(!word == "t.co") %>% 
  count(word, sort = T)%>% 
  top_n(200) %>% 
  wordcloud2(size = .6)

Joining, by = "word"
Selecting by n

Cardi B Voice: CAH-ROHNA VI-RUSS shit is real shit is getting real.

bing_words <- get_sentiments("bing")

usatoday_words %>% 
  inner_join(bing_words) %>% 
  filter(!word == "trump") %>% 
  count(word, sentiment, sort = TRUE) %>% 
  group_by(sentiment) %>% 
  top_n(10) %>% 
  ungroup() %>% 
  mutate(word = reorder(word, n)) %>% 
  ggplot(aes(word, n, fill = sentiment)) +
  geom_col(show.legend = FALSE) + 
  facet_wrap(vars(sentiment), scales = "free") +
  labs(y = "Bing Sentiment Contribution by Word (USA Today Tweets)",
       x = NULL) +
  coord_flip() +
  theme_minimal()

Joining, by = "word"
Selecting by n

I was right: this is depressing. All the positive words are like an overly cheerful person telling you to just “stop being sad,” and the negative words are all about dying. Great.

nrc_words <- get_sentiments("nrc")

usatoday_words %>% 
  inner_join(nrc_words) %>% 
  filter(!word == "trump") %>% 
  count(word, sentiment, sort = TRUE) %>% 
  group_by(sentiment) %>% 
  top_n(5) %>% 
  ungroup() %>% 
  mutate(word = reorder(word, n)) %>% 
  ggplot(aes(word, n, fill = sentiment)) +
  geom_col(show.legend = FALSE) + 
  facet_wrap(vars(sentiment), scales = "free") +
  labs(y = "NRC Sentiment Contribution by Word (USA Today Tweets)",
       x = NULL) +
  coord_flip() +
  theme_minimal()

Joining, by = "word"
Selecting by n

We’re all so scared. “Pandemic” shows up at the top for negative, sadness, and fear.

remove_words = c("https", "t.co")

usatoday_tweets %>% 
  select(text) %>% 
  unnest_tokens(words, text, token = "ngrams", n = 2) %>% 
  separate(words, c("word1", "word2"), sep = " ") %>% 
  filter(!word1 %in% stop_words$word) %>% 
  filter(!word2 %in% stop_words$word) %>% 
  filter(!word1 %in% remove_words) %>% 
  filter(!word2 %in% remove_words) %>% 
  unite(words, word1, word2, sep = " ") -> usatoday_bigrams

usatoday_bigrams %>% 
  count(words, sort = T)

This would be the ideal project to see the spread of tweets/interaction regarding the presidential candidates right before election, but now the world is ending and all we want to talk about is COVID.

usatoday_bigrams %>% 
  count(words, sort = T) %>% 
  top_n(100) %>% 
  wordcloud2(size = .5)

Selecting by n

See? Even with the limited coverage surrounding the 2020 presidential bid, it’s still cool to see how it could have been.

first_words <- c("biden", "sanders")

usatoday_bigrams %>% 
  count(words, sort = TRUE) %>% 
  separate(words, c("word1", "word2"), sep = " ") %>% 
  filter(word1 %in% first_words) %>% 
  count(word1, word2, wt = n, sort = TRUE) %>% 
  rename(total = nn)

Let’s do this one time with DNC candidates. Fairly positive.

first_words <- c("corona", "coronavirus")

usatoday_bigrams %>% 
  count(words, sort = TRUE) %>% 
  separate(words, c("word1", "word2"), sep = " ") %>% 
  filter(word1 %in% first_words) %>% 
  count(word1, word2, wt = n, sort = TRUE) %>% 
  rename(total = nn)

Oh, look. We’re sad again.