R Notebook

Kaitlin Kavlie PSYC-541

Lab #9: Text Analysis of Tweets

1. I extracted 5,000 tweets from CNN’s Twitter, unnested the words of the tweets, removed stop words and weird web related terms, and created a table and a word cloud of the top words.

I extracted the tweets using this first code.

info_tweets <- get_timeline("cnn", n = 5000)

Then I unnested the words of the tweets with this code below.

info_words <- info_tweets %>% 
  unnest_tokens(word, text) %>% 
  select(screen_name, word)

Using the code chunk below I removed stop words and weird words, as well as created a table of the top words.

info_words %>% 
  anti_join(stop_words) %>% 
  count(word, sort = T) %>%
  filter(!word == "https") %>%
  filter(!word == "t.co")

Joining, by = "word"

With this last code chunk I created a word cloud of the top words.

info_words %>% 
  anti_join(stop_words) %>% 
  count(word, sort = T) %>%
  filter(!word == "https") %>%
  filter(!word == "t.co") %>%
  top_n(100) %>%
  wordcloud2(size = .5)

Joining, by = "word"
Selecting by n

2. I conducted a sentiment analysis using bing, removed multiple errors, and created a graph of the words that contribute the most to each sentiment.

I ran the sentiment analysis with bing by using the first code below.

bing <- get_sentiments("bing")
bing

Then I removed multiple word errors with the following code.

info_words %>% 
  inner_join(bing) %>% 
  count(word, sentiment, sort = TRUE) %>%
  filter(!word == "trump") %>%
  filter(!word == "like") %>%
  filter(!word == "top")

Joining, by = "word"

Using this last code I created a graph of the words that contribute the most to each sentiment.

info_words %>% 
  inner_join(bing) %>% 
  count(word, sentiment, sort = TRUE) %>%
  filter(!word == "trump") %>%
  filter(!word == "like") %>%
  filter(!word == "top") %>%
  group_by(sentiment) %>%
  top_n(10) %>%
  ungroup() %>%
  mutate(word = reorder(word, n)) %>%
  ggplot(aes(word, n, fill = sentiment)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(vars(sentiment), scales = "free") +
  labs(y = "News headlines: Words that contribute the most to each sentiment",
       x = NULL) +
  coord_flip() +
  theme_minimal()

Joining, by = "word"
Selecting by n

3. I unnested the tweets as bigrams, removed stop words and errors, and created a table and word cloud of the most common bigrams.

This first code chunk was used to unnest the tweets as bigrams.

info_tweets %>%
  select(text) %>%                                                 
  unnest_tokens(words, text, token = "ngrams", n = 2) %>%
  count(words, sort = T)

This next code filtered out stop words.

info_tweets %>%
  select(text) %>%                                                 
  unnest_tokens(words, text, token = "ngrams", n = 2) %>% 
  separate(words, c("word1", "word2"), sep = " ") %>%          
  filter(!word1 %in% stop_words$word) %>%                      
  filter(!word2 %in% stop_words$word) %>% 
  filter(!word1 %in% remove_words) %>%                         
  filter(!word2 %in% remove_words) %>%
  unite(words, word1, word2, sep = " ") 

Then this code was used to filter out web terms.

remove_words = c("https", "t.co")

info_tweets %>%
  select(text) %>%                                                 
  unnest_tokens(words, text, token = "ngrams", n = 2) %>% 
  separate(words, c("word1", "word2"), sep = " ") %>%          
  filter(!word1 %in% stop_words$word) %>%                      
  filter(!word2 %in% stop_words$word) %>%                        
  filter(!word1 %in% remove_words) %>%                         
  filter(!word2 %in% remove_words) %>%                         
  unite(words, word1, word2, sep = " ") -> info_bigrams                      

This code created a table of the most common bigrams.

info_bigrams %>%
  count(words, sort = T)

Then this code was used to create a word cloud of the most common bigrams.

info_bigrams %>%
  count(words, sort = T) %>%
  top_n(100) %>%
  wordcloud2(size = .5)

Selecting by n

4. Above in question 3 I created bigrams of the tweets, removed the stopwords, and created a table and word cloud of the most common bigrams. I believe this question 4 is a repeat of question 3.

5. I used the bigram method and found the most common words that come after ‘ukraine’ and ‘russia’.

firstinfo_word <- c("ukraine", "russia")                                  

info_bigrams %>%             
  count(words, sort = TRUE) %>%
  separate(words, c("word1", "word2"), sep = " ") %>%     
  filter(word1 %in% firstinfo_word) %>%                          
  count(word1, word2, wt = n, sort = TRUE)

After finding the most common words that come after ‘ukraine’ and ‘russia’, I created bar graph displaying the results for each word.

firstinfo_word <- c("ukraine", "russia")                                  

info_bigrams %>%             
  count(words, sort = TRUE) %>%
  separate(words, c("word1", "word2"), sep = " ") %>%       
  filter(word1 %in% firstinfo_word) %>%                          
  count(word1, word2, wt = n, sort = TRUE) %>%
  mutate(word2 = factor(word2, levels = rev(unique(word2)))) %>%     
  group_by(word1) %>% 
  top_n(5) %>% 
  ggplot(aes(word2, n, fill = word1)) +                          
  scale_fill_viridis_d() +                                           
  geom_col(show.legend = FALSE) +
  labs(x = NULL, y = NULL, title = "Word following:") +
  facet_wrap(~word1, scales = "free") +
  coord_flip()

Selecting by n