Text Analysis

library(tidyverse)
library(tidytext)
library(wordcloud2)

marina <- read_csv("matd.txt")

Warning: One or more parsing issues, see `problems()` for details

Rows: 658 Columns: 1
-- Column specification --------------------------------------------------------------------------------------------------------------
Delimiter: ","
chr (1): lyrics

i Use `spec()` to retrieve the full column specification for this data.
i Specify the column types or set `show_col_types = FALSE` to quiet this message.

1. Here I chose the album ’Froot” by Marina and the diamonds and unnested the lyrics.

marina_words <- marina %>%
  unnest_tokens(word, lyrics) %>% 
  select(word)

marina_words

There are 10 songs each of over 3 minutes long on the album so there are lots of words here.

2. Here I removed the stopwords, and then created a table and word cloud with the words counts.

marina_words %>% 
  anti_join(stop_words) %>% 
  count(word, sort = T) %>%
  top_n(100) %>%
  wordcloud2(size = .5)

Joining, by = "word"
Selecting by n

“Yeah” is used the most at 56 times.

3. Here is a sentiment analysis using bing and a graph of the words that contribute most to each sentiment.

bing <- get_sentiments("bing")
bing

marina_words %>% 
  inner_join(bing) %>% 
  count(word, sentiment, sort = TRUE) %>%
  group_by(sentiment) %>%
  top_n(10) %>%
  ungroup() %>%
  mutate(word = reorder(word, n)) %>%
  ggplot(aes(word, n, fill = sentiment)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(vars(sentiment), scales = "free") +
  labs(y = "Marina's Froot album: Words that contribute the most to each sentiment ",
       x = NULL) +
  scale_fill_viridis_d() +
  coord_flip() +
  theme_minimal()

Joining, by = "word"
Selecting by n

By the scales at the bottom you can see that there are many more positive words than negative words.

4. Here is a table and word cloud of the most common bigrams.

marina_bigrams %>%
    separate(bigram, c("word1", "word2"), sep = " ") %>% 
  filter(!word1 %in% stop_words$word) %>%
  filter(!word2 %in% stop_words$word) %>% 
  unite(bigram, word1, word2, sep = " ") %>%
  count(bigram, sort = T)

NA

marina_bigrams %>% 
  separate(bigram, c("word1", "word2"), sep = " ") %>% 
  filter(!word1 %in% stop_words$word) %>%
  filter(!word2 %in% stop_words$word) %>% 
  unite(bigram, word1, word2, sep = " ") %>%
  count(bigram, sort =T) %>%
  filter(n > 1) %>% 
  wordcloud2(size = .5)

I find it interesting that there are many bigrams used 6 times. Also “NA NA” is a funny bigram.

5. Here I used the bigram method to find the most common words that come after the words I and you

first_word <- c("i", "you")                                  

marina_bigrams %>% 
  count(bigram, sort = T) %>% 
  separate(bigram, c("word1", "word2"), sep = " ") %>%       
  filter(word1 %in% first_word) %>%                          
  count(word1, word2, wt = n, sort = TRUE) %>%
  mutate(word2 = factor(word2, levels = rev(unique(word2)))) %>%     
  group_by(word1) %>% 
  top_n(5) %>% 
  ggplot(aes(word2, n, fill = word1)) +                          
  scale_fill_viridis_d() +                                        
  geom_col(show.legend = FALSE) +
  labs(x = NULL, y = NULL, title = "Word following:") +
  facet_wrap(~word1, scales = "free") +
  coord_flip() +
  theme_minimal()

Selecting by n

I find it interesting both can and can’t follow “you”.