WordCloud and Quick Analysis: Donald Trumps tweets

Today, the New York Times released their compiled list of Donald Trump’s Twitter Feed, compiled by person the tweet was directed against. Since we have already looked at acceptance speeches here, we thought it might be worthwhile to take 2 minutes of analysis* against this particular dataset. This quicklook only considers his tweets against his opponent, Secretary Hillary Clinton.

There are really three parts to doing text analysis; they are:

1. Getting the data
2. Putting the data in a usable format
3. Making the picture you want to show.

In this very quick analysis, putting the data in a usable format was the challenge. Step by step:

1. Getting the data. I used the tried-and-true method of ctrl+A and ctrl+C against the subject Website

2. Putting it in usable format. This was a dance that involved doing analysis in R, some formatting in notepad, and… MS Word (!?!?). That’s right - the raw data had a lot of " marks, which were easily converted to whitespace in Word by using the ‘search and replace’ function. This isn’t elegant, but it was fast and effective.

3. Making the picture. This was straightforward, and used code that was previously developed for the Acceptance Speeches article, with some minor tweaks.

After uploading the text document using the read_delim function from the readr package, we proceed as follows. Note the copious use of the %<>% operator from the magrittr package. This really streamlines the code where you perform a number of transforms against a fixed dataset.

library(tm); library(SnowballC); library(wordcloud)
library(magrittr)
TC = Corpus(VectorSource(Trump))
TC %<>% tm_map( PlainTextDocument)
TC %<>% tm_map(removePunctuation)
TC %<>% tm_map(removeWords, stopwords('english'))
TC %<>% tm_map(stemDocument)
dtm = TermDocumentMatrix(TC)
m = as.matrix(dtm)
v = sort(rowSums(m), decreasing = TRUE)
d = data.frame(word = names(v), freq = v)
wordcloud(words = d$word, freq = d$freq, min.freq = 3, random.order = FALSE, rot.per = .35, colors = brewer.pal(8, "Dark2")) 

Figure 1: Donald Trump’s Tweets targeting Hillary Clinton. For some reason, they look better in pastel

Having the Document Term Matrix (DTM) allows for other interesting things to be quickly performed as well. For example, the word ‘crook’ is featured prominently in the center of the word cloud. We can quickly examine measures of interest, such as the proportion of times the word ‘Crook’ is used:

 (d$freq[which(d$word == "crook")]/sum(d$freq)) %>% multiply_by(100) %>%  format(digits = 0) %>% paste("%") %>% print() 

[1] "23 %"

Wow. Fully 23% of the non stopwords in this sequence of tweets are ‘Crook’.

In fact, his use of the word “Crook” dwarfs all other words in his tweets, as shown:

barplot(d$freq[1:5], main = "Five most common words", xaxt = "n")
axis(1, labels = d$word[1:5], at = 1:5)

Figure 2: Barplot of the 5 most common words in Donald Trump’s Tweets about Secretary Clinton. Note that ‘presid’ is the stem for “President”, “Presidency” and “Presidential”

*why 2 minutes? Because we used a lot of code from the last Five Minute Analyst.