Post 5: STM: Further Analysis, Interpretations and Clusters
Isha Akshita Mahajan
In the previous I ran a a structural topic model with K = 6. This was deciphered by using the searchK() function and looking at the coherence values for statistical fit. For meta data, I used organization so I can decipher which words occur in each publication. As seen in in the previous visualization, topic 2 seemed to be the most prevalent in across all documents. The words that were common in this topic were prison, torture, death, arrests, sentences, charges. From an initial glace, this looks like a topic that would cover articles that include writings on prisoner arrests charges or perhaps prisoners of war. The other topic that seemed distinct was looked like topic 6 where women, children and civilians have emphasis in the topic.
To move in this a step further, I use some clustering to decipher which topics are similar in nature.
#begin by taking out correlations in the model
corr <- topicCorr(model)library(factoextra)## Welcome! Want to learn more? See two factoextra-related books at https://goo.gl/ve3WBahc <- eclust(scale(corr$cor), "hclust", nboot = 500, k = 3)## Warning: `guides(<scale> = FALSE)` is deprecated. Please use `guides(<scale> =
## "none")` instead.fviz_dend(hc, rect = TRUE)## Warning: `guides(<scale> = FALSE)` is deprecated. Please use `guides(<scale> =
## "none")` instead.
Given that our model is run on only 6 topics, I thought that 3 clusters might be a good point to explore hierarchical cluster dendogram. From the dendogram, we see that topics 4,1,3 are in a cluster together, 5 in another, and 2 and 6 in the last.
Here , I plot the clusters in wordlouds to better understand what the features of eacch topic are.
Cluster 1 - topic 4,1,3
cloud(model, topic = 4)## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : organ could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : without could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : public could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : newspap could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : journalist could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : offic could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : polit could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : ngos could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : protest could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : number could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : media could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : person could not be fit on page. It will not be plotted.
cloud(model, topic = 1)## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : women could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : worker could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : howev could not be fit on page. It will not be plotted.
cloud(model, topic = 3)## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : provid could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : case could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : person could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : general could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : approxim could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : worker could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : independ could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : without could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : discrimin could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : individu could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : freedom could not be fit on page. It will not be plotted.
Cluster 2 - topic 5
cloud(model, topic = 5)## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : public could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : european could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : parliament could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : septemb could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : administr could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : octob could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : time could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : ethnic could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : reform could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : region could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : minor could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : access could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : increas could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : crime could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : foreign could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : critic could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : defend could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : investig could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : polit could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : month could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : detent could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : monitor could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : abus could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : return could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : group could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : april could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : organ could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : presid could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : made could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : develop could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : attack could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : mani could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : support could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : protect could not be fit on page. It will not be plotted.## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : case could not be fit on page. It will not be plotted.
Cluster 3 - topic 2 and 6
cloud(model, topic= 2)## Warning in wordcloud::wordcloud(words = vocab, freq = vec, max.words =
## max.words, : tortur could not be fit on page. It will not be plotted.
cloud(model, topic = 6)