The above graph illustrates the measures of lexical density for each author. Auvinen (.35%) and Breivik’s (.28%) letters demonstrated the highest percentages for lexical density.
`stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
manifestos |>mutate(word_length =nchar(word)) |>ggplot(aes(word_length)) +facet_wrap(vars(author), scales ="free_y") +geom_histogram(binwidth =2) +labs(title ="Average Word Length of Serial Killer's Manifestos")
The above graph illustrates the average word length of each serial killer’s manifestos.
manifestos |>count(word, sort = T)
# A tibble: 3,863 × 2
word n
<chr> <int>
1 the 1189
2 of 724
3 and 617
4 to 603
5 a 455
6 i 392
7 in 354
8 that 284
9 my 269
10 is 238
# ℹ 3,853 more rows
# A tibble: 5,879 × 3
# Groups: author [6]
author word n
<chr> <chr> <int>
1 Breivik the 387
2 Rodger the 322
3 Rodger i 273
4 Unabomber the 245
5 Breivik of 225
6 Rodger my 208
7 Unabomber of 193
8 Rodger to 181
9 Rodger a 175
10 Rodger was 174
# ℹ 5,869 more rows
manifestos |>group_by(author) |>count(word, sort = T) |>top_n(5) |>ungroup() |>mutate(word =reorder(word, n)) |>ggplot(aes(word, n, fill = author)) +geom_col(show.legend =FALSE) +coord_flip() +facet_wrap(~author, scales ="free") +# creates separate graphs for each authorscale_fill_viridis_d() +# uses a nicer color schemetheme_minimal() +# removes the gray backgroundlabs(x =NULL, y ="Most common words")
Selecting by n
The above graph illustrates the most common words used by each serial killer in their manifestos.
stop_words
# A tibble: 1,149 × 2
word lexicon
<chr> <chr>
1 a SMART
2 a's SMART
3 able SMART
4 about SMART
5 above SMART
6 according SMART
7 accordingly SMART
8 across SMART
9 actually SMART
10 after SMART
# ℹ 1,139 more rows
# A tibble: 3,740 × 2
word n
<chr> <int>
1 will 102
2 life 68
3 school 67
4 one 66
5 people 66
6 time 65
7 like 58
8 society 57
9 can 53
10 black 51
# ℹ 3,730 more rows
# A tibble: 5,314 × 3
# Groups: author [6]
author word n
<chr> <chr> <int>
1 Rodger time 44
2 Breivik cultural 42
3 Breivik political 42
4 Rodger school 39
5 Breivik marxism 37
6 Breivik correctness 36
7 Unabomber society 36
8 Rodger life 34
9 Rodger first 33
10 Rodger house 31
# ℹ 5,304 more rows
manifestos |>anti_join(snowball) |>group_by(author) |>count(word, sort = T) |>top_n(5) |>ungroup() |>mutate(word =reorder(word, n)) |>ggplot(aes(word, n, fill = author)) +geom_col(show.legend =FALSE) +labs(x =NULL, y ="Most common words") +facet_wrap(vars(author), scales ="free") +scale_fill_viridis_d() +theme_minimal() +coord_flip()
Joining with `by = join_by(word)`
Selecting by n
The above graph illustrates the most common words without stop words used by each serial killer in their manifestos.
manifestos_word_counts <- manifestos_text |># This counts each word per authorunnest_tokens(word, text) |>count(author, word, sort =TRUE) total_words <- manifestos_word_counts |># This counts total words per authorgroup_by(author) |>summarize(total =sum(n))manifestos_word_counts <-left_join(manifestos_word_counts, total_words) # Joins the two
# A tibble: 5,879 × 7
author word n total tf idf tf_idf
<chr> <chr> <int> <int> <dbl> <dbl> <dbl>
1 Cho you 130 1971 0.0660 0.405 0.0267
2 Cho your 34 1971 0.0173 1.10 0.0190
3 Cho fuck 19 1971 0.00964 1.79 0.0173
4 Breivik cultural 42 5186 0.00810 1.79 0.0145
5 Breivik marxism 37 5186 0.00713 1.79 0.0128
6 Auvinen intelligent 10 1750 0.00571 1.79 0.0102
7 Auvinen minded 10 1750 0.00571 1.79 0.0102
8 Cho jesus 18 1971 0.00913 1.10 0.0100
9 Unabomber leftist 27 5100 0.00529 1.79 0.00949
10 Breivik frankfurt 27 5186 0.00521 1.79 0.00933
# ℹ 5,869 more rows
The above table illustrates how often a word appears in each author’s document, divided by how often it appears in other documents.
manifestos_tf_idf |>arrange(-tf_idf) |>mutate(word =factor(word, levels =rev(unique(word)))) |>group_by(author) |>top_n(5) |>ggplot(aes(word, tf_idf, fill = author)) +geom_col(show.legend =FALSE) +labs(x =NULL, y ="tf-idf") +facet_wrap(~author, scales ="free") +theme_minimal() +scale_fill_viridis_d() +labs(title ="Most distinctive words in each authors manifestos")
Selecting by tf_idf
coord_flip()
<ggproto object: Class CoordFlip, CoordCartesian, Coord, gg>
aspect: function
backtransform_range: function
clip: on
default: FALSE
distance: function
expand: TRUE
is_free: function
is_linear: function
labels: function
limits: list
modify_scales: function
range: function
render_axis_h: function
render_axis_v: function
render_bg: function
render_fg: function
setup_data: function
setup_layout: function
setup_panel_guides: function
setup_panel_params: function
setup_params: function
train_panel_guides: function
transform: function
super: <ggproto object: Class CoordFlip, CoordCartesian, Coord, gg>
The above graphs illustrates how often a word appears in each author’s document, divided by how often it appears in other documents.
# A tibble: 652 × 3
word sentiment n
<chr> <chr> <int>
1 like positive 58
2 good positive 26
3 modern positive 21
4 fuck negative 19
5 enough positive 18
6 work positive 18
7 great positive 16
8 weak negative 16
9 important positive 15
10 well positive 15
# ℹ 642 more rows
manifestos |>inner_join(bing) |>count(word, sentiment, sort =TRUE) |>group_by(sentiment) |>top_n(10) |>ungroup() |>mutate(word =reorder(word, n)) |>ggplot(aes(word, n, fill = sentiment)) +geom_col(show.legend =FALSE) +facet_wrap(vars(sentiment), scales ="free") +labs(y ="Manifestos: Words that contribute the most to each sentiment",x =NULL) +scale_fill_viridis_d() +coord_flip() +theme_minimal()
Joining with `by = join_by(word)`
Selecting by n
The above graph illustrates a sentiment analysis of each author’s manifestos. Specifically, which positive and negative words were used in each document.
manifestos_text |>unnest_tokens(bigram, text, token ="ngrams", n =2) |>select(bigram)
# A tibble: 21,806 × 1
bigram
<chr>
1 industrial society
2 society and
3 and its
4 its future
5 future introduction
6 introduction 1
7 1 the
8 the industrial
9 industrial revolution
10 revolution and
# ℹ 21,796 more rows
manifestos_text |>unnest_tokens(bigram, text, token ="ngrams", n =2) |>select(bigram) -> manifestos_bigrams
manifestos_bigrams |>count(bigram, sort = T)
# A tibble: 14,954 × 2
bigram n
<chr> <int>
1 of the 159
2 in the 81
3 to the 69
4 i was 60
5 and the 56
6 to be 43
7 political correctness 37
8 i had 34
9 it is 34
10 it was 30
# ℹ 14,944 more rows
# A tibble: 4,861 × 1
bigram
<chr>
1 industrial society
2 future introduction
3 introduction 1
4 industrial revolution
5 human race
6 greatly increased
7 life expectancy
8 advanced countries
9 destabilized society
10 made life
# ℹ 4,851 more rows
# A tibble: 4,115 × 2
bigram n
<chr> <int>
1 political correctness 37
2 frankfurt school 22
3 jesus christ 17
4 cultural marxism 16
5 black man 12
6 human race 12
7 modern leftism 9
8 power process 9
9 surrogate activities 9
10 will never 9
# ℹ 4,105 more rows
The above table illustrates words that appear together in each author’s manifestos. Further, it demonstrates the number of bigrams with stop words removed.
first_word <-c("the", "industrial") # these need to be lowercasemanifestos_bigrams |>count(bigram, sort = T) |>separate(bigram, c("word1", "word2"), sep =" ") |># separate the two wordsfilter(word1 %in% first_word) |># find first words from our listcount(word1, word2, wt = n, sort =TRUE)
# A tibble: 623 × 3
word1 word2 n
<chr> <chr> <int>
1 the world 24
2 the black 22
3 the first 22
4 the frankfurt 20
5 the time 17
6 the most 13
7 the system 13
8 the oversocialized 12
9 the leftist 11
10 the power 11
# ℹ 613 more rows
The above table demonstrates the words that follow “the,” and “industrial,” in each serial killer’s manifestos.
first_word <-c("the", "industrial") # these need to be lowercasemanifestos_bigrams |>count(bigram, sort = T) |>separate(bigram, c("word1", "word2"), sep =" ") |># separate the two wordsfilter(word1 %in% first_word) |># find first words from our listcount(word1, word2, wt = n, sort =TRUE) |>mutate(word2 =factor(word2, levels =rev(unique(word2)))) |># put the words in ordergroup_by(word1) |>top_n(5) |>ggplot(aes(word2, n, fill = word1)) +scale_fill_viridis_d() +# set the color palettegeom_col(show.legend =FALSE) +labs(x =NULL, y =NULL, title ="Word following:") +facet_wrap(~word1, scales ="free") +coord_flip() +theme_minimal()
Selecting by n
The above graph demonstrates the words that follow “the,” and “industrial,” in each serial killer’s manifestos.
