By TF-IDF Scores

# A tibble: 677,436 × 2
   title               bigram             
   <chr>               <chr>              
 1 Pride and Prejudice there is           
 2 Pride and Prejudice is an              
 3 Pride and Prejudice an illustrated     
 4 Pride and Prejudice illustrated edition
 5 Pride and Prejudice edition of         
 6 Pride and Prejudice of this            
 7 Pride and Prejudice this title         
 8 Pride and Prejudice title which        
 9 Pride and Prejudice which may          
10 Pride and Prejudice may viewed         
# … with 677,426 more rows

Before “love”

# A tibble: 167 × 4
   title               word1 word2     n
   <chr>               <chr> <chr> <int>
 1 Middlemarch         in    love     42
 2 Pride and Prejudice in    love     34
 3 Dracula             i     love     14
 4 Middlemarch         of    love     11
 5 Middlemarch         my    love      9
 6 Pride and Prejudice my    love      8
 7 Middlemarch         his   love      7
 8 Pride and Prejudice of    love      7
 9 Dracula             you   love      6
10 Middlemarch         to    love      6
# … with 157 more rows

After “love”

# A tibble: 173 × 4
   title               word1 word2     n
   <chr>               <chr> <chr> <int>
 1 Middlemarch         love  with     28
 2 Pride and Prejudice love  with     17
 3 Middlemarch         love  for      13
 4 Middlemarch         love  and      12
 5 Dracula             love  and      10
 6 Dracula             love  him       9
 7 Dracula             love  you       9
 8 Middlemarch         love  of        8
 9 Pride and Prejudice love  and       8
10 Middlemarch         love  me        7
# … with 163 more rows

Before “wedding”

# A tibble: 21 × 4
   title               word1    word2       n
   <chr>               <chr>    <chr>   <int>
 1 Pride and Prejudice the      wedding     6
 2 Middlemarch         a        wedding     5
 3 Middlemarch         her      wedding     4
 4 Middlemarch         the      wedding     4
 5 Cranford            her      wedding     2
 6 Dracula             my       wedding     2
 7 Dracula             our      wedding     2
 8 Pride and Prejudice her      wedding     2
 9 Pride and Prejudice sister's wedding     2
10 Cranford            mother's wedding     1
# … with 11 more rows

After “wedding”

# A tibble: 25 × 4
   title               word1   word2       n
   <chr>               <chr>   <chr>   <int>
 1 Middlemarch         wedding journey     9
 2 Pride and Prejudice wedding clothes     4
 3 Middlemarch         wedding clothes     3
 4 Middlemarch         wedding ring        2
 5 Pride and Prejudice wedding day         2
 6 Pride and Prejudice wedding took        2
 7 Cranford            wedding day         1
 8 Cranford            wedding ring        1
 9 Cranford            wedding with        1
10 Dracula             wedding be          1
# … with 15 more rows

Analysis

According to Bigram Count Distribution,the distribution of bigrams in these four books is relatively even with each other. And their range distribution is consistent with the previous radar chart to a very hitch degree. Here we can see that the writing style of all four authors is careful and the choice of words is in harmony with the overall text.

About word pairs Analysis

As you can see from the graph, the word love is preceded or followed most often by a preposition, followed by a pronoun. This indicates that writers have similar styles when expressing the emotion of love. The word “wedding” is mostly preceded by a qualifier, such as “the”. After the word wedding, most of the words are “clothes”, “journey”. This shows that the authors tend to describe the layout of the wedding. ### About Bigram Analysis To cite Wikipedia:

A bigram or digram is a sequence of two adjacent elements from a string of tokens, which are typically letters, syllables, or words. A bigram is an n-gram for n=2. The frequency distribution of every bigram in a string is commonly used for simple statistical analysis of text in many applications, including in computational linguistics, cryptography, speech recognition, and so on.

Bigram analyses focus on word pairs, including pair-associations and order of precedence. This can provide us with valuable information about the patterns and even thematic concerns of a text corpus. ### word pairs about “love”

The Four Books CORS

# A tibble: 7,281,902 × 3
   item1   item2   correlation
   <chr>   <chr>         <dbl>
 1 helsing van           0.976
 2 van     helsing       0.976
 3 adam    fitz          0.946
 4 fitz    adam          0.946
 5 brunoni signor        0.894
 6 signor  brunoni       0.894
 7 bourgh  de            0.796
 8 de      bourgh        0.796
 9 court   stone         0.763
10 stone   court         0.763
# … with 7,281,892 more rows

Pride and Prejudice Cors

# A tibble: 726,756 × 3
   item1     item2     correlation
   <chr>     <chr>           <dbl>
 1 bourgh    de              0.938
 2 de        bourgh          0.938
 3 william   sir             0.736
 4 sir       william         0.736
 5 pounds    thousand        0.700
 6 thousand  pounds          0.700
 7 catherine lady            0.678
 8 lady      catherine       0.678
 9 forster   colonel         0.591
10 colonel   forster         0.591
# … with 726,746 more rows

###Top 150 Word Correlations in Pride and Prejudice

Analysis

As shown in the figure, the most frequent pairings in these four books are about the names or titles of people. This indicates that the authors centered their creations around the characters. And in Mystery and Prejudice, in addition to the names of people, the correelation of a thousand pounds appears most often.It also fits in well with the theme of the book, as it is a story about two young people falling in love across class, where the differences in wealth levels that exist cannot be ignored.

About Word Correlations

Word Pairs vs. Word Corrs: Word Pairs consider adjacent words: term1, term2. This has a linear order: the way we naturally read a text. For some examples from Nathaniel Hawthorne’s The Scarlet Letter, designating term2 as “child”: “elf child”, “strange child”, “naughty child”, and “poor child”. In contrast, Word Correlations consider document sections {6}. If termX appears in a given section, what other terms are likely to appear in that same section? This can be anywhere in the section: any place before or after termX, not just adjacent.

Word Corr Findings : So a Word Correlation analysis tells us what words are associated with termX, and the strength of those associations {6}. These word clusters, centered on termX, reveal linkages of language and thought that might otherwise escape our attention since we typically read in a linear fashion. In the case of an individual author, they help show that author’s linguistic habits – the unconscious as well as conscious mind at work. For a corpus of different authors, they can help reveal underlying assumptions – assumptions perhaps even unknown to the authors being studied!

Word Correlation analysis provides insight as the likelihood of one term occurring shortly before or after another term. This shows us word associations which provides insight to the themes and concerns of the text corpus author or authors. We can also use this track changing word associations over time, and so see stylistic differences on the micro level or historical changes in sensibility on the macro level.

WP Network Graph

A network graph can reveal the structure of relationships among the word pairs, and not just a simple count of frequency or even tf-idf value. It potentially reveals connections and linkages that we might otherwise miss if the word pairs were plotted in a bar graph, or listed in a table, or if the text were read in a natural linear fashion.

Although relational, the Word Pairs network graph is also directional: it proceeds from term 1 to term 2, as indicated by the line and arrow. Indeed, as Silge & Robinson point out: the Word Pairs network graph is also a “visualization of a Markov chain, a common model in text processing. In a Markov chain, each choice of word depends only on the previous word” {8}.

In brief, the Word Pairs network graph provides us with an overview of key word-pairs that also indicates their relationships within the the document (or documents).

(See also Silge & Robinson, “Relationships between words: n-grams and correlations”).

WC Network Graph

Word Correlation analyses discover which terms co-occur with each other, as per a specified section of a document, and the importance (statistical significance) of that co-occurrence as measured by the correlation value {9}.

A network graph of word correlations displays linkages and associations that are not as easily captured in a bar plot or data table, and are typically missed when reading the text in a natural linear fashion. Unlike a Word Pairs network graph, a Word Correlations network graph is not directional: rather, it is cluster-centered depending on the strength of correlation.

In brief, the Word Correlations network graph reveals valuable term clusters (empirically determined word associations) which provide information about the deeper layers of language-usage and thought in the document (or documents).

(See also Silge & Robinson, “Relationships between words: n-grams and correlations”).

Source: CLA 3206A TM Guide. TJ Haslam, CC-BY-4.0, 2020. In-text citations by URL link.

He-She terms in four Books

after_he_she_plot

analysis

The portrayal of the two genders in these four books is very different. When describing he, the writers use the word interested the most. When describing women, the writers use the word recoverd the most. This may reveal the writers’ perceptions of gender. It also reveals differences in how the two genders should be represented.

About gender analysis

This approach to gender analysis was inspired by blog post | study by Julia Silge, author of textbook co-author and one of the creators of Tidytext. We freely use the code she provided – thank you, Julia!

Silge in turn credits the break-through academic study by Professors Matthew Jockers and Gabi Kirilloff, “Understanding Gender and Character Agency in the 19th Century Novel”, Journal of Cultural Analytics (2017).

Approaching the topic of “character identity [as realized through] character action”, Jockers and Kirilloff (2017) examined “character agency in the context of character gender” by examining “trends in behavior associated with male and female characters” in their macro-analytic study .

We borrow their insights also.

By seeing what words – and particularly verbs – follow “he” or “she”, we can gain insight into how these texts portray gender and hence gender roles: “character agency in the context of character gender”, as Jockers and Kirilloff (2017) put it.