Lab 12
2026-11-12
install.packages(“tidytext”) install.packages(“tidyverse”) install.packages(“janeaustenr”)
library(tidytext)
library(tidyverse)## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.2.1 ✔ readr 2.2.0
## ✔ forcats 1.0.1 ✔ stringr 1.6.0
## ✔ ggplot2 4.0.3 ✔ tibble 3.3.1
## ✔ lubridate 1.9.5 ✔ tidyr 1.3.2
## ✔ purrr 1.2.2
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errorslibrary(janeaustenr)text <- c(
"Because I could not stop for Death -",
"He kindly stopped for me -",
"The Carriage held but just Ourselves -",
"and Immortality"
)
text## [1] "Because I could not stop for Death -"
## [2] "He kindly stopped for me -"
## [3] "The Carriage held but just Ourselves -"
## [4] "and Immortality"library(dplyr)
text_df <- tibble(
line = 1:4,
text = text
)
text_df## # A tibble: 4 × 2
## line text
## <int> <chr>
## 1 1 Because I could not stop for Death -
## 2 2 He kindly stopped for me -
## 3 3 The Carriage held but just Ourselves -
## 4 4 and Immortalitylibrary(tidytext)
tidy_poem <- text_df %>%
unnest_tokens(word, text)
tidy_poem## # A tibble: 20 × 2
## line word
## <int> <chr>
## 1 1 because
## 2 1 i
## 3 1 could
## 4 1 not
## 5 1 stop
## 6 1 for
## 7 1 death
## 8 2 he
## 9 2 kindly
## 10 2 stopped
## 11 2 for
## 12 2 me
## 13 3 the
## 14 3 carriage
## 15 3 held
## 16 3 but
## 17 3 just
## 18 3 ourselves
## 19 4 and
## 20 4 immortalityIn this section, I learned how to use the unnest_tokens() function to separate text into individual words. The original poem had four lines, but after tokenization, each word was placed into its own row. This makes it easier to organize and analyze the text.
1.3 Tidying the Works of Jane Austen
library(janeaustenr)
library(dplyr)
library(stringr)
original_books <- austen_books() %>%
group_by(book) %>%
mutate(
linenumber = row_number(),
chapter = cumsum(
str_detect(
text,
regex("^chapter [\\divxlc]",
ignore_case = TRUE)
)
)
) %>%
ungroup()
original_books## # A tibble: 73,422 × 4
## text book linenumber chapter
## <chr> <fct> <int> <int>
## 1 "SENSE AND SENSIBILITY" Sense & Sensibility 1 0
## 2 "" Sense & Sensibility 2 0
## 3 "by Jane Austen" Sense & Sensibility 3 0
## 4 "" Sense & Sensibility 4 0
## 5 "(1811)" Sense & Sensibility 5 0
## 6 "" Sense & Sensibility 6 0
## 7 "" Sense & Sensibility 7 0
## 8 "" Sense & Sensibility 8 0
## 9 "" Sense & Sensibility 9 0
## 10 "CHAPTER 1" Sense & Sensibility 10 1
## # ℹ 73,412 more rowstidy_books <- original_books %>%
unnest_tokens(word, text)
tidy_books## # A tibble: 725,064 × 4
## book linenumber chapter word
## <fct> <int> <int> <chr>
## 1 Sense & Sensibility 1 0 sense
## 2 Sense & Sensibility 1 0 and
## 3 Sense & Sensibility 1 0 sensibility
## 4 Sense & Sensibility 3 0 by
## 5 Sense & Sensibility 3 0 jane
## 6 Sense & Sensibility 3 0 austen
## 7 Sense & Sensibility 5 0 1811
## 8 Sense & Sensibility 10 1 chapter
## 9 Sense & Sensibility 10 1 1
## 10 Sense & Sensibility 13 1 the
## # ℹ 725,054 more rowsdata(stop_words)
tidy_books <- tidy_books %>%
anti_join(stop_words, by = "word")
tidy_books## # A tibble: 217,602 × 4
## book linenumber chapter word
## <fct> <int> <int> <chr>
## 1 Sense & Sensibility 1 0 sense
## 2 Sense & Sensibility 1 0 sensibility
## 3 Sense & Sensibility 3 0 jane
## 4 Sense & Sensibility 3 0 austen
## 5 Sense & Sensibility 5 0 1811
## 6 Sense & Sensibility 10 1 chapter
## 7 Sense & Sensibility 10 1 1
## 8 Sense & Sensibility 13 1 family
## 9 Sense & Sensibility 13 1 dashwood
## 10 Sense & Sensibility 13 1 settled
## # ℹ 217,592 more rowsword_counts <- tidy_books %>%
count(word, sort = TRUE)
head(word_counts, 10)## # A tibble: 10 × 2
## word n
## <chr> <int>
## 1 miss 1855
## 2 time 1337
## 3 fanny 862
## 4 dear 822
## 5 lady 817
## 6 sir 806
## 7 day 797
## 8 emma 787
## 9 sister 727
## 10 house 699library(ggplot2)
tidy_books %>%
count(word, sort = TRUE) %>%
filter(n > 600) %>%
mutate(word = reorder(word, n)) %>%
ggplot(aes(n, word)) +
geom_col(fill = "steelblue") +
labs(
title = "Most Common Words in Jane Austen's Novels",
x = "Word Frequency",
y = "Words"
) +
theme_minimal()
After analyzing Jane Austen’s novels, I noticed that words such as “miss,” “time,” “dear,” and “lady” appeared frequently. I think this shows how word frequency can help identify common themes and patterns within text. This is also useful in marketing because companies can analyze customer reviews or social media posts to see which words people use most often. For example, if customers frequently mention words like “price,” “quality,” or “service,” businesses can use that information to better understand what matters to their customers. Overall, this exercise helped me understand how text mining can turn written information into data that businesses can use to make decisions.