Homework 4: Word Clouds and TF-IDF
Sheyla Perez
3/4/2026
The Text Data
I collected comp reviews for three Cold War historical fiction novels: A Shadow in Moscow by Katherine Reay, The Secrets We Kept by Lara Prescott, and Our Woman in Moscow by Beatriz Williams. Reviews include both positive (5-star) and negative/less positive (1-2 star) ratings from Goodreads.
Let’s take a glance at the dataset.
reviews <- read.xlsx("Comps Data Collection Part 2 .xlsx", sheet=1)
reviews[1:5,]## Author Title Series
## 1 Katherine Reay A Shadow in Moscow Standalone
## 2 Katherine Reay A Shadow in Moscow Standalone
## 3 Katherine Reay A Shadow in Moscow Standalone
## 4 Katherine Reay A Shadow in Moscow Standalone
## 5 Katherine Reay A Shadow in Moscow Standalone
## Link
## 1 https://www.goodreads.com/book/show/63340978-a-shadow-in-moscow
## 2 https://www.goodreads.com/book/show/63340978-a-shadow-in-moscow
## 3 https://www.goodreads.com/book/show/63340978-a-shadow-in-moscow
## 4 https://www.goodreads.com/book/show/63340978-a-shadow-in-moscow
## 5 https://www.goodreads.com/book/show/63340978-a-shadow-in-moscow
## Description
## 1 Two women spies operating in Moscow during different Cold War eras. Ingrid Bauer, an Austrian who marries into the Soviet system in 1954, becomes MI6's best Soviet agent. Anya Kadinova, a Georgetown student in 1980, returns to Moscow and becomes the CIA's newest recruit. Both women risk everything to fight against the oppressive Soviet regime.
## 2 Two women spies operating in Moscow during different Cold War eras. Ingrid Bauer, an Austrian who marries into the Soviet system in 1954, becomes MI6's best Soviet agent. Anya Kadinova, a Georgetown student in 1980, returns to Moscow and becomes the CIA's newest recruit. Both women risk everything to fight against the oppressive Soviet regime.
## 3 Two women spies operating in Moscow during different Cold War eras. Ingrid Bauer, an Austrian who marries into the Soviet system in 1954, becomes MI6's best Soviet agent. Anya Kadinova, a Georgetown student in 1980, returns to Moscow and becomes the CIA's newest recruit. Both women risk everything to fight against the oppressive Soviet regime.
## 4 Two women spies operating in Moscow during different Cold War eras. Ingrid Bauer, an Austrian who marries into the Soviet system in 1954, becomes MI6's best Soviet agent. Anya Kadinova, a Georgetown student in 1980, returns to Moscow and becomes the CIA's newest recruit. Both women risk everything to fight against the oppressive Soviet regime.
## 5 Two women spies operating in Moscow during different Cold War eras. Ingrid Bauer, an Austrian who marries into the Soviet system in 1954, becomes MI6's best Soviet agent. Anya Kadinova, a Georgetown student in 1980, returns to Moscow and becomes the CIA's newest recruit. Both women risk everything to fight against the oppressive Soviet regime.
## Average_Stars N_Reviews Price
## 1 4.32 876 16.99
## 2 4.32 876 16.99
## 3 4.32 876 16.99
## 4 4.32 876 16.99
## 5 4.32 876 16.99
## Genre
## 1 Historical Fiction, \nFiction, \nRussia, \nHistorical, Mystery, \nEspionage, \nSuspense
## 2 Historical Fiction, \nFiction, \nRussia, \nHistorical, Mystery, \nEspionage, \nSuspense
## 3 Historical Fiction, \nFiction, \nRussia, \nHistorical, Mystery, \nEspionage, \nSuspense
## 4 Historical Fiction, \nFiction, \nRussia, \nHistorical, Mystery, \nEspionage, \nSuspense
## 5 Historical Fiction, \nFiction, \nRussia, \nHistorical, Mystery, \nEspionage, \nSuspense
## Review_stars
## 1 5/5 stars
## 2 5/5 stars
## 3 5/5 stars
## 4 5/5 stars
## 5 5/5 stars
## Headline
## 1 Beautifully Written and Seamlessly Told”
## 2 A New Favorite: A Shadow in Moscow Might Be Katherine Reay’s Best Yet!
## 3 Secrets, Sacrifice, and a Shocking Connection
## 4 A Cold War Masterpiece That Deserves a Place in Every School
## 5 A Slow Burn That Builds Into an Unforgettable Finish
## Review
## 1 I read this book before I listened to the audio version, and let me just say that the two narrators nailed it!! Beautifully done.\nAnd now for the story...\nBeautiful prose, and wonderfully written. This book carried me back and forth from World War II at the end of the war into Moscow during the Cold War with the Soviet Union.. The transitions from the timelines were seamless, and I turned the pages effortlessly until the very end. In fact, I had a book hangover for days. I have picked a Shadow In Moscow as my next book club pick. If you are looking for a historical book about the Cold War, spies, espionage, and romance, then this book is for you.\nThis book is probably one of the best books I have read this year. I truly enjoyed it.\n*I was given a copy of this book by Harper Muse, and this is my honest opinion.
## 2 WHOA. I loooooved this book! I've been a Katherine Reay fan since her debut novel, Dear Mr. Knightley, and probably would've called that one my longstanding favorite of hers...but now A Shadow in Moscow might just top the list. Both time periods in this novel were completely enthralling and I was both intrigued and emotionally invested all the way through. I loved it so much that after reading the ARC I immediately pre-ordered a hard copy...and I've no doubt I'll reread it as soon as it arrives. A true masterpiece!
## 3 This book slowly pulled me in and then did not let go. Two seemingly separate stories are told: One around Ingrid, a woman who lost everyone dear to her in the last days of WWII and the other around Anya, a young woman who grew up in Moscow only to get her college education in the States. I love the London House for the way it's dual timelines wove into a seemless story, and A Shadow in Moscow stunned me at the 60% mark when I saw the connection. This is a spy novel set in the dark days of the Cold War. It's a race to see who can survive, and with men like Aldrich Ames in the story, you know not everyone can. But it's a story of love, of sacrifice, of looking for truth and beauty in dark places. It's a book filled with the search for goodness and hope in points you can't expect to find it, and with the subtlest overlay of James Bond. I highly recommend it.
## 4 Edited...i just read this book last year when it first came out and absolutely loved it. I just listened to it again because it was my book club pick for May. The ladies and I all agree it's a book to be read in all schools.\nBeautiful prose, and wonderfully written. This book carried me back and forth from World War II at the end of the war into Moscow during the cold war with the Soviet Union.. The transitions from the time lines were seamless and I turned the pages effortlessly until the very end. In fact, I had a book hangover for days. I have picked A Shadow In Moscow as my next book club pick. If you are looking for a historical book about the cold war, spies, espionage and romance, then this book is for you.\nThis book is probably one of the best books I have read this year. I truly enjoyed it.\n*I was given a copy of this book by Harper Muse and this is my honest opinion.\nI also listened to the audio version and the two narrators nailed it!! Great job by the author and narrators. They brought the cold war story to life.\n
## 5 It took me a bit to get into this novel split between the perspectives of the first and second Cold War eras. But when I did, I couldn't put it down.\nReay is a master storyteller, but it's always the family relationships in her books that make me teary-eyed.\nI didn't see the ending coming, but I loved each heroine and was in awe of the part she played.\nA must-read for lovers of historical fiction.Check the structure of the dataset.
str(reviews)## 'data.frame': 30 obs. of 12 variables:
## $ Author : chr "Katherine Reay" "Katherine Reay" "Katherine Reay" "Katherine Reay" ...
## $ Title : chr "A Shadow in Moscow" "A Shadow in Moscow" "A Shadow in Moscow" "A Shadow in Moscow" ...
## $ Series : chr "Standalone" "Standalone" "Standalone" "Standalone" ...
## $ Link : chr "https://www.goodreads.com/book/show/63340978-a-shadow-in-moscow" "https://www.goodreads.com/book/show/63340978-a-shadow-in-moscow" "https://www.goodreads.com/book/show/63340978-a-shadow-in-moscow" "https://www.goodreads.com/book/show/63340978-a-shadow-in-moscow" ...
## $ Description : chr "Two women spies operating in Moscow during different Cold War eras. Ingrid Bauer, an Austrian who marries into "| __truncated__ "Two women spies operating in Moscow during different Cold War eras. Ingrid Bauer, an Austrian who marries into "| __truncated__ "Two women spies operating in Moscow during different Cold War eras. Ingrid Bauer, an Austrian who marries into "| __truncated__ "Two women spies operating in Moscow during different Cold War eras. Ingrid Bauer, an Austrian who marries into "| __truncated__ ...
## $ Average_Stars: num 4.32 4.32 4.32 4.32 4.32 4.32 4.32 4.32 4.32 4.32 ...
## $ N_Reviews : num 876 876 876 876 876 876 876 876 876 876 ...
## $ Price : chr "16.99" "16.99" "16.99" "16.99" ...
## $ Genre : chr "Historical Fiction, \nFiction, \nRussia, \nHistorical, Mystery, \nEspionage, \nSuspense" "Historical Fiction, \nFiction, \nRussia, \nHistorical, Mystery, \nEspionage, \nSuspense" "Historical Fiction, \nFiction, \nRussia, \nHistorical, Mystery, \nEspionage, \nSuspense" "Historical Fiction, \nFiction, \nRussia, \nHistorical, Mystery, \nEspionage, \nSuspense" ...
## $ Review_stars : chr "5/5 stars" "5/5 stars" "5/5 stars" "5/5 stars" ...
## $ Headline : chr "Beautifully Written and Seamlessly Told”" "A New Favorite: A Shadow in Moscow Might Be Katherine Reay’s Best Yet!" "Secrets, Sacrifice, and a Shocking Connection" "A Cold War Masterpiece That Deserves a Place in Every School" ...
## $ Review : chr "I read this book before I listened to the audio version, and let me just say that the two narrators nailed it!!"| __truncated__ "WHOA. I loooooved this book! I've been a Katherine Reay fan since her debut novel, Dear Mr. Knightley, and prob"| __truncated__ "This book slowly pulled me in and then did not let go. Two seemingly separate stories are told: One around Ingr"| __truncated__ "Edited...i just read this book last year when it first came out and absolutely loved it. I just listened to it "| __truncated__ ...Label reviews as Positive or Negative
reviews <- reviews %>%
mutate(Sentiment = ifelse(grepl("^5", Review_stars), "Positive Review", "Negative Review"))
table(reviews$Sentiment)##
## Negative Review Positive Review
## 17 13table(reviews$Title)##
## A Shadow in Moscow Our Woman in Moscow The Secrets We Kept
## 10 10 10Standard Stop Words
We remove common English stop words first.
data(stop_words)
words <- reviews %>%
unnest_tokens(word, Review) %>%
anti_join(stop_words) %>%
count(word, sort = TRUE)
words[1:10,]## word n
## 1 book 69
## 2 story 34
## 3 read 26
## 4 spy 19
## 5 war 19
## 6 characters 18
## 7 author 15
## 8 cold 15
## 9 historical 15
## 10 moscow 15Word Cloud Before Custom Stop Words
Let’s see the initial word cloud to identify noise we may want to remove.
words %>%
with(wordcloud(word, n, max.words = 100))
Custom Stop Words
After examining the word cloud, we remove author names, character names, book titles, and other words that appear across all reviews but carry no analytical meaning for comparing how readers describe these books.
custom <- data.frame(word = c(
# Author names
"katherine", "reay", "lara", "prescott", "beatriz", "williams",
# Book/character names specific to these titles
"moscow", "shadow", "secrets", "zhivago", "pasternak", "boris",
"irina", "sally", "iris", "ruth", "olga", "ingrid", "anya", "sasha", "digby",
# Generic book/review words
"book", "books", "read", "reading", "novel", "story", "author",
# Numbers
"1", "2", "3", "4", "5",
# Contractions that survive tokenization
"it's", "i've", "don't", "didn't", "i'm", "i'd", "wasn't",
# Platform/review terms
"goodreads", "review", "reviews", "arc", "netgalley",
# Star rating artifacts
"stars", "star"
))Revised Word Cloud
After removing custom stop words, the word cloud more clearly reflects how readers describe the books.
words_revised <- reviews %>%
unnest_tokens(word, Review) %>%
anti_join(stop_words) %>%
anti_join(custom) %>%
count(word, sort = TRUE)
words_revised %>%
with(wordcloud(word, n, max.words = 100))
Comparing Word Clouds: Positive vs. Negative Reviews
Do positive and negative reviewers use different language across these comp titles?
Positive Reviews Word Cloud
reviews %>%
filter(Sentiment == "Positive Review") %>%
unnest_tokens(word, Review) %>%
anti_join(stop_words) %>%
anti_join(custom) %>%
count(word, sort = TRUE) %>%
with(wordcloud(word, n, max.words = 90))
Negative/Less Positive Reviews Word Cloud
reviews %>%
filter(Sentiment == "Negative Review") %>%
unnest_tokens(word, Review) %>%
anti_join(stop_words) %>%
anti_join(custom) %>%
count(word, sort = TRUE) %>%
with(wordcloud(word, n, max.words = 90))
TF-IDF Analysis
TF-IDF identifies words that are especially distinctive to one group compared to the other. Here we compare Positive vs. Negative reviews across all three comp titles.
Set Up Word Counts by Sentiment
series_words <- reviews %>%
unnest_tokens(word, Review) %>%
anti_join(stop_words) %>%
anti_join(custom) %>%
count(Sentiment, word, sort = TRUE)
total_words <- series_words %>%
group_by(Sentiment) %>%
summarize(total = sum(n))
series_words <- left_join(series_words, total_words)
series_words[1:10,]## Sentiment word n total
## 1 Negative Review characters 14 951
## 2 Positive Review war 13 587
## 3 Negative Review spy 12 951
## 4 Positive Review love 9 587
## 5 Negative Review fiction 8 951
## 6 Negative Review historical 8 951
## 7 Negative Review spies 8 951
## 8 Negative Review woman 8 951
## 9 Positive Review cold 8 587
## 10 Negative Review cold 7 951Calculate TF-IDF
series_tf_idf <- series_words %>%
bind_tf_idf(word, Sentiment, n)
series_tf_idf %>%
select(-total) %>%
arrange(desc(tf_idf)) %>%
slice_max(tf_idf, n = 10)## Sentiment word n tf idf tf_idf
## 1 Positive Review days 5 0.008517888 0.6931472 0.005904150
## 2 Negative Review dialogue 7 0.007360673 0.6931472 0.005102030
## 3 Negative Review ussr 7 0.007360673 0.6931472 0.005102030
## 4 Positive Review beautiful 4 0.006814310 0.6931472 0.004723320
## 5 Positive Review masterpiece 4 0.006814310 0.6931472 0.004723320
## 6 Positive Review muse 4 0.006814310 0.6931472 0.004723320
## 7 Negative Review pretty 5 0.005257624 0.6931472 0.003644307
## 8 Positive Review absolutely 3 0.005110733 0.6931472 0.003542490
## 9 Positive Review engaging 3 0.005110733 0.6931472 0.003542490
## 10 Positive Review listened 3 0.005110733 0.6931472 0.003542490
## 11 Positive Review literary 3 0.005110733 0.6931472 0.003542490
## 12 Positive Review pick 3 0.005110733 0.6931472 0.003542490
## 13 Positive Review readers 3 0.005110733 0.6931472 0.003542490Visualize the Most Distinctive Words per Group
series_tf_idf %>%
group_by(Sentiment) %>%
slice_max(tf_idf, n = 12) %>%
ungroup() %>%
ggplot(aes(tf_idf, fct_reorder(word, tf_idf), fill = Sentiment)) +
geom_col(show.legend = FALSE) +
facet_wrap(~Sentiment, ncol = 2, scales = "free") +
labs(x = "TF-IDF", y = NULL,
title = "Most Distinctive Words by Review Sentiment",
subtitle = "Words that differentiate Positive from Negative/Less Positive reviews")