Text Analysis & NLP - Word Co-occurrence Analysis
Heather Cox Richardson
Roci Barnes - adapted from Zhenning “Jimmy” Xu, Ph.D.
July 07, 2026
library(tidyverse)
library(tidytext)
library(widyr)
library(igraph)
library(ggraph)
library(Matrix)
library(checkdown)Objective:
To perform a word co-occurrence analysis using the reviews or comments scraped in the Bluesky project about Heather Cox Richardson. The goal is to read the file and identify words that often come together in the comments.
Step 1: Load the Raw Text
HCR_lines <- readLines("HCR_coocurence.txt", encoding = "UTF-8")
gsub("^\\s+|\\s+$", "", HCR_lines, useBytes = TRUE)## [1] "text"
## [2] "Senator Heather Cox Richardson has a nice ring to it. ????"
## [3] "Did you read Dr. Stacey Patton's essay on Heather Cox Richardson? Pretty spot on. (I'm a fan of Richardson and have been following her for years.)"
## [4] "Senator Heather Cox Richardson has a nice ring to it. ????"
## [5] "Senator Heather Cox Richardson has a nice ring to it. ????"
## [6] "Senator Heather Cox Richardson has a nice ring to it. ????"
## [7] "\"The Replacement contenders?"
## [8] ""
## [9] "Maine Governor, Janet Mills"
## [10] "Maine Resident, Professor Heather Cox Richardson\""
## [11] "\"Heather Cox Richardson. Calm, honest, enlightened, ethical, with over 2 million followers. Why go back to teaching?\""
## [12] "Heather Cox-Richardson would be amazing!"
## [13] "Maine needs to draft Heather Cox Richardson."
## [14] "\"Heather Cox Richardson"
## [15] "For Maine US senate seat\""
## [16] "Jess Craven wants Heather Cox Richardson"
## [17] "\"Heather Cox Richardson"
## [18] "For Maine US senate seat\""
## [19] "\"Why do \"\"opening borders\"\" and \"\"non-citizens voting\"\" come to mind? Can you point to something?"
## [20] ""
## [21] "Recently, I've seen at least two people with platforms, including Heather Cox Richardson, describe democratic socialism and neither spoke of either of those two things."
## [22] ""
## [23] "www.youtube.com/watch?v=_Nt2...\""
## [24] "\"Heather Cox Richardson outdid herself in this 16 minute episode on our 250th. Will anyone listen?"
## [25] ""
## [26] "substack.com/@heathercoxr...\""
## [27] "\"Yea, stick a fork in him, he is done. As I\x83??ve said before, if I lived in Maine, I\x83??d hold my nose and vote for him, the senate is at stake"
## [28] ""
## [29] "This revelation, on top of everything else, no, I would not vote for him (or Collins), I\x83??d write in Heather Cox Richardson"
## [30] ""
## [31] "I guess we chalk this seat as a loss\""
## [32] "Heather Cox Richardson"
## [33] "\"Just saw this: could a ground swell of public opinion lead to Heather Cox Richardson agreeing to run in ME. She's from ME, but is she currently a resident?\""
## [34] "I don\x83??t get it. Why is Maine so short on Democrats who can run for Senator? What about Heather Cox Richardson running as an independent? I\x83??d contribute to that campaign."
## [35] "Someone suggested Heather Cox Richardson if she would do it. I think it\x83??s a brilliant possibility."
## [36] "\"Shes on n Bluesky, too!"
## [37] ""
## [38] "A fantastic newsletter with loads of citations & links."
## [39] ""
## [40] "Free subscription, too!"
## [41] ""
## [42] "(11) Letters from an American | Heather Cox Richardson | Substack share.google/YWPgXmnBXntO...\""
## [43] "Platner is toast. Paging Heather Cox Richardson."
## [44] "\"Ai says :Heather Cox Richardson does not identify as a liberal; she describes herself as a \"\"Lincoln Republican\"\" and historically identifies as a registered Republican. Was the former Governor that bad a choice ?\""
## [45] "\"Who steps in?"
## [46] "Janet Mills"
## [47] "Chellie Pingree"
## [48] "Matt Dunlap"
## [49] ""
## [50] "Love the Heather Cox Richardson suggestion\""
## [51] "100% support the idea of Heather Cox Richardson @hcrichardson.bsky.social running for Senate. She would be AMAZING! I would donate $ today."
## [52] "\"Janet Mills has been a great governor for the past eight years. She's old, but not in the way."
## [53] ""
## [54] "But, yes, Heather Cox Richardson would be a great choice!!!!!!!!!!!\""
## [55] "\"Heather Cox Richardson for Maine Senate."
## [56] ""
## [57] "If she is willing to do it.\""
## [58] "Heather Cox Richardson!!!"
## [59] "How about Heather Cox Richardson?"
## [60] "How about Heather Cox Richardson? You don't get any more Maine than her and she is loved."
## [61] "No. He won't. How about Heather Cox Richardson? If you're a Mainer....contact her."
## [62] "Heather Cox Richardson would be so much better."
## [63] "Lunch Money with @pkrugman.bsky.social & @hcrichardson.bsky.social \x83?? streaming live Wednesday at 12pm ET / 9am PT. Save this link: www.youtube.com/live/uJxQbfb..."
## [64] "\"HEATHER COX RICHARDSON"
## [65] "7/5/26"
## [66] ""
## [67] "\"\"Democratic Socialists are not communists or socialists, who want to see the end of private property.\"\""
## [68] ""
## [69] "open.substack.com/pub/heatherc...\""
## [70] "Heather Cox Richardson. Your country needs you."
## [71] "Heather Cox Richardson!!!"
## [72] "\"6 words:"
## [73] ""
## [74] "Heather Cox Richardson as a write-in.\""
## [75] "Write in Heather Cox Richardson"
## [76] "\"How The South Won The Civil War, Heather Cox Richardson.\""
## [77] "\"A lot of people object to Substack \x83??on principle\x83?\x9d due to its partial VC (venture capital) ownership & allowing \x83??nazi content.\"\""
## [78] "I value access to Meidas, The Contrarian, Heather Cox Richardson, Aaron Parnas, and the list goes on."
## [79] "You can pick them all up on YouTube if you prefer.\""
## [80] "\"After 1950s Red Scare, Sen. Joe McCarthy's lawyer Roy Cohn became Trump's fixer."
## [81] "Trump just threatened his white nationalist base: \"\"You can be a communist or you can be a patriot. You cannot be both.\x83?\x9d Communism, by the way, is not Democratic Socialism. It's Trump who has his own red streak.\""
## [82] "\"Historian Heather Cox Richardson breaks down the historical evolution of the word socialism in American politics:"
## [83] "youtu.be/gMwcdV2YXeY?...\""
## [84] "\"There is a series of 1-minute(ish) videos that highlight many of our accomplishments as Americans over 250 years. It\x83??s call \x83??250 to 250\x83?\x9d on YouTube and is on the page of Heather Cox Richardson. They are still being released a few at a time."
## [85] ""
## [86] "I do recommend! ???\xa7????"
## [87] ""
## [88] "youtube.com/playlist?lis...\""
## [89] "Heather Cox Richardson spoke to this in her YouTube June 25 \x83??It\x83??s time to Prepare for the November Midterms\x83??"
## [90] "I agree with this but i feel like the timing is off. All of these people got big during Trump I and I feel like you never hear about heather cox richardson anymore!"
## [91] "\"July 5, 2026"
## [92] ""
## [93] "thoughtful insights from Heather Cox Richardson, as usual-- through the lens of our longer history."
## [94] ""
## [95] "I gag though, at mention of 37+% approval of Trump still."
## [96] ""
## [97] "how?"
## [98] ""
## [99] "sad...\""
## [100] "\"His advice to ... \x83??Join my very special Trump community. GET YOUR $TRUMP NOW!\x83?\x9d cost (his followers) dearly as the coin slid from trading at $75.35 to trading at $1.76, a drop of 97%. - Heather Cox Richardson\""
## [101] "Heather Cox Richardson explains it all. www.youtube.com/live/_Nt2CQR..."
## [102] "\"Heather Cox Richardson appeared on BBC News to talk about the future of the USA. As usual she is brilliant. For example: Why democrat socialism is similar to main stream republican policies in the 50s. The current republicans have abandoned checks and balances since the 90s."
## [103] "youtu.be/oRIgO-VFIBg?...\""
## [104] "\"\"\"what today\x83??s Democratic Socialists call for is much more limited than what the Republicans under President Dwight D. Eisenhower wanted in 1956, when the top income tax bracket in the United States was 91%.\"\""
## [105] "-Heather Cox Richardson, Jul 06, 2026\""
## [106] "\"\"\"Democratic Socialists call for a robust system of private enterprise, alongside government control of the aspects of society required for people to participate in the economy on a level playing field... schools, medical care, or roads, should not be profit-making industries.\"\""
## [107] "Heather Cox Richardson\""
## [108] "\"\"\"Democrats on the House Committee on Natural Resources released a report accusing Trump of cheating the American people at large by diverting donations that donors intended to make to the nonpartisan America250 program to his own Freedom 250 organization.\"\""
## [109] "-Heather Cox Richardson\""
## [110] "\"\"\"the New York Times reported that as of the end of June, nearly a million people who bought Trump\x83??s memecoin lost a total of $3.81 billion while Trump walked away with $636 million. Trump took transaction fees up front, so he made money no matter what happened with the coin.\"\""
## [111] "Heather Cox Richardson\""
## [112] "\"\"\"Trump went on to say that the nature of the presidency means that his children \x83??have inside information\x83?\x9d about almost any business decision they make... Almost anything they do, if they want to buy a truck, if they buy an energy efficient truck, they have inside information.\x83?\x9d"
## [113] "Heather Cox Richardson\""
## [114] "\"\"\"in the U.S., where farmers were hit with skyrocketing fertilizer prices during planting season... 70% of respondents said they couldn\x83??t afford all the fertilizer they needed for the season... just one of the reasons that consumers will see higher food prices this fall.\"\""
## [115] "-Heather Cox Richardson\""
## [116] "\"\"\"Communists call for the end of private ownership of the means of production, giving the state control of private enterprises. In today\x83??s America, it is actually Trump, himself, who is taking government stakes in private enterprises.\"\""
## [117] ""
## [118] "Heather Cox Richardson\""
## [119] "Heather Cox Richardson excerpt"
## [120] "Heather Cox Richardson excerpt"
## [121] "Heather Cox Richardson excerpt"
## [122] "Heather Cox Richardson excerpt"
## [123] "Heather Cox Richardson excerpt"
## [124] "Heather Cox Richardson excerpt"
## [125] "Heather Cox Richardson:"
## [126] "\"Communists call for the end of private ownership of the means of production, giving the state control of private enterprises. In today\x83??s America, it is actually Trump, himself, who is taking government stakes in private enterprises."
## [127] ""
## [128] "Heather Cox Richardson\""
## [129] "\"Heather Cox Richardson turned a daily political-history Facebook practice into a Substack newsletter that now has over three million subscribers."
## [130] "\""
## [131] "\"Heather Cox Richardson"
## [132] "Jul 06, 2026"
## [133] "July 5, 2026"
## [134] "On July 3, Fifty Plus One, which tracks Trump\x83??s job approval rating, reported that 59.1% of Americans disapprove of his performance while only 37.5% approve."
## [135] "And so, on Friday night, the eve of the 250th anniver..."
## [136] "open.substack.com/pub/heatherc...\""
## [137] "\"Heather Cox Richardson is spot on. Mitch McConnell and Senate Republicans could have stopped DJT but most chose party over country."
## [138] ""
## [139] "During his first impeachment trial in 2020, only Sen. Mitt Romney voted to convict DJT."
## [140] ""
## [141] "Seven Republican senators voted to convict"
## [142] ""
## [143] "P 1/2\""
## [144] "\"July 5, 2026"
## [145] "Heather Cox Richardson open.substack.com/pub/heatherc...\""
## [146] "\"July 6, 2026"
## [147] "Heather Cox Richardson"
## [148] ""
## [149] "Going into the weekend during which Americans celebrated the 250th anniversary of the day on which the Second Continental Congress accepted the Declaration of Independence, President Donald J. Trump was facing a whole lot of bad news......\""
## [150] "\"I heard Heather Cox Richardson the other day, on a BBC podcast about the 250th, say that these Democratic Socialist candidates and elected officials are actually to the right of the Republican Party of 1956.\""
## [151] "\"I am grateful to have found you and Heather Cox Richardson and others whose work I have followed for the past year or so. I learn from you every day. And I know, in my heart and in my mind, when the history of these times is written, your names will be writ large."
## [152] "You have my eternal gratitude."
## [153] "###\""
## [154] "Heather Cox Richardson is a very intelligent woman. A future Democratic president would do well to tap her for advice on how to stop all this bullshit from happening again."
## [155] "Heather Cox Richardson did a video about this a week or so ago. I don\x83??t think the title reflects that necessarily though"
## [156] "\"Heather Cox Richardson: America's 250 Great (and Awful) Years"
## [157] "open.substack.com/pub/thebulwa...\""
## [158] "\"\x83??Once you admit the principle that some people don\x83??t have the same rights as others, you\x83??ve given up the whole game.\x83?\x9d --- Heather Cox Richardson"
## [159] "youtube.com/shorts/QWhxd...\""
## [160] "\"open.substack.com/pub/heatherc..."
## [161] ""
## [162] "For those of you that need some US history, professor Heather Cox Richardson is the best!\""
## [163] "\"Or there's this excellent little diddy from Heather Cox Richardson. ????"
## [164] ""
## [165] "\"\"We have to get some of our terms under control...\"\""
## [166] ""
## [167] "www.youtube.com/watch?v=_Nt2...\""
## [168] "\"From \"\"The Second Trump Election Broke Something in America (w/ @hcrichardson.bsky.social ),\"\" The Focus Group"
## [169] ""
## [170] "@sarahlongwell25.bsky.social @thebulwark.com"
## [171] ""
## [172] "youtu.be/xBE68EKHq2c?..."
## [173] ""
## [174] "Full carousel on Instagram."
## [175] "www.instagram.com/p/DabI8vBltW...\""
## [176] "\"The billionaires are trying to steal our national parks and buy them cheaply or take over managing them for themselves and then develop them ."
## [177] ""
## [178] "They have an aggressive list to take"
## [179] ""
## [180] "Lies, Threats, and Looting"
## [181] "Heather Cox Richardson"
## [182] "Jun 28, 2026 heathercoxrichardson.substack.com/p/lies-threa...\""
## [183] "\"HEATHER COX RICHARDSON"
## [184] ""
## [185] "Week Six in 250 to 250"
## [186] "open.substack.com/pub/heatherc...\""
## [187] "\"If anyone has been following the news from across the pond, the country we stopped from occupying is concerned and curious. Boston College history professor Heather Cox Richardson had some excellent points on where we stand in this polarizing time in our history:"
## [188] ""
## [189] "open.spotify.com/episode/5Of1...\""
## [190] "\"Trump is stealing and grifting the American people blind!"
## [191] "The White House is Not a Business Opportunity"
## [192] "Heather Cox Richardson"
## [193] "Jul 02, 2026 heathercoxrichardson.substack.com/p/the-white-...\""
## [194] "Heather Cox Richardson is a propagandist who never mentioned the genocide in Gaza during the Biden administration."
## [195] "\"You might want to listen to what Heather Cox Richardson has to say about what is left of center these days."
## [196] ""
## [197] "Got to go!\""
## [198] "\"If you want to really understand the Declaration of Independence, Heather Cox Richardson reads it and gives it real meaning."
## [199] "www.youtube.com/live/RCsf4t2...\""
## [200] "\"I say a BIG cake for some of our most important democracy defenders Joyce White Vance, Marc Elias, Heather Cox Richardson, the Meisalas brothers, Leigh McGowan, Glenn Kirschner, and so many more!!\""
## [201] "\"Why? I already have showed #AnyOther as a net for Dems. Heather Cox Richardson made it understandable. Fer mah Dah, who's 67 y/o - a Never Trump voter. He wants *ACCOUNTABILITY* and punishment for the malfeasance of this regime. And he's on Medicare *AND* SSDI. In Downey, California 90241...\""
## [202] "By Heather Cox Richardson - Jul 5"
## [203] "\"Political historian and author Heather Cox Richardson notes that the rapid leaking of information is a tactic being used as leading figures inside the current administration lose power. As individuals attempt to grab power, distance themselves from certain agendas, or avoid prosecution, they are act\""
## [204] "OMG ???\xfd This is AMAZING! Everyone needs to listen to Heather Cox Richardson explain the Declaration of Independence! \x83\x9d?\x8b?\x8f???\xad????????"
## [205] "\"Another one who didn\x83??t vote in 2024, gave us Trump and now is covering the guilt by claiming something new which really has been the position of Dems for years. Start reading the daily historical perspective of Heather Cox Richardson. You will learn a lot.\""
## [206] "\"It\x83??s a repeat of McCarthy-ism. After the Robber Barrons of the early 1900\x83??s were forced to conduct business in a more civil manor and FDR rolled out programs to help people, the backlash was the Communism/Red Scare BS. Heather Cox Richardson explained it well. Will try to find it.\""
## [207] "Excellent summation of some of Trump's corrupt activities. We are being looted. Join me in following and supporting Heather Cox Richardson."
## [208] "\"Biden will go down in history as one of the best presidents ever. Trump will be remembered as the worst president ever."
## [209] ""
## [210] "www.presidency.ucsb.edu/documents/fa...\""
## [211] "Haven\x83??t read it summed up better than this. Honestly. Was literally just reading a comment from someone ragging on Heather Cox Richardson of all people. As if she\x83??s the Antichrist. It\x83??s exhausting."
## [212] "\"open.substack.com/pub/heatherc..."
## [213] ""
## [214] "Many thanks to Heather Cox Richardson and her team for all of the 250 snippets of history. THAT is how you celebrate democracy.\""
## [215] "How Six People Are Destroying Democracy | Explainer \x83?? Heather Cox Richardson themindlesspraetorianblog.blogspot.com/2026/07/how-..."
## [216] "The BBC's Americast team interviews historian Heather Cox Richardson to explore how the 18th-century vision of the Founding Fathers compares to the realities of 2026."
## [217] "\"So, Facebook, X, and Fox"
## [218] "www.nytimes.com/interactive/...\""
## [219] "\"This is echoed by Heather Cox Richardson on the Focus Group podcast with Sarah Longwell."
## [220] ""
## [221] "www.youtube.com/watch?v=xBE6...\""
## [222] "\"Heather Cox Richardson, historian\""HCR_text <- tibble(
id = seq_along(HCR_lines),
text = HCR_lines
)
HCR_text## # A tibble: 222 × 2
## id text
## <int> <chr>
## 1 1 "text"
## 2 2 "Senator Heather Cox Richardson has a nice ring to it. ????"
## 3 3 "Did you read Dr. Stacey Patton's essay on Heather Cox Richardson? Pr…
## 4 4 "Senator Heather Cox Richardson has a nice ring to it. ????"
## 5 5 "Senator Heather Cox Richardson has a nice ring to it. ????"
## 6 6 "Senator Heather Cox Richardson has a nice ring to it. ????"
## 7 7 "\"The Replacement contenders?"
## 8 8 ""
## 9 9 "Maine Governor, Janet Mills"
## 10 10 "Maine Resident, Professor Heather Cox Richardson\""
## # ℹ 212 more rowsStep 2: Tokenize and Clean
To find the coocurence of words in my data, it is important that I remove my subject’s name from the data. First, the name will dominate all the word searches. Also, the name is three words, therefore it is very likely that it will have multiple intances of coocurrence. I also removed references to social media platforms.
custom_stop_words <- bind_rows(
stop_words,
tibble(word = c("im", "ive", "dont", "didnt", "isnt", "lol", "heather", "cox", "richardson", "the", "and", "substack", "youtube", "bsky", "pub", "heatherc", "hcr", "www", "hcrichardson"), lexicon = "custom")
)
HCR_words <- HCR_text %>%
mutate(text = str_replace_all(text, "[^[:alnum:][:space:]$]", " ")) %>%
unnest_tokens(word, text, token = "words") %>%
filter(!str_detect(word, "^[0-9]+$")) %>% # drop pure numbers
anti_join(custom_stop_words, by = "word")
HCR_words %>% count(word, sort = TRUE) %>% head(15)## # A tibble: 15 × 2
## word n
## <chr> <int>
## 1 trump 22
## 2 people 11
## 3 maine 9
## 4 history 8
## 5 private 8
## 6 democratic 7
## 7 excerpt 6
## 8 senate 6
## 9 call 5
## 10 july 5
## 11 republican 5
## 12 senator 5
## 13 social 5
## 14 250th 4
## 15 america 4Step 3: Count Word Pairs
HCR_pairs <- HCR_words %>%
pairwise_count(word, id, sort = TRUE, upper = FALSE)
HCR_pairs %>% head(15)## # A tibble: 15 × 3
## item1 item2 n
## <chr> <chr> <dbl>
## 1 senator nice 4
## 2 senator ring 4
## 3 nice ring 4
## 4 maine senate 4
## 5 people trump 4
## 6 janet mills 3
## 7 democratic socialists 3
## 8 communists private 3
## 9 private call 3
## 10 private government 3
## 11 call government 3
## 12 private control 3
## 13 call control 3
## 14 government control 3
## 15 trump means 3Step 4: Build the Co-occurrence Matrix
top_words <- HCR_words %>%
count(word, sort = TRUE) %>%
slice_max(n, n = 15) %>%
pull(word)
HCR_matrix <- HCR_pairs %>%
filter(item1 %in% top_words, item2 %in% top_words) %>%
bind_rows(HCR_pairs %>% rename(item1 = item2, item2 = item1) %>%
filter(item1 %in% top_words, item2 %in% top_words)) %>%
cast_sparse(item1, item2, n) %>%
as.matrix()
HCR_matrix## nice ring senate trump socialists call control america democratic
## senator 4 4 0 0 0 0 0 0 0
## nice 0 4 0 0 0 0 0 0 0
## maine 0 0 4 0 0 0 0 0 0
## people 0 0 0 4 1 1 1 0 2
## democratic 0 0 0 1 3 2 1 0 0
## private 0 0 0 2 2 3 3 2 2
## call 0 0 0 2 2 0 3 2 2
## trump 0 0 0 0 0 2 2 3 1
## socialists 0 0 0 0 0 2 1 0 3
## american 0 0 0 2 0 0 0 0 0
## control 0 0 0 2 1 3 0 2 1
## enterprises 0 0 0 2 0 2 2 2 0
## million 0 0 0 1 0 0 0 0 0
## socialism 0 0 0 1 0 0 0 0 2
## 250th 0 0 0 1 0 0 0 0 1
## senate 0 0 0 0 0 0 0 0 0
## social 0 0 1 1 0 0 0 1 0
## history 0 0 0 1 0 0 0 0 0
## white 0 0 0 1 0 0 0 0 1
## ring 4 0 0 0 0 0 0 0 0
## america 0 0 0 3 0 2 2 0 0
## don 0 0 0 0 0 0 0 0 0
## president 0 0 0 2 1 1 0 0 2
## republican 0 0 0 0 0 0 0 0 1
## live 0 0 0 0 0 0 0 0 0
## historian 0 0 0 0 0 0 0 0 0
## july 0 0 0 1 0 0 0 0 0
## jul 0 0 0 0 0 0 0 0 0
## socialism don american private president enterprises maine people
## senator 0 1 0 0 0 0 1 0
## nice 0 0 0 0 0 0 0 0
## maine 0 2 0 0 0 0 0 0
## people 1 1 2 1 0 0 0 0
## democratic 2 0 0 2 2 0 0 2
## private 0 0 0 0 0 2 0 1
## call 0 0 0 3 1 2 0 1
## trump 1 0 2 2 2 2 0 4
## socialists 0 0 0 2 1 0 0 1
## american 1 0 0 0 0 0 0 2
## control 0 0 0 3 0 2 0 1
## enterprises 0 0 0 2 0 0 0 0
## million 0 0 0 0 0 0 0 1
## socialism 0 0 1 0 0 0 0 1
## 250th 0 0 0 0 1 0 0 0
## senate 0 0 0 0 0 0 4 0
## social 0 0 0 0 0 0 0 0
## history 0 0 0 0 1 0 0 0
## white 1 0 0 0 0 0 0 0
## ring 0 0 0 0 0 0 0 0
## america 0 0 0 2 0 2 0 0
## don 0 0 0 0 0 0 2 1
## president 0 0 0 0 0 0 0 0
## republican 1 0 0 0 0 0 0 0
## live 0 0 0 0 0 0 0 0
## historian 1 0 1 0 0 0 0 0
## july 0 0 0 0 0 0 0 0
## jul 0 0 0 0 0 0 0 0
## 250th republican social live white historian july history jul
## senator 0 0 0 0 0 0 0 0 0
## nice 0 0 0 0 0 0 0 0 0
## maine 0 0 0 0 0 0 0 0 0
## people 0 0 0 0 0 0 0 0 0
## democratic 1 1 0 0 1 0 0 0 0
## private 0 0 0 0 0 0 0 0 0
## call 0 0 0 0 0 0 0 0 0
## trump 1 0 1 0 1 0 1 1 0
## socialists 0 0 0 0 0 0 0 0 0
## american 0 0 0 0 0 1 0 0 0
## control 0 0 0 0 0 0 0 0 0
## enterprises 0 0 0 0 0 0 0 0 0
## million 0 0 0 0 0 0 0 1 0
## socialism 0 1 0 0 1 1 0 0 0
## 250th 0 1 0 0 0 0 0 0 0
## senate 0 0 1 0 0 0 0 0 0
## social 0 0 0 1 0 0 0 0 0
## history 0 0 0 0 0 0 0 0 0
## white 0 0 0 0 0 0 0 0 1
## ring 0 0 0 0 0 0 0 0 0
## america 0 0 1 0 0 0 0 0 0
## don 0 0 0 0 0 0 0 0 0
## president 1 0 0 0 0 0 0 1 0
## republican 1 0 0 0 0 0 0 0 0
## live 0 0 1 0 0 0 0 0 0
## historian 0 0 0 0 0 0 0 0 0
## july 0 0 0 0 0 0 0 0 0
## jul 0 0 0 0 1 0 0 0 0
## senator million
## senator 0 0
## nice 4 0
## maine 1 0
## people 0 1
## democratic 0 0
## private 0 0
## call 0 0
## trump 0 1
## socialists 0 0
## american 0 0
## control 0 0
## enterprises 0 0
## million 0 0
## socialism 0 0
## 250th 0 0
## senate 0 0
## social 0 0
## history 0 1
## white 0 0
## ring 4 0
## america 0 0
## don 1 0
## president 0 0
## republican 0 0
## live 0 0
## historian 0 0
## july 0 0
## jul 0 0Step 5: Visualize the Strongest Co-occurrences
To keep the network readable, we’ll only keep pairs that occurred together more than once.
set.seed(580)
HCR_pairs %>%
filter(n > 1) %>%
graph_from_data_frame() %>%
ggraph(layout = "fr") +
geom_edge_link(aes(edge_alpha = n, edge_width = n), color = "firebrick") +
geom_node_point(size = 4, color = "steelblue") +
geom_node_text(aes(label = name), repel = TRUE, size = 3.5) +
theme_void() +
labs(
title = "Word Co-occurrence Network: HCR Bluesky Comments",
subtitle = "Edge thickness = number of sentences containing both words"
)
Reflection questions for you (add your responses below)
Question 1 Which additional words did you add to the stopword list manually? Why does it make sense to remove those words as stopwords? Also, does the final network graph (see the last code chunk) change after removing these additional stopwords? If so, please briefly explain how and why.**
I added several words that I felt dominated the search, but provided very little context to how people feel about Heather Cox Richardson.
, “heather”, “cox”, “richardson”, “the”, “and”, “substack”, “youtube”, “bsky”, “pub”, “heatherc”, “hcr”, “www”, “hcrichardson”
Removing these words changed the list, and the focus became Trump, lol
Question 2 Change “n” in the function, “filter(n > 1)”, in step 5 to a different value. What is your final value of n? Will it give you a better result?
When I changed it to 5, the graph disappeared. I think that I need a longer than 100 comments scrapped in order to get a better feel of the people’s sentiment. I left it at 1, since my sample is so small.
Additional discussion Questions (try to use external sources to support your responses)
Question 3 On my BlueSky post here: https://bsky.app/profile/did:plc:jbmmqoxfdgpoavycm7fz4k3r/post/3mphyccr2vc2g, I had two co-occurence network graphs. Which one do you like better? Why?
I preferer the first graph. It provides the interconnections accross the other words, which the second graph primarily direct connections. My guess is that this is the difference between the filter(n > 1)
Question 4 Which word pairs surprised you?
I was surprised of the scattered clusters in the data. I expected to see trump as a major point with a heavy connection, but I did not expect the cluster around “senate” an “senator”. I also expected more about sentiments around history, since that is her primary occupation.
Question 5 Should we treat keep HCR as one token instead of treating it as individual parts of her name?
No HCR is as common as each part of her name, when it comes to posts on social media. It should be removed in the tokenization to ensure other words that are more context can float to the top.
Question 6 What’s the risk of drawing conclusions from a co-occurrence network built on only a small sample of comments? What is a good sample size? How would you communicate that limitation in a business report?
A major risk in pulling a small sample of any social media is that it will be dominated by what is occuring on that day. Social media is a large and limiting data will cause a bias. A good sample size should be determined by the subject and it’s footpring. It can also be determined on an event sorrouning the subject.
When presenting this type of data we must always informated the user that the data a represntative of a range, whether date, population size, or vent to ensure the data is interpreted correctly.
Question 7 Is there a way to analyze comments or reviews on topics that interest you in relation to a significant event? If so, how would you identify the event? Why is that event significant, and how might it influence the comments or reviews you analyze?
In marketing, events that predict consumer spending are always important to capture. For example it is imperative that businesses be able to predict the sales and needed inventory for the end of year holiday season. This is the type of event that I would chose to explore.
Wrap-Up:
Working with the Heather Cox Richardson dataset helped me see how the same text analysis techniques from our introductory examples apply to real-world public sentiment. The earlier examples made it easier to understand how word relationships form because the patterns were simple and predictable. In contrast, the HCR data contained much more natural and complex language, making the connections less obvious. Even though the dataset was messier, I followed the same workflow throughout the project: tokenize the text, clean and normalize the words, identify word pairs, build a co-occurrence matrix, and visualize the network. Completing this project showed me that the same analytical process can be used to explore public perception and sentiment in authentic text, even when the patterns are not immediately apparent.
References
Boost Brand Credibility with Co Occurrence SEO. https://www.linkedin.com/posts/umer-abid-78045131a_co-occurrence-seo-is-a-powerful-concept-that-activity-7421759779892809728-MGh_/
https://seopressor.com/blog/why-co-citation-and-co-occurrence-are-such-big-deal/
Kong, J., Scott, A., & Goerg, G. M. (2016). Improving topic clustering on search queries with word co-occurrence and bipartite graph coclustering. https://research.google/pubs/improving-topic-clustering-on-search-queries-with-word-co-occurrence-and-bipartite-graph-co-clustering/
Colladon, A. F. (2018). The semantic brand score. Journal of Business Research, 88, 150-160.
Silge, J., & Robinson, D. (2017). Text Mining with R: A Tidy Approach. O’Reilly Media. https://www.tidytextmining.com/
Robinson, D. (2021). widyr: Widen, process, then re-tidy data [R package documentation]. https://CRAN.R-project.org/package=widyr
Csardi, G., & Nepusz, T. (2006). The igraph software package for complex network research. InterJournal, Complex Systems, 1695. https://igraph.org
Pedersen, T. L. (2024). ggraph: An implementation of grammar of graphics for graphs and networks [R package documentation]. https://CRAN.R-project.org/package=ggraph