Text Analysis & NLP - Word Co-occurrence Analysis (World Cup 2026 Comments)
Special Topics in Business Analytics
Zhenning “Jimmy” Xu, Ph.D., adapted by Lennin Jaime Avila
July 08, 2026
library(tidyverse)
library(tidytext)
library(widyr)
library(igraph)
library(ggraph)
library(Matrix)
library(checkdown)Part 2: Real-World Example — World Cup 2026 YouTube Comments
Im using the data i scraped in my previous exercise. I used a youtube video for the World Cup game between Mexico and Czechia ### Step 1: Load the Raw Text
worldcup_raw <- read_csv("/Users/okjaime/src/msba_biz_analytics/580/labs/lab4/worldcup2026_comments_raw.csv")
worldcup_text <- worldcup_raw %>%
mutate(id = row_number()) %>%
select(id, text)
worldcup_text## # A tibble: 100 × 2
## id text
## <int> <chr>
## 1 1 I’m not Mexican but dayum Mexicans are hyped asf! 😂
## 2 2 4:56-5:10 & 5:50-6:20 it’s crazy Romo got that pass off setting up tha…
## 3 3 Im from Nicaragua but love Quienonez game
## 4 4 Man, my pops would have been so proud of this team.... RIP Pops & to a…
## 5 5 ⚽️⚽️⚽️🎁🏆🎁⚽️⚽️⚽️
## 6 6 MEXICO IS GOING TO WIN
## 7 7 This group is so in sync. They are 100% playing for each other and the…
## 8 8 Stylish really stylish
## 9 9 VIVA MEXICO CABRONES
## 10 10 ..
## # ℹ 90 more rowsEach comment is treated as its own “document” (id), kind of like how the fruit example did it.
Step 2: Tokenize and Clean
this is real-world text and comments so it needs to be cleaned up. seeing as its youtube comments, there are a lot of abbreviations, slang, and misspellings and emojis.
Tasks for you:
Add your own stopwords: Look at the word tibble on my tutorial. Find 1–2 words that feel too generic to be meaningful, words that are frequent only because they appear in most sentences, not because they signal something meaninful.
Add those words to the tibble(word = c()) line in the chunk below, then re-run that chunk only without publishing the document.
custom_stop_words <- bind_rows(
stop_words,
tibble(word = c("im", "ive", "dont", "didnt", "isnt", "lol"), lexicon = "custom")
)
worldcup_words <- worldcup_text %>%
mutate(text = str_replace_all(text, "[^[:alnum:][:space:]]", " ")) %>%
unnest_tokens(word, text, token = "words") %>%
filter(!str_detect(word, "^[0-9]+$")) %>% # drop pure numbers
anti_join(custom_stop_words, by = "word")
worldcup_words %>% count(word, sort = TRUE) %>% head(15)## # A tibble: 15 × 2
## word n
## <chr> <int>
## 1 mexico 31
## 2 mexican 13
## 3 game 12
## 4 viva 12
## 5 ochoa 11
## 6 team 11
## 7 memo 8
## 8 goal 5
## 9 love 5
## 10 beautiful 4
## 11 congrats 4
## 12 country 4
## 13 crazy 4
## 14 cup 4
## 15 de 4Step 3: Count Word Pairs
worldcup_pairs <- worldcup_words %>%
pairwise_count(word, id, sort = TRUE, upper = FALSE)
worldcup_pairs %>% head(15)## # A tibble: 15 × 3
## item1 item2 n
## <chr> <chr> <dbl>
## 1 viva mexico 9
## 2 team mexico 7
## 3 mexican team 5
## 4 love mexico 4
## 5 world cup 4
## 6 mexico congrats 4
## 7 mexican viva 3
## 8 proud viva 3
## 9 team viva 3
## 10 mexican mexico 3
## 11 game mexico 3
## 12 mexican memo 3
## 13 team memo 3
## 14 game watching 3
## 15 mexican games 3Step 4: Build the Co-occurrence Matrix
Same idea as the fruit example. Again, because this is real data and World Cup vocabulary is much larger than the fruit example, I’m restricting so it can be readable still.
top_words <- worldcup_words %>%
count(word, sort = TRUE) %>%
slice_max(n, n = 15) %>%
pull(word)
worldcup_matrix <- worldcup_pairs %>%
filter(item1 %in% top_words, item2 %in% top_words) %>%
bind_rows(worldcup_pairs %>% rename(item1 = item2, item2 = item1) %>%
filter(item1 %in% top_words, item2 %in% top_words)) %>%
cast_sparse(item1, item2, n) %>%
as.matrix()
worldcup_matrix## mexico team cup congrats viva memo watching players goal winning game
## viva 9 3 0 0 0 1 0 0 0 0 0
## team 7 0 0 1 3 3 1 0 0 1 1
## mexican 3 5 0 1 3 3 0 3 0 0 0
## love 4 0 0 2 1 0 0 0 0 0 1
## world 1 0 4 0 0 0 0 1 0 1 0
## mexico 0 7 1 4 9 0 1 0 0 2 3
## game 3 1 0 1 0 0 3 0 2 2 0
## crazy 0 0 0 0 0 0 0 0 2 2 2
## goal 0 0 0 0 0 1 0 0 0 1 2
## winning 2 1 1 0 0 0 0 0 1 0 2
## ochoa 1 2 2 1 0 1 1 2 0 0 1
## de 0 0 0 0 0 0 0 0 0 0 0
## beautiful 2 1 0 0 1 0 0 1 0 0 0
## stadium 1 0 1 0 0 0 0 0 0 0 0
## soccer 0 0 1 0 0 0 0 0 0 0 0
## watching 1 1 0 1 0 0 0 0 0 0 3
## cup 1 0 0 0 0 0 0 1 0 1 0
## congrats 4 1 0 0 0 0 1 0 0 0 1
## memo 0 3 0 0 1 0 0 0 1 0 0
## players 0 0 1 0 0 0 0 0 0 0 0
## country 2 1 0 0 1 0 0 0 1 0 1
## la 1 0 0 0 1 0 0 0 0 0 0
## beautiful ochoa stadium world country la love de mexican crazy soccer
## viva 1 0 0 0 1 1 1 0 3 0 0
## team 1 2 0 0 1 0 0 0 5 0 0
## mexican 2 2 0 0 0 0 1 0 0 0 0
## love 0 0 0 0 0 0 0 0 1 0 0
## world 0 2 1 0 0 0 0 0 0 0 1
## mexico 2 1 1 1 2 1 4 0 3 0 0
## game 0 1 0 0 1 0 1 0 0 2 0
## crazy 0 0 0 0 0 0 0 0 0 0 0
## goal 0 0 0 0 1 0 0 0 0 2 0
## winning 0 0 0 1 0 0 0 0 0 2 0
## ochoa 1 0 2 2 0 1 0 0 2 0 0
## de 0 0 0 0 0 2 0 0 0 0 1
## beautiful 0 1 0 0 0 0 0 0 2 0 0
## stadium 0 2 0 1 0 0 0 0 0 0 0
## soccer 0 0 0 1 0 0 0 1 0 0 0
## watching 0 1 0 0 0 0 0 0 0 0 0
## cup 0 2 1 4 0 0 0 0 0 0 1
## congrats 0 1 0 0 0 0 2 0 1 0 0
## memo 0 1 0 0 0 0 0 0 3 0 0
## players 1 2 0 1 0 0 0 0 3 0 0
## country 0 0 0 0 0 0 0 0 0 0 0
## la 0 1 0 0 0 0 0 2 0 0 0Step 5: Visualize the Strongest Co-occurrences
To keep the network readable, we’ll only keep pairs that occurred together more than once.
set.seed(580)
worldcup_pairs %>%
filter(n > 1) %>%
graph_from_data_frame() %>%
ggraph(layout = "fr") +
geom_edge_link(aes(edge_alpha = n, edge_width = n), color = "firebrick") +
geom_node_point(size = 4, color = "steelblue") +
geom_node_text(aes(label = name), repel = TRUE, size = 3.5) +
theme_void() +
labs(
title = "Word Co-occurrence Network: World Cup 2026 Comments",
subtitle = "Edge thickness = number of comments containing both words"
)
Reflection questions
Question 1 Which additional words did you add to the stopword list manually? Why does it make sense to remove those words as stopwords? Also, does the final network graph (see the last code chunk) change after removing these additional stopwords? If so, please briefly explain how and why.**
I used words that could inflate the co-occurrence counts, such as “im”, “ive”, “dont”, “didnt”, “isnt”, and “lol”. These words will not provide meaningful insight into the content of the comments.
Question 2 Change “n” in the function, “filter(n > 1)”, in step 5 to a different value. What is your final value of n? Will it give you a better result?
lowering the threshold to n>1 showed more pairs, but also a lot more noise. the refinment and “context clarity” were lost.
Additional discussion Questions (try to use external sources to support your responses)
Question 3 On my BlueSky post here: https://bsky.app/profile/did:plc:jbmmqoxfdgpoavycm7fz4k3r/post/3mphyccr2vc2g, I had two co-occurence network graphs. Which one do you like better? Why?
I like them both for separate reasons. Graph 1 (the busier one) is good from an SEO perspective. It allows me to make a better decision as to the verbage I should use for my coffee website. Graph 2 (the simple one) I appreciate from a data consumer perspective. It’s clear, simple, to the point. It doesn’t leave me with a lack of confidence.
Question 4 Which word pairs surprised you?
Honestly, none really. It was pretty predictable and aligned with what the reality of the video and the fans of boths teams would say. So maybe the finding is that everything is as expected.
Question 5 Should we treat variants of words as the same token instead of separate ones?
Yes. Or maybe it depends. There is value in being explicit when we’re talking about SEO and things like that, but there is value in being concise for analysis.
Question 6 What’s the risk of drawing conclusions from a co-occurrence network built on only a small sample of comments? What is a good sample size? How would you communicate that limitation in a business report?
A small sample risks negating the population truth. That’s true for any analysis though, not just this. In a business report I would use my intuition and experience to plainly say how confident I am of the findings. If there is a means to represent scale, I would also do that as well.
Question 7 Is there a way to analyze comments or reviews on topics that interest you in relation to a significant event? If so, how would you identify the event? Why is that event significant, and how might it influence the comments or reviews you analyze?
Yes of course. The implication of an “event” is time. So based on that time, you could get data around it (pre, during, and post) and analyze that.
Wrap-Up: From Toy Example to Business Insight
it was very good to see how word relationships are more meaningful than word frequencies for analysis of this kind. simple frequency is not good enough to create context.