Lab 4 - YouTube Scrape
Jaason Loucks
2026-06-26
Part 1: 5-Sentence Reflection
Based on the Chapter 12 reading, “Watching Competitors,” it reveals that competitive intelligence is a mundane but essential process of observing, studying, and learning about rivals to anticipate their future actions. Generally speaking, there is a wealth of competitive data readily available in the public domain through corporate websites, social media, and financial databases. Through that social media, one is able to gauge the public’s overall sentiment toward that company, public figure, band, etc. However, one should take caution when “scraping” these comments/sentiment as they can come from bots/inauthentic sources.
It’s important to remember that while manual searching is possible, automated data acquisition via web crawling and scraping is far more efficient for gathering this information at scale, especially when comments can reach up into the thousands.
Load Libraries
library(tuber)
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(readr)
library(tidytext)
library(ggplot2)
library(wordcloud)## Loading required package: RColorBrewerlibrary(RColorBrewer)
library(tm)## Loading required package: NLP##
## Attaching package: 'NLP'## The following object is masked from 'package:ggplot2':
##
## annotatelibrary(httpuv)
library(stringr)Authenticate with yt_oauth()
app_id <- Sys.getenv("app_id")
app_secret <- Sys.getenv("app_secret")
yt_oauth(app_id, app_secret, token = ".httr-oauth")Retrieving the Video ID
# The Damned Things - We've Got a Situation Here
## https://www.youtube.com/watch?v=XMANR-wrg7o&list=RDXMANR-wrg7o&start_radio=1
video_id <- "XMANR-wrg7o"
# Pulling Comments
comments_raw <- get_all_comments(video_id = video_id)
head(comments_raw)## authorDisplayName
## 1 @Heavenhellhaleema
## 2 @ElectroIsElectro
## 3 @chickenstevens
## 4 @seansexton1616
## 5 @Pennyroyal-tea1993z
## 6 @RikkiB221
## authorProfileImageUrl
## 1 https://yt3.ggpht.com/ytc/AIdro_m3k0tI4OZTkxWzlzCDDU6NYz-P4QH2X-zW_o4d_Kk=s48-c-k-c0x00ffffff-no-rj
## 2 https://yt3.ggpht.com/-unqhZ2eAT4SOBxbAJbwCP9ozpTR6IIA0ezgyfHm5iy1kPy3YgmI9vSqnjUlie8mZRBJw04_XA=s48-c-k-c0x00ffffff-no-rj
## 3 https://yt3.ggpht.com/ytc/AIdro_kY4ZDRYpJbrAm85JYdop_hevgAvwOwRPVrQVW8TJnT6B8=s48-c-k-c0x00ffffff-no-rj
## 4 https://yt3.ggpht.com/ytc/AIdro_lOBiL515Ne7-qB7gjA76dYD8rp_tsoVtdi0otKXXM=s48-c-k-c0x00ffffff-no-rj
## 5 https://yt3.ggpht.com/QOd7dbdvrlf8vnxhnZ2kjFEQGw8XHeX8lv_Es6WCYaFPaDWZU1gWWdjLWE1dWeaRI-acVjFuwzw=s48-c-k-c0x00ffffff-no-rj
## 6 https://yt3.ggpht.com/skRTROT3_L01QsPDzikN9lp6cKsc-MeUZgN-fa284woThRgvf7DxVJgIL3Hqver1X-O7FnsfEg=s48-c-k-c0x00ffffff-no-rj
## authorChannelUrl authorChannelId.value
## 1 http://www.youtube.com/@Heavenhellhaleema UCDSXWFpcnrtkA4gvcWq7hKQ
## 2 http://www.youtube.com/@ElectroIsElectro UC2C8yegEF0okucQ2VqPzULA
## 3 http://www.youtube.com/@chickenstevens UCAnagQOLpxl4Jy7NZA-1XXA
## 4 http://www.youtube.com/@seansexton1616 UCxPVV_BpVr49UHx7EJh3PaA
## 5 http://www.youtube.com/@Pennyroyal-tea1993z UCC5UdwdAZczqbT-S7to_QBw
## 6 http://www.youtube.com/@RikkiB221 UCTQSgH3iRn-wJ9peIFa9O5w
## videoId
## 1 XMANR-wrg7o
## 2 XMANR-wrg7o
## 3 XMANR-wrg7o
## 4 XMANR-wrg7o
## 5 XMANR-wrg7o
## 6 XMANR-wrg7o
## textDisplay
## 1 Reading None of this Rocks by Joe Trohman and finally inspired to check out this project. <br>This does, indeed, rock 👍🏾
## 2 I think i need to lock in and actual listen to their albums bc their stuff sounds pretty freaking cool 👁️👁️
## 3 We need a new album please!
## 4 They were supposed to be the Messiahs of Rock and were done before they got started. I love this song!! Brought me back to Thrax
## 5 Amazing combination of good musicians -> amazing good music
## 6 2025 baby!!!
## textOriginal
## 1 Reading None of this Rocks by Joe Trohman and finally inspired to check out this project. \nThis does, indeed, rock 👍🏾
## 2 I think i need to lock in and actual listen to their albums bc their stuff sounds pretty freaking cool 👁️👁️
## 3 We need a new album please!
## 4 They were supposed to be the Messiahs of Rock and were done before they got started. I love this song!! Brought me back to Thrax
## 5 Amazing combination of good musicians -> amazing good music
## 6 2025 baby!!!
## canRate viewerRating likeCount publishedAt updatedAt
## 1 TRUE none 0 2026-06-03T02:44:27Z 2026-06-03T02:44:27Z
## 2 TRUE none 1 2026-03-31T10:03:55Z 2026-03-31T10:03:55Z
## 3 TRUE none 2 2026-02-13T20:00:11Z 2026-02-13T20:00:11Z
## 4 TRUE none 1 2026-01-31T04:15:07Z 2026-01-31T04:15:07Z
## 5 TRUE none 1 2026-01-16T12:03:45Z 2026-01-16T12:03:45Z
## 6 TRUE none 0 2025-10-30T04:06:28Z 2025-10-30T04:06:28Z
## id moderationStatus parentId
## 1 UgxX7YLU8nRrzwJG3Ix4AaABAg <NA> <NA>
## 2 UgxpjtpA9Occ7hpDzkl4AaABAg <NA> <NA>
## 3 Ugw45jv2GooFTMUtt7R4AaABAg <NA> <NA>
## 4 Ugy3Sj7EweUn0bji-MB4AaABAg <NA> <NA>
## 5 UgzzQwkhWdB6nwC1qzR4AaABAg <NA> <NA>
## 6 UgwdARmx-ctnhaeRtpB4AaABAg <NA> <NA>
##
## --- Tuber Metadata ---
## function: get_all_comments api_calls: 66 results_found: 1377 timestamp: 2026-06-25 20:23:47
## (Use tuber_info() for full metadata)glimpse(comments_raw)## Rows: 1,377
## Columns: 15
## $ authorDisplayName <chr> "@Heavenhellhaleema", "@ElectroIsElectro", "@chi…
## $ authorProfileImageUrl <chr> "https://yt3.ggpht.com/ytc/AIdro_m3k0tI4OZTkxWzl…
## $ authorChannelUrl <chr> "http://www.youtube.com/@Heavenhellhaleema", "ht…
## $ authorChannelId.value <chr> "UCDSXWFpcnrtkA4gvcWq7hKQ", "UC2C8yegEF0okucQ2Vq…
## $ videoId <chr> "XMANR-wrg7o", "XMANR-wrg7o", "XMANR-wrg7o", "XM…
## $ textDisplay <chr> "Reading None of this Rocks by Joe Trohman and f…
## $ textOriginal <chr> "Reading None of this Rocks by Joe Trohman and f…
## $ canRate <lgl> TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, …
## $ viewerRating <chr> "none", "none", "none", "none", "none", "none", …
## $ likeCount <dbl> 0, 1, 2, 1, 1, 0, 2, 1, 2, 0, 2, 0, 3, 0, 0, 1, …
## $ publishedAt <chr> "2026-06-03T02:44:27Z", "2026-03-31T10:03:55Z", …
## $ updatedAt <chr> "2026-06-03T02:44:27Z", "2026-03-31T10:03:55Z", …
## $ id <chr> "UgxX7YLU8nRrzwJG3Ix4AaABAg", "UgxpjtpA9Occ7hpDz…
## $ moderationStatus <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, …
## $ parentId <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, …Clean Version
comments_clean <- comments_raw %>%
as_tibble() %>%
distinct(id, .keep_all = TRUE) %>%
select(authorDisplayName, textOriginal, publishedAt, likeCount, id)
write_csv(comments_clean, "comments_clean.csv")Word Frequency Table
comments_words <- comments_clean %>%
select(textOriginal) %>%
unnest_tokens(word, textOriginal) %>%
anti_join(stop_words, by = "word") %>%
filter(!str_detect(word, "^[0-9]+$"))
word_counts <- comments_words %>%
count(word, sort = TRUE)
head(word_counts, 20)## # A tibble: 20 × 2
## word n
## <chr> <int>
## 1 video 164
## 2 band 125
## 3 music 118
## 4 scott 112
## 5 boy 110
## 6 fall 107
## 7 ian 90
## 8 song 89
## 9 love 86
## 10 time 85
## 11 rock 83
## 12 andy 79
## 13 awesome 79
## 14 anthrax 70
## 15 fob 70
## 16 lol 70
## 17 metal 67
## 18 fucking 62
## 19 joe 61
## 20 guys 56Bar Chart: Most Common Words
word_counts %>%
slice_max(n, n = 20) %>%
ggplot(aes(x = reorder(word, n), y = n)) +
geom_col() +
coord_flip() +
labs(
title = "Top 20 Most Common Words in YouTube Comments",
x = "Word",
y = "Frequency"
)
Word Cloud
set.seed(123)
bad_words <- c("fucking")
word_counts_clean <- word_counts %>%
filter(!word %in% bad_words)
wordcloud(
words = word_counts_clean$word,
freq = word_counts_clean$n,
max.words = 20,
scale = c(3, 0.6),
random.order = FALSE,
colors = brewer.pal(8, "Dark2")
)
Sentiment Analysis
# Break comments into individual words
comment_words <- comments_clean %>%
select(id, textOriginal) %>%
unnest_tokens(word, textOriginal)
# Join words with positive/negative sentiment dictionary
sentiment_words <- comment_words %>%
inner_join(get_sentiments("bing"), by = "word")
# Count positive and negative words
sentiment_counts <- sentiment_words %>%
count(sentiment)
sentiment_counts## # A tibble: 2 × 2
## sentiment n
## <chr> <int>
## 1 negative 888
## 2 positive 1134Sentiment Graph
sentiment_counts %>%
ggplot(aes(x = sentiment, y = n, fill = sentiment)) +
geom_col(show.legend = FALSE) +
labs(
title = "Sentiment of YouTube Comments",
x = "Sentiment",
y = "Word Count"
)
Discussion
Key Themes and Insights
It seems like the majority of people are discussing aspects of the video, albeit the over-the-top theatrics of it. The next thing that came up were the members of the band because it had members of Fall Out Boy, Anthrax, and Every Time I Die. However, due to personal conflicts with certain members in their respective bands, that’s where the comments turn negative. Of the comments scanned, they yielded results of 888 negative to 1134 positive.