The AI was genuinely useful for the mechanical work. It wrote working
tidytext code for tokenizing, joining three sentiment
lexicons, and building the plotly chart much faster than I
could have, and the verification checks all passed (for example, my hand
calculation of the Wildwood review matched the code exactly:
Bing +5, AFINN +11). But “the code runs” turned out to be different from
“the results are right.” The automated tools made several mistakes I
only caught by reading the posts myself. The language detector labeled
an English list of Godzilla titles as Polish and missed a French post.
The political keyword filter removed posts that were really about movies
(for example, an “Election Day Movies” noir list). The lexicons scored
the filler word “like” as positive (it was Bing’s #1 positive word),
read horror words like “bloody” and “evil” as negative even in fan
posts, gave a glowing review an NRC score of 0, and missed sarcasm like
“the movies themselves? sooo bad lmao.” In my 8-post spot check, each
lexicon matched my reading only 3 times, and the three lexicons agreed
with each other on just 42% of posts.
Human judgment still decided most of what mattered. I had to define what counted as spam or off-topic, which no package does for you, and decide that timestamps with problems should be flagged instead of deleted. I also had to notice the CSV was saved in the wrong encoding and recognize that the original “engagement” column left out replies. Most importantly, I had to interpret the results honestly. The chart shows no real relationship between sentiment and engagement (ρ ≈ 0.02), and a 1-hour, 153-post sample from Election Day night isn’t enough to make strong claims about movie audiences. If I had trusted the AI output without checking it, I would have reported cleaner-looking but less accurate findings. The AI is a fast first draft. The analyst is still responsible for knowing the data, questioning the method, and deciding what the numbers actually mean for a marketing decision.
AI disclosure: R code was drafted with Claude (Anthropic) from my prompts, then run, checked, and edited by me. Prompts and responses are posted to the discussion forum.
This chunk installs any missing packages automatically, so nothing has to be run in the Console first.
pkgs <- c("tidyverse", "tidytext", "textdata", "plotly", "cld2", "scales")
missing_pkgs <- pkgs[!pkgs %in% rownames(installed.packages())]
if (length(missing_pkgs) > 0) {
install.packages(missing_pkgs, repos = "https://cloud.r-project.org")
}
library(tidyverse) # data cleaning (dplyr, stringr, readr) + ggplot2
library(tidytext) # tokenizing, stop words, sentiment lexicons
library(textdata) # AFINN and NRC lexicon downloads
library(plotly) # interactive chart
library(cld2) # language detection
Loading the lexicons. Bing comes built into
tidytext. AFINN and NRC come from textdata,
which normally asks a Yes/No question before downloading. A knit can’t
answer that question, so it stops with an error. The function below
downloads each lexicon’s official file (the same source
textdata uses), lets textdata process it with
no prompt, and saves it so later knits skip the download. If the
official site is down, it falls back to a public GitHub copy of the same
lexicon version.
load_lexicon <- function(name) {
folder <- if (name == "afinn") lexicon_afinn(return_path = TRUE) else lexicon_nrc(return_path = TRUE)
rds <- file.path(folder, if (name == "afinn") "afinn_111.rds" else "NRCWordEmotion.rds")
if (file.exists(rds)) return(read_rds(rds)) # already downloaded on an earlier knit
dir.create(folder, recursive = TRUE, showWarnings = FALSE)
official <- tryCatch({
if (name == "afinn") {
download.file("http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/6010/zip/imm6010.zip",
file.path(folder, "imm6010.zip"), mode = "wb", quiet = TRUE)
lexicon_afinn(manual_download = TRUE) # process the file, no Yes/No prompt
} else {
download.file("http://saifmohammad.com/WebDocs/Lexicons/NRC-Emotion-Lexicon.zip",
file.path(folder, "NRC-Emotion-Lexicon.zip"), mode = "wb", quiet = TRUE)
unzip(file.path(folder, "NRC-Emotion-Lexicon.zip"), exdir = folder)
lexicon_nrc(manual_download = TRUE)
}
}, error = function(e) NULL)
if (!is.null(official)) return(official)
# Fallback: public GitHub copies of the same versions (AFINN-111, NRC v0.92)
message("Official ", name, " download failed; using GitHub copy.")
if (name == "afinn") {
lex <- read_tsv("https://raw.githubusercontent.com/fnielsen/afinn/master/afinn/data/AFINN-111.txt",
col_names = c("word", "value"), col_types = "cd", quote = "")
} else {
lex <- read_tsv("https://raw.githubusercontent.com/dinbav/LeXmo/master/NRC-Emotion-Lexicon-Wordlevel-v0.92.txt",
col_names = c("word", "sentiment", "flag"), col_types = "ccd") %>%
filter(flag == 1) %>% select(word, sentiment)
}
write_rds(lex, rds)
lex
}
The CSV was saved from Excel, so it uses
Windows-1252 encoding. Reading it as UTF-8 (the
default) turns apostrophes and accents into �. Emojis were
already lost before export: they show up as ? and can’t be
recovered.
raw <- read_csv("Gian Abundo -Scrape Movies 11-3(Sheet1)(in).csv",
locale = locale(encoding = "windows-1252"),
show_col_types = FALSE) %>%
mutate(post_id = row_number())
glimpse(raw)
## Rows: 200
## Columns: 9
## $ text <chr> "www.peoplesworld.org/article/desp...", "Alors, il y a d…
## $ author_handle <chr> "charleskeener.bsky.social", "lecheikh.bsky.social", "tu…
## $ created_at <chr> "11/3/2026 23:30", "11/3/2026 23:30", "11/3/2026 23:30",…
## $ like_count <dbl> 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,…
## $ repost_count <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
## $ reply_count <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,…
## $ labels <lgl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, …
## $ engagement <dbl> 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,…
## $ post_id <int> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 1…
Each step is logged so the number of posts removed at every stage can be checked (see Section 6).
log_step <- function(log, step, data) {
bind_rows(log, tibble(step = step, posts_remaining = nrow(data)))
}
cleaning_log <- log_step(tibble(), "0. Raw scrape", raw)
posts <- raw %>%
mutate(
text = replace_na(text, ""),
nsfw = str_detect(str_to_lower(text), "nsfw"), # flag before hashtags are stripped
clean_text = text %>%
str_replace_all("[\u2018\u2019]", "'") %>% # curly -> straight apostrophes
str_replace_all("[\u201C\u201D]", '"') %>%
str_replace_all("[\u2013\u2014]", " - ") %>% # en/em dashes
str_replace_all("\u2026", " ") %>% # ellipsis character
str_replace_all(""", '"') %>%
str_replace_all("@[\\w.-]+", " ") %>% # @mentions (before links, so handles go too)
str_replace_all("https?://\\S+|www\\.\\S+", " ") %>% # full URLs
str_replace_all("\\b[a-z0-9.-]+\\.(com|org|net|co|to|in|uk|be|social|app|io|online|jp|us|gy)(/\\S*)?", " ") %>% # bare links (youtu.be/..., wix.to/...)
str_replace_all("#\\S+", " ") %>% # hashtags (mostly #movies, #filmsky tags)
str_replace_all("\\?{2,}|(^|\\s)\\?+", " ") %>% # emoji placeholders: "??" or "?" at the start of a word
str_squish(),
n_words = str_count(clean_text, "[A-Za-z']+"),
# Original 'engagement' column = likes + reposts only (replies left out)
engagement_total = like_count + repost_count + reply_count,
created_at = mdy_hm(created_at),
time_flag = case_when(
is.na(created_at) ~ "missing timestamp",
created_at > mdy_hm("11/3/2026 23:30") ~ "outside scrape window", # scrape ran 22:35-23:30
TRUE ~ "ok")
)
count(posts, time_flag)
Timestamps are flagged, not removed, because the chart doesn’t use time. Rows with missing or out-of-window timestamps still have usable text and engagement.
posts <- posts %>% filter(n_words >= 3)
cleaning_log <- log_step(cleaning_log, "1. Remove empty / link-only / <3-word posts", posts)
cld2 detects language automatically, but it made
two mistakes on this data. It labeled an English list of
Godzilla titles as Polish, and it returned no language (NA) for a French
post. Both are corrected below with a documented manual review.
posts <- posts %>% mutate(lang = detect_language(clean_text))
# What cld2 did NOT call English, for human review
posts %>% filter(is.na(lang) | lang != "en") %>%
select(author_handle, lang, clean_text)
english_override <- c("sethhyneswriter.bsky.social") # list of Godzilla titles; cld2 guessed Polish
non_english_manual <- c("lecheikh.bsky.social") # French; cld2 returned NA
posts <- posts %>%
filter((lang == "en" | is.na(lang) | author_handle %in% english_override),
!author_handle %in% non_english_manual)
cleaning_log <- log_step(cleaning_log, "2. Remove non-English posts", posts)
These accounts post automated news links, ads, job listings, or self-promotion rather than opinions about movies. They were chosen by reading their posts, not by a rule.
spam_accounts <- tribble(
~author_handle, ~reason,
"justpastemovies.bsky.social", "Automated news-link reposts (4 posts)",
"jazzcomposer.bsky.social", "Music promo, off-topic (3 identical posts)",
"bot-tan.com", "Bot account",
"nowbreezing.ntw.app", "Trending-words bot",
"emme-gray.bsky.social", "Art shop promotion",
"nearskill.bsky.social", "Job ad",
"videogamedepot.bsky.social", "Store ad",
"hollywoodmetrics.bsky.social", "Product promotion",
"plot-hole-game.bsky.social", "Game promotion",
"classicfilms.app", "Automated anniversary post",
"florestario.bsky.social", "Amazon link promo (non-English)"
)
spam_accounts
posts <- posts %>% anti_join(spam_accounts, by = "author_handle")
cleaning_log <- log_step(cleaning_log, "3. Remove spam / bot / promo accounts", posts)
# Exact duplicate text after cleaning (cross-posts, copy-paste trend posts)
posts <- posts %>% distinct(clean_text, .keep_all = TRUE)
cleaning_log <- log_step(cleaning_log, "4. Remove duplicate posts", posts)
The scrape ran on Election Day (Nov 3), so many posts talk about politics and mention movies only in passing. A keyword filter removes those posts.
political_terms <- c("election", "elect", "trump", "maga", "nazi", "nazis", "republican",
"democrat", "democracy", "senate", "midterm", "vote", "voting",
"voters", "polls", "gop", "gov", "governor", "politic", "political",
"kornacki")
political_regex <- paste0("\\b(", paste(political_terms, collapse = "|"), ")\\b")
removed_political <- posts %>% filter(str_detect(str_to_lower(clean_text), political_regex))
removed_political %>% select(author_handle, clean_text) # review what the filter caught
# Human review: the keyword filter caught 2 posts that are really ABOUT movies -> keep them
political_keep <- c("hmuncut.bsky.social", # film news: thriller about "political persecution"
"semetrarhodes.bsky.social") # "Election Day Movies" noir recommendation
posts <- posts %>%
filter(!str_detect(str_to_lower(clean_text), political_regex) | author_handle %in% political_keep,
!nsfw)
cleaning_log <- log_step(cleaning_log, "5. Remove political / off-topic / NSFW posts", posts)
cleaning_log
Final clean dataset: 153 posts (from 200 raw).
Standard stop words are removed, plus the search keyword (“movies”/“movie”), which appears in nearly every post by design and would otherwise top the list.
tokens <- posts %>%
select(post_id, author_handle, clean_text, engagement_total) %>%
unnest_tokens(word, clean_text) # lowercase, strip punctuation
custom_stop <- tibble(word = c("movies", "movie", "film", "films", "it's", "i'm", "don't"))
top_words <- tokens %>%
anti_join(stop_words, by = "word") %>%
anti_join(custom_stop, by = "word") %>%
filter(!str_detect(word, "^\\d+$")) %>% # drop bare numbers
count(word, sort = TRUE)
top_words %>%
slice_max(n, n = 15, with_ties = FALSE) %>%
ggplot(aes(x = n, y = reorder(word, n))) +
geom_col(fill = "#2C7FB8") +
labs(title = "Top 15 Words in BlueSky 'Movies' Posts (Nov 3, 2026)",
x = "Count", y = NULL) +
theme_minimal()
bigrams <- posts %>%
unnest_tokens(bigram, clean_text, token = "ngrams", n = 2) %>%
filter(!is.na(bigram)) %>%
separate(bigram, into = c("w1", "w2"), sep = " ") %>%
filter(!w1 %in% stop_words$word, !w2 %in% stop_words$word) %>%
unite(bigram, w1, w2, sep = " ") %>%
count(bigram, sort = TRUE)
head(bigrams, 10)
| Lexicon | What it gives | Post score used here |
|---|---|---|
| Bing | Each word = positive or negative | # positive − # negative words |
| AFINN | Each word = integer from −5 to +5 | Sum of word values |
| NRC | Each word = 8 emotions + positive/negative | Emotion counts; # positive − # negative |
Stop words are not removed before sentiment scoring, so words like “love” or “bad” are never dropped by accident. A post with no lexicon words gets a score of 0 (neutral).
bing <- get_sentiments("bing") # built into tidytext
afinn <- load_lexicon("afinn") # 2,477 words, scored -5 to +5
nrc <- load_lexicon("nrc") # ~14,000 word-emotion pairs
bing_scores <- tokens %>%
inner_join(bing, by = "word") %>%
count(post_id, sentiment) %>%
pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
transmute(post_id, bing_score = positive - negative)
afinn_scores <- tokens %>%
inner_join(afinn, by = "word") %>%
group_by(post_id) %>%
summarise(afinn_score = sum(value), .groups = "drop")
nrc_scores <- tokens %>%
inner_join(filter(nrc, sentiment %in% c("positive", "negative")), by = "word",
relationship = "many-to-many") %>%
count(post_id, sentiment) %>%
pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
transmute(post_id, nrc_score = positive - negative)
label <- function(x) case_when(x > 0 ~ "Positive", x < 0 ~ "Negative", TRUE ~ "Neutral")
scored <- posts %>%
select(post_id, author_handle, clean_text, like_count, repost_count,
reply_count, engagement_total) %>%
left_join(bing_scores, by = "post_id") %>%
left_join(afinn_scores, by = "post_id") %>%
left_join(nrc_scores, by = "post_id") %>%
mutate(across(c(bing_score, afinn_score, nrc_score), ~ replace_na(.x, 0)),
bing_label = label(bing_score),
afinn_label = label(afinn_score),
nrc_label = label(nrc_score))
scored %>%
pivot_longer(c(bing_label, afinn_label, nrc_label),
names_to = "lexicon", values_to = "label") %>%
count(lexicon, label) %>%
pivot_wider(names_from = label, values_from = n, values_fill = 0)
agreement <- scored %>%
summarise(
bing_vs_afinn = mean(bing_label == afinn_label),
bing_vs_nrc = mean(bing_label == nrc_label),
afinn_vs_nrc = mean(afinn_label == nrc_label),
all_three = mean(bing_label == afinn_label & afinn_label == nrc_label)
) %>%
mutate(across(everything(), scales::percent))
agreement
# Posts where at least one lexicon disagrees with the others
scored %>%
filter(!(bing_label == afinn_label & afinn_label == nrc_label)) %>%
select(clean_text, bing_score, afinn_score, nrc_score) %>%
mutate(clean_text = str_trunc(clean_text, 110))
nrc_emotions <- tokens %>%
inner_join(filter(nrc, !sentiment %in% c("positive", "negative")), by = "word",
relationship = "many-to-many") %>%
count(sentiment, sort = TRUE)
ggplot(nrc_emotions, aes(x = n, y = reorder(sentiment, n))) +
geom_col(fill = "#41AB5D") +
labs(title = "NRC Emotion Word Counts", x = "Word count", y = NULL) +
theme_minimal()
# Which words drive the top emotions? (useful for verification)
tokens %>%
inner_join(filter(nrc, sentiment %in% c("fear", "anger")), by = "word",
relationship = "many-to-many") %>%
count(sentiment, word, sort = TRUE) %>%
group_by(sentiment) %>% slice_head(n = 6)
tokens %>%
inner_join(bing, by = "word") %>%
count(sentiment, word, sort = TRUE) %>%
group_by(sentiment) %>% slice_head(n = 8) %>%
ggplot(aes(x = n, y = reorder(word, n), fill = sentiment)) +
geom_col(show.legend = FALSE) +
facet_wrap(~ sentiment, scales = "free_y") +
scale_fill_manual(values = c(negative = "#D7301F", positive = "#2C7FB8")) +
labs(title = "Top Words Contributing to Bing Sentiment", x = "Count", y = NULL) +
theme_minimal()
Bing and AFINN both score “like” as positive (Bing +1, AFINN +2). In these posts it’s almost always filler or a comparison (“flew by like it was nothing”, “Like im going to be fr”), not “I like this.” It’s Bing’s most common positive word here, so it inflates positive sentiment. Re-scoring Bing without it:
bing_no_like <- tokens %>%
filter(word != "like") %>%
inner_join(bing, by = "word") %>%
count(post_id, sentiment) %>%
pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
transmute(post_id, bing_no_like = positive - negative)
like_check <- scored %>%
left_join(bing_no_like, by = "post_id") %>%
mutate(bing_no_like = replace_na(bing_no_like, 0),
label_no_like = label(bing_no_like))
like_check %>% count(original = bing_label, without_like = label_no_like)
like_changed <- sum(like_check$bing_label != like_check$label_no_like)
Dropping one word changes the Bing label of 13 posts (8%). A lexicon can’t tell what a word means in context; a human reader can.
checks <- tibble(
check = c(
"Original 'engagement' = likes + reposts (replies excluded)",
"No duplicate post IDs after cleaning",
"Every clean post has a score (no NA) in all 3 lexicons",
"AFINN word values are all between -5 and +5",
"Scored rows = clean rows (joins didn't add/drop posts)",
"No spam accounts remain"
),
passed = c(
all(raw$engagement == raw$like_count + raw$repost_count),
!any(duplicated(scored$post_id)),
!anyNA(select(scored, bing_score, afinn_score, nrc_score)),
all(between(afinn$value, -5, 5)),
nrow(scored) == nrow(posts),
!any(scored$author_handle %in% spam_accounts$author_handle)
)
)
checks
stopifnot(all(checks$passed))
# Share of posts with zero lexicon matches (scored "Neutral" by default)
scored %>% summarise(
no_bing_words = scales::percent(mean(!post_id %in% bing_scores$post_id)),
no_afinn_words = scales::percent(mean(!post_id %in% afinn_scores$post_id)))
Post checked by hand: evethepuppy’s review of Wildwood.
hand_post <- scored %>% filter(author_handle == "evethepuppy.bsky.social")
hand_post$clean_text
## [1] "70 - Wildwood (2026) 9/10 Absolutely incredible film. Two hours flew by like it was nothing. One of those movies where you're constantly left wondering how every bit of it was made. Great voice acting, great story, great everything. A must-see in theaters. Honestly it might be a 10/10 someday."
hand_tokens <- tokens %>% filter(post_id == hand_post$post_id)
# Every word in the post that is in each lexicon
hand_tokens %>% inner_join(bing, by = "word") %>% count(word, sentiment)
hand_tokens %>% inner_join(afinn, by = "word") %>% count(word, value) %>%
mutate(subtotal = n * value)
Working it out by hand (reading the post word by word against each lexicon):
| Word | Bing | AFINN |
|---|---|---|
| great (×3) | +1 ×3 = +3 | +3 ×3 = +9 |
| incredible | +1 | not in AFINN (0) |
| like (“flew by like it was nothing”) | +1 | +2 |
| Total | +5 | +11 |
The code’s scores match the hand calculation (Bing = 5, AFINN = 11), so the joins and sums are working correctly. But the hand check also shows two problems the code can’t see. “Like” adds points even though it isn’t positive here; without it, the true scores would be Bing +4 and AFINN +9. And NRC scores this glowing review 0, because “great” and “incredible” aren’t in its positive list. Words like “must-see” and “10/10” aren’t in any of the three lexicons.
stopifnot(hand_post$bing_score == 5,
hand_post$afinn_score == 11)
"Hand calculation matches code output"
## [1] "Hand calculation matches code output"
set.seed(4000)
spot <- scored %>% slice_sample(n = 8) %>%
select(post_id, clean_text, bing_label, afinn_label, nrc_label)
# My human judgment of each post's sentiment, after reading the full text
# (joined by post_id, so each label stays with the right post)
human_labels <- tribble(
~post_id, ~human_label, ~why,
16, "Neutral", "Listicle headline - no opinion",
101, "Positive", "Praises middle films of trilogies",
51, "Neutral", "Data / informational post",
19, "Negative", "'the movies themselves? sooo bad' - sarcasm missed by all 3",
120, "Positive", "Excited for Godzilla Minus Zero (no sentiment words)",
94, "Neutral", "Recommendation; 'Love', 'Success' are in a film TITLE",
91, "Positive", "Calls Deja Vu the best new-to-me movie",
121, "Positive", "'Masterpiece! One of the best movies ever'"
)
spot <- spot %>% left_join(human_labels, by = "post_id")
spot %>% mutate(clean_text = str_trunc(clean_text, 140)) %>%
select(clean_text, human_label, bing_label, afinn_label, nrc_label, why)
spot %>% summarise(
bing_correct = sum(bing_label == human_label),
afinn_correct = sum(afinn_label == human_label),
nrc_correct = sum(nrc_label == human_label),
out_of = n())
Hover over a point to see the post. Engagement = likes + reposts + replies.
plot_data <- scored %>%
mutate(hover = paste0("<b>@", author_handle, "</b><br>",
str_wrap(str_trunc(clean_text, 220), 60) %>% str_replace_all("\n", "<br>"),
"<br><br>AFINN: ", afinn_score, " | Bing: ", bing_score,
"<br>Likes: ", like_count, " | Reposts: ", repost_count,
" | Replies: ", reply_count),
bing_label = factor(bing_label, levels = c("Positive", "Neutral", "Negative")))
plot_ly(plot_data,
x = ~afinn_score, y = ~engagement_total,
color = ~bing_label,
colors = c(Positive = "#2C7FB8", Neutral = "#969696", Negative = "#D7301F"),
type = "scatter", mode = "markers",
marker = list(size = 9, opacity = 0.7, line = list(width = 0.5, color = "white")),
text = ~hover, hoverinfo = "text") %>%
layout(title = list(text = "Sentiment vs. Engagement: BlueSky 'Movies' Posts (Nov 3, 2026)",
x = 0.02, xanchor = "left"),
margin = list(t = 70),
xaxis = list(title = "AFINN sentiment score (negative <-> positive)", zeroline = TRUE),
yaxis = list(title = "Total engagement (likes + reposts + replies)"),
legend = list(title = list(text = "Bing label")))
# Spearman (rank) correlation: engagement is heavily skewed, so Pearson would be driven by a few posts
cor.test(scored$afinn_score, scored$engagement_total, method = "spearman", exact = FALSE)
##
## Spearman's rank correlation rho
##
## data: scored$afinn_score and scored$engagement_total
## S = 582549, p-value = 0.7679
## alternative hypothesis: true rho is not equal to 0
## sample estimates:
## rho
## 0.02404934
scored %>%
group_by(bing_label) %>%
summarise(posts = n(),
mean_engagement = round(mean(engagement_total), 1),
median_engagement = median(engagement_total),
.groups = "drop")
Limitations: single ~1-hour scrape (22:35-23:30 on Election Day), n = 153 after cleaning, emojis lost in export, and engagement counts captured at scrape time (recent posts had less time to collect likes).