1. Data collection

Posts about each general-election candidate, Xavier Becerra (D) and Steve Hilton (R), were collected from Bluesky and saved to one Excel file per candidate. Both files are read here without re-scraping, so every knit uses the same dataset and the results stay reproducible. Only the fields needed for analysis are kept, so account names and other author fields never enter the analysis. No usernames or individual posts are printed in this report.

becerra_file <- "Xavier_Becerra.xlsx"   # .xlsx or .csv
hilton_file  <- "Steve_Hilton.xlsx"     # (the Hilton workbook's second sheet is empty)

read_posts <- function(path, sheet = 1) {
  raw <- if (grepl("\\.csv$", path, ignore.case = TRUE)) {
    readr::read_csv(path, show_col_types = FALSE)
  } else {
    readxl::read_excel(path, sheet = sheet)
  }
  raw %>%
    select(uri, text, indexed_at,
           like_count, repost_count, reply_count, quote_count)   # no author fields
}

raw <- bind_rows(read_posts(becerra_file), read_posts(hilton_file))
nrow(raw)
## [1] 391

2. Data cleaning

The cleaning rules are:

  • Duplicates: the same post can appear in both files or in repeated pulls, so each post is kept once by its unique ID.
  • Posts naming both candidates: these cannot be assigned to one candidate without reading them, so they are dropped.
  • Same-name ambiguity: “Hilton” appears in the hotel brand, Paris Hilton, Hilton Head, and so on, so those posts are removed. Posts must also contain election context.
  • Lexicon bias: in the Bing lexicon, trump is coded positive. Since President Trump endorsed Hilton, every Trump mention would inflate positivity, so it is removed from the lexicon.
  • Non-English posts: Bing is English-only, so Spanish-language posts score neutral. This is a known limitation, especially for Becerra.
  • URLs, @mentions, and hashtags are stripped before tokenizing.
  • Time window: only posts after the June 2, 2026 primary are kept.
noise   <- "paris hilton|hilton hotel|hilton honors|hilton head|hilton garden|doubletree|hampton inn|perez hilton|conrad hilton"
context <- "xavier|steve|governor|\\bgov\\b|california|\\bca\\b|calif|newsom|election|vote|ballot|campaign|debate|sacramento|republican|democrat|gop"

step0 <- raw %>% filter(!is.na(text))
step1 <- step0 %>% distinct(uri, .keep_all = TRUE)
step2 <- step1 %>%
  mutate(text_l = str_to_lower(text),
         date   = as.Date(indexed_at),
         has_b  = str_detect(text_l, "becerra"),
         has_h  = str_detect(text_l, "hilton")) %>%
  filter(!str_detect(text_l, noise))
step3 <- step2 %>% filter(str_detect(text_l, context))
step4 <- step3 %>% filter(xor(has_b, has_h))               # exactly one candidate
step5 <- step4 %>% filter(date >= as.Date("2026-06-02"))   # after the primary

posts <- step5 %>%
  mutate(candidate  = if_else(has_b, "Becerra", "Hilton"),
         engagement = like_count + repost_count + reply_count + quote_count,
         post_id    = row_number(),
         clean_txt  = text_l %>%
           str_remove_all("https?://\\S+") %>%
           str_remove_all("@\\S+") %>%
           str_remove_all("#")) %>%
  select(post_id, candidate, date, clean_txt, text, engagement)

tibble(`Cleaning step` = c("Posts with text", "Unique posts", "Brand/name noise removed",
                           "Election context present", "Exactly one candidate named",
                           "Posted after June 2 primary"),
       Posts = c(nrow(step0), nrow(step1), nrow(step2), nrow(step3), nrow(step4), nrow(step5))) %>%
  kable(caption = "Posts remaining after each cleaning step")
Posts remaining after each cleaning step
Cleaning step Posts
Posts with text 378
Unique posts 335
Brand/name noise removed 334
Election context present 311
Exactly one candidate named 238
Posted after June 2 primary 238 
posts %>% count(candidate, name = "posts_after_cleaning")
candidate posts_after_cleaning
Becerra 140
Hilton 98

3. AI prompt (exact text used)

I am doing a marketing analysis by scraping Bluesky posts to forecast the 2026 California governor's race (Xavier Becerra vs. Steve Hilton). I have two Excel-compatible files, one per candidate, with post text, and engagement counts. Write one commented R Markdown report in R (tidyverse, readxl, tidytext) that imports both files, labels posts by candidate, and builds a `posts` tibble with no author fields. Remove duplicates, posts naming both candidates, unrelated  "Hilton" posts, and posts before the June 2, 2026 primary. Scores sentiment with the Bing lexicon: tokenize, remove stop words, drop "trump", "trumps", and "trumped" from the lexicon, compute net = positive - negative per post (keeping zero-match posts), and label each post positive, negative, or neutral. Summarizes by candidate: post volume, volume share, positive, negative, and neutral shares, net sentiment, mean net score, and mean engagement. Randomly selects 20 posts (seed 2026) for blind manual stance coding (support / oppose / neutral), then reports agreement accuracy with a 95% CI, Cohen's kappa, and a confusion matrix. Predicts the winner and vote share using positive share as a proxy, with volume share and net sentiment as comparisons.Runs 1,000 bootstrap resamples for 95% intervals and win probabilities. Creates kable tables and charts: tone by candidate, top sentiment words(aggregate counts), the bootstrap histogram, and a comparison of the three rules.

4. Sentiment scoring (tidytext + Bing)

Posts are tokenized into words, stop words are removed, and the remaining words are matched to the Bing lexicon. A post is positive if positive words outnumber negative words, negative if negative words outnumber positive, and neutral otherwise (including posts with no lexicon matches).

bing <- get_sentiments("bing") %>%
  filter(!word %in% c("trump", "trumps", "trumped"))

word_scores <- posts %>%
  unnest_tokens(word, clean_txt) %>%
  anti_join(stop_words, by = "word") %>%
  inner_join(bing, by = "word", relationship = "many-to-many") %>%
  count(post_id, sentiment) %>%
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0)

scored <- posts %>%
  left_join(word_scores, by = "post_id") %>%
  mutate(across(c(positive, negative), ~ coalesce(., 0)),
         net   = positive - negative,
         label = case_when(net > 0 ~ "positive", net < 0 ~ "negative", TRUE ~ "neutral"))

summary_tbl <- scored %>%
  group_by(candidate) %>%
  summarise(posts          = n(),
            pct_positive   = mean(label == "positive"),
            pct_negative   = mean(label == "negative"),
            pct_neutral    = mean(label == "neutral"),
            net_sentiment  = pct_positive - pct_negative,
            mean_net       = mean(net),
            mean_engagement = mean(engagement)) %>%
  mutate(volume_share = posts / sum(posts))

kable(summary_tbl, digits = 3,
      caption = "Post volume and sentiment by candidate (aggregate only)")
Post volume and sentiment by candidate (aggregate only)
candidate posts pct_positive pct_negative pct_neutral net_sentiment mean_net mean_engagement volume_share
Becerra 140 0.107 0.614 0.279 -0.507 -0.893 2.293 0.588
Hilton 98 0.194 0.439 0.367 -0.245 -0.459 1.908 0.412 
scored %>%
  count(candidate, label) %>%
  group_by(candidate) %>% mutate(share = n / sum(n)) %>%
  mutate(label = factor(label, levels = c("negative", "neutral", "positive"))) %>%
  ggplot(aes(candidate, share, fill = label)) +
  geom_col() + coord_flip() +
  geom_text(aes(label = percent(share, 1)), position = position_stack(vjust = 0.5),
            color = "white", size = 3.5) +
  scale_y_continuous(labels = percent) +
  scale_fill_manual(values = c(negative = "#B22222", neutral = "#9AA0A6", positive = "#2E8B57")) +
  labs(title = "Sentiment distribution by candidate", x = NULL, y = "Share of posts", fill = NULL)

# Aggregate word counts only (words used at least 5 times), no individual posts
posts %>%
  unnest_tokens(word, clean_txt) %>%
  anti_join(stop_words, by = "word") %>%
  inner_join(bing, by = "word", relationship = "many-to-many") %>%
  count(candidate, sentiment, word) %>%
  filter(n >= 5) %>%
  group_by(candidate, sentiment) %>% slice_max(n, n = 6, with_ties = FALSE) %>% ungroup() %>%
  ggplot(aes(reorder_within(word, n, list(candidate, sentiment)), n, fill = sentiment)) +
  geom_col(show.legend = FALSE) + coord_flip() + scale_x_reordered() +
  scale_fill_manual(values = c(positive = "#2E8B57", negative = "#B22222")) +
  facet_wrap(candidate ~ sentiment, scales = "free_y", ncol = 2) +
  labs(title = "Words driving the scores", x = NULL, y = "Count")

5. Validation: hand-labeled audit

Twenty random posts are labeled by hand for stance toward the candidate (support / oppose / neutral). Post text is labeled offline and is not printed here, to protect privacy. The file you label does not show the code’s label, so your coding is blind.

set.seed(2026)
audit <- scored %>% slice_sample(n = 20) %>%
  select(post_id, candidate, text) %>%
  mutate(my_stance = "")          # fill in: support / oppose / neutral

# Written once and never overwritten. Label it, then save as audit_labeled.csv.
if (!file.exists("audit_to_label.csv")) write_csv(audit, "audit_to_label.csv")

Steps: open audit_to_label.csv, fill my_stance with support, oppose, or neutral (the author’s position toward the candidate, not the mood of the words), save it as audit_labeled.csv, and knit again. Keep both files local and out of any published output.

stance_levels <- c("support", "oppose", "neutral")
validation_ready <- FALSE

if (file.exists("audit_labeled.csv")) {
  labeled <- read_csv("audit_labeled.csv", show_col_types = FALSE) %>%
    mutate(my_stance = str_to_lower(str_trim(my_stance))) %>%
    inner_join(scored %>% select(post_id, code_label = label), by = "post_id") %>%
    mutate(code_stance = recode(code_label, positive = "support",
                                negative = "oppose", neutral = "neutral"),
           match = code_stance == my_stance)

  validation_ready <- nrow(labeled) > 0 && all(labeled$my_stance %in% stance_levels)
}

if (validation_ready) {
  n_val    <- nrow(labeled)
  accuracy <- mean(labeled$match)
  acc_ci   <- binom.test(sum(labeled$match), n_val)$conf.int
  pe       <- sum(prop.table(table(factor(labeled$my_stance,  stance_levels))) *
                  prop.table(table(factor(labeled$code_stance, stance_levels))))
  kappa    <- (accuracy - pe) / (1 - pe)

  cat("Validation accuracy:", percent(accuracy), "on", n_val, "posts",
      "(95% CI", percent(acc_ci[1], 1), "to", percent(acc_ci[2], 1),
      "| Cohen's kappa", round(kappa, 2), ")\n")

  kable(table(Hand = labeled$my_stance, Code = labeled$code_stance),
        caption = "Confusion matrix (hand stance vs. code sentiment)")
} else {
  cat("Manual labeling is not finished yet: fill in `my_stance`, save as audit_labeled.csv, and knit again.\n")
}
## Manual labeling is not finished yet: fill in `my_stance`, save as audit_labeled.csv, and knit again.

6. Prediction and bootstrap

Three prediction rules each give Becerra’s share of the two-candidate vote:

  • Volume share: Becerra posts ÷ all posts
  • Positive share: Becerra positive posts ÷ all positive posts
  • Net sentiment: (Becerra positive + Hilton negative) ÷ all non-neutral posts

The project’s proxy is the positive-share rule, but all three are shown for comparison.

rule <- "positive"   # <- choose: "volume", "positive", or "net"

share_rules <- function(cand, lab) {
  nb <- sum(cand == "Becerra");  nh <- sum(cand == "Hilton")
  pb <- sum(cand == "Becerra" & lab == "positive"); ph <- sum(cand == "Hilton" & lab == "positive")
  gb <- sum(cand == "Becerra" & lab == "negative"); gh <- sum(cand == "Hilton" & lab == "negative")
  c(volume   = nb / (nb + nh),
    positive = pb / (pb + ph),
    net      = (pb + gh) / (pb + gb + ph + gh))
}

point  <- share_rules(scored$candidate, scored$label)
winner <- if_else(point[[rule]] > 0.5, "Becerra", "Hilton")
cat("Rule:", rule, "| Predicted winner:", winner,
    "| Becerra", percent(point[[rule]], 0.1), "- Hilton", percent(1 - point[[rule]], 0.1), "\n")
## Rule: positive | Predicted winner: Hilton | Becerra 44.1% - Hilton 55.9%

Bootstrap (1,000 resamples)

Posts are resampled with replacement and every rule is recomputed. Resampling all posts together (rather than within each candidate) lets post volume vary too, so the volume rule gets a real interval.

set.seed(2026)
n <- nrow(scored)
boots <- t(replicate(1000, {
  i <- sample.int(n, replace = TRUE)
  share_rules(scored$candidate[i], scored$label[i])
})) %>% as_tibble()

ci <- quantile(boots[[rule]], c(0.025, 0.975), na.rm = TRUE)
cat("95% bootstrap CI for Becerra share:", percent(ci, 0.1),
    "| P(Becerra > 50%):", percent(mean(boots[[rule]] > 0.5, na.rm = TRUE)), "\n")
## 95% bootstrap CI for Becerra share: 25.8% 60.0% | P(Becerra > 50%): 20%
rules_tbl <- map_dfr(names(point), function(r) {
  q <- quantile(boots[[r]], c(0.025, 0.975), na.rm = TRUE)
  tibble(Rule = r, `Becerra share` = percent(point[[r]], 0.1),
         `Hilton share` = percent(1 - point[[r]], 0.1),
         `95% CI (Becerra)` = paste0(percent(q[1], 0.1), " to ", percent(q[2], 0.1)),
         `P(Becerra > 50%)` = percent(mean(boots[[r]] > 0.5, na.rm = TRUE), 1),
         Winner = if_else(point[[r]] > 0.5, "Becerra", "Hilton"))
})
kable(rules_tbl, caption = "All three prediction rules with bootstrap uncertainty")
All three prediction rules with bootstrap uncertainty
Rule Becerra share Hilton share 95% CI (Becerra) P(Becerra > 50%) Winner
volume 58.8% 41.2% 52.1% to 65.1% 99% Becerra
positive 44.1% 55.9% 25.8% to 60.0% 20% Hilton
net 35.6% 64.4% 28.2% to 43.1% 0% Hilton 
ggplot(tibble(share = boots[[rule]]), aes(share)) +
  geom_histogram(bins = 40, fill = "#2C6E9B", color = "white") +
  geom_vline(xintercept = 0.5, linetype = "dashed") +
  geom_vline(xintercept = ci, color = "grey30") +
  scale_x_continuous(labels = percent) +
  labs(title = "Bootstrap distribution of Becerra's predicted vote share",
       subtitle = paste("Rule:", rule, "| dashed = 50%, solid = 95% interval"),
       x = "Becerra share", y = "Count")

boots %>% pivot_longer(everything(), names_to = "Rule", values_to = "share") %>%
  ggplot(aes(share, Rule)) +
  stat_summary(fun.data = function(x) data.frame(y = mean(x, na.rm = TRUE),
                                                 ymin = quantile(x, .025, na.rm = TRUE),
                                                 ymax = quantile(x, .975, na.rm = TRUE)),
               geom = "pointrange", orientation = "y") +
  geom_vline(xintercept = 0.5, linetype = "dashed") +
  scale_x_continuous(labels = percent) +
  labs(title = "Becerra's predicted share under each rule", x = "Becerra share (95% bootstrap interval)", y = NULL)

7. Reflection

Who is on Bluesky. Bluesky users are not a random sample of California voters. The platform skews younger and more politically left, partly because many users left X after its ownership change. This likely inflates positive talk about Becerra and negative talk about Hilton, so any Becerra lead in this forecast is probably overstated relative to the actual electorate. Older, rural, and Spanish-speaking voters are also underrepresented, and Spanish posts score neutral under the English-only Bing lexicon.

Sarcasm, tone vs. stance. Tone is the emotional color of the words; stance is where the author stands. My validation accuracy was [X%] on [N] posts. The most common error was [describe the most common error once the audit is labeled]. Negative words are often used to attack one candidate while supporting the other, which the code scores as negative, and sarcastic posts that use positive words to mock a candidate are read as support. A dictionary method counts words and cannot read intent.

Volume is not support. Becerra had 59% of posts, and those posts were mostly negative (11% positive, 61% negative). More posts can mean more criticism or a viral controversy rather than more votes. Volume measures attention, not support.

What the bootstrap captures. My predicted Becerra share was 44.1%, with a 95% bootstrap interval of 25.8% to 60.0%. That interval reflects sampling noise only. It does not correct for who is on Bluesky, lexicon errors, sarcasm, or non-English posts, so the true uncertainty is larger than the interval suggests.

Other limits. The dataset is a snapshot from the search API covering only a few days. Bots, reposts, and a small number of prolific accounts can inflate counts, and dictionary methods are less accurate than context-aware models. The predicted share is a descriptive indicator of relative positive sentiment on Bluesky, not a forecast of the election result.