1. Overview

Question. Can public Bluesky posts forecast the winner of the November 3, 2026 California governor’s race between Xavier Becerra (D) and Steve Hilton (R)?

Approach. Collect public posts, clean them (including removing Hilton Hotel and Paris Hilton posts), score each post with the tidytext Bing lexicon, summarize by candidate, apply a prediction rule chosen before looking at results, and bootstrap the result. I then validate the code against my own hand labels and reflect on the limits.

Privacy. Everything reported here is aggregate. No usernames, post text, author identifiers, or profile links are published. The scraper never stores handles; it stores only a one-way scrambled (hashed) author code, used solely so the bootstrap can treat each account as one unit.

The headline results are collected in Section 8 (Bottom line), after the methods.

2. Data collection

I collected the posts with a separate R script (run once, before knitting) that logs in to Bluesky with an app password typed into a prompt, searches the API (app.bsky.feed.searchPosts) for each query term, English only, sorted by latest, pages through the results, and saves them to data/raw_posts.csv. The script never saves handles; it stores only a one-way hashed account code. This report reads that saved file. The chunk below is the equivalent collection code and runs only if no saved file exists.

Things to keep in mind about this data:

  • It is a convenience sample of searchable posts, not a random sample of Californians or a census of Bluesky.
  • A search hit can come from linked previews or attached content, so a returned post does not necessarily name the candidate in its own text. Cleaning handles this below.
  • The same post can be returned by more than one query; posts are de-duplicated by ID.
  • Each query has a result cap per run, and “latest” sorting means the most recent days can be over-represented.
bsky_login <- function() {
  request("https://bsky.social/xrpc/com.atproto.server.createSession") |>
    req_body_json(list(identifier = Sys.getenv("BSKY_HANDLE"),
                       password   = Sys.getenv("BSKY_APP_PASSWORD"))) |>
    req_perform() |>
    resp_body_json() |>
    pluck("accessJwt")
}

search_posts <- function(q, jwt, since, until, max_pages = 30) {
  out <- list(); cursor <- NULL
  for (i in seq_len(max_pages)) {
    res <- request("https://bsky.social/xrpc/app.bsky.feed.searchPosts") |>
      req_auth_bearer_token(jwt) |>
      req_url_query(q = q, since = since, until = until, lang = "en",
                    limit = 100, sort = "latest", cursor = cursor) |>
      req_retry(max_tries = 3) |>
      req_perform() |>
      resp_body_json()
    if (length(res$posts) == 0) break
    out <- c(out, lapply(res$posts, function(p)
      tibble(uri        = p$uri,
             # one-way hash of the account ID: no handle is ever stored
             author_id  = rlang::hash(paste0("ca-gov-lab-2026-", p$author$did)),
             created_at = p$record$createdAt %||% NA_character_,
             text       = p$record$text %||% NA_character_,
             query      = q)))
    cursor <- res$cursor
    if (is.null(cursor)) break
    Sys.sleep(0.5)
  }
  bind_rows(out)
}

jwt <- bsky_login()
raw <- map_dfr(queries, ~ search_posts(.x, jwt, since, until))
write_csv(raw, raw_path)
raw_in <- read_csv(raw_path, show_col_types = FALSE) |>
  rename_with(~ str_to_lower(str_replace_all(.x, "[^A-Za-z0-9]+", "_")))

# Accept exports from the earlier lab / the atrrr package, which name columns differently
first_col <- function(df, candidates) {
  nm <- intersect(candidates, names(df))
  if (length(nm)) as.character(df[[nm[1]]]) else rep(NA_character_, nrow(df))
}
stopifnot("The data must have a text column (`text` or `post_text`)" =
            any(c("text", "post_text") %in% names(raw_in)))

raw <- tibble(
  text       = first_col(raw_in, c("text", "post_text")),
  uri        = first_col(raw_in, c("uri", "post_id")),
  created_at = first_col(raw_in, c("created_at", "createdat", "indexed_at",
                                   "timestamp_original", "timestamp", "created")),
  query      = first_col(raw_in, "query"),
  author_id  = first_col(raw_in, "author_id")
)
raw$uri <- coalesce(raw$uri, as.character(seq_len(nrow(raw))))

# Anonymous author code: use the supplied one, else hash the account ID (DID) found in the post URI.
# Handles and display names are discarded here and never used again.
if (all(is.na(raw$author_id))) {
  did <- str_extract(raw$uri, "did:[a-z]+:[A-Za-z0-9._%-]+")
  raw$author_id <- map_chr(did, ~ if (is.na(.x)) NA_character_
                                  else rlang::hash(paste0("ca-gov-lab-2026-", .x)))
}
author_available <- any(!is.na(raw$author_id))
if (!author_available) raw$author_id <- raw$uri   # last resort: each post is its own "author"
raw$author_id <- coalesce(raw$author_id, raw$uri)
rm(raw_in)

3. Data cleaning and audit

3.1 What the AI said to check

I asked an AI assistant: “I scraped Bluesky posts about Xavier Becerra and Steve Hilton for the California governor’s race. Besides removing Hilton Hotel / Paris Hilton posts, what other cleaning issues should I handle?” Its list, and what I did about each item:

Issue flagged Why it matters What I did
Hilton Hotels / Paris Hilton and relatives / Hilton Head / Hilton Honors Brand and celebrity posts are not about the candidate Regex exclusion (below)
Other people named Becerra, or “Hilton” without “Steve” Surname-only searches pull unrelated posts Required political-context words or the full name “Steve Hilton”
Same post returned by several queries Double counting inflates volume De-duplicated by post ID
Copy-paste / bot-like duplicate text One message repeated many times looks like many supporters Dropped exact duplicate text (kept one)
Posts mentioning both candidates Sentiment cannot be attributed to one candidate Excluded from the candidate comparison
URLs, @mentions, hashtags, HTML entities Noise for tokenizing; @mentions contain usernames Stripped before analysis
Very short posts Too few words to score Dropped posts under 4 words
Non-English posts The Bing lexicon is English-only Requested English in the search
Missing or out-of-window dates Keeps the sample on the campaign period Filtered on timestamp (when dates are available)
Lexicon traps (e.g., “trump” is coded positive in Bing) Would add false positive sentiment to political posts Removed from the lexicon
One account posting many times Heavy posters can dominate counts Bootstrap resamples accounts, not posts

3.2 Cleaning pipeline

hotel_pat <- regex(paste(
  "paris hilton", "nicky hilton", "kathy hilton", "perez hilton", "conrad hilton",
  "hilton hotel", "hilton worldwide", "hilton honors", "hilton head", "hilton garden",
  "hilton (inn|suites|resort|grand vacations)", "hampton (inn )?by hilton",
  "doubletree", "waldorf astoria", "hilton (hotels?|stay|lobby|bar|buffet|breakfast)",
  sep = "|"), ignore_case = TRUE)

ctx_pat <- regex(paste(
  "xavier", "governor", "gubernatorial", "\\bgov\\b", "california", "\\bca\\b", "election",
  "\\bvote", "ballot", "\\bhhs\\b", "attorney general", "democrat", "republican", "\\bgop\\b",
  "newsom", "campaign", "debate", "\\bprop\\b", "bianco", "fox news", "trump", "sacramento",
  sep = "|"), ignore_case = TRUE)

becerra_pat   <- regex("becerra", ignore_case = TRUE)
hilton_full   <- regex("steve hilton|stephen hilton", ignore_case = TRUE)
hilton_single <- regex("\\bhilton\\b", ignore_case = TRUE)

clean_text <- function(x) {
  x |>
    str_replace_all("https?://\\S+", " ") |>       # URLs
    str_replace_all("\\b[a-z0-9-]+\\.(com|org|net|co|io|us)\\S*", " ") |>  # bare domains
    str_replace_all("@[A-Za-z0-9._-]+", " ") |>    # @mentions (contain usernames)
    str_replace_all("&amp;", "&") |>
    str_replace_all("#", " ") |>
    str_squish()
}

log_step <- function(a, label, df)
  add_row(a, step = label, remaining = nrow(df),
          removed = a$remaining[nrow(a)] - nrow(df))

audit <- tibble(step = "Raw posts collected", remaining = nrow(raw), removed = NA_integer_)

d <- raw |> filter(!is.na(text), str_squish(text) != "")
audit <- log_step(audit, "Dropped empty or missing text", d)

d <- d |> distinct(uri, .keep_all = TRUE)
audit <- log_step(audit, "Dropped duplicate post IDs (same post returned by multiple queries)", d)

d <- d |> mutate(text_clean = clean_text(text))

d <- d |> distinct(text_key = str_to_lower(text_clean), .keep_all = TRUE) |> select(-text_key)
audit <- log_step(audit, "Dropped exact duplicate text (reposts, copy-paste, bot-like)", d)

d <- d |> filter(!str_detect(text, hotel_pat))
audit <- log_step(audit, "Dropped Hilton Hotel / Paris Hilton / other Hilton-brand posts", d)

# Candidate and date flags (computed before filtering so they can be reported)
d <- d |>
  mutate(created = coalesce(ymd_hms(created_at, quiet = TRUE), ymd(created_at, quiet = TRUE)),
         is_b = str_detect(text_clean, becerra_pat) & str_detect(text_clean, ctx_pat),
         is_h = str_detect(text_clean, hilton_full) |
                (str_detect(text_clean, hilton_single) & str_detect(text_clean, ctx_pat)),
         short_post = str_count(text_clean, "\\S+") < 4)

flags <- tibble(
  Flag = c("Rows entering the flag check",
           "Missing or unparseable date",
           "Not confidently about either candidate",
           "Names both candidates",
           "Under 4 words after cleaning"),
  Posts = c(nrow(d),
            sum(is.na(d$created)),
            sum(!d$is_b & !d$is_h),
            sum(d$is_b & d$is_h),
            sum(d$short_post)))

dates_available <- any(!is.na(d$created))
if (dates_available) {
  d <- d |> filter(!is.na(created), created >= ymd_hms(since), created < ymd_hms(until))
  audit <- log_step(audit, "Dropped posts with missing dates or outside the date window", d)
}

d_about <- d |> filter(is_b | is_h)
audit <- log_step(audit, "Dropped posts not confidently about either candidate", d_about)

d_one <- d_about |> filter(xor(is_b, is_h))
audit <- log_step(audit, "Dropped posts mentioning both candidates", d_one)

posts <- d_one |>
  filter(!short_post) |>
  mutate(candidate = if_else(is_b, "Becerra", "Hilton"),
         post_id = row_number()) |>
  select(post_id, uri, author_id, candidate, created, text, text_clean)
audit <- log_step(audit, "Dropped posts under 4 words", posts)

kable(audit, col.names = c("Cleaning step", "Posts remaining", "Posts removed"),
      caption = "Cleaning audit: posts remaining and removed at each step")
Cleaning audit: posts remaining and removed at each step
Cleaning step Posts remaining Posts removed
Raw posts collected 4764 NA
Dropped empty or missing text 4734 30
Dropped duplicate post IDs (same post returned by multiple queries) 2722 2012
Dropped exact duplicate text (reposts, copy-paste, bot-like) 2430 292
Dropped Hilton Hotel / Paris Hilton / other Hilton-brand posts 2428 2
Dropped posts with missing dates or outside the date window 2428 0
Dropped posts not confidently about either candidate 1865 563
Dropped posts mentioning both candidates 1520 345
Dropped posts under 4 words 1510 10 
kable(flags, col.names = c("Problem flag", "Posts"),
      caption = "Flag counts (before filtering). Flags can overlap, so they should not be added together.")
Flag counts (before filtering). Flags can overlap, so they should not be added together.
Problem flag Posts
Rows entering the flag check 2428
Missing or unparseable date 0
Not confidently about either candidate 563
Names both candidates 345
Under 4 words after cleaning 77 
n_by_cand <- posts |> count(candidate)
print(kable(n_by_cand, col.names = c("Candidate", "Posts after cleaning"),
            caption = "Final analysis sample"))
Final analysis sample
Candidate Posts after cleaning
Becerra 711
Hilton 799 
cat("\n\n")
if (!dates_available) cat("*Note: the data had no usable dates, so the date-window filter was skipped.*\n\n")
if (!author_available) cat("*Note: the data had no author IDs, so the bootstrap in Section 5.2 resamples individual posts instead of accounts.*\n\n")
if (nrow(n_by_cand) < 2 || min(n_by_cand$n) < 100) {
  cat("**Warning:** fewer than 100 posts for at least one candidate; treat all estimates as very rough.\n")
}

The final sample has 1001 distinct accounts across 1510 posts (accounts identified only by an anonymous code).

4. Sentiment scoring

4.1 The AI prompt I used (saved exactly)

The R code in this section was written by an AI assistant (Claude) in response to the following prompt:

I’m writing R code for a class lab that forecasts the 2026 California governor’s race (Xavier Becerra, Democrat, vs. Steve Hilton, Republican) from public Bluesky posts.

DATA: a data frame posts with one row per post and these columns: post_id (integer), uri (character), author_id (anonymous hashed account code), candidate (“Becerra” or “Hilton”), created (POSIXct), text_clean (character; URLs and @mentions already removed). The posts were already cleaned for Hilton-hotel / Paris Hilton false positives and duplicates.

METHOD: Tokenize text_clean with tidytext::unnest_tokens, join to get_sentiments("bing"), count positive and negative words per post, and compute net = positive - negative. Classify each post as “positive” if net > 0, “negative” if net < 0, “balanced” if net == 0 but it has lexicon matches, and “no_match” if it has no lexicon words. Keep all posts. Remove the words “trump”, “like” and “right” from the lexicon (Bing codes them as sentiment words, but here they are a name, a common verb/comparison, and a direction or political label). Summarize by candidate: posts, positive, negative, balanced, no-match, mean net score, and net sentiment = (positive - negative) / (positive + negative). Explain that word sentiment does not necessarily measure stance toward a candidate.

PREDICTION: Before computing results, define the rule as each candidate’s count of positive posts divided by the combined positive posts for both candidates. Label it an uncalibrated classroom vote-share estimate, not a representative voter estimate.

BOOTSTRAP: Resample anonymous authors with replacement 2000 times, keeping all of each sampled author’s posts. Handle resamples with zero positive posts explicitly (mark as undefined and report how many).

CONSTRAINTS: Use only tidyverse and tidytext. Use the native pipe |>. No loops over rows. Never print post text, usernames, or author IDs. The code must run top to bottom inside an R Markdown chunk, with a brief comment on each step. Do not attempt negation handling; I will discuss that as a limitation.

4.2 Code produced

# Bing lexicon, minus words that are mostly not sentiment in this setting:
#  "trump" (a name), "like" (usually a verb or comparison, e.g. "looks like"),
#  "right" (a direction or political "right", not approval)
bing <- get_sentiments("bing") |> filter(!word %in% c("trump", "like", "right"))

# Tokenize and join to lexicon
post_scores <- posts |>
  unnest_tokens(word, text_clean) |>
  inner_join(bing, by = "word", relationship = "many-to-many") |>
  count(post_id, sentiment) |>
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0)

# Make sure both columns exist even if one sentiment never appears
for (col in c("positive", "negative")) {
  if (!col %in% names(post_scores)) post_scores[[col]] <- 0L
}

# Attach scores; keep every post, separating "no evidence" from "balanced evidence"
posts <- posts |>
  left_join(post_scores, by = "post_id") |>
  mutate(positive = replace_na(positive, 0),
         negative = replace_na(negative, 0),
         net = positive - negative,
         match_type = case_when(net > 0 ~ "positive",
                                net < 0 ~ "negative",
                                positive + negative == 0 ~ "no_match",
                                TRUE ~ "balanced"),
         code_stance = case_when(match_type == "positive" ~ "support",
                                 match_type == "negative" ~ "oppose",
                                 TRUE ~ "neutral"))

# Summary by candidate
summ <- posts |>
  group_by(candidate) |>
  summarise(n_posts  = n(),
            support  = sum(match_type == "positive"),
            oppose   = sum(match_type == "negative"),
            balanced = sum(match_type == "balanced"),
            no_match = sum(match_type == "no_match"),
            mean_net = mean(net),
            .groups = "drop") |>
  mutate(volume_share = n_posts / sum(n_posts),
         net_sentiment = (support - oppose) / (support + oppose))

summ |>
  transmute(Candidate = candidate, Posts = n_posts, Positive = support, Negative = oppose,
            `Balanced matches` = balanced, `No matches` = no_match,
            `Mean net score` = round(mean_net, 3),
            `Net sentiment` = round(net_sentiment, 3),
            `Volume share` = percent(volume_share, 0.1)) |>
  kable(caption = "Sentiment summary by candidate (aggregate only)")
Sentiment summary by candidate (aggregate only)
Candidate Posts Positive Negative Balanced matches No matches Mean net score Net sentiment Volume share
Becerra 711 236 187 66 222 0.135 0.116 47.1%
Hilton 799 164 323 94 218 -0.441 -0.326 52.9%

How to read this. The Bing lexicon measures the tone of words, not necessarily stance toward a candidate. Positive words can describe an opponent, appear in sarcasm, or be unrelated to the candidate; a supportive post can contain harsh words aimed at someone else. “No matches” means the dictionary had no evidence, which is not the same as a neutral stance, and “balanced matches” means mixed evidence. The method does not interpret negation, sarcasm, emojis, or conversational context.

posts |>
  count(candidate, match_type) |>
  mutate(match_type = factor(match_type, levels = c("positive", "negative", "balanced", "no_match"))) |>
  ggplot(aes(candidate, n, fill = match_type)) +
  geom_col(position = "dodge") +
  scale_fill_manual(values = c(positive = "#2c7fb8", negative = "#d95f0e",
                               balanced = "grey60", no_match = "grey85")) +
  labs(x = NULL, y = "Posts", fill = "Lexicon result",
       title = "Posts by candidate and lexicon-based sentiment") +
  theme_minimal()

4.3 Lexicon sanity check

The most common sentiment words per candidate (aggregate word counts only) help catch lexicon mistakes, such as political words that Bing scores as emotional. In an earlier run of this check, “like” and “right” were among the most common “positive” words for both candidates even though they are mostly not praise (“looks like”, “the right”), so I added them to the removal list along with “trump”. This is a measurement fix, not a change to the prediction rule, and I made it before hand-labeling so my validation compares against the final code. Other words that remain, such as “win” and “safe”, can also be political jargon rather than praise; I left them in because they can also express real approval, and I note this as a limitation.

posts |>
  unnest_tokens(word, text_clean) |>
  inner_join(bing, by = "word", relationship = "many-to-many") |>
  count(candidate, word, sentiment, sort = TRUE) |>
  group_by(candidate) |>
  slice_head(n = 8) |>
  ungroup() |>
  kable(caption = "Top 8 lexicon words per candidate")
Top 8 lexicon words per candidate
candidate word sentiment n
Becerra good positive 36
Becerra win positive 24
Becerra better positive 23
Becerra safe positive 21
Becerra well positive 19
Becerra thank positive 18
Becerra support positive 16
Becerra corruption negative 14
Hilton creepy negative 43
Hilton weird negative 28
Hilton bad negative 25
Hilton good positive 22
Hilton win positive 21
Hilton innuendo negative 18
Hilton fuck negative 17
Hilton support positive 16

5. Prediction

5.1 Rule chosen (defined before looking at results)

I chose the positive-post share rule: a candidate’s predicted two-party vote share is their count of positive posts divided by the combined count of positive posts for both candidates. This is an uncalibrated classroom estimate, not a measurement of voter preferences.

  • Volume share rewards raw attention, and attention includes outrage, jokes, and news links, so more posts does not mean more support.
  • Net sentiment ignores how many people are talking, while elections count voters.
  • Positive-post share combines both: it counts only posts with positive wording, a rough proxy for favorable talk. Only two candidates are on the ballot, so the two shares sum to 100%. This normalization ignores turnout and abstention.

I keep this rule even if the result looks surprising; changing the rule after seeing the output would defeat the purpose.

pos_b <- summ$support[summ$candidate == "Becerra"]
pos_h <- summ$support[summ$candidate == "Hilton"]
pred_share <- pos_b / (pos_b + pos_h)           # Becerra's predicted share
winner <- if (pred_share > 0.5) "Becerra" else "Hilton"
winner_share <- max(pred_share, 1 - pred_share)

rules <- tibble(
  Rule = c("Volume share (Becerra)", "Positive-post share (Becerra) - CHOSEN",
           "Net sentiment, Becerra minus Hilton"),
  Value = c(percent(summ$volume_share[summ$candidate == "Becerra"], 0.1),
            percent(pred_share, 0.1),
            sprintf("%+.3f", summ$net_sentiment[summ$candidate == "Becerra"] -
                             summ$net_sentiment[summ$candidate == "Hilton"])))
kable(rules, caption = "Point predictions under the three candidate rules")
Point predictions under the three candidate rules
Rule Value
Volume share (Becerra) 47.1%
Positive-post share (Becerra) - CHOSEN 59.0%
Net sentiment, Becerra minus Hilton +0.442

There were 400 positive posts in total: 236 for Becerra and 164 for Hilton. Neutral and negative posts are not in the denominator.

Predicted winner under the chosen rule: Becerra, with 59.0% of the two-party estimate (Becerra 59.0%, Hilton 41.0%).

5.2 Bootstrap: how much could be sampling noise?

I resampled accounts with replacement 2000 times, keeping every post from each sampled account (an account drawn twice contributes its posts twice). This respects the fact that posts from the same account are not independent, so one heavy poster cannot make the result look more certain than it is. Candidate groups were not resampled separately. In each replicate I recomputed the positive-post share, and a replicate with no positive posts is marked undefined and excluded (and counted below).

set.seed(SEED)

# One row per anonymous account: its positive posts for each candidate
auth <- posts |>
  group_by(author_id) |>
  summarise(pb = sum(candidate == "Becerra" & match_type == "positive"),
            ph = sum(candidate == "Hilton"  & match_type == "positive"),
            .groups = "drop")

boot_share <- replicate(N_BOOT, {
  i  <- sample.int(nrow(auth), replace = TRUE)
  pb <- sum(auth$pb[i]); ph <- sum(auth$ph[i])
  if (pb + ph == 0) NA_real_ else pb / (pb + ph)
})

n_undefined <- sum(is.na(boot_share))
valid <- boot_share[!is.na(boot_share)]

ci        <- quantile(valid, c(0.025, 0.975))
boot_se   <- sd(valid)
p_b_ahead <- mean(valid > 0.5)
p_h_ahead <- mean(valid < 0.5)
p_tie     <- mean(valid == 0.5)

kable(tibble(`Point estimate (Becerra)` = percent(pred_share, 0.1),
             `Bootstrap SE` = percent(boot_se, 0.1),
             `95% interval (Becerra)` = paste0(percent(ci[1], 0.1), " to ", percent(ci[2], 0.1)),
             `Becerra ahead` = percent(p_b_ahead, 0.1),
             `Hilton ahead` = percent(p_h_ahead, 0.1),
             `Tied` = percent(p_tie, 0.1),
             `Undefined replicates` = n_undefined),
      caption = "Account-level bootstrap results")
Account-level bootstrap results
Point estimate (Becerra) Bootstrap SE 95% interval (Becerra) Becerra ahead Hilton ahead Tied Undefined replicates
59.0% 2.8% 53.6% to 64.4% 100.0% 0.0% 0.0% 0 
ggplot(tibble(share = valid), aes(share)) +
  geom_histogram(bins = 40, fill = "#2c7fb8", color = "white") +
  geom_vline(xintercept = 0.5, linetype = "dashed") +
  geom_vline(xintercept = pred_share, color = "#d95f0e", linewidth = 1) +
  scale_x_continuous(labels = percent) +
  labs(x = "Becerra share of positive posts", y = "Bootstrap resamples",
       title = "Bootstrap distribution of the prediction",
       subtitle = "Orange = point estimate; dashed = 50%") +
  theme_minimal()

“Becerra ahead” and “Hilton ahead” are frequencies under this resampling procedure, not election win probabilities. The bootstrap measures only sampling instability among the accounts I observed. It cannot correct for errors in sentiment scoring, missing search coverage, coordinated or correlated accounts, or the mismatch between Bluesky users and the electorate. An interval that excludes 50% can still surround a biased and invalid proxy.

5.3 Sanity check against polling

For context, one public polling average (PollCheck, updated October 27, 2026) put Becerra at about 56.5% and Hilton at about 35.7% among all respondents, or roughly 61% of the two-party total (56.5 / (56.5 + 35.7)). Source: https://www.pollcheck.co.uk/us-midterms-2026/governor/california. Polls are not part of my model; they are an outside benchmark for how far the Bluesky signal sits from other evidence.

poll_two_party <- 56.5 / (56.5 + 35.7)
kable(tibble(Source = c("Bluesky positive-post share (Becerra)", "Polling average, two-party equivalent (Becerra)"),
             Share = percent(c(pred_share, poll_two_party), 0.1)))
Source Share
Bluesky positive-post share (Becerra) 59.0%
Polling average, two-party equivalent (Becerra) 61.3%

6. Validation: my hand labels vs. the code

I took a seeded random sample of 20 cleaned posts and saved them to a local sheet (data/hand_label_sheet.csv) without the code’s score, so my labels were blind to it. For each post I labeled stance toward the named candidate as support, oppose, or neutral (neutral = news, a question, or unclear). For sarcasm I labeled the intended stance. The sheet stays on my computer and is not published.

Labeling disclosure: I labeled these posts myself, without AI suggestions and without seeing the machine scores.

For comparison only, the code’s positive posts are mapped to support, negative to oppose, and everything else (no matches or balanced matches) to neutral. This mapping is a proxy; it does not claim that sentiment and stance are the same thing.

if (!file.exists(sheet_path)) {
  set.seed(SEED)
  posts |>
    slice_sample(n = min(N_LABEL, nrow(posts))) |>
    transmute(uri, candidate, text, hand_stance = NA_character_) |>
    write_csv(sheet_path)
}

lab <- read_csv(sheet_path, show_col_types = FALSE,
                col_types = cols(.default = col_character())) |>
  mutate(hand = str_to_lower(str_trim(hand_stance))) |>
  select(uri, hand) |>
  inner_join(posts |> select(uri, candidate, code = code_stance), by = "uri")

labels_ready <- nrow(lab) > 0 && all(lab$hand %in% c("support", "oppose", "neutral"))
if (!labels_ready) {
  cat("**Hand labels are not complete yet.** Open `data/hand_label_sheet.csv`, fill the `hand_stance` column with support / oppose / neutral for every row, save as CSV, and knit again.\n")
} else {
  lv <- c("support", "oppose", "neutral")
  n_lab   <- nrow(lab)
  correct <- sum(lab$hand == lab$code)
  val_acc <- correct / n_lab
  acc_ci  <- prop.test(correct, n_lab)$conf.int

  # Baseline: a classifier that always answers the most common hand label
  baseline_neutral <- mean(lab$hand == "neutral")
  majority_label   <- names(which.max(table(factor(lab$hand, lv))))
  baseline_major   <- mean(lab$hand == majority_label)

  cm <- table(`Hand label` = factor(lab$hand, lv), `Code label` = factor(lab$code, lv))
  po <- sum(diag(cm)) / sum(cm)
  pe <- sum(rowSums(cm) * colSums(cm)) / sum(cm)^2
  kappa <- (po - pe) / (1 - pe)

  both_dir <- lab |> filter(hand != "neutral", code != "neutral")
  dir_acc <- if (nrow(both_dir) > 0) mean(both_dir$hand == both_dir$code) else NA_real_

  cat(sprintf("**Overall agreement:** %d of %d posts (%s; 95%% CI %s to %s).\n\n",
              correct, n_lab, percent(val_acc, 0.1), percent(acc_ci[1], 1), percent(acc_ci[2], 1)))
  cat(sprintf("**Baseline for comparison:** a classifier that always answered \"%s\" (my most common label) would agree with %s of these labels. Always answering \"neutral\" would agree with %s.\n\n",
              majority_label, percent(baseline_major, 0.1), percent(baseline_neutral, 0.1)))
  cat(sprintf("**Cohen's kappa (agreement beyond chance):** %.2f.\n\n", kappa))
  cat(sprintf("**Direction-only agreement** (posts where both my label and the code's were support/oppose): %s (n = %d).\n\n",
              if (is.na(dir_acc)) "n/a" else percent(dir_acc, 0.1), nrow(both_dir)))
  cat("Confusion matrix (rows = my label, columns = code's label):\n\n")
  print(kable(as.data.frame.matrix(cm)))
  cat("\n\nWith only", n_lab, "labeled posts, this accuracy estimate is imprecise; a larger independently labeled sample would give a firmer number.\n")
}

Overall agreement: 9 of 20 posts (45.0%; 95% CI 24% to 68%).

Baseline for comparison: a classifier that always answered “support” (my most common label) would agree with 45.0% of these labels. Always answering “neutral” would agree with 15.0%.

Cohen’s kappa (agreement beyond chance): 0.08.

Direction-only agreement (posts where both my label and the code’s were support/oppose): 60.0% (n = 15).

Confusion matrix (rows = my label, columns = code’s label):

support oppose neutral
support 6 3 0
oppose 3 3 2
neutral 3 0 0

With only 20 labeled posts, this accuracy estimate is imprecise; a larger independently labeled sample would give a firmer number.

7. Reflection: limits of this forecast

Who is on Bluesky. Bluesky users are not a random sample of California voters. The platform tends to skew toward people who are politically engaged, heavy internet users, and more left-leaning than the electorate overall, and many California voters, especially older, rural, and less politically engaged ones, are not on it. This dataset does not establish users’ locations, ages, eligibility to vote in California, or likelihood of turning out, and English-only collection leaves out discussion in other languages. Even a perfect reading of every Bluesky post would describe Bluesky, not California. That is a coverage problem the bootstrap cannot fix. A result favoring Becerra would therefore be unsurprising and not very informative, and a result favoring Hilton would more likely signal measurement problems, a burst of posting by a few accounts, or a news cycle than real support.

Sarcasm and tone versus stance. The Bing lexicon counts words, so it measures tone, not stance. A furious post attacking Hilton scores negative but is pro-Becerra, and a lexicon-based stance mapping files it under Hilton’s “oppose” count without crediting Becerra. Sarcasm (“great job, genius”) is scored as praise, negation (“not good”) is ignored, and words like “attack” or “crisis” appear in neutral news posts. My hand validation in Section 6 quantifies this: the code agreed with my labels on 45% of the sampled posts, compared with 15% for a classifier that always says “neutral”. Because many posts also contain no lexicon words, absence of matches is lack of evidence, not a neutral stance.

Does more posts mean more support? No. Posts can be hate-reading, mockery, news sharing, fact-checking, or reactions to a viral moment, and a controversial candidate can lead in volume while being unpopular. I used positive-post share rather than raw volume for that reason, but it still treats a positive post as a supporter. Resampling accounts instead of posts partly addresses the problem of one account posting many times, but it cannot detect bots or coordinated accounts. My search terms, the “latest” sort, and the API’s result caps also determine which days and posts are over-represented.

What the bootstrap does and does not tell us. The interval in Section 5.2 shows how stable the estimate is across the accounts I observed. It does not cover non-representative users, lexicon errors, sarcasm, or search design. A narrow interval around an invalid measure is still invalid, so I treat this as a demonstration of the method and a rough read of Bluesky conversation, not a replacement for polling.

What I would do with more time. Collect a larger independently labeled sample and measure accuracy for each candidate separately, handle negation and tone with a stance classifier, compare reasonable prediction rules without choosing one because of its outcome, and test the approach against representative polls or past election results.

Ethics. All results are aggregate counts and shares. No usernames, post text, author codes, or profile links are published, the scraper never stored handles, and the raw data file is kept local.

8. Bottom line

cat(sprintf("Under the rule I defined in advance (positive-post share), the forecast selects **%s** with **%s** of the two-candidate estimate (Becerra %s, Hilton %s). The 95%% account-level bootstrap interval for Becerra's share is %s to %s.\n\n",
            winner, percent(winner_share, 0.1), percent(pred_share, 0.1), percent(1 - pred_share, 0.1),
            percent(ci[1], 0.1), percent(ci[2], 0.1)))

Under the rule I defined in advance (positive-post share), the forecast selects Becerra with 59.0% of the two-candidate estimate (Becerra 59.0%, Hilton 41.0%). The 95% account-level bootstrap interval for Becerra’s share is 53.6% to 64.4%.

if (labels_ready) {
  cat(sprintf("However, the sentiment code agreed with my hand labels on %s of the %d sampled posts, versus %s for an always-neutral baseline. %s\n\n",
              percent(val_acc, 0.1), nrow(lab), percent(baseline_neutral, 0.1),
              if (val_acc < 0.6) "That is weak agreement, so this output is a poor measure of candidate support and should not be treated as an election forecast."
              else "That is moderate agreement, but the sample is small and the sampling and coverage limits above still apply."))
} else {
  cat("Hand-label validation is not finished yet (see Section 6).\n\n")
}

However, the sentiment code agreed with my hand labels on 45.0% of the 20 sampled posts, versus 15.0% for an always-neutral baseline. That is weak agreement, so this output is a poor measure of candidate support and should not be treated as an election forecast.

cat("These percentages are shares of positive-scoring posts, not measured voter preferences.\n")

These percentages are shares of positive-scoring posts, not measured voter preferences.

Method reference

Silge, J., and Robinson, D. Text Mining with R, Chapter 2: Sentiment analysis with tidy data. https://www.tidytextmining.com/sentiment

Appendix: reproducibility

  • Collection window: 2026-09-01 to 2026-11-03 (UTC), English language, queries: Becerra; Xavier Becerra; Steve Hilton; Hilton governor; Hilton California governor.
  • Random seed: 2026; bootstrap resamples: 2000; hand-labeled posts: 20.
  • To rerun: set BSKY_HANDLE and BSKY_APP_PASSWORD in ~/.Renviron (use a Bluesky app password), delete data/raw_posts.csv to rescrape, and knit.
sessionInfo()
## R version 4.6.1 (2026-06-24)
## Platform: x86_64-pc-linux-gnu
## Running under: Ubuntu 24.04.5 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
## LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0
## 
## locale:
##  [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
##  [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
##  [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
## [10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   
## 
## time zone: UTC
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] knitr_1.52      scales_1.4.0    httr2_1.3.0     tidytext_0.4.3 
##  [5] lubridate_1.9.5 forcats_1.0.1   stringr_1.6.0   dplyr_1.2.1    
##  [9] purrr_1.2.2     readr_2.2.0     tidyr_1.3.2     tibble_3.3.1   
## [13] ggplot2_4.0.3   tidyverse_2.0.0
## 
## loaded via a namespace (and not attached):
##  [1] janeaustenr_1.0.0  sass_0.4.10        generics_0.1.4     stringi_1.8.9     
##  [5] lattice_0.22-9     hms_1.1.4          digest_0.6.39      magrittr_2.0.5    
##  [9] evaluate_1.0.5     grid_4.6.1         timechange_0.4.0   RColorBrewer_1.1-3
## [13] fastmap_1.2.0      jsonlite_2.0.0     Matrix_1.7-5       jquerylib_0.1.4   
## [17] cli_3.6.6          crayon_1.5.3       rlang_1.3.0        tokenizers_0.3.0  
## [21] bit64_4.8.6        withr_3.0.3        cachem_1.1.0       yaml_2.3.12       
## [25] otel_0.2.0         parallel_4.6.1     tools_4.6.1        tzdb_0.5.0        
## [29] vctrs_0.7.3        R6_2.6.1           lifecycle_1.0.5    bit_4.6.0         
## [33] vroom_1.7.1        pkgconfig_2.0.3    pillar_1.11.1      bslib_0.12.0      
## [37] gtable_0.3.6       glue_1.8.1         Rcpp_1.1.2         xfun_0.61         
## [41] tidyselect_1.2.1   rstudioapi_0.19.0  farver_2.1.2       htmltools_0.5.9   
## [45] SnowballC_0.7.1    labeling_0.4.3     rmarkdown_2.32     compiler_4.6.1    
## [49] S7_0.2.2