Results at a glance

  • Sample: 290 unique posts, collected October 22–October 29, 2026.
  • Predicted winner: Xavier Becerra.
  • Predicted two-candidate share: Becerra 52.4%; Hilton 47.6%.
  • Becerra’s 95% bootstrap interval: 43.5%–61.5%.
  • Audit label agreement: 20.0%.

The bootstrap interval includes 50%, so the resampling results allow either candidate to lead. These shares are uncalibrated social-media proxies, not representative estimates of voters’ preferences.

Research question and collection

I used public Bluesky posts to make a classroom forecast for the November 3, 2026 California governor election between Xavier Becerra and Steve Hilton. My prediction rule uses each candidate’s share of positively scored posts. I treat that share as a two-candidate vote-share proxy for the assignment, although it has not been calibrated to actual voting.

I used RStudio and atrrr, following the earlier scraping lab. I searched Becerra and Hilton separately, using English-language search and the same seven-day window for both. Each search used sort="latest" and daily intervals with a 5,000-result safety limit. The collection code stops if any interval reaches that limit, rather than silently comparing capped samples. The API may still omit indexed or unavailable posts, so this is a search sample rather than a census.

The common window ran from 2026-10-22 14:52:42 PDT to 2026-10-29 14:52:42 PDT (end excluded). Collection finished at 2026-10-29 14:52:52 PDT. Retained post creation times ranged from 2026-10-22 14:57:18 PDT to 2026-10-28 16:59:12 PDT.

query daily_searches returned_search_rows any_cap_reached
Becerra 7 294 FALSE
Hilton 7 1006 FALSE

Exact AI prompt used

The following is the exact specification I gave ChatGPT to generate the scoring and reporting code. The preparation code in this document adapts the raw atrrr result into this structure.

Write R code for a Bluesky classroom forecasting assignment comparing Xavier Becerra and Steve Hilton. My cleaned data has one row per unique post and these columns: post_id (character), author_group (anonymous integer for private clustering), text (character), score_text (text with URLs and @mentions removed), becerra and hilton (logical candidate-mention flags). Use tidytext::unnest_tokens() and tidytext::get_sentiments(“bing”). Count positive and negative word occurrences, define net as positive minus negative, and label posts positive when net > 0, negative when net < 0, and neutral when net = 0. Keep posts with no matching Bing words, assigning zero counts and neutral, and report their number. Score each unique post once, then attach that score to each candidate explicitly mentioned; disclose that this is overall tone, not candidate-specific stance. Summarize counts, positive rate, and mean net by candidate. Prepare a reproducible, blinded random hand-label sample of up to 10 posts per candidate (10 to 20 total observations) with labels positive, negative, neutral, or mixed/unclear. Do not invent human labels. Compare completed human stance labels with the automated tone labels using accuracy, majority-class baseline, and a confusion table; count mixed/unclear as disagreement. Fix the prediction rule before examining results: candidate positive-post count divided by the positive-post counts summed across both candidates, treated explicitly as an uncalibrated two-candidate vote-share proxy. Use 2,000 author-cluster bootstrap resamples, sampling authors jointly across both candidates with replacement, preserving posts mentioning both. Report percentile 95% intervals, invalid zero-denominator replicates, and how often each candidate leads. Stop on an undefined prediction. Use fixed seeds. Publish only aggregate results, with no usernames, handles, author IDs, post identifiers, post text, or post links.

For cleaning, I also asked:

How should I clean English Bluesky searches for Becerra and Hilton before a California governor analysis? Remove Hilton Hotel and Paris Hilton posts. Identify other issues, including duplicate search hits, copied text, unrelated Hilton references, dates, language, spam, repeated authors, posts mentioning both candidates, sarcasm, and tone versus candidate stance. Explain what can be automated and what still needs human judgment. Do not silently treat all mentions as support.

Cleaning decisions and audit results

I removed repeated post URIs, empty text, missing or out-of-window creation times, hotel/Paris Hilton/tourism references, and search hits without a candidate reference in the post text. Hilton needed either the full name Steve Hilton or Hilton together with a political-context keyword. Becerra needed the surname. These asymmetric name-disambiguation rules can miss real posts or retain unrelated ones, which is a limitation.

I then removed repeated lowercased, whitespace-normalized text across the combined dataset, keeping the earliest copy. This reduces copied-message inflation but can also remove independently written identical messages. URLs and account mentions were removed only for sentiment scoring. English was an API search filter, not an independently verified language classification. I did not automatically classify accounts as bots or remove short posts, negative posts, negators, or repeated authors.

Cleaning step / outcome Posts
Raw search rows 1300
Repeated URI search hits removed 60
Empty text 49
Hotel / Paris Hilton / other Hilton tourism 122
Missing/out-of-window creation time 116
No candidate name/context in post text 608
Repeated normalized text removed 55
Unique posts retained 290

Raw search rows minus repeated URI hits equals the pool of distinct posts. The later removal categories are mutually exclusive and, together with retained posts, account for that pool.

There were 40 retained posts mentioning both candidates. Each contributes one observation to each candidate. Therefore, the candidate totals can exceed the number of unique posts. The bootstrap preserves this overlap. Overall tone is assigned to both candidates in those posts, even if the writer supports one and opposes the other; that is a known measurement problem.

The private cleaning-review CSV supports inspection of the rules and removed posts. The audit identified one false match about Erika Hilton rather than Steve Hilton. It remains flagged as mixed/unclear in the fixed audit and is counted as a disagreement; this is a known unresolved relevance error, rather than evidence about Steve Hilton. The hand-label audit below checks stance classification among retained posts; it does not estimate the relevance of every excluded post. Additional issues AI identified were off-topic surnames, copied campaigns, spam, repeat authors, search-index omissions, embedded content not present in plain text, negation, sarcasm, and sentiment directed at someone other than the named candidate. Automated rules address only part of this list.

Bing sentiment results

Each positive word contributes +1 and each negative word contributes -1. A zero score is labeled neutral, including ties and posts with no matching words. No-match counts are reported separately because a zero dictionary score does not prove neutral stance. The implementation uses the following scoring steps:

counts <- clean %>%
  select(post_id, score_text) %>%
  tidytext::unnest_tokens(word, score_text) %>%
  inner_join(tidytext::get_sentiments("bing"), by="word") %>%
  group_by(post_id) %>%
  summarise(positive=sum(sentiment=="positive"),
            negative=sum(sentiment=="negative"), .groups="drop")
scored <- clean %>% left_join(counts, by="post_id") %>%
  mutate(positive=coalesce(positive,0L), negative=coalesce(negative,0L),
         net=positive-negative,
         code_label=case_when(net>0 ~ "positive", net<0 ~ "negative",
                              TRUE ~ "neutral"))
Candidate Posts Positive Negative Neutral
Steve Hilton 157 50 44 63
Xavier Becerra 173 55 42 76
Candidate No dictionary matches Mean net score Positive rate
Steve Hilton 44 0.013 31.8%
Xavier Becerra 50 0.127 31.8%

Stance audit and validation accuracy

ChatGPT supplied initial stance labels and explanations for 20 randomly selected candidate-post observations, using seed 20261029 and up to 10 per candidate. The automated label was hidden during labeling. A post appearing in both candidate samples was evaluated separately for its stance toward each target. This is a stratified sample, so overall accuracy weights candidates approximately equally rather than matching their full-data volume.

The audit labels describe stance toward the named candidate: positive means support, negative means opposition, neutral means neither clear support nor opposition, and mixed/unclear means the stance could not be reduced to those categories. I counted mixed/unclear as an automated disagreement and did not drop those observations. The automated output measures tone, so this comparison tests how well tone approximates stance.

candidate audited correct accuracy
Steve Hilton 10 2 20.0%
Xavier Becerra 10 2 20.0%
Overall 20 4 20.0%

Overall validation accuracy was 20.0%. Predicting the most common human label for every audit observation would achieve 35.0% on this same sample. That is a descriptive majority-label baseline, not a separately trained classifier. Only 10 to 20 observations were audited, so accuracy is imprecise.

positive negative neutral
positive 1 3 3
negative 2 2 3
neutral 1 1 1
mixed/unclear 0 2 1

Prediction and bootstrap

I fixed positive share as my rule: each candidate’s count of positively scored candidate-posts divided by the sum of those counts for both candidates. This uses positive counts, not each candidate’s within-candidate positive rate, and does not force equal numbers of collected posts. Neutral and negative posts do not enter the rule’s denominator. Posts mentioning both candidates can contribute to both sides.

Predicted winner: Xavier Becerra. The table reports the assignment’s predicted two-candidate vote-share proxy and bootstrap interval. These are not polling estimates.

candidate predicted_vote_share_proxy bootstrap_low bootstrap_high
Xavier Becerra 52.4% 43.5% 61.5%
Steve Hilton 47.6% 38.5% 56.5%

I resampled 201 author groups with replacement 2,000 times, using seed 20261030. When an author was sampled, all their retained observations for both candidates traveled together, including repeated copies when an author was drawn multiple times. I recalculated the same positive-share rule in every replicate and used its 2.5th and 97.5th percentiles for the intervals.

0 replicates had no positive posts and were undefined; intervals use 2000 valid replicates. Becerra led in 67.2%, Hilton led in 28.8%, and they tied in 3.9% of valid replicates. These frequencies describe resampling stability, not probabilities of winning the election.

The intervals describe sampling variability under this empirical author-resampling model. They do not account for platform selection, missing search results, cleaning errors, unobserved campaign coordination across accounts, or systematic sentiment errors. Even a narrow interval cannot make the sample representative of California voters.

Reflection

This forecast shows which candidate received more positively worded attention in my Bluesky search sample. I would be careful about treating that as an election result. People on Bluesky are a self-selected group, and my data does not establish that they are California residents, eligible voters, or likely voters. Political activity and platform preferences can make their views different from the electorate’s. I have no demographic weights or verified voter information to correct that.

The Bing dictionary also misses context. A sarcastic compliment could score positive even when the writer dislikes the candidate. A supporter could describe a serious problem using negative words, while a news report could include positive language without endorsing anyone. Negation, quoted speech, links, images, and posts discussing both candidates make this harder. The audit labels measure stance, while the code measures tone; the audit shows how closely those match in a small sample.

More posts do not automatically mean more support. Criticism, controversy, news coverage, campaign activity, and highly active accounts can all increase volume. Removing duplicate text and clustering the bootstrap by author helps with some repetition, but it does not solve those problems. My vote-share number is a classroom conversion of positive-post share, with no evidence that a one-point change equals a one-point change in the vote. I would use representative polling and other election evidence before relying on it for a real forecast.

Privacy and reproducibility

Only aggregate counts, accuracy, confusion tables, forecasts, and plots are published. Usernames, handles, author groups, post IDs, post text, and post links are omitted. Raw data and hand-label files stay private. Collection is frozen before auditing so knitting does not scrape new posts or replace the hand-label sample.

package version
atrrr 0.2.0
dplyr 1.2.1
tidyr 1.3.2
stringr 1.6.0
readr 2.2.0
purrr 1.2.2
tidytext 0.4.3
rmarkdown 2.32
knitr 1.52
ggplot2 4.0.3

Method references: atrrr documentation, tidytext sentiment lexicons, and Text Mining with R: Sentiment analysis.