For this project, I used public Bluesky posts to create a social-media-based forecast of the California governor’s race between Xavier Becerra and Steve Hilton. I collected posts about both candidates, cleaned the data, analyzed sentiment using the Bing sentiment lexicon, manually checked a sample of posts, and used bootstrap resampling to examine uncertainty in my results.
I collected public Bluesky posts using the atrrr package
in R. I searched separately for “Xavier Becerra” and “Steve Hilton.” I
collected 198 posts for Becerra and 193 posts for Hilton.
Only aggregate results are reported in this document. Usernames are not published.
library(atrrr)
becerra_posts <- search_post(
q = "Xavier Becerra",
limit = 100
)
hilton_posts <- search_post(
q = "Steve Hilton",
limit = 100
)I checked the data for missing text, duplicate posts, and irrelevant results. For Steve Hilton, I specifically checked for unrelated references to Paris Hilton, Hilton Hotels, Hilton Honors, and similar uses of “Hilton.” Because I searched for the full name “Steve Hilton,” these false matches were limited.
After removing blank and duplicate post text and applying the Hilton-related filter, 125 Becerra posts and 159 Hilton posts remained.
library(dplyr)
library(stringr)
becerra_clean2 <- becerra_clean %>%
filter(!is.na(text), str_trim(text) != "") %>%
distinct(text, .keep_all = TRUE)
hilton_clean2 <- hilton_clean %>%
filter(!is.na(text), str_trim(text) != "") %>%
filter(!str_detect(
str_to_lower(text),
"paris hilton|hilton hotel|hilton hotels|hilton honors"
)) %>%
distinct(text, .keep_all = TRUE)I used ChatGPT to help write the R code for the sentiment analysis. The exact prompt I used was:
I have two cleaned Bluesky datasets in R called
becerra_clean2andhilton_clean2. Each dataset contains one row per unique Bluesky post and has atextcolumn containing the post text. I want to conduct sentiment analysis using thetidytextpackage and the Bing sentiment lexicon. Please write R code that tokenizes each post into individual words, joins the words with the Bing lexicon, calculates positive and negative word counts for each post, assigns each post a sentiment score (positive words minus negative words), and then summarizes the results by candidate. I need the summary to show the number of posts, positive posts, negative posts, neutral posts, positive share, and average/net sentiment for Xavier Becerra and Steve Hilton. Do not include or display usernames in the results.
I used the Bing sentiment lexicon through the tidytext
package. A post’s sentiment score was calculated by subtracting its
number of negative words from its number of positive words.
The results were:
| Candidate | Posts | Positive | Negative | Neutral | Positive Share | Net Sentiment | Average |
|---|---|---|---|---|---|---|---|
| Steve Hilton | 159 | 32 | 50 | 77 | 20.1% | -34 | -0.21 |
| Xavier Becerra | 125 | 26 | 29 | 70 | 20.8% | +4 | +0.03 |
Although Hilton had more posts, his overall sentiment was more negative. Becerra had a slightly higher positive share and a positive net sentiment.
I randomly selected 15 posts and manually labeled their stance toward the candidate as Pro, Anti, or Neutral. I then compared my labels with the Bing classifications by treating Positive as Pro, Negative as Anti, and Neutral as Neutral.
Bing matched my manual stance label for 4 out of 15 posts, giving an accuracy of 26.7%.
The low accuracy shows an important limitation of word-based sentiment analysis. A post can contain negative words about a candidate’s opponent while actually supporting the candidate. News headlines, sarcasm, context, and political criticism can also cause sentiment and stance to differ.
I chose positive sentiment balance as my prediction rule. For this rule, I calculated the percentage of non-neutral posts that were positive for each candidate.
Becerra’s sentiment balance was 47.3%, while Hilton’s was 39.0%. I normalized these two scores into a two-candidate forecast:
Based on this rule, my predicted winner is Xavier Becerra.
This percentage is a sentiment-based forecast and should not be interpreted as a scientific estimate of actual voter intention.
I used 1,000 bootstrap samples to measure how much my predicted vote share could change because of sampling noise. For each bootstrap sample, I resampled the posts with replacement, recalculated each candidate’s positive sentiment balance, and converted the results into a two-candidate predicted share.
Across the 1,000 bootstrap samples, Becerra’s average predicted share was approximately 54.7%. The 95% bootstrap interval ranged from approximately 44.8% to 64.4%. Becerra received more than 50% of the predicted share in 82.3% of the bootstrap samples.
These results generally favor Becerra, but there is still considerable uncertainty. Because the 95% interval includes values below 50%, the bootstrap shows that sampling noise could change the predicted winner in some samples.
There are several limitations to this forecast. First, Bluesky users do not represent all California voters. People who use Bluesky may have different political views and demographics than the overall voting population, so these results cannot be treated the same way as a representative election poll.
Another major limitation is the difference between sentiment and stance. My manual validation accuracy was only 26.7%, showing that Bing often had difficulty identifying whether a post actually supported or opposed a candidate. Sarcasm, jokes, news headlines, and criticism directed at another person can all affect the sentiment score.
More posts also do not necessarily mean more support. Hilton had more posts in my cleaned dataset, but his overall sentiment was more negative. A candidate could receive a lot of attention because people are criticizing them rather than supporting them.
Overall, Bluesky data can provide information about online political discussion, but it has major limitations as an election forecasting tool. My analysis predicts Xavier Becerra as the winner, but the low validation accuracy and bootstrap uncertainty mean the forecast should be interpreted cautiously.