Overview

This report scores the sentiment of Bluesky posts about two candidates, Xavier Becerra and Steve Hilton, using the Bing lexicon in the tidytext package. Each post’s score is the number of positive words minus the number of negative words. The results are then summarized by candidate.

AI prompt

the R code that scores each post’s sentiment with tidytext and the Bing lexicon, and then summarizes the results by candidate. the two candidates are Xavier Becerra and the other is Steve Hilton. Finally, pick a prediction rule (volume share, positive share, or net sentiment), name a predicted winner and vote share, and run a bootstrap to measure how much of your result could be sampling noise. Close with a short written reflection on the limits of the forecast, including who is on Bluesky, sarcasm and tone versus stance, and whether more posts means more support. can you add this in the code and convert it to a rmd doc

# install.packages(c("tidyverse", "tidytext", "knitr"))  # run once if needed
library(tidyverse)
library(tidytext)
library(knitr)

1. Load data

Place Xavier.csv and Steve.csv in the same folder as this .Rmd file (or edit the paths below).

xavier <- read_csv("Xavier.csv", show_col_types = FALSE) %>%
  mutate(candidate = "Xavier Becerra")

steve <- read_csv("Steve.csv", show_col_types = FALSE) %>%
  mutate(candidate = "Steve Hilton")

posts <- bind_rows(xavier, steve) %>%
  filter(!is.na(text)) %>%
  distinct(uri, candidate, .keep_all = TRUE) %>%   # drop duplicate posts
  select(uri, candidate, author_handle, text,
         like_count, repost_count, reply_count, created_at)

posts %>% count(candidate) %>% kable(caption = "Posts per candidate")
Posts per candidate
candidate n
Steve Hilton 183
Xavier Becerra 195

2. Clean text

URLs, @mentions, hashtags, and non-letter characters are removed before tokenizing.

posts <- posts %>%
  mutate(
    clean_text = text %>%
      str_remove_all("https?://\\S+") %>%   # URLs
      str_remove_all("@\\S+") %>%           # mentions
      str_remove_all("#\\S+") %>%           # hashtags
      str_replace_all("[^[:alpha:]'\\s]", " ")
  )

3. Tokenize and score with Bing

Bing codes some words that are neutral in a political context as sentiment-bearing (for example, “trump” is labeled positive). Add any such words to custom_drop.

bing <- get_sentiments("bing")

custom_drop <- c("trump")

word_sentiment <- posts %>%
  select(uri, clean_text) %>%
  unnest_tokens(word, clean_text) %>%
  filter(!word %in% custom_drop) %>%
  inner_join(bing, by = "word", relationship = "many-to-many")

post_scores <- word_sentiment %>%
  count(uri, sentiment) %>%
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0)

# make sure both columns exist even if one is absent
if (!"positive" %in% names(post_scores)) post_scores$positive <- 0L
if (!"negative" %in% names(post_scores)) post_scores$negative <- 0L

# Posts with no lexicon words get a neutral score of 0
scored <- posts %>%
  left_join(post_scores, by = "uri") %>%
  mutate(
    positive = replace_na(positive, 0L),
    negative = replace_na(negative, 0L),
    score    = positive - negative,
    label    = case_when(score > 0 ~ "Positive",
                         score < 0 ~ "Negative",
                         TRUE      ~ "Neutral")
  )

4. Summary by candidate

summary_by_candidate <- scored %>%
  group_by(candidate) %>%
  summarise(
    n_posts        = n(),
    total_positive = sum(positive),
    total_negative = sum(negative),
    net_sentiment  = total_positive - total_negative,
    mean_score     = mean(score),
    median_score   = median(score),
    sd_score       = sd(score),
    pct_positive   = mean(label == "Positive") * 100,
    pct_negative   = mean(label == "Negative") * 100,
    pct_neutral    = mean(label == "Neutral")  * 100,
    .groups = "drop"
  )

summary_by_candidate %>%
  mutate(across(where(is.numeric), ~ round(.x, 2))) %>%
  kable(caption = "Sentiment summary by candidate")
Sentiment summary by candidate
candidate n_posts total_positive total_negative net_sentiment mean_score median_score sd_score pct_positive pct_negative pct_neutral
Steve Hilton 183 114 180 -66 -0.36 0 1.40 16.94 35.52 47.54
Xavier Becerra 195 217 269 -52 -0.27 0 1.26 14.36 38.46 47.18 
label_counts <- scored %>%
  count(candidate, label) %>%
  group_by(candidate) %>%
  mutate(pct = n / sum(n) * 100) %>%
  ungroup()

label_counts %>%
  mutate(pct = round(pct, 1)) %>%
  kable(caption = "Post counts by sentiment label")
Post counts by sentiment label
candidate label n pct
Steve Hilton Negative 65 35.5
Steve Hilton Neutral 87 47.5
Steve Hilton Positive 31 16.9
Xavier Becerra Negative 75 38.5
Xavier Becerra Neutral 92 47.2
Xavier Becerra Positive 28 14.4

5. Is the difference meaningful?

A Welch t-test compares mean scores between candidates; the Wilcoxon rank-sum test is a non-parametric check since scores are discrete and may be skewed.

t_test <- t.test(score ~ candidate, data = scored)
t_test
## 
##  Welch Two Sample t-test
## 
## data:  score by candidate
## t = -0.68476, df = 365.76, p-value = 0.4939
## alternative hypothesis: true difference in means between group Steve Hilton and group Xavier Becerra is not equal to 0
## 95 percent confidence interval:
##  -0.3639038  0.1759257
## sample estimates:
##   mean in group Steve Hilton mean in group Xavier Becerra 
##                   -0.3606557                   -0.2666667
wilcox <- wilcox.test(score ~ candidate, data = scored)
wilcox
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  score by candidate
## W = 17960, p-value = 0.9059
## alternative hypothesis: true location shift is not equal to 0

6. Most common sentiment words

top_words <- word_sentiment %>%
  left_join(select(posts, uri, candidate), by = "uri") %>%
  count(candidate, sentiment, word, sort = TRUE) %>%
  group_by(candidate, sentiment) %>%
  slice_max(n, n = 10, with_ties = FALSE) %>%
  ungroup()

top_words %>% kable(caption = "Top 10 positive and negative words per candidate")
Top 10 positive and negative words per candidate
candidate sentiment word n
Steve Hilton negative attack 12
Steve Hilton negative rail 12
Steve Hilton negative biting 9
Steve Hilton negative hack 6
Steve Hilton negative sharply 6
Steve Hilton negative breaking 4
Steve Hilton negative despair 4
Steve Hilton negative rival 4
Steve Hilton negative bloated 3
Steve Hilton negative difficult 3
Steve Hilton positive hopeful 10
Steve Hilton positive like 8
Steve Hilton positive won 6
Steve Hilton positive friendly 5
Steve Hilton positive good 5
Steve Hilton positive win 5
Steve Hilton positive work 5
Steve Hilton positive astonishing 4
Steve Hilton positive efficient 3
Steve Hilton positive endorsement 3
Xavier Becerra negative corruption 75
Xavier Becerra negative cruelty 73
Xavier Becerra negative dishonor 49
Xavier Becerra negative rail 10
Xavier Becerra negative hack 6
Xavier Becerra negative sharply 6
Xavier Becerra negative breaking 4
Xavier Becerra negative rival 4
Xavier Becerra negative sorry 3
Xavier Becerra negative blasted 2
Xavier Becerra positive good 75
Xavier Becerra positive safe 73
Xavier Becerra positive like 9
Xavier Becerra positive work 5
Xavier Becerra positive astonishing 4
Xavier Becerra positive hopeful 4
Xavier Becerra positive lead 4
Xavier Becerra positive great 3
Xavier Becerra positive supporting 3
Xavier Becerra positive well 3

7. Plots

Distribution of scores

ggplot(scored, aes(x = score, fill = candidate)) +
  geom_histogram(binwidth = 1, position = "dodge", color = "white") +
  labs(title = "Distribution of post sentiment scores (Bing)",
       x = "Score (positive words - negative words)", y = "Posts",
       fill = NULL) +
  theme_minimal()

Share of posts by label

ggplot(label_counts, aes(x = candidate, y = pct, fill = label)) +
  geom_col() +
  scale_fill_manual(values = c(Negative = "#d95f02",
                               Neutral  = "grey70",
                               Positive = "#1b9e77")) +
  labs(title = "Share of posts by sentiment label",
       x = NULL, y = "% of posts", fill = NULL) +
  theme_minimal()

Top sentiment words

top_words %>%
  mutate(word = reorder_within(word, n, list(candidate, sentiment))) %>%
  ggplot(aes(x = n, y = word, fill = sentiment)) +
  geom_col(show.legend = FALSE) +
  scale_y_reordered() +
  facet_wrap(candidate ~ sentiment, scales = "free_y") +
  labs(title = "Top sentiment words", x = "Count", y = NULL) +
  theme_minimal()

8. Save outputs

write_csv(scored, "scored_posts.csv")
write_csv(summary_by_candidate, "sentiment_summary_by_candidate.csv")

9. Forecast: prediction rule and bootstrap

The rule

Three simple rules could turn post data into a forecast:

  • Volume share: who gets more posts.
  • Positive share: who gets a larger share of the positive posts.
  • Net sentiment: who has the higher average score (positive minus negative words).

I use positive share as the prediction rule:

Predicted Becerra share = positive posts about Becerra / (positive posts about Becerra + positive posts about Hilton)

This is treated as a two-candidate share, so whoever is above 50% is the predicted winner. I chose it over volume because the number of posts per candidate was fixed when the data were collected (roughly 200 each), so volume share carries almost no information here, and over net sentiment because it converts directly to a vote-share-style number. Volume share and net sentiment difference are reported below for comparison.

scored_small <- scored %>% select(candidate, score, label)

calc_stats <- function(d) {
  b     <- d$candidate == "Xavier Becerra"
  n_b   <- sum(b);  n_h <- sum(!b)
  pos_b <- sum(d$label[b]  == "Positive")
  pos_h <- sum(d$label[!b] == "Positive")
  c(volume_share   = n_b / (n_b + n_h),
    positive_share = pos_b / (pos_b + pos_h),
    net_diff       = mean(d$score[b]) - mean(d$score[!b]))
}

obs <- calc_stats(scored_small)

predicted_share  <- obs[["positive_share"]]
predicted_winner <- ifelse(predicted_share > 0.5, "Xavier Becerra", "Steve Hilton")
winner_share     <- ifelse(predicted_share > 0.5, predicted_share, 1 - predicted_share)

Bootstrap

To see how much of this result could be sampling noise, I resample posts with replacement within each candidate 5,000 times, recompute the prediction each time, and look at how much it moves. This captures noise from having only a limited set of posts. It does not capture bias in who posts or how posts were collected.

set.seed(2026)
B    <- 5000
is_b <- scored_small$candidate == "Xavier Becerra"

boot_once <- function() {
  idx <- c(sample(which(is_b),  replace = TRUE),
           sample(which(!is_b), replace = TRUE))
  calc_stats(scored_small[idx, ])
}

boot <- t(replicate(B, boot_once()))

ci_pos    <- quantile(boot[, "positive_share"], c(0.025, 0.975))
ci_net    <- quantile(boot[, "net_diff"],       c(0.025, 0.975))
se_pos    <- sd(boot[, "positive_share"])
p_becerra <- mean(boot[, "positive_share"] > 0.5)
p_hilton  <- 1 - p_becerra
tibble(
  rule = c("Positive share (chosen rule): Becerra share",
           "Volume share: Becerra share",
           "Net sentiment: Becerra minus Hilton mean score"),
  estimate = c(obs[["positive_share"]], obs[["volume_share"]], obs[["net_diff"]]),
  ci_low   = c(ci_pos[[1]], NA, ci_net[[1]]),
  ci_high  = c(ci_pos[[2]], NA, ci_net[[2]])
) %>%
  mutate(across(where(is.numeric), ~ round(.x, 3))) %>%
  kable(caption = "Observed values and 95% bootstrap intervals (volume share is fixed by data collection, so it has no interval)")
Observed values and 95% bootstrap intervals (volume share is fixed by data collection, so it has no interval)
rule estimate ci_low ci_high
Positive share (chosen rule): Becerra share 0.475 0.357 0.594
Volume share: Becerra share 0.516 NA NA
Net sentiment: Becerra minus Hilton mean score 0.094 -0.174 0.373 
ggplot(tibble(share = boot[, "positive_share"]), aes(share)) +
  geom_histogram(bins = 40, fill = "#1b9e77", color = "white") +
  geom_vline(xintercept = 0.5, linetype = "dashed") +
  geom_vline(xintercept = ci_pos, color = "#d95f02") +
  labs(title = "Bootstrap distribution of predicted Becerra share (positive-share rule)",
       subtitle = "Dashed line = 50% tie; orange lines = 95% interval",
       x = "Predicted Becerra share", y = "Bootstrap replicates") +
  theme_minimal()

Prediction

Using the positive-share rule, the predicted winner is Steve Hilton with a predicted two-candidate share of about 52.5%. Xavier Becerra’s predicted share is 47.5% (95% bootstrap interval 35.7% to 59.4%; bootstrap standard error 6.1 points).

Across 5,000 resamples, Becerra finishes ahead in 32.6% of them and Hilton in 67.4%. The closer these are to 50/50, or the more the interval straddles 50%, the less the post sample can distinguish the candidates from sampling noise alone.

10. Reflection

Who is on Bluesky. Bluesky users are not a random sample of voters. The platform has tended to skew younger, more online, and more left-leaning than the electorate, and a small number of very active accounts can produce a large share of the posts. A model fit to this population can say something about the conversation on Bluesky but very little about how all registered voters will vote. Any lean in this sample is likely to reflect who is on the platform at least as much as who is winning. The bootstrap treats posts as independent, but many posts come from the same authors, so the true uncertainty is larger than the interval above suggests.

Sarcasm and tone versus stance. The Bing lexicon counts individual words. It cannot detect sarcasm, negation (“not good”), quotes of opponents, or news-style neutral reporting, and it knows nothing about politics: a post that angrily attacks one candidate can contain negative words about the attacker’s target and positive words about the alternative, or the reverse. Tone is also not stance. A negative post may come from a supporter angry at the other side, and a positive post may be a joke or a link to an article. Positive-share therefore measures the emotional vocabulary of posts that mention a candidate, not support for that candidate.

More posts does not mean more support. Volume can reflect controversy, scandal, news cycles, bots, or organized campaigns as easily as enthusiasm. Here volume was also fixed by how many posts were collected, so it says nothing about popularity. Even with equal volume, the posts were selected by search, which shapes who appears in the sample.

Bottom line. The bootstrap quantifies only one source of error, sampling noise within this particular set of posts. It says nothing about the larger errors from platform bias, lexicon limits, and the gap between online sentiment and actual votes. The result should be treated as a descriptive snapshot of sentiment among these Bluesky posts, not as a calibrated election forecast.