library(tidyverse)
library(tidytext)
library(httr2)
library(wordcloud)
library(RColorBrewer)
library(writexl)
library(knitr)

Part 1: Reading Reflection (5 Sentences)

Collecting user-generated text through APIs — whether NewsAPI headlines or BlueSky posts — lets marketers measure brand narratives in near real time before those signals fully appear in sales or survey data (Xu, 2026). The Week 3 NewsAPI workflow shows that cleaning, deduplication, and stop-word removal are as important as the API call itself, because noisy text produces misleading word-frequency and sentiment results (Silge & Robinson, 2017). Tokenizing with tidytext and ranking the most common terms is a simple but powerful first cut at theme discovery: hubs like valuation, launch, or competition often reveal how audiences frame a brand. Visualizations such as frequency bar charts and word clouds turn those counts into managerial stories that non-technical stakeholders can grasp quickly. The main limitation is that raw frequencies ignore context and sarcasm, so frequency analysis should be treated as exploratory brand listening rather than a definitive measure of public opinion (Liu, 2012).

Part 2: Option 2 — BlueSky Scraping & Word Frequency Analysis

Brief description of the data source

I chose Option 2 (BlueSky) and collected public posts mentioning SpaceX / Starship / SPCX, which continues the SpaceX theme used in earlier MSBA 580 labs and matches the industry focus in the course YouTube scraping tutorial (Xu, syntax_for_lab4). BlueSky’s AppView search endpoint (app.bsky.feed.searchPosts) provides open access to public posts — useful as researchers move away from restricted X/Twitter APIs (assignment micro-learning note).

Data collection process (API code)

`%||%` <- function(x, y) if (is.null(x)) y else x

fetch_bsky <- function(q, limit = 100) {
  req <- request("https://api.bsky.app/xrpc/app.bsky.feed.searchPosts") |>
    req_url_query(q = q, limit = limit) |>
    req_headers(`User-Agent` = "Mozilla/5.0 MSBA580-Lab4") |>
    req_error(is_error = function(resp) FALSE)

  resp <- req_perform(req)
  if (resp_status(resp) != 200) {
    warning(paste("Query", q, "returned status", resp_status(resp)))
    return(tibble())
  }

  posts <- resp_body_json(resp, simplifyVector = FALSE)$posts
  map_dfr(posts, function(p) {
    tibble(
      uri = p$uri %||% NA_character_,
      author = p$author$handle %||% NA_character_,
      display_name = p$author$displayName %||% NA_character_,
      text = p$record$text %||% NA_character_,
      created_at = p$record$createdAt %||% NA_character_,
      like_count = as.integer(p$likeCount %||% 0),
      reply_count = as.integer(p$replyCount %||% 0),
      repost_count = as.integer(p$repostCount %||% 0),
      query = q
    )
  })
}

# Prefer a saved snapshot for reproducible knitting; refresh live if missing
if (file.exists("bluesky_spacex_posts.rds")) {
  posts_raw <- readRDS("bluesky_spacex_posts.rds")
} else {
  posts_raw <- bind_rows(
    fetch_bsky("SpaceX"),
    fetch_bsky("Starship"),
    fetch_bsky("SPCX stock")
  ) %>%
    distinct(uri, .keep_all = TRUE) %>%
    filter(!is.na(text), nchar(text) > 0)

  saveRDS(posts_raw, "bluesky_spacex_posts.rds")
  write_csv(posts_raw, "bluesky_spacex_posts.csv")
  write_xlsx(posts_raw, "bluesky_spacex_posts.xlsx")
}

glimpse(posts_raw)
## Rows: 283
## Columns: 9
## $ uri          <chr> "at://did:plc:bdxufr7cirpo7bp5zdbke5jo/app.bsky.feed.post…
## $ author       <chr> "newsstocks.bsky.social", "newsfromaneighbor.bsky.social"…
## $ display_name <chr> "", "News From A Neighbor", "", "TechBroBot Billionaire",…
## $ text         <chr> "SpaceX Valuation Factors Highlighted by Bernstein Analys…
## $ created_at   <chr> "2026-08-02T20:02:41.125Z", "2026-08-02T20:01:05.579Z", "…
## $ like_count   <int> 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 1, 0, 0, …
## $ reply_count  <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, …
## $ repost_count <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, …
## $ query        <chr> "SpaceX", "SpaceX", "SpaceX", "SpaceX", "SpaceX", "SpaceX…

Screenshot / evidence of collection

The chunk above is the data-collection process. The saved files in this project folder document the scrape:

  • bluesky_spacex_posts.csv / .xlsx — attachable scraped dataset
  • bluesky_spacex_posts.rds — R snapshot used for reproducible analysis
cat("Number of unique posts:", nrow(posts_raw), "\n")
## Number of unique posts: 283
cat("Unique authors:", n_distinct(posts_raw$author), "\n")
## Unique authors: 236
cat("Date range:",
    as.character(min(as.Date(posts_raw$created_at), na.rm = TRUE)), "to",
    as.character(max(as.Date(posts_raw$created_at), na.rm = TRUE)), "\n")
## Date range: 2026-07-27 to 2026-08-02
posts_raw %>%
  select(author, created_at, like_count, text) %>%
  slice_head(n = 8) %>%
  mutate(text = str_trunc(text, 120)) %>%
  kable(caption = "Sample of collected BlueSky posts")
Sample of collected BlueSky posts
author created_at like_count text
newsstocks.bsky.social 2026-08-02T20:02:41.125Z 0 SpaceX Valuation Factors Highlighted by Bernstein Analysis

Bernstein has identified four key topics that are essenti… | |newsfromaneighbor.bsky.social |2026-08-02T20:01:05.579Z | 0|In today’s Daily Updates we cover: SpaceX rocket to hit moon, Indonesia ferry on fire, helicopter crash in Greece, an… | |zephyr7591.bsky.social |2026-08-02T19:59:36.968Z | 0|www.space.com/astronomy/mo… | |techbrobot.bsky.social |2026-08-02T19:50:24.098872+00:00 | 0|Earth is a cradle, not a cage. Becoming a multi-planetary species isn’t just ambition, it’s an insurance policy for h… | |jdkiff.bsky.social |2026-08-02T19:46:03.545Z | 0|#ThoughtForTheDay On the SpaceX moon crash When life hands you lemons, grab tequila and salt! | |pwrdbyrog.bsky.social |2026-08-02T19:42:03.835Z | 1|Musk: Just unveiled Starship V2.0: 100% pure American tech, completely explosion-proof & failure-free. No more “rapid… | |auto-blog.bsky.social |2026-08-02T19:40:08Z | 0|Elon Musk denied a report that Tesla is considering a merger with SpaceX, a deal that could face scrutiny over Tesla’… | |exwifenews.bsky.social |2026-08-02T19:37:00.604Z | 0|Tesla reportedly might sell its my ex-wife business ahead of a SpaceX merger - TechCrunch |

Clean and prepare the text

Following the Week 3 NewsAPI / tidytext pattern: lowercase, strip URLs and punctuation, tokenize, remove stop words and numbers (Silge & Robinson, 2017; Xu, 2026).

custom_stop_words <- bind_rows(
  stop_words,
  tibble(
    word = c(
      "https", "http", "t.co", "amp", "rt", "na",
      "it's", "i'm", "don't", "spacexs"
    ),
    lexicon = "custom"
  )
)

post_words <- posts_raw %>%
  mutate(
    text_clean = str_replace_all(text, "https?://\\S+", " "),
    text_clean = str_replace_all(text_clean, "@\\w+", " "),
    text_clean = str_replace_all(text_clean, "#", " "),
    text_clean = str_replace_all(text_clean, "[^[:alnum:][:space:]]", " "),
    text_clean = str_squish(str_to_lower(text_clean))
  ) %>%
  select(uri, author, text_clean) %>%
  unnest_tokens(word, text_clean) %>%
  filter(!str_detect(word, "^[0-9]+$")) %>%
  anti_join(custom_stop_words, by = "word")

word_freq <- post_words %>%
  count(word, sort = TRUE)

word_freq %>%
  slice_head(n = 20) %>%
  kable(caption = "Top 20 words in BlueSky SpaceX-related posts")
Top 20 words in BlueSky SpaceX-related posts
word n
spacex 167
starship 80
stock 78
spcx 65
musk 41
elon 34
rocket 30
space 30
www 30
de 29
ipo 27
en 19
moon 19
tesla 19
time 18
news 17
earnings 16
money 16
troopers 15
investors 14

Word frequency visualization

top_n <- 20

word_freq %>%
  slice_max(n, n = top_n) %>%
  mutate(word = reorder(word, n)) %>%
  ggplot(aes(n, word)) +
  geom_col(fill = "#1d9bf0") +
  labs(
    title = "Most Common Terms in BlueSky Posts about SpaceX / Starship",
    subtitle = paste("Based on", nrow(posts_raw), "unique posts"),
    x = "Frequency",
    y = NULL
  ) +
  theme_minimal(base_size = 12)

Word cloud visualization

set.seed(580)
par(mar = rep(0, 4))
wordcloud(
  words = word_freq$word,
  freq = word_freq$n,
  min.freq = 2,
  max.words = 80,
  random.order = FALSE,
  rot.per = 0.15,
  colors = brewer.pal(8, "Dark2")
)
title("Word Cloud: BlueSky SpaceX Conversation")

Short interpretation of findings

Across 283 BlueSky posts, the highest-frequency terms cluster around SpaceX, Starship, launch/mission language, and adjacent competitive or market framing. That pattern is consistent with social listening research: brand mentions on open platforms often co-occur with product milestones and speculative investor language rather than with detailed technical discussion alone (Liu, 2012; Silge & Robinson, 2017). For a marketing or investor-relations analyst, the practical takeaway is that BlueSky conversation currently amplifies launch / vehicle narratives and market interest signals — so campaign or IR messaging timed to launch windows is likely to meet an already-activated audience. Frequency analysis cannot separate sarcasm from praise, and search-based samples over-represent users who chose those keywords, so these results should be framed as directional theme discovery, ideally followed by sentiment coding or a larger multi-week scrape (Xu, 2026).

Appendix A — Option 1 path (YouTube / tuber) for reference

The class syntax file walks through Google Cloud OAuth + tuber::get_all_comments() for the SpaceX video xH8FM8oepkM. That path requires interactive browser authentication, so it is shown here for documentation rather than executed in this knit.

# From syntax_for_lab4.txt (Xu) — run interactively in RStudio after OAuth setup
library(tuber)
library(dplyr)
library(readr)

app_id     <- Sys.getenv("YOUTUBE_CLIENT_ID")
app_secret <- Sys.getenv("YOUTUBE_CLIENT_SECRET")
yt_oauth(app_id, app_secret, token = "")

video_id <- "xH8FM8oepkM"
comments_raw <- get_all_comments(video_id = video_id)

comments_clean <- comments_raw %>%
  as_tibble() %>%
  distinct(id, .keep_all = TRUE) %>%   # column is often `id`, not comment_id
  select(any_of(c("authorDisplayName", "textOriginal", "publishedAt", "likeCount", "id")))

write_csv(comments_clean, "comments1.csv")

Part 3: Peer Responses

(Complete on the discussion board after publishing: reply to two classmates with specific suggestions on their scraping choice, cleaning steps, or interpretation.)

References

Liu, B. (2012). Sentiment Analysis and Opinion Mining. Morgan & Claypool.

Silge, J., & Robinson, D. (2017). Text Mining with R: A Tidy Approach. O’Reilly. https://www.tidytextmining.com/

Xu, Z. (2026). Mining the News: Real-time Sentiment and Text Analysis of Headlines with NewsAPI. MSBA 580 tutorial.

Xu, Z. (n.d.). Scraping Comments from a SpaceX YouTube Video Using R (syntax_for_lab4.txt / tuber + YouTube Data API v3).

Bluesky API. app.bsky.feed.searchPosts — https://docs.bsky.app/