R Markdown

This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.

When you click the Knit button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this:

AI Usage Disclaimer

I used AI tools to assist in creating and recreating the original R Markdown (RMD) file for this project. AI was also used to help organize the code, improve documentation, and troubleshoot potential errors. The analysis, interpretation of results, and final review were conducted to ensure the accuracy and relevance of the project.

2 paragraph reflection

I used an AI assistant to write R code that combined five Bluesky datasets, cleaned the text, ran a sentiment analysis with the bing lexicon, and built an interactive plotly chart. The AI’s first version produced a working structure, but it didn’t account for problems in our data until I looked at the files closely. The created_at column had been corrupted into values like “07:23.7,” so I couldn’t use it for any time analysis. The files also contained about 19 duplicated posts, a few blank posts, and many non-English posts (mostly Portuguese in week 9), which the English-only bing lexicon can’t score. I also noticed that week 12 covered a different topic from the other four files, so I kept a topic column to compare them instead of merging them into one. After cleaning, the dataset dropped from 517 posts to 324.

The AI’s code also showed me what it can’t do on its own. A lexicon counts words without understanding context, so a neutral news headline about a “lawsuit” or “censorship” scores as negative, and a sarcastic post like “lowkey an under appreciated factor of linear tv” can be scored wrong. Each search returned only about 100 posts, collected in different weeks, so our results describe a small sample and shouldn’t be read as public opinion about Meta or commercials. Human judgment was needed to notice the mixed topics, decide to filter out non-English posts, question whether the scores matched what the posts actually said, and decide what the chart can and can’t claim. The AI helped me write code faster, but I had to check it and interpret the findings.

library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.2.1     ✔ readr     2.2.0
## ✔ forcats   1.0.1     ✔ stringr   1.6.0
## ✔ ggplot2   4.0.4     ✔ tibble    3.3.1
## ✔ lubridate 1.9.5     ✔ tidyr     1.3.2
## ✔ purrr     1.2.2     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(tidytext)
library(plotly)
## 
## Attaching package: 'plotly'
## 
## The following object is masked from 'package:ggplot2':
## 
##     last_plot
## 
## The following object is masked from 'package:stats':
## 
##     filter
## 
## The following object is masked from 'package:graphics':
## 
##     layout
files <- c("Joelyne_Week9(in).csv", "Joelyne_Week10(in).csv",
           "Joelyne_Week11(in).csv", "Tais_week12(in).csv", "my_post (3).csv")

posts <- map_dfr(files, ~ read_csv(.x, col_types = cols(.default = "c")) %>%
                   mutate(source = .x)) %>%
  select(uri, text, langs, like_count, source) %>%
  mutate(
    like_count = as.numeric(like_count),
    topic = if_else(str_detect(source, "week12"), "commercials", "meta")
  )

# Clean
posts_clean <- posts %>%
  filter(!is.na(text)) %>%
  distinct(text, .keep_all = TRUE) %>%
  filter(str_detect(langs, "^en")) %>%
  mutate(
    post_id = row_number(),
    text_clean = text %>%
      str_remove_all("http\\S+|www\\.\\S+") %>%
      str_remove_all("@\\S+") %>%
      str_remove_all("#\\S+") %>%
      str_to_lower()
  )

nrow(posts); nrow(posts_clean)   # sanity check: how many removed?
## [1] 517
## [1] 324
# Tokenize and remove stop words
data(stop_words)
words <- posts_clean %>%
  unnest_tokens(word, text_clean) %>%
  anti_join(stop_words, by = "word")

# Most common words per topic
words %>% count(topic, word, sort = TRUE) %>% group_by(topic) %>% slice_max(n, n = 10)
## # A tibble: 22 × 3
## # Groups:   topic [2]
##    topic       word            n
##    <chr>       <chr>       <int>
##  1 commercials commercials    93
##  2 commercials tv             12
##  3 commercials watch           8
##  4 commercials 2               7
##  5 commercials people          7
##  6 commercials ai              6
##  7 commercials music           6
##  8 commercials games           5
##  9 commercials insurance       5
## 10 commercials wii             5
## # ℹ 12 more rows
# Sentiment per post (bing)
post_sentiment <- words %>%
  inner_join(get_sentiments("bing"), by = "word") %>%
  count(post_id, topic, sentiment) %>%
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>%
  mutate(net = positive - negative)

# Interactive plotly chart: average net sentiment by topic
summary_df <- post_sentiment %>%
  group_by(topic) %>%
  summarise(avg_net = mean(net), posts = n())

plot_ly(summary_df, x = ~topic, y = ~avg_net, type = "bar",
        text = ~paste("Posts scored:", posts), hoverinfo = "text+y") %>%
  layout(title = "Average net sentiment by topic",
         yaxis = list(title = "Avg (positive - negative words)"))