Week 12 | Data Dive — Time-based Data

Loading the Billboard Hot 100 Number Ones Dataset

library(tidyverse)
library(dplyr)
library(xts)
library(tsibble)

tuesdata <- tidytuesdayR::tt_load(2025, week = 34)

billboard <- tuesdata$billboard
topics <- tuesdata$topics

Preview of the Dataset

head(billboard)
## # A tibble: 6 × 105
##   song   artist date                weeks_at_number_one non_consecutive rating_1
##   <chr>  <chr>  <dttm>                            <dbl>           <dbl>    <dbl>
## 1 Poor … Ricky… 1958-08-04 00:00:00                   2               0        4
## 2 Nel B… Domen… 1958-08-18 00:00:00                   5               1        7
## 3 Littl… The E… 1958-08-25 00:00:00                   1               0        5
## 4 It's … Tommy… 1958-09-29 00:00:00                   6               0        3
## 5 It's … Conwa… 1958-11-10 00:00:00                   2               1        7
## 6 Tom D… The K… 1958-11-17 00:00:00                   1               0        5
## # ℹ 99 more variables: rating_2 <dbl>, rating_3 <dbl>, overall_rating <dbl>,
## #   divisiveness <dbl>, label <chr>, parent_label <chr>, cdr_genre <chr>,
## #   cdr_style <chr>, discogs_genre <chr>, discogs_style <chr>,
## #   artist_structure <dbl>, featured_artists <chr>,
## #   multiple_lead_vocalists <dbl>, group_named_after_non_lead_singer <dbl>,
## #   talent_contestant <chr>, posthumous <dbl>, artist_place_of_origin <chr>,
## #   front_person_age <dbl>, artist_male <dbl>, artist_white <dbl>, …
head(topics)
## # A tibble: 6 × 1
##   lyrical_topics   
##   <chr>            
## 1 Addiction        
## 2 Anger            
## 3 Appreciation     
## 4 Badassery        
## 5 Bad Behavior     
## 6 Bad Relationships

Selecting a Column of Data that Encodes Time and Converting it into a Date in R

Column of Data that Encodes Time:

  • date — First week for a song to hit number one
class(billboard$date)
## [1] "POSIXct" "POSIXt"
billboard <- billboard |>
  dplyr::mutate(date = as.Date(date))
class(billboard$date)
## [1] "Date"

Choosing a Column of Data to Analyze Over Time

Column of Data to Analyze Over Time - Response-like Variable of Interest:

  • overall_rating — Sample mean of ratings 1, 2, and 3 as provided by judges 1, 2, and 3

Creating a tsibble Object Made up of new_date & overall_rating

billboard_ts <- billboard |>
  select(date, overall_rating) |>
  as_tibble(index = date)

Plotting overall_rating Over Time (Full Range of Time for Dataset: 1958-2025)

library(ggplot2)

billboard_ts |>
  ggplot(aes(x = date, y = overall_rating)) +
  geom_point(alpha = 0.6) +
  geom_line(alpha = 0.4) +
  labs(
    title = "Overall Rating for Billboard Hot 100 #1s Over Time",
    x = "Year",
    y = "Overall Rating"
  )

Based on the full range plot above (1958-2025), we can see that the ratings seem to fluctuate throughout the period as a whole where a majority of the ratings sit roughly between 5-8 and don’t have a common trend from start to finish. There are the occasional very low/very high outliers, but there’s no exact longer term movement either up or down. Lastly, the density of the points seems pretty consistent from start to finish, but the modern era (2000-2025) contains more drastic jumps between songs which could mean modern music is more diverse, or judging has changed a bit in this era specifically compared to the other eras.

Plotting overall_rating Over Time (Early Window: 1958-1980)

billboard_ts |>
  filter(date < as.Date("1980-01-01")) |>
  ggplot(aes(date, overall_rating)) +
  geom_point(alpha = 0.7) +
  geom_line(alpha = 0.4) +
  labs(
    title = "Overall Rating (1958–1980)",
    x = "Year",
    y = "Overall Rating"
  )

The ratings for this earlier period of the dataset (1958-1980) seem consistent and clustered, where a majority of songs sit between an overall rating of 5-7.5, of course with the occasional song reaching above or falling below that range. There also seems to be fewer drastic jumps or falls compared to the previous plot, which could just mean judging in this era was more consistent overall, and the smoother pattern could mean this was an era where the judges’ criteria for a #1 song was more consistent as well.

Plotting overall_rating Over Time (Middle Window: 1980-2000)

billboard_ts |>
  filter(date >= as.Date("1980-01-01"),
         date < as.Date("2000-01-01")) |>
  ggplot(aes(date, overall_rating)) +
  geom_point(alpha = 0.7) +
  geom_line(alpha = 0.4) +
  labs(
    title = "Overall Rating (1980–2000)",
    x = "Year",
    y = "Overall Rating"
  )

This middle era window (1980-2000) definitely shows a more noticeable spread in the overall ratings of songs. Many songs still fall in the 5-7.5 rating range, but there are many outliers, both high and low, compared to the previous era. These drastic changes may be caused due to large shifts in genre and what was deemed as “popular” music in this era, for example, the come up of hip-hip, R&B, or even alternative, and may have thrown judges for a loop, with either positive or negative reactions. This pattern could suggest the introduction of many new styles/genres of music, and judgement could be thrown off a bit as judges didn’t know how to actually score these new genres compared to the previously known and familiar genres.

Plotting overall_rating Over Time (Modern Window: 2000-2025)

billboard_ts |>
  filter(date >= as.Date("2000-01-01")) |>
  ggplot(aes(date, overall_rating)) +
  geom_point(alpha = 0.7) +
  geom_line(alpha = 0.4) +
  labs(
    title = "Overall Rating (2000–2025)",
    x = "Year",
    y = "Overall Rating"
  )

In this modern era plot (2000-2025), we can see that ratings swing dramatically from song to song with many songs reaching a higher range of overall ratings, as well as falling to the lower range of overall ratings a bit more frequently compared to the previous eras. The jumps in this modern era are far greater than the jumps in any of the previous eras shown above. This may suggest that there are rapid changes in production trends overall, as well as how the introduction of music streaming services has changed the game as personal music selection and tracking has grown immensely. Again, similar to the previous middle era window, judgement criteria may have been thrown for a loop once again given these technological changes. Of all the windows, this modern era is definitely the least stable ratings-wise.

Utilizing LOESS Smoothing to Detect a Season in the Data, Incorporating PACF to Illustrate Seasonality

billboard_ts |>
  ggplot(aes(x = date, y = overall_rating)) +
  geom_point(alpha = 0.4) +
  geom_smooth(method = "loess", span = 0.2, se = FALSE, color = "blue", linewidth = 1) +
  labs(
    title = "Smoothed Trend of Overall Ratings Over Time",
    x = "Year",
    y = "Overall Rating"
  )
## `geom_smooth()` using formula = 'y ~ x'

ratings_ts <- ts(billboard_ts$overall_rating)

pacf(ratings_ts, main = "PACF of Overall Ratings")

Interpreting the Results of LOESS and PACF

Utilizing LOESS for smoothing and PACF for illustration of seasonality, both implementations show little to no seasonality for overall rating for the Billboard Hot 100 #1 songs dataset. LOESS shows simply only long-term fluctuations that don’t repeat at consistent intervals, so true seasons aren’t exactly present in the dataset in this instance. The PACF plot also supports this as there are no significant partial autocorrelations at any of the lag points, and there is no consistent repeating pattern. With these results, it is clear that overall ratings behave more like noise rather than a seasonal time series, which in hindsight makes sense as the #1 songs don’t necessarily occur at fixed time intervals.