Taylor Swift and Billboard Comparison

Final Project

Author

Meredith Briggs

Intro to the Project

With the popularity of the Eras Tour, I wanted to take a deeper look into Taylor Swift’s music and look for some trends in her music over the years. Additionally, I was wondering how Taylor’s popularity was reflected on the Billboard Top 100.

Below are the packages I will use to manipulate the data throughout this document.

Taylor Swift

Taylor Swift is pop artist who has 14 albums. She recent ended her Era’s Tour which honored her music from all of her albums. I wanted to take a deeper look into Taylor Swift’s discography to see how she might have changed over time.

The Spotify API wasused to gather data of every Taylor Swift song. Below is a data dictionary of the Spotify variables for each song.

Data Dictionary

Variable	Description
track_id	Unique track identifier
track_name	Name of track or song
album_id	Unique album identifier
album_name	Name of the album
duration_ms	Time length in milliseconds
time_signature	Indicates beats per measure
danceability	Measure from 0.0 to 1.0 and represents likelihood to dance based on musical components
energy	Measure from 0.0 to 1.0 and represents a perceptual measure of intensity and activity
loudness	Measure of how loud a song is by adjusting the Loudness Units Relative to Full Scale down to -14
explicit	Use of explicit words (TRUE/FALSE)
key_mode	Combination of key_name and mode_name
popularity	Measure of song popularity 0 to 100 with larger numbers being more popular
artist_id	Unique artist identifier
artist_name	Name of the artsit

Trends and Analysis

How long is the discography?

Knowing how many albums Taylor has made, I was wondering how long it would take to listen to her whole discography.

# A tibble: 1 × 1
  total_time_hr
          <dbl>
1          38.5

Based on my calculations, it would take 38.5 hours to listen to every single song with Taylor as the main artist. This means if you listened to every song with no break, it would take more than 1 and half days to listen to everything.

Energy per Version

Taylor famously re-recorded some her original albums due to issues with her original record lable. This cuased me to think if the energy for the original albums is different than the re-recorded albums.

The bars are grouped and colored by Original and Taylor’s Version. We can see that the original albums did have higher energy than Taylor’s Version. This could be attributed to Taylor’s older age and small tweaks when recording the newer versions.

Favorite Musical Key

Many artists tend to use the same musical key in order to show case their singing ability. I will find Taylor’s most used musical key to see if there are certain notes Taylor prefers singing.

This chart indicates that Taylor definitely has a favorite key to write in. C major has over 100 tracks while the next highest key use is G major with around 100 tracks. The C major and G major keys must be very comfortable to Taylor to sing, as well as a comfortable key for her fans to sing along.

Most Danceable Album

All of Taylor’s albums have a variety of upbeat or more sad songs. When thinking about the popular music on the radio, I would think songs with a higher danceability would appear more. So I will create a chart comparing the danceability of all her albums.

For this analysis, I will only use a list of Taylor’s 11 main albums and only use one version of said albums.

Based on the boxplots, we can see that the Lover album had the highest danceability. Folklore has one of the lowest danceabilities. This is something to keep in mind when looking at which albums appear on the Billboard Top 100.

Most Popular Songs

Now that we have looked into Taylor’s albums, I want to find her most popular songs on Spotify. Since Spotify measures popularity over time, I would expect to see a mix of songs from her her albums that have not been re-recorded becasue the listens will not be split between the two. Also, I would expect to see songs from Lover since it had the highest danceability.

# A tibble: 10 × 5
   track_id...1           track_name              popularity danceability energy
   <chr>                  <chr>                        <dbl>        <dbl>  <dbl>
 1 1BxfuPKGuaTgP7aM0Bbdwr Cruel Summer                    90        0.552  0.702
 2 2OzhQlSqBEmt7hmkYxfT6m Fortnight (feat. Post …         86        0.504  0.386
 3 3hUxzQpSfdDqwM3ZTFQY0K august                          85        0.532  0.623
 4 1dGr1c8CrMLDpV6mPbImSI Lover                           84        0.359  0.543
 5 4q5YezDOIPcoLr8R81x9qy I Can Do It With a Bro…         84        0.701  0.751
 6 1R0a2iXumgCiFb7HEZ7gUE Don‚Äôt Blame Me                83        0.615  0.534
 7 4R2kfaDFhslZEMJqAFNpdd cardigan                        83        0.613  0.581
 8 0V3wPSX9ygBnCm8psDIegu Anti-Hero                       82        0.633  0.648
 9 1MgV7FIyNxIG7WzMRJV5HC my tears ricochet               79        0.469  0.263
10 1P17dC1amhFzptugyAO7Il Look What You Made Me …         79        0.766  0.709

As suspected, most of the songs come from non re-recorded albums. This list contains songs from both Lover and Folklore which indicates that danceability is not the only predictor of popularity.

The Billboard Top 100

The Billboard Top 100 is a list of the top 100 most popular songs on the radio. I want to see how Taylor Swift appears on this chart for the past year.

In order to scrape my data, I used the Billboard Top 100 website (https://www.billboard.com/charts/hot-100/) and my user agent was my personal identifier. I scraped the songs, artists, position on the chart, position on the chart in the previous week, weeks on chart, the peak position on the chart, and the week of the chart.

Data Preparation

After scraping the data, I want to be able to use the billboard_week as an actual data with the lubridate package. This means I have to clean the data by removing the “week of” string and covert the rest of the string into a date.

Trends and Analysis

How many Taylor Swift songs occurred on the Top 100?

# A tibble: 33 × 1
   billboard_song                      
   <chr>                               
 1 I Can Do It With A Broken Heart     
 2 Down Bad                            
 3 Who's Afraid Of Little Old Me?      
 4 Guilty As Sin?                      
 5 But Daddy I Love Him                
 6 My Boy Only Breaks His Favorite Toys
 7 So Long, London                     
 8 The Smallest Man Who Ever Lived     
 9 The Tortured Poets Department       
10 loml                                
# ℹ 23 more rows

Taylor had 33 different songs on the Billboard Top 100 in 2024. Upon reviewing this list, majority of the songs are from Taylor’s most recent album, Tortured Poets Department. This is expected since the album was released earlier this year.

What 10 songs have been on the chart the longest?

In order to find Taylor’s most popular songs on the radio, I will filter to create a list of the 10 songs that have been on the Billboard Top 100 the longest. By being on the Top 100 longer, it would imply the song is more popular and liked by people who listen to the radio.

# A tibble: 10 × 2
   billboard_song                                             billboard_weeks_on
   <chr>                                                                   <dbl>
 1 Cruel Summer                                                               54
 2 I Can Do It With A Broken Heart                                            31
 3 Is It Over Now? (Taylor's Version) [From The Vault]                        22
 4 Down Bad                                                                   12
 5 Who's Afraid Of Little Old Me?                                             11
 6 Guilty As Sin?                                                             10
 7 But Daddy I Love Him                                                        8
 8 My Boy Only Breaks His Favorite Toys                                        8
 9 So Long, London                                                             8
10 Now That We Don't Talk (Taylor's Version) [From The Vault]                  8

Again, the Tortured Poets Department album has more occurrences because of its recent release. However, “Cruel Summer” from the Lover album has been on the chart the longest with 54 weeks, which shows older Taylor songs are still popular. Below is the trend of “Cruel Summer” during 2024.

Cruel Summer

Cruel Summer appeared at the top of both popularity rankings for Billboard and Spotify. Therefore I want to track its position on the Billboard Chart.

We can see that its popularity began to fall throughout 2024, and has not on the Billboard chart since May. However, being on the chart for 54 weeks is still impressive and could be seen as the most popular Taylor song.

Now that we have narrowed down the Billboard Songs to 10 most popular Taylor Swift songs, we can conduct so comparisons between the other top 10.

Comparison of Top 10 Songs

From the Billboard analysis, I have found the top 10 Taylor Swift songs the radio listens to. Something to keep in mind is that the demographic of people who listen to the radio can differ from those who pay for and use a Spotify Premium account exclusively. Therefore, I will compare this to the top 10 songs Taylor has per Spotify. AThis comparison will show which of her songs is currently the most popular per each of these demographics.

Data Preparation

First I will assign the track id from the Taylor Data Set to the matching songs in the Billboard Top 100 Data Set.

Then I will create variables for numeric ranking and origin of Top 10 position.

Finally I will join the tables so that it will be easier to compare.

Trends and Analysis

Popularity of Billboard

Now that we have gathered the Spotify data for the Billboard songs, I want to know where these songs fall in the Spotify popularity rankings

# A tibble: 10 × 3
   track_name                                                position popularity
   <chr>                                                        <dbl>      <dbl>
 1 Cruel Summer                                                     1         90
 2 I Can Do It With a Broken Heart                                  2         76
 3 Guilty as Sin?                                                   6         76
 4 Is It Over Now? (Taylor's Version) (From The Vault)              3         75
 5 Down Bad                                                         4         75
 6 Who‚Äôs Afraid of Little Old Me?                                 5         75
 7 My Boy Only Breaks His Favorite Toys                             8         75
 8 Now That We Don't Talk (Taylor's Version) (From The Vaul…       10         75
 9 But Daddy I Love Him                                             7         74
10 So Long, London                                                  9         74

As we know, Cruel Summer appeared at the top of popularity for both lists. We can also see that the rest of the songs were still fairly popular songs on the Spotify data. This discrepency could originate from the fact that one of Spotify key measures for popularity is total listens ever, not just this year like the Billboard data.

Based on these results, we can see that Billboard does have a slightly higher average danceability, as expected since more upbeat songs are on the radio. Suprisingly, Spotify has a much higher energy rating than Billboard. I expected the higher energy, or Spotify measure of intensity, songs to be more popular on the radio than Spotify.

Overall, there are no true predictors of a song or albums popularity. Despite this fact, Taylor Swift has produced many popular songs over the years with a variety of music keys, danceability, and energy. Her ability to create music with a variety of features could be an attribute of her success and popularity.

--- title: "Taylor Swift and Billboard Comparison" subtitle: "Final Project" author: "Meredith Briggs" format: html: # Options related to HTML output. code-tools: TRUE # Allow the code tools option showing in the output. embed-resources: TRUE # Embeds all components into a single HTML file. execute: # Options related to the execution of code chunks. warning: FALSE # FALSE: Code chunk sarnings are hidden by default. message: FALSE # FALSE: Code chunk messages are hidden by default. echo: FALSE # TRUE: Show all code in the output. editor: visual --- # Intro to the Project With the popularity of the Eras Tour, I wanted to take a deeper look into Taylor Swift's music and look for some trends in her music over the years. Additionally, I was wondering how Taylor's popularity was reflected on the Billboard Top 100. Below are the packages I will use to manipulate the data throughout this document. ```{r} #| label: setup #| include: TRUE library(tidyverse) # For all the tidy things library(lubridate) # For dealing with dates in a polite way library(skimr) # For reviewing data easily library(dplyr) # For easy manipulation of data library(knitr) # Useful tools when 'knitting' (rendering) Quarto documents. ``` # Taylor Swift Taylor Swift is pop artist who has 14 albums. She recent ended her Era's Tour which honored her music from all of her albums. I wanted to take a deeper look into Taylor Swift's discography to see how she might have changed over time. The Spotify API wasused to gather data of every Taylor Swift song. Below is a data dictionary of the Spotify variables for each song. ```{r} #| label: loading taylor data #| include: FALSE taylor_discography<- read_csv("https://myxavier-my.sharepoint.com/:x:/g/personal/briggsm4_xavier_edu/Efap7GcnifxFvhXC6JJ-NIIBD8UJ24PiBkOjKoZ9mJD6kg?download=1") ``` ## Data Dictionary | Variable | Description | |---------------------------|---------------------------------------------| | track_id | Unique track identifier | | track_name | Name of track or song | | album_id | Unique album identifier | | album_name | Name of the album | | duration_ms | Time length in milliseconds | | time_signature | Indicates beats per measure | | danceability | Measure from 0.0 to 1.0 and represents likelihood to dance based on musical components | | energy | Measure from 0.0 to 1.0 and represents a perceptual measure of intensity and activity | | loudness | Measure of how loud a song is by adjusting the Loudness Units Relative to Full Scale down to -14 | | explicit | Use of explicit words (TRUE/FALSE) | | key_mode | Combination of key_name and mode_name | | popularity | Measure of song popularity 0 to 100 with larger numbers being more popular | | artist_id | Unique artist identifier | | artist_name | Name of the artsit | ## Trends and Analysis ### How long is the discography? Knowing how many albums Taylor has made, I was wondering how long it would take to listen to her whole discography. ```{r} #| label: how long to listen #| include: TRUE taylor_discography %>% summarize( total_time_ms = sum(duration_ms, na.rm = TRUE), total_time_min = total_time_ms / 60000, total_time_hr = total_time_min / 60) %>% select(total_time_hr) ``` Based on my calculations, it would take 38.5 hours to listen to every single song with Taylor as the main artist. This means if you listened to every song with no break, it would take more than 1 and half days to listen to everything. ### Energy per Version Taylor famously re-recorded some her original albums due to issues with her original record lable. This cuased me to think if the energy for the original albums is different than the re-recorded albums. ```{r} #| label: compare energy #| include: TRUE taylor_discography %>% group_by(album_name) %>% filter(str_detect(album_name, "Fearless") | str_detect(album_name, "Red") | str_detect(album_name, "Speak Now") | str_detect(album_name, "1989") )%>% filter(!str_detect(album_name, "Deluxe"), !str_detect(album_name, "Live"), !str_detect(album_name, "Platinum")) %>% summarize(average_energy = mean(energy, na.rm = TRUE)) %>% select(album_name,average_energy) %>% ggplot(aes(y=album_name,x=average_energy,fill=album_name))+ geom_col()+ scale_fill_manual(values = c("lightblue","steelblue","gold","gold4","red", "darkred","purple", "purple4"))+ labs(title="Average Energy in Original vs Taylor's Version", x="Album Name", y="Average Energy")+ theme(legend.position = "none") ``` The bars are grouped and colored by Original and Taylor's Version. We can see that the original albums did have higher energy than Taylor's Version. This could be attributed to Taylor's older age and small tweaks when recording the newer versions. ### Favorite Musical Key Many artists tend to use the same musical key in order to show case their singing ability. I will find Taylor's most used musical key to see if there are certain notes Taylor prefers singing. ```{r} #| label: musical key use #| include: TRUE taylor_discography %>% group_by(key_mode) %>% filter(!is.na(key_mode)) %>% summarize(total=sum(n())) %>% ggplot(aes(y=reorder(key_mode,total), x=total))+ geom_col(fill="seagreen")+ labs(title="Musical Key Use", y="Musical Key", x="Count of Use") ``` This chart indicates that Taylor definitely has a favorite key to write in. C major has over 100 tracks while the next highest key use is G major with around 100 tracks. The C major and G major keys must be very comfortable to Taylor to sing, as well as a comfortable key for her fans to sing along. ### Most Danceable Album All of Taylor's albums have a variety of upbeat or more sad songs. When thinking about the popular music on the radio, I would think songs with a higher danceability would appear more. So I will create a chart comparing the danceability of all her albums. For this analysis, I will only use a list of Taylor's 11 main albums and only use one version of said albums. ```{r} #| label: compare dancebility #| include: TRUE taylor_discography %>% group_by(album_name) %>% filter(album_id %in% c( "7mzrIsaAjnXihW3InKjlC3", "4hDok0OAJd57SGIT8xuWJH", "5AEDGbliTTfjOB8TSm1sxt", "6kZ42qRrzov54LcAk4onW9", "64LU4c1nfjz1t4VnGhagcg", "6DEjYFkNZh67HP7R9PSZvv", "1NAmidJlEaVgA3MpcPFYGq", "2fenSS68JI1h4Fo296JfGr", "2Xoteh7uEpea4TohMxjtaq", "151w1FgRZfnKZA9FEcg9Z3", "5H7ixXZfsNMGbIE5OBSpcb")) %>% # created list of albums by hand because otherwise filtering would be very extensive ggplot(aes(y = album_name, x = danceability)) + geom_boxplot() + labs( title = "Danceability of Each Album", x = "Album Name", y = "Danceability") ``` Based on the boxplots, we can see that the Lover album had the highest danceability. Folklore has one of the lowest danceabilities. This is something to keep in mind when looking at which albums appear on the Billboard Top 100. ### Most Popular Songs Now that we have looked into Taylor's albums, I want to find her most popular songs on Spotify. Since Spotify measures popularity over time, I would expect to see a mix of songs from her her albums that have not been re-recorded becasue the listens will not be split between the two. Also, I would expect to see songs from Lover since it had the highest danceability. ```{r} #| label: most popular #| include: TRUE top_10_taylor <- taylor_discography %>% arrange(desc(popularity)) %>% slice_head(n = 10) %>% select(`track_id...1`, track_name,popularity, danceability, energy) top_10_taylor ``` As suspected, most of the songs come from non re-recorded albums. This list contains songs from both Lover and Folklore which indicates that danceability is not the only predictor of popularity. # The Billboard Top 100 The Billboard Top 100 is a list of the top 100 most popular songs on the radio. I want to see how Taylor Swift appears on this chart for the past year. In order to scrape my data, I used the Billboard Top 100 website (<https://www.billboard.com/charts/hot-100/>) and my user agent was my personal identifier. I scraped the songs, artists, position on the chart, position on the chart in the previous week, weeks on chart, the peak position on the chart, and the week of the chart. ```{r} #| label: loading billboard data #| include: FALSE #| echo: FALSE billboard_top_100<- read_csv("https://myxavier-my.sharepoint.com/:x:/g/personal/briggsm4_xavier_edu/EbLqb8jH9uFMrVNdMJdPPj4BjoKr79Tuz733O7hIhqE5Hw?download=1") ``` ## Data Preparation After scraping the data, I want to be able to use the billboard_week as an actual data with the lubridate package. This means I have to clean the data by removing the "week of" string and covert the rest of the string into a date. ```{r} #| label: data manipulation #| include: FALSE #| echo: FALSE billboard_top_100$billboard_week<- mdy(str_extract(billboard_top_100$billboard_week, "\\w+ \\d{1,2}, \\d{4}")) ``` ## Trends and Analysis ### How many Taylor Swift songs occurred on the Top 100? ```{r} #| label: Taylor Songs on Chart #| include: TRUE billboard_top_100 %>% filter(billboard_artist=="Taylor Swift") %>% distinct(billboard_song) ``` Taylor had 33 different songs on the Billboard Top 100 in 2024. Upon reviewing this list, majority of the songs are from Taylor's most recent album, Tortured Poets Department. This is expected since the album was released earlier this year. ### What 10 songs have been on the chart the longest? In order to find Taylor's most popular songs on the radio, I will filter to create a list of the 10 songs that have been on the Billboard Top 100 the longest. By being on the Top 100 longer, it would imply the song is more popular and liked by people who listen to the radio. ```{r} #| label: longest on chart top_10_billboard <- billboard_top_100 %>% filter(billboard_artist=="Taylor Swift") %>% arrange(desc(billboard_weeks_on)) %>% distinct(billboard_song, billboard_artist, .keep_all = TRUE) %>% select(billboard_song, billboard_weeks_on) %>% slice_head(n = 10) top_10_billboard ``` Again, the Tortured Poets Department album has more occurrences because of its recent release. However, "Cruel Summer" from the Lover album has been on the chart the longest with 54 weeks, which shows older Taylor songs are still popular. Below is the trend of "Cruel Summer" during 2024. ### Cruel Summer Cruel Summer appeared at the top of both popularity rankings for Billboard and Spotify. Therefore I want to track its position on the Billboard Chart. ```{r} #| label: Trend of Cruel Summer position on chart #| include: TRUE billboard_top_100 %>% filter(billboard_song=="Cruel Summer") %>% ggplot(aes(x=billboard_week,y=billboard_position))+ geom_line()+ geom_point()+ scale_y_reverse() + labs(title="Trend of Cruel Summer", x="Week", y="Billboard Position") ``` We can see that its popularity began to fall throughout 2024, and has not on the Billboard chart since May. However, being on the chart for 54 weeks is still impressive and could be seen as the most popular Taylor song. Now that we have narrowed down the Billboard Songs to 10 most popular Taylor Swift songs, we can conduct so comparisons between the other top 10. # Comparison of Top 10 Songs From the Billboard analysis, I have found the top 10 Taylor Swift songs the radio listens to. Something to keep in mind is that the demographic of people who listen to the radio can differ from those who pay for and use a Spotify Premium account exclusively. Therefore, I will compare this to the top 10 songs Taylor has per Spotify. AThis comparison will show which of her songs is currently the most popular per each of these demographics. ## Data Preparation First I will assign the track id from the Taylor Data Set to the matching songs in the Billboard Top 100 Data Set. ```{r} #| label: find track_id top_10_billboard$track_id...1 <- c( "1BxfuPKGuaTgP7aM0Bbdwr", "2fPvQfGQEZOKtJ9qXeL4x8", "1Iq8oo9XkmmvCQiGOfORiz", "1kbEbBdEgQdQeLXCJh28pJ", "2d8UxVNhJinc8uat9PoM9y", "799KrpEbhZp0MHeiA8YK9P", "4QMgEffJQuKtjCNvqfRZ0m", "7uGYWMwRy24dm7RUDDhUlD", "7wAkQFShJ27V8362MqevQr", "5KD6AEm19QnMbfWpfoOHMl") # i tried using joins but the taylor discography containing multiples mad this very difficult so it was easier to search the track ids by had and insert the variable ``` Then I will create variables for numeric ranking and origin of Top 10 position. ```{r} #| label: add positions top_10_billboard$position <- c(1,2,3,4,5,6,7,8,9,10) top_10_taylor$position <- c(1,2,3,4,5,6,7,8,9,10) top_10_billboard$origin <- "Billboard" top_10_taylor$origin <- "Spotify" ``` Finally I will join the tables so that it will be easier to compare. ```{r} #| label: top 10 creation top_10_billboard_full <- left_join(top_10_billboard, taylor_discography, by = "track_id...1")%>% select(`track_id...1`, track_name,popularity, danceability, energy, position, origin) top_10_all <- bind_rows(top_10_taylor,top_10_billboard_full) ``` ## Trends and Analysis ### Popularity of Billboard Now that we have gathered the Spotify data for the Billboard songs, I want to know where these songs fall in the Spotify popularity rankings ```{r} #| label: billboard popularity top_10_all %>% filter(origin=="Billboard") %>% arrange(desc(popularity)) %>% select(track_name, position, popularity) ``` As we know, Cruel Summer appeared at the top of popularity for both lists. We can also see that the rest of the songs were still fairly popular songs on the Spotify data. This discrepency could originate from the fact that one of Spotify key measures for popularity is total listens ever, not just this year like the Billboard data. ::: panel-tabset ### Danceability Comparison ```{r} #| label: Danceability Comparison top_10_all %>% group_by(origin) %>% ggplot(aes(x = origin, y = danceability)) + geom_boxplot() + labs(title = "Danceability Comparison", x = "Origin", y = "Danceability") ``` ### Energy Comparison ```{r} #| label: Energy Comparison top_10_all %>% group_by(origin) %>% ggplot(aes(x = origin, y = energy )) + geom_boxplot() + labs(title = "Energy Comparison", x = "Origin", y = "Energy") ``` ::: Based on these results, we can see that Billboard does have a slightly higher average danceability, as expected since more upbeat songs are on the radio. Suprisingly, Spotify has a much higher energy rating than Billboard. I expected the higher energy, or Spotify measure of intensity, songs to be more popular on the radio than Spotify. Overall, there are no true predictors of a song or albums popularity. Despite this fact, Taylor Swift has produced many popular songs over the years with a variety of music keys, danceability, and energy. Her ability to create music with a variety of features could be an attribute of her success and popularity.