Project 2

Wiktor Lalik

Introduction Essay

Topic. This project looks at what makes a song popular on Spotify. The data is the “Spotify 2000” dataset, a Kaggle dataset of about 2,000 well known songs with audio features pulled from Spotify’s API. I chose it because music is a big part of my daily life, and I wanted to know whether things like loudness, tempo, or song length are linked to how popular a song is.

Variables. I kept eight variables from the original file:

• artist (categorical, nominal): the performer.
• title (categorical, nominal): the song name.
• genre (categorical, nominal): Spotify’s top genre label for the song.
• year (quantitative, discrete): release year.
• bpm (quantitative, continuous): beats per minute, or tempo.
• loudness (quantitative, continuous): average loudness in decibels (dB). Values are negative, and closer to 0 is louder.
• length (quantitative, continuous): song duration in seconds.
• popularity (quantitative, continuous): a 0 to 100 score, where higher means more popular.

Cleaning. I loaded the CSV with read_csv(). I selected only the eight variables above and renamed them to short, clean names. length was read as text because a few values contain commas (for example “1,412”), so I removed the commas with gsub() and converted it to a number with as.numeric(). I checked for missing values (there were none). I then used filter() to keep only songs from 2000 onward and from the five most common genres in that group. This leaves fewer than 800 observations and makes the plots easier to read.

Load Libraries and Data

# Load packages
library(tidyverse)

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.2.1     ✔ readr     2.2.0
✔ forcats   1.0.1     ✔ stringr   1.6.0
✔ ggplot2   4.0.3     ✔ tibble    3.3.1
✔ lubridate 1.9.5     ✔ tidyr     1.3.2
✔ purrr     1.2.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

library(plotly)

Attaching package: 'plotly'

The following object is masked from 'package:ggplot2':

    last_plot

The following object is masked from 'package:stats':

    filter

The following object is masked from 'package:graphics':

    layout

library(RColorBrewer)

library(GGally)

# Read the CSV with readr
spotify <- readr::read_csv("Spotify-2000.csv")

Rows: 1994 Columns: 15
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr  (3): Title, Artist, Top Genre
dbl (11): Index, Year, Beats Per Minute (BPM), Energy, Danceability, Loudnes...
num  (1): Length (Duration)

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

glimpse(spotify)

Rows: 1,994
Columns: 15
$ Index                    <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14…
$ Title                    <chr> "Sunrise", "Black Night", "Clint Eastwood", "…
$ Artist                   <chr> "Norah Jones", "Deep Purple", "Gorillaz", "Fo…
$ `Top Genre`              <chr> "adult standards", "album rock", "alternative…
$ Year                     <dbl> 2004, 2000, 2001, 2007, 2002, 2004, 2002, 200…
$ `Beats Per Minute (BPM)` <dbl> 157, 135, 168, 173, 106, 99, 102, 137, 148, 1…
$ Energy                   <dbl> 30, 79, 69, 96, 82, 46, 71, 96, 92, 67, 5, 46…
$ Danceability             <dbl> 53, 50, 66, 43, 58, 54, 71, 37, 36, 91, 44, 7…
$ `Loudness (dB)`          <dbl> -14, -11, -9, -4, -5, -9, -6, -5, -4, -3, -16…
$ Liveness                 <dbl> 11, 17, 7, 3, 10, 14, 13, 12, 10, 24, 11, 26,…
$ Valence                  <dbl> 68, 81, 52, 37, 87, 14, 54, 21, 23, 66, 31, 3…
$ `Length (Duration)`      <dbl> 201, 207, 341, 269, 256, 247, 257, 366, 223, …
$ Acousticness             <dbl> 94, 17, 2, 0, 1, 0, 6, 0, 0, 0, 88, 1, 2, 18,…
$ Speechiness              <dbl> 3, 7, 17, 4, 3, 2, 3, 14, 8, 7, 4, 8, 3, 10, …
$ Popularity               <dbl> 71, 39, 69, 76, 59, 45, 74, 69, 77, 82, 49, 7…

Cleaning the Data

# Keep only the 8 needed variables and give them short names
spotify <- spotify |>
  select(artist = Artist, title = Title, genre = `Top Genre`, year = Year,
         bpm = `Beats Per Minute (BPM)`, loudness = `Loudness (dB)`,
         length = `Length (Duration)`, popularity = Popularity)

# Length has commas in some rows (e.g. "1,412"), so remove them and make it numeric
spotify$length <- as.numeric(gsub(",", "", spotify$length))

# Check for missing values in each column
colSums(is.na(spotify))

    artist      title      genre       year        bpm   loudness     length 
         0          0          0          0          0          0          0 
popularity 
         0 

Exploring the Variables

# Summary of the quantitative variables
summary(spotify)

       artist           title            genre           year     
 Length   :1994   Length   :1994   Length   :1994   Min.   :1956  
 N.unique : 731   N.unique :1958   N.unique : 149   1st Qu.:1979  
 N.blank  :   0   N.blank  :   0   N.blank  :   0   Median :1993  
 Min.nchar:   2   Min.nchar:   2   Min.nchar:   3   Mean   :1993  
 Max.nchar:  47   Max.nchar: 132   Max.nchar:  27   3rd Qu.:2007  
                                                    Max.   :2019  
      bpm           loudness           length         popularity    
 Min.   : 37.0   Min.   :-27.000   Min.   :  93.0   Min.   : 11.00  
 1st Qu.: 99.0   1st Qu.:-11.000   1st Qu.: 212.0   1st Qu.: 49.25  
 Median :119.0   Median : -8.000   Median : 245.0   Median : 62.00  
 Mean   :120.2   Mean   : -9.009   Mean   : 262.4   Mean   : 59.53  
 3rd Qu.:136.0   3rd Qu.: -6.000   3rd Qu.: 289.0   3rd Qu.: 71.00  
 Max.   :206.0   Max.   : -2.000   Max.   :1412.0   Max.   :100.00  

# Histogram of popularity (quantitative)
ggplot(spotify, aes(x = popularity)) +
  geom_histogram(binwidth = 5, fill = "forestgreen", color = "white") +
  labs(title = "Distribution of Popularity", x = "Popularity", y = "Number of Songs")

# Bar chart of the 10 most common genres (categorical)
spotify |>
  count(genre) |>
  arrange(desc(n)) |>
  head(10) |>
  ggplot(aes(x = reorder(genre, n), y = n)) +
  geom_col(fill = "lightblue") +
  coord_flip() +
  labs(title = "Top 10 Genres", x = "Genre", y = "Number of Songs")

Inclusion/Exclusion Criteria

# Find the 5 most common genres among songs from 2000 onward
top_genres <- spotify |>
  filter(year >= 2000) |>
  count(genre) |>
  arrange(desc(n)) |>
  head(5)

# Keep songs from 2000 onward in those 5 genres
spotify_small <- spotify |>
  filter(year >= 2000, genre %in% top_genres$genre)

# Number of observations (must be 800 or fewer)
nrow(spotify_small)

[1] 271

Multiple Linear Regression

# Correlation plot to pick predictors for popularity
ggpairs(spotify_small, columns = c("popularity", "bpm", "loudness", "length", "year"))

Loudness has the strongest correlation with popularity. BPM, length, and year are weakly correlated with it. I include all four quantitative variables in the model so I can test each one.

# Multiple linear regression
model <- lm(popularity ~ loudness + bpm + length + year, data = spotify_small)
summary(model)

Call:
lm(formula = popularity ~ loudness + bpm + length + year, data = spotify_small)

Residuals:
    Min      1Q  Median      3Q     Max 
-46.165 -12.810   0.287  12.740  38.632 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept) 420.670412 368.480369   1.142    0.255    
loudness      2.859436   0.404118   7.076 1.32e-11 ***
bpm           0.003659   0.033632   0.109    0.913    
length        0.006282   0.015053   0.417    0.677    
year         -0.173625   0.182990  -0.949    0.344    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 16.9 on 266 degrees of freedom
Multiple R-squared:  0.1626,    Adjusted R-squared:   0.15 
F-statistic: 12.91 on 4 and 266 DF,  p-value: 1.271e-09

# Pull out the coefficients for the equation
b <- round(coef(model), 3)

Equation: Popularity = 420.67 + 2.859(loudness) + 0.004(bpm) + 0.006(length) + -0.174(year)

Analysis. Loudness has a very small p-value, so it is a significant predictor: each 1 dB increase in loudness goes with roughly a 3-point increase in popularity, holding the other variables constant. BPM, length, and year have large p-values (above 0.05), so they are not significant. The adjusted R^2 is low, which means the model explains only a small share of the variation in popularity. Loudness matters most out of all other variables.

Visualization 1: Loudness vs. Popularity

# Scatterplot colored by genre, with a trend line and an annotation
p1 <- ggplot(spotify_small, aes(x = loudness, y = popularity, color = genre,
                                text = paste(title, "-", artist))) +
  geom_point(alpha = 0.7, size = 2) +
  geom_smooth(aes(group = 1), method = "lm", se = FALSE, color = "black", linetype = "dashed") +
  annotate("text", x = -12, y = 95, label = "Louder songs trend more popular", size = 3.5) +
  scale_color_brewer(palette = "Set1") +
  theme_minimal() +
  labs(title = "Louder Songs Tend to Be More Popular Across Five Genres (2000 Onward)",
       x = "Loudness (dB, closer to 0 is louder)",
       y = "Popularity (0-100)",
       color = "Top Genre",
       caption = "Source: Spotify 2000 dataset (Kaggle)")

# Make it interactive with mouseover
ggplotly(p1, tooltip = c("text", "x", "y"))

`geom_smooth()` using formula = 'y ~ x'

Warning: The following aesthetics were dropped during statistical transformation: text.
ℹ This can happen when ggplot fails to infer the correct grouping structure in
  the data.
ℹ Did you forget to specify a `group` aesthetic or to convert a numerical
  variable into a factor?

Visualization 2: Average Popularity by Genre

# Average popularity per genre
genre_avg <- spotify_small |>
  group_by(genre) |>
  summarize(avg_popularity = mean(popularity), .groups = "drop")

ggplot(genre_avg, aes(x = reorder(genre, avg_popularity), y = avg_popularity, fill = genre)) +
  geom_col() +
  geom_text(aes(label = round(avg_popularity, 1)), hjust = -0.2) +
  coord_flip() +
  ylim(0, 85) +
  scale_fill_brewer(palette = "Dark2") +
  theme_bw() +
  labs(title = "Dance Pop and Modern Rock Songs Are the Most Popular on Average",
       x = "Top Genre",
       y = "Average Popularity (0-100)",
       fill = "Top Genre",
       caption = "Source: Spotify 2000 dataset (Kaggle)")

What the Visualizations Show

The first plot shows each song as a point, colored by genre, with loudness on the x-axis and popularity on the y-axis. Hovering over a point shows the song and artist. The dashed line shows an upward trend: louder songs tend to be more popular, which matches the regression result that loudness was the only significant predictor. The second plot shows average popularity by genre. Dance pop and modern rock have the highest averages, and dutch indie has the lowest. One surprise was how weak the other variables were. Tempo, length, and release year barely related to popularity at all. Given more time, I would have added a shiny app with a sliderInput() for year.

Sources

https://www.kaggle.com/datasets/iamsumat/spotify-top-2000s-mega-dataset/data
image: https://www.google.com/url?sa=t&source=web&rct=j&url=https%3A%2F%2Fcommons.wikimedia.org%2Fwiki%2FFile%3ASpotify_icon.svg&ved=0CBcQjRxqFwoTCNiyqbzc_pcDFQAAAAAdAAAAABA5&opi=89978449