Data Project 2

Topic: Finding the Relationship Between Song Characteristics and Spotify Popularity

Music is a major part of everyday life, and different songs have characteristics that make them appealing to listeners. Some songs have faster tempos, higher energy, or greater danceability, while others have more acoustic or instrumental elements. I wanted to explore whether these musical characteristics have any relationship with how popular a song becomes. This project uses a dataset containing 2,000 songs and 18 variables, including artist, song title, genre, release year, popularity, danceability, energy, loudness, tempo, and other audio features.

The dataset includes six categorical variables and twelve quantitative variables. The categorical variables describe information such as the artist, song title, genre, explicit content, musical key, and mode. The quantitative variables measure characteristics such as song duration, popularity, danceability, energy, acousticness, valence, and tempo. The dataset covers songs released between 1998 and 2020. I plan to focus on danceability, energy, valence, tempo, loudness, and other audio characteristics to investigate their relationships. I chose this topic because I enjoy listening to music and wanted to understand whether the musical qualities I notice in songs are related to their popularity.

The photo is from https://thesuffolkjournal.com/45105/opinion/the-state-of-pop-music-needs-a-shuffle/#photo and the artist is Brooklyn Leighton

Dataset: “C:/Users/akeeb/Downloads/spotifysongs.csv” I found this dataset in the class datasets folder. https://drive.google.com/file/d/1CRGzZNh8BaCRIhvUU6CH9SJVDhpZb-E5/view

library(tidyverse)

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.2.1     ✔ readr     2.2.0
✔ forcats   1.0.1     ✔ stringr   1.6.0
✔ ggplot2   4.0.3     ✔ tibble    3.3.1
✔ lubridate 1.9.5     ✔ tidyr     1.3.2
✔ purrr     1.2.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

library(readr)
library(ggplot2)
library(plotly)

Attaching package: 'plotly'

The following object is masked from 'package:ggplot2':

    last_plot

The following object is masked from 'package:stats':

    filter

The following object is masked from 'package:graphics':

    layout

library(RColorBrewer)

# Load the Spotify dataset
spotify <- read_csv("C:/Users/akeeb/Downloads/spotifysongs.csv")

Rows: 2000 Columns: 18
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr  (3): artist, song, genre
dbl (14): duration_ms, year, popularity, danceability, energy, key, loudness...
lgl  (1): explicit

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

# View the dataset
head(spotify)

# A tibble: 6 × 18
  artist   song  duration_ms explicit  year popularity danceability energy   key
  <chr>    <chr>       <dbl> <lgl>    <dbl>      <dbl>        <dbl>  <dbl> <dbl>
1 Britney… Oops…      211160 FALSE     2000         77        0.751  0.834     1
2 blink-1… All …      167066 FALSE     1999         79        0.434  0.897     0
3 Faith H… Brea…      250546 FALSE     1999         66        0.529  0.496     7
4 Bon Jovi It's…      224493 FALSE     2000         78        0.551  0.913     0
5 *NSYNC   Bye …      200560 FALSE     2000         65        0.614  0.928     8
6 Sisqo    Thon…      253733 TRUE      1999         69        0.706  0.888     2
# ℹ 9 more variables: loudness <dbl>, mode <dbl>, speechiness <dbl>,
#   acousticness <dbl>, instrumentalness <dbl>, liveness <dbl>, valence <dbl>,
#   tempo <dbl>, genre <chr>

# Check the dimensions
dim(spotify)

[1] 2000   18

# Check variable names
names(spotify)

 [1] "artist"           "song"             "duration_ms"      "explicit"        
 [5] "year"             "popularity"       "danceability"     "energy"          
 [9] "key"              "loudness"         "mode"             "speechiness"     
[13] "acousticness"     "instrumentalness" "liveness"         "valence"         
[17] "tempo"            "genre"           

# Check data types
str(spotify)

spc_tbl_ [2,000 × 18] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
 $ artist          : chr [1:2000] "Britney Spears" "blink-182" "Faith Hill" "Bon Jovi" ...
 $ song            : chr [1:2000] "Oops!...I Did It Again" "All The Small Things" "Breathe" "It's My Life" ...
 $ duration_ms     : num [1:2000] 211160 167066 250546 224493 200560 ...
 $ explicit        : logi [1:2000] FALSE FALSE FALSE FALSE FALSE TRUE ...
 $ year            : num [1:2000] 2000 1999 1999 2000 2000 ...
 $ popularity      : num [1:2000] 77 79 66 78 65 69 86 68 75 77 ...
 $ danceability    : num [1:2000] 0.751 0.434 0.529 0.551 0.614 0.706 0.949 0.708 0.713 0.72 ...
 $ energy          : num [1:2000] 0.834 0.897 0.496 0.913 0.928 0.888 0.661 0.772 0.678 0.808 ...
 $ key             : num [1:2000] 1 0 7 0 8 2 5 7 5 6 ...
 $ loudness        : num [1:2000] -5.44 -4.92 -9.01 -4.06 -4.81 ...
 $ mode            : num [1:2000] 0 1 1 0 0 1 0 1 0 1 ...
 $ speechiness     : num [1:2000] 0.0437 0.0488 0.029 0.0466 0.0516 0.0654 0.0572 0.0322 0.102 0.0379 ...
 $ acousticness    : num [1:2000] 0.3 0.0103 0.173 0.0263 0.0408 0.119 0.0302 0.0267 0.273 0.00793 ...
 $ instrumentalness: num [1:2000] 1.77e-05 0.00 0.00 1.35e-05 1.04e-03 9.64e-05 0.00 0.00 0.00 2.93e-02 ...
 $ liveness        : num [1:2000] 0.355 0.612 0.251 0.347 0.0845 0.07 0.0454 0.467 0.149 0.0634 ...
 $ valence         : num [1:2000] 0.894 0.684 0.278 0.544 0.879 0.714 0.76 0.861 0.734 0.869 ...
 $ tempo           : num [1:2000] 95.1 148.7 136.9 120 172.7 ...
 $ genre           : chr [1:2000] "pop" "rock, pop" "pop, country" "rock, metal" ...
 - attr(*, "spec")=
  .. cols(
  ..   artist = col_character(),
  ..   song = col_character(),
  ..   duration_ms = col_double(),
  ..   explicit = col_logical(),
  ..   year = col_double(),
  ..   popularity = col_double(),
  ..   danceability = col_double(),
  ..   energy = col_double(),
  ..   key = col_double(),
  ..   loudness = col_double(),
  ..   mode = col_double(),
  ..   speechiness = col_double(),
  ..   acousticness = col_double(),
  ..   instrumentalness = col_double(),
  ..   liveness = col_double(),
  ..   valence = col_double(),
  ..   tempo = col_double(),
  ..   genre = col_character()
  .. )
 - attr(*, "problems")=<pointer: 0x000001f1c08fdce0> 

# Explore the dataset
summary(spotify)

       artist            song       duration_ms      explicit      
 Length   :2000   Length   :2000   Min.   :113000   Mode :logical  
 N.unique : 835   N.unique :1879   1st Qu.:203580   FALSE:1449     
 N.blank  :   0   N.blank  :   0   Median :223280   TRUE :551      
 Min.nchar:   2   Min.nchar:   1   Mean   :228748                  
 Max.nchar:  41   Max.nchar: 114   3rd Qu.:248133                  
                                   Max.   :484146                  
      year        popularity     danceability        energy      
 Min.   :1998   Min.   : 0.00   Min.   :0.1290   Min.   :0.0549  
 1st Qu.:2004   1st Qu.:56.00   1st Qu.:0.5810   1st Qu.:0.6220  
 Median :2010   Median :65.50   Median :0.6760   Median :0.7360  
 Mean   :2009   Mean   :59.87   Mean   :0.6674   Mean   :0.7204  
 3rd Qu.:2015   3rd Qu.:73.00   3rd Qu.:0.7640   3rd Qu.:0.8390  
 Max.   :2020   Max.   :89.00   Max.   :0.9750   Max.   :0.9990  
      key            loudness            mode         speechiness     
 Min.   : 0.000   Min.   :-20.514   Min.   :0.0000   Min.   :0.02320  
 1st Qu.: 2.000   1st Qu.: -6.490   1st Qu.:0.0000   1st Qu.:0.03960  
 Median : 6.000   Median : -5.285   Median :1.0000   Median :0.05985  
 Mean   : 5.378   Mean   : -5.512   Mean   :0.5535   Mean   :0.10357  
 3rd Qu.: 8.000   3rd Qu.: -4.168   3rd Qu.:1.0000   3rd Qu.:0.12900  
 Max.   :11.000   Max.   : -0.276   Max.   :1.0000   Max.   :0.57600  
  acousticness       instrumentalness       liveness         valence      
 Min.   :0.0000192   Min.   :0.0000000   Min.   :0.0215   Min.   :0.0381  
 1st Qu.:0.0140000   1st Qu.:0.0000000   1st Qu.:0.0881   1st Qu.:0.3867  
 Median :0.0557000   Median :0.0000000   Median :0.1240   Median :0.5575  
 Mean   :0.1289549   Mean   :0.0152260   Mean   :0.1812   Mean   :0.5517  
 3rd Qu.:0.1762500   3rd Qu.:0.0000683   3rd Qu.:0.2410   3rd Qu.:0.7300  
 Max.   :0.9760000   Max.   :0.9850000   Max.   :0.8530   Max.   :0.9730  
     tempo              genre     
 Min.   : 60.02   Length   :2000  
 1st Qu.: 98.99   N.unique :  59  
 Median :120.02   N.blank  :   0  
 Mean   :120.12   Min.nchar:   3  
 3rd Qu.:134.27   Max.nchar:  37  
 Max.   :210.85                   

# Check for missing values
colSums(is.na(spotify))

          artist             song      duration_ms         explicit 
               0                0                0                0 
            year       popularity     danceability           energy 
               0                0                0                0 
             key         loudness             mode      speechiness 
               0                0                0                0 
    acousticness instrumentalness         liveness          valence 
               0                0                0                0 
           tempo            genre 
               0                0 

# Check for duplicate rows
sum(duplicated(spotify))

[1] 59

# Check the different genres
count(spotify, genre, sort = TRUE)

# A tibble: 59 × 2
   genre                              n
   <chr>                          <int>
 1 pop                              428
 2 hip hop, pop                     277
 3 hip hop, pop, R&B                244
 4 pop, Dance/Electronic            221
 5 pop, R&B                         178
 6 hip hop                          124
 7 hip hop, pop, Dance/Electronic    78
 8 rock                              58
 9 rock, pop                         43
10 Dance/Electronic                  41
# ℹ 49 more rows

# Check the number of songs per year
count(spotify, year, sort = TRUE)

# A tibble: 23 × 2
    year     n
   <dbl> <int>
 1  2012   115
 2  2017   111
 3  2001   108
 4  2010   107
 5  2018   107
 6  2005   104
 7  2014   104
 8  2011    99
 9  2015    99
10  2016    99
# ℹ 13 more rows

# Clean and transform the dataset
spotify_clean <- spotify %>%
  mutate(
    explicit = factor(explicit),
    genre = factor(genre),
    mode = factor(mode),
    key = factor(key),
    duration_minutes = duration_ms / 60000
  )

# Explore the most common music genres
spotify_clean %>%
  count(genre, sort = TRUE) %>%
  head(10) %>%
  ggplot(aes(x = reorder(genre, n), y = n)) +
  geom_col(fill = "blue") +
  coord_flip() +
  labs(
    title = "Top 10 Most Common Music Genres",
    x = "Genre",
    y = "Number of Songs"
  ) +
  theme_minimal()

# Explore the distribution of danceability
ggplot(spotify_clean, aes(x = danceability)) +
  geom_histogram(
    bins = 30,
    fill = "purple",
    color = "white"
  ) +
  labs(
    title = "Distribution of Song Danceability",
    x = "Danceability Score",
    y = "Number of Songs"
  ) +
  theme_minimal()

# Explore the distribution of energy
ggplot(spotify_clean, aes(x = energy)) +
  geom_histogram(
    bins = 30,
    fill = "orange",
    color = "white"
  ) +
  labs(
    title = "Distribution of Song Energy",
    x = "Energy Score",
    y = "Number of Songs"
  ) +
  theme_minimal()

# Create a multiple linear regression model
model <- lm(
  danceability ~ valence + energy + tempo +
    speechiness + acousticness + liveness + loudness,
  data = spotify_clean
)

# View the regression results
summary(model)

Call:
lm(formula = danceability ~ valence + energy + tempo + speechiness + 
    acousticness + liveness + loudness, data = spotify_clean)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.49317 -0.07052  0.00470  0.08096  0.32338 

Coefficients:
               Estimate Std. Error t value Pr(>|t|)    
(Intercept)   0.8359025  0.0278086  30.059  < 2e-16 ***
valence       0.2979256  0.0129183  23.062  < 2e-16 ***
energy       -0.2908496  0.0255238 -11.395  < 2e-16 ***
tempo        -0.0007102  0.0001006  -7.061 2.28e-12 ***
speechiness   0.1624508  0.0280633   5.789 8.22e-09 ***
acousticness -0.1284572  0.0171954  -7.470 1.19e-13 ***
liveness     -0.1105398  0.0192385  -5.746 1.06e-08 ***
loudness      0.0033060  0.0018175   1.819   0.0691 .  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.119 on 1992 degrees of freedom
Multiple R-squared:  0.2846,    Adjusted R-squared:  0.282 
F-statistic: 113.2 on 7 and 1992 DF,  p-value: < 2.2e-16

# Examine correlations between quantitative variables
spotify_clean %>%
  select(danceability, valence, energy, tempo,
         speechiness, acousticness, liveness, loudness) %>%
  cor()

             danceability     valence      energy       tempo   speechiness
danceability   1.00000000  0.40317765 -0.10403836 -0.17341799  0.1455896851
valence        0.40317765  1.00000000  0.33447443 -0.02507583  0.0736050031
energy        -0.10403836  0.33447443  1.00000000  0.15371909 -0.0570181923
tempo         -0.17341799 -0.02507583  0.15371909  1.00000000  0.0577474709
speechiness    0.14558969  0.07360500 -0.05701819  0.05774747  1.0000000000
acousticness  -0.06542889 -0.12812789 -0.44546852 -0.10365999  0.0003941595
liveness      -0.12641287  0.01903963  0.15676071  0.02863569  0.0611721377
loudness      -0.03331486  0.23215045  0.65101552  0.08070880 -0.0763881859
              acousticness    liveness    loudness
danceability -0.0654288929 -0.12641287 -0.03331486
valence      -0.1281278858  0.01903963  0.23215045
energy       -0.4454685177  0.15676071  0.65101552
tempo        -0.1036599909  0.02863569  0.08070880
speechiness   0.0003941595  0.06117214 -0.07638819
acousticness  1.0000000000 -0.11004306 -0.31003918
liveness     -0.1100430562  1.00000000  0.10215898
loudness     -0.3100391798  0.10215898  1.00000000

# Visualization 1: Danceability and Valence

p1 <- ggplot(
  spotify_clean,
  aes(x = valence, y = danceability, color = explicit)
) +
  geom_point(alpha = 0.6) +
  geom_smooth(
    method = "lm",
    se = FALSE,
    color = "black"
  ) +
  scale_color_brewer(palette = "Set1") +
  labs(
    title = "Relationship Between Song Positivity and Danceability",
    x = "Valence",
    y = "Danceability Score",
    color = "Explicit Content",
  ) +
  theme_minimal()

ggplotly(p1)

`geom_smooth()` using formula = 'y ~ x'

References:

Dataset: https://drive.google.com/file/d/1CRGzZNh8BaCRIhvUU6CH9SJVDhpZb-E5/view

Image: https://thesuffolkjournal.com/45105/opinion/the-state-of-pop-music-needs-a-shuffle/#photo