Introduction

• In this project, we aim to explore trends and patterns in music track data, focusing on the popular songs on Spotify.
• Music streaming platforms like Spotify provide a wealth of data that can be analyzed to uncover insights about the music industry, artist popularity, and listener preferences.
• One of the key aspects of music analysis is understanding the musical elements that contribute to a song’s popularity. By analyzing various attributes such as the key, bpm (beats per minute), danceability, energy, and other features, we aim to identify patterns that might explain why certain keys are more prevalent in popular music.
• This project will specifically investigate the significant difference in the mean bpm (beats per minute) between songs in specific keys as well as the relationship between the keys and modes.

Problem Statement

• Objectives:

1. Identify the most common key used in popular Spotify tracks.
2. Analyze and visualize the relationship between the key of a song and other musical attributes such as bpm, danceability, and energy. 3.Investigate whether there is a significant difference in the mean, bpm(beats per minutes) between songs in the key notes through Hypothesis testing.
3. Determine if there is an association between a song’s key and mode through Chi-Square testing.
4. Provide insights into the musical attributes that characterize popular songs based on the analysis.

RPubs link information

• This presentation has also been published to RPubs and is available at the link here.

Data

The dataset used for this analysis is sourced from Kaggle.com (Kaggle 2024). It includes 24 attributes for each of the 953 tracks.

music <- read_csv("C:/Users/preya/Documents/MAnalytics/AA/A2/Popular_Spotify_Songs.csv")

• Some of the variables which will be focused on for the analysis are:

track name: The name of the track.
artist(s) name: The name(s) of the artist(s) who performed the track.
artist count: The number of artists involved in the track.
streams: The number of streams the track has received.
bpm: The beats per minute of the track.
key: The key in which the track is composed.
mode: The modality (major or minor) of the track.
danceability%: A measure of how suitable a track is for dancing.

Data

• The data includes track data from various streaming services, namely Spotify, Apple Music, Deezer and Shazam.
  
• For the purpose of our report we are focusing on Spotify data only, so below we have taken steps to remove the columns pertaining to the other platforms by using the select() function.
  To simplify data analysis and visualization, using complete.cases() the rows containing missing or NA values have been removed.

#removing columns with apple music, deezer and shazam music data
music_select <- music %>% select(-matches("apple"),-matches("deezer"),-matches("shazam"))

#removing rows that contain no missing data by subsetting complete observations
complete.cases(music_select) %>% sum()
music_new <- music_select[complete.cases(music_select), ]

• There are 19 remaining variables and 857 observations after preprocessing the data, of which 14 are numeric and 5 are of character type.
  For the numeric variables, we have different scales of data along the dataset, like year and percentage, along with normal integer values.

• Out of the 5 character variables, 2 of them - key and mode - has been converted to factors having 11 and 2 levels respectively.

music_new$key <- as.factor(music_new$key) 
music_new$key %>% levels()

##  [1] "A"  "A#" "B"  "C#" "D"  "D#" "E"  "F"  "F#" "G"  "G#"

music_new$mode <- as.factor(music_new$mode) 
music_new$mode %>% levels()

## [1] "Major" "Minor"

Descriptive Statistics and Visualisation

• There are 24 keys in music (Classic FM 2024) split into 12 major and 12 minor keys.
• The 858 popular music tracks included in our sample are each created using one key. The breakdown of the top 6 keys can be seen below.

music_new$key_mode <- paste(music_new$key, music_new$mode, sep=" ")
key_counts <- music_new %>% count(key_mode) 
key_counts %>% arrange(desc(n)) %>% head() %>% kable()

• Using paste(), a new column key_mode was created by merging the key and mode values with a space separator.

• In conclusion, C#Major is the most frequently used key by the songs in the data set.

• The distribution of keys used in the popular Spotify tracks are also depicted in the bar graph on the next page. Using the ggplot package we have created a bar plot to visualize the amount of times each key is used across the songs in the data set.
  geom_bar(): Adds a bar plot and counts the occurrences happening on each key.
  scale_fill_brewer(): Applies the “Paired” colour palette for better visual distinction.

ggplot(music_new, aes(x = key, fill = key)) + geom_bar() +  
  scale_fill_brewer(palette = "Paired" ) +  theme_minimal() + labs(title = "Most Common Key Used in Popular Spotify Tracks", x = "Key", y = "Count")

• In conclusion, the C# which is highlighted in green has the highest key count with 73 and D# which is highlighted in red has the lowest key count with 12.

• Now we look into the BPM (Beats per Minute) variable. Below is a summary including the min, Q1, median, Q3, max, mean and standard deviation of the variable.

music_new %>% summarise(Min = min(bpm), Q1 = quantile(bpm,probs = .25),
                        Median = median(bpm, na.rm = TRUE),
                        Q3 = quantile(bpm,probs = .75,na.rm = TRUE),
                        Max = max(bpm,na.rm = TRUE),
                        Mean = mean(bpm, na.rm = TRUE),
                        SD = sd(bpm, na.rm = TRUE)) %>% kable()

• Additionally, we are also visualizing the Tempo Distribution of the Spotify Popular tracks, wherein we are creating a histogram with the help of ggplot2 package.
  geom_histogram(): Adds a histogram to the plot, which represents the distribution of the bpm values.

ggplot(music_new, aes(x = bpm)) +  geom_histogram(binwidth = 5, fill = "skyblue", color = "black") +  
  theme_minimal() + labs(title = "Tempo Distribution of Spotify Popular Tracks", x = "Tempo (BPM)", y = "Count")

• Also, we are visualizing the quartiles of distribution by using the boxplot() function which creates a horizontal boxplot of the bpm.

boxplot(music_new$bpm, main="Box Plot of BPM(Beats per Minute", xlab="BPM",horizontal=TRUE, col = "lightgreen")
music_new$bpm %>% mean() %>% abline(v=.,col='black', lty = "dashed")

• In the below graph, we create a scatter plot to visualize the relationship between danceability % and the number of streams of Spotify popular tracks using the geom_point() function which adds a layer of points to the plot, with each point representing a track.

music_new$streams_mil <- as.numeric(music_new$streams)/10000000
ggplot(music_new, aes(x = `danceability_%`, y=streams_mil)) +  geom_point(size=0.7) +  
  labs(title = "Danceability Percentage of Spotify Popular Tracks", x = "Danceability (%)", y = "Streams (in millions)")

• In conclusion, there seems to be no direct correlation between danceability% and the no. of streams. It is not the only attribute of a song listeners are looking for in the music they listen to.

Hypothesis Testing

# Filter the dataset for songs in key C and key G
songs_C <- subset(music_new, key == 'C#')$bpm
songs_G <- subset(music_new, key == 'G')$bpm

# Check for normality
shapiro_test_C <- shapiro.test(songs_C)
shapiro_test_G <- shapiro.test(songs_G)

# Check for equal variances
var_test <- var.test(songs_C, songs_G)

# Perform t-test based on variance test result
t_test_result <- t.test(songs_C, songs_G, var.equal = var_test$p.value > 0.05)

# Extract confidence interval
conf_interval <- t_test_result$conf.int

# Print results
print(shapiro_test_C)

## 
##  Shapiro-Wilk normality test
## 
## data:  songs_C
## W = 0.95763, p-value = 0.0008201

Hypothesis Testing

print(shapiro_test_G)

## 
##  Shapiro-Wilk normality test
## 
## data:  songs_G
## W = 0.95775, p-value = 0.00357

print(var_test)

## 
##  F test to compare two variances
## 
## data:  songs_C and songs_G
## F = 0.99917, num df = 119, denom df = 95, p-value = 0.9908
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
##  0.6781117 1.4600121
## sample estimates:
## ratio of variances 
##          0.9991732

Hypothesis Testing

print(t_test_result)

## 
##  Two Sample t-test
## 
## data:  songs_C and songs_G
## t = 0.034956, df = 214, p-value = 0.9721
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -7.385157  7.651824
## sample estimates:
## mean of x mean of y 
##  122.3417  122.2083

print(conf_interval)

## [1] -7.385157  7.651824
## attr(,"conf.level")
## [1] 0.95

• In the above R code section, it is investigated whether there is a significant difference in the mean bpm (beats per minute) between songs in the key C# and G. As the dataset does not include genre variable, the key of the song is taken into account. The keys C# and G are taken specifically because most of the pop dancemusic songs are made in the key C# whereas G is a common key for most of the rock songs. By using these two keys, the analysis compares two different genres of the music indirectly.

Hypothesis Testing

• The null and alternate hypotheses in this scenario are:
  Null Hypothesis (H0): Songs in C# have the same mean bpm as songs in G.
  Alternative Hypothesis (HA): The mean beats per minute (bpm) of songs in C# differs from those in G.

• Shapiro-Wilk normality test:
  Key C#: W = 0.95763; p-value = 0.0008201. Key G: W = 0.95775; p-value = 0.00357.
  The p-values for both keys are less than 0.05, demonstrating that the bpm for songs in both keys deviates from the normal distribution.

• F-Test with Equal Variances:
  F = 0.99917; p-value = 0.9908.
  The p-value is much larger than 0.05, showing that the bpm variances for keys C# and G songs are equivalent.

• Two-Sample T-Test:
  t = 0.034956; df = 214; p-value = 0.9721. 95% confidence interval: [-7.385157, 7.651824]. Means: Key C# = 122.3417; Key G = 122.2083.
  The p-value is 0.9721, significantly larger than 0.05, indicating we cannot reject the null hypothesis. There is no discernible variation in mean bpm between key C# and G songs.

• After hypothesis testing, it was discovered that the bpm for songs in keys C# and G are not regularly distributed, and the bpm variances between the two groups are identical. The t-test and the confidence interval that includes zero show no significant difference in mean bpm between songs in keys C# and G. The data imply that the song’s key has no major effect on its bpm.

Categorical association

• When analyzing music data, understanding the relationship between different musical elements can provide valuable insights into the composition and structure of songs. One such relationship that is of particular interest is the association between the key and mode of a track.

• By investigating the association between key and mode, we can explore whether certain keys are more commonly used in major or minor modes.

• To determine if there is a significant association between key and mode in the music data, we can perform a categorical association analysis using the chi-squared test of association.

• The hypotheses and assumptions are as follows:
  Null Hypothesis (H0): There is no association between the key of a song and whether it is in major or minor mode.
  Alternative Hypothesis (HA): There is an association between the key of a song and its mode.

keymodetab <- table(music_new$key,music_new$mode)

chi <- chisq.test(keymodetab)
chi

## 
##  Pearson's Chi-squared test
## 
## data:  keymodetab
## X-squared = 75.841, df = 10, p-value = 3.263e-12

Categorical association

chi$observed

##     
##      Major Minor
##   A     41    33
##   A#    27    30
##   B     35    46
##   C#    73    47
##   D     66    15
##   D#    12    21
##   E     17    45
##   F     44    45
##   F#    30    43
##   G     66    30
##   G#    63    28

chi$expected

##     
##         Major    Minor
##   A  40.92882 33.07118
##   A# 31.52625 25.47375
##   B  44.80047 36.19953
##   C# 66.37106 53.62894
##   D  44.80047 36.19953
##   D# 18.25204 14.74796
##   E  34.29172 27.70828
##   F  49.22520 39.77480
##   F# 40.37573 32.62427
##   G  53.09685 42.90315
##   G# 50.33139 40.66861

Categorical association

chi$p.value

## [1] 3.263108e-12

• The p-value(p) is very small and lower than significance level of 0.05.
  Thus we can reject the H0.

• This means there is strong evidence of an association between the key and mode of a song.

• In the next page, visualisation of the relationship between the two variables is depicted.

Categorical association

keymode_prop <- music_new %>% group_by(key, mode) %>%  summarise(n = n()) %>% mutate(prop = n / sum(n)) 

ggplot(data = keymode_prop, aes(x = key, y = prop, fill=mode)) +
  geom_bar(position="dodge", stat="identity") + scale_fill_brewer(palette = "PuBu") +   theme_classic() 

Discussion

Major Findings

• Most Common Key: The analysis revealed that C# is the most frequent key used in popular Spotify tracks, with a notable prevalence over other keys.

• Tempo Distribution: The tempo distribution of popular tracks and boxplot of BPM shows that the average bpm pf popular Spotify tracks is 122.84 and indicate a wide variety of tempos are popular on Spotify.

• Danceability vs. Streams: The scatter plot showed no direct correlation between danceability percentage and the number of streams, suggesting that danceability is not the sole factor in a track’s popularity.

• Hypothesis Testing: The t-test comparing the mean BPM of songs in keys C# and G found no significant difference, implying that the key of a song does not significantly affect its tempo.

• Key and Mode Association: The chi-squared test showed a significant association between key and mode, suggesting that certain keys are more likely to be used in either major or minor modes.

Strengths and Limitations

Strengths

1.Comprehensive Dataset: The dataset from Kaggle provided a rich source of information, including various musical attributes that allowed for in-depth analysis.

2.Robust Statistical Methods: The use of multiple statistical tests (e.g., t-test, chi-squared test) ensured that the findings were rigorously validated.

3.Visualizations: Effective use of visualizations (bar plots, histograms, scatter plots) made it easier to identify and communicate key trends and patterns.

Limitations

1.Dataset Size: The dataset, although extensive, was limited to 953 tracks, which may not be fully representative of all popular music on Spotify.

2.Genre Information: The lack of genre information in the dataset limited the ability to analyze trends across different music genres.

3.Temporal Scope: The dataset did not account for changes over time, such as shifts in musical trends or listener preferences.

Future Directions

1.Genre-Specific Analysis: Future investigations could include genre information to explore how trends vary across different types of music.

2.Temporal Analysis: Analyzing how musical attributes and their popularity change over time could provide insights into evolving trends in the music industry.

3.Additional Attributes: Including more attributes, such as lyrics sentiment or production techniques, could enrich the analysis and uncover new patterns.

Broader Dataset: Expanding the dataset to include more tracks and other music attributes like genre could provide a more comprehensive view of the music landscape.

Conclusion

• The investigation into Spotify’s popular tracks uncovered significant insights into the musical attributes that characterize popular songs. While C# emerged as the most common key, the tempo distribution and danceability analysis showed that a variety of musical elements contribute to a track’s success. The key and mode association highlighted the nuanced relationship between these musical aspects. Overall, this study provides valuable information for music producers and artists aiming to understand and leverage the attributes that make songs popular.

• The key of a song, particularly C#, plays a significant role in popular music, but it is the combination of various musical attributes, including tempo, danceability, and mode, that collectively influence a track’s success on Spotify.

References

• Zee solver (2024) ‘Spotify Songs Album’, Kaggle, viewed 2 June 2024, https://www.kaggle.com/datasets/zeesolver/spotfy

• Classic FM (2024) ‘What’s a musical ’key’ – and how do composers know which one to write in?’, Classic FM, viewed 2 June 2024, https://www.classicfm.com/discover-music/music-theory/what-are-musical-keys/