Music streaming platforms such as Spotify have transformed how people discover and experience music. With millions of songs available, listeners are no longer limited to traditional music genres. Personalized playlist, algorithm-driven recommendations, and social media trends have encouraged people to explore music across different styles and categories.
However, traditional genre labels such as Pop, Rap, Rock, Latin, R&B, and EDM may not fully capture the similarities between songs. For example, a Latin song and a Pop song might share similar levels of danceability, energy, and tempo despite belonging to different genres. This raises an important question: Can measurable audio characteristics reveal meaningful connections between songs that traditional genre classifications overlook?
This project investigates whether Spotify’s audio features can identify patterns and similarities across different music genres. Rather than relying exclusively on predefined genre labels, we aim to explore whether machine learning can uncover natural groupings of songs based on how they sound.
Understanding these relationships is relevant because music discovery plays an important role in the streaming experience. Identifying similarities across genres could provide useful insights into how songs are organized, how playlists are developed, and how listeners might discover music beyond their usual preferences.
The primary objective of this project is to examine how Spotify’s measurable audio characteristics vary across music genres and determine whether unsupervised machine learning can identify meaningful groups of songs that extend beyond traditional genre classifications.
Specific Research Objectives
Analyze audio characteristics across music genres. Examine how danceability, energy, valence, tempo, and acousticness differ among Pop, Rap, Rock, Latin, R&B, and EDM.
Identify similarities and differences between genres. Determine which genres share similar audio profiles and which exhibit more distinctive musical characteristics.
Discover hidden patterns using machine learning. Apply K-means clustering to group songs based on their audio features without using genre labels to form the clusters.
Compare machine learning results with traditional genre classifications. Investigate whether the resulting clusters primarily contain songs from one genre or combine songs from multiple genres.
Explore practical applications for music discovery. Evaluate how audio-based groupings could inform playlist organizations and provide a foundation for exploring alternative music recommendation strategies.
This study will use the Spotify Songs dataset published through the TidyTuesday project on January 21, 2020. The dataset was originally collected using the spotifyr package and contains information about songs, playlist genres, and measurable audio characteristics.
Our analysis will focus on six playlist genres: Pop, Rap, Rock, Latin, R&B, and EDM. The primary variables of interest include danceability, energy, valence, tempo, and acousticness.
The analysis will be conducted in R and will consist of three main stages.
Stage 1: Data Preparation and Exploratory Data Analysis
Import and clean the dataset, examine missing values and duplicate observations, and calculate descriptive statistics. Visualizations such as boxplots, bar charts, and heatmaps will help compare audio characteristics across genres.
Stage 2: Machine Learning and Cluster Analysis
Use K-means clustering, an unsupervised machine learning technique that groups observations according to their similarities. Audio features will be standardized before clustering to account for differences in measurement scales. We will investigate an appropriate number of clusters and examine their characteristics to identify patterns in the data.
Stage 3: Interpretation and Genre Comparison
Compare the machine learning clusters with Spotify’s existing playlist genre classifications. This will help us evaluate whether audio-based groupings reflect traditional genre categories or reveal similarities across genre boundaries.
The results of this project could provide valuable insights for music streaming platforms, playlist curators, music marketers, and music listeners. For streaming platforms, understanding similarities between songs could help inform alternative approaches to organizing music libraries and developing playlists based on audio characteristics rather than genre classifications alone.
Playlist curators could potentially use these findings to identify songs from different genres that share similar musical qualities. For example, a workout playlist could include energetic songs from Pop, Latin, EDM, and Rap, even though they belong to different categories. Music marketers and artists could also benefit from understanding how musical characteristics vary across genres and where different styles overlap.
For listeners, the potential value lies in discovering music outside their familiar genres while maintaining similarities in musical characteristics they may enjoy. Although the data set does not include individual listening preferences or recommendation outcomes, this analysis can provide an exploratory foundation for future research into audio-based music discovery.
Ultimately, this project aims to demonstrate how business analytics and machine learning can reveal patterns that are not immediately visible through traditional music classifications.
To analyze the Spotify Songs data set, we will use several R packages that support data preparation, exploratory data analysis, visualization, and machine learning.
These packages will allow us to examine the musical characteristics of songs across different genres, identify patterns within the dataset, and explore similarities using clustering techniques.
library(tidyverse)
library(janitor)
library(skimr)
library(knitr)
library(lubridate)
library(cluster)tidyverse is used for data manipulation and
visualization.
janitor helps examine data quality, clean variable
names, and create frequency tables.
skimr provides details descriptive statistics and
summaries of the data set.
knitr formats tables and generates the final R Markdown
report.
lubridate is used for data manipulation specifically for
dates and time.
cluster provides tools for evaluating clustering
results, including silhouette analysis.
The data set used in this project is the Spotify Songs data set, published through the TidyTuesday project on January 21, 2020. The data originated from Spotify’s API and was collected using the spotifyr R package. The data set includes song metadata, playlist genre classifications, and numerical audio features that describe different musical characteristics.
The original data set is publicly available through the TidyTuesday GitHub repository
This data set was selected because it provides the information necessary to compare audio characteristics across music genres and explore whether machine learning can identify meaningful similarities between songs from different categories.
The data set contains 32,833 observations and 23 variables. For this study, we will focus primarily on five audio characteristics:
Danceability: A score from 0 to 1 describing how suitable a song is for dancing.
Energy: A score from 0 to 1 measuring musical intensity and activity.
Valence: A score from 0 to 1 representing the musical positivity of a song.
Tempo: The estimated speed of a song, measured in beats per minute.
Acousticness: A score from 0 to 1 representing confidence that a track is acoustic.
The data set also contains repeated track IDs, indicating that some songs appear in multiple playlist entries. Because playlist genres are used as reference categories in this study, these repeated entries require careful treatment.
These characteristics will guide our data preparation decisions before conducting exploratory analysis and clustering.
Importing the data:
spotify <-
read.csv("C:/Users/ximew/OneDrive/Desktop/MSBA-660-01 Data Wrangling/Midterm Project/spotify_songs.csv")View the structure of the data set:
## 'data.frame': 32833 obs. of 23 variables:
## $ track_id : chr "6f807x0ima9a1j3VPbc7VN" "0r7CVbZTWZgbTCYdfa2P31" "1z1Hg7Vb0AhHDiEmnDE79l" "75FpbthrwQmzHlBJLuGdC7" ...
## $ track_name : chr "I Don't Care (with Justin Bieber) - Loud Luxury Remix" "Memories - Dillon Francis Remix" "All the Time - Don Diablo Remix" "Call You Mine - Keanu Silva Remix" ...
## $ track_artist : chr "Ed Sheeran" "Maroon 5" "Zara Larsson" "The Chainsmokers" ...
## $ track_popularity : int 66 67 70 60 69 67 62 69 68 67 ...
## $ track_album_id : chr "2oCs0DGTsRO98Gh5ZSl2Cx" "63rPSO264uRjW1X5E6cWv6" "1HoSmj2eLcsrR0vE9gThr4" "1nqYsOef1yKKuGOVchbsk6" ...
## $ track_album_name : chr "I Don't Care (with Justin Bieber) [Loud Luxury Remix]" "Memories (Dillon Francis Remix)" "All the Time (Don Diablo Remix)" "Call You Mine - The Remixes" ...
## $ track_album_release_date: chr "2019-06-14" "2019-12-13" "2019-07-05" "2019-07-19" ...
## $ playlist_name : chr "Pop Remix" "Pop Remix" "Pop Remix" "Pop Remix" ...
## $ playlist_id : chr "37i9dQZF1DXcZDD7cfEKhW" "37i9dQZF1DXcZDD7cfEKhW" "37i9dQZF1DXcZDD7cfEKhW" "37i9dQZF1DXcZDD7cfEKhW" ...
## $ playlist_genre : chr "pop" "pop" "pop" "pop" ...
## $ playlist_subgenre : chr "dance pop" "dance pop" "dance pop" "dance pop" ...
## $ danceability : num 0.748 0.726 0.675 0.718 0.65 0.675 0.449 0.542 0.594 0.642 ...
## $ energy : num 0.916 0.815 0.931 0.93 0.833 0.919 0.856 0.903 0.935 0.818 ...
## $ key : int 6 11 1 7 1 8 5 4 8 2 ...
## $ loudness : num -2.63 -4.97 -3.43 -3.78 -4.67 ...
## $ mode : int 1 1 0 1 1 1 0 0 1 1 ...
## $ speechiness : num 0.0583 0.0373 0.0742 0.102 0.0359 0.127 0.0623 0.0434 0.0565 0.032 ...
## $ acousticness : num 0.102 0.0724 0.0794 0.0287 0.0803 0.0799 0.187 0.0335 0.0249 0.0567 ...
## $ instrumentalness : num 0.00 4.21e-03 2.33e-05 9.43e-06 0.00 0.00 0.00 4.83e-06 3.97e-06 0.00 ...
## $ liveness : num 0.0653 0.357 0.11 0.204 0.0833 0.143 0.176 0.111 0.637 0.0919 ...
## $ valence : num 0.518 0.693 0.613 0.277 0.725 0.585 0.152 0.367 0.366 0.59 ...
## $ tempo : num 122 100 124 122 124 ...
## $ duration_ms : int 194754 162600 176616 169093 189052 163049 187675 207619 193187 253040 ...View the dimensions of the data set:
## [1] 32833 23View the summary statistics of the data set:
## track_id track_name track_artist track_popularity
## Length :32833 Length :32833 Length :32833 Min. : 0.00
## N.unique :28356 N.unique :23449 N.unique :10692 1st Qu.: 24.00
## N.blank : 0 N.blank : 0 N.blank : 0 Median : 45.00
## Min.nchar: 22 Min.nchar: 1 Min.nchar: 2 Mean : 42.48
## Max.nchar: 22 Max.nchar: 144 Max.nchar: 69 3rd Qu.: 62.00
## NAs : 5 NAs : 5 Max. :100.00
## track_album_id track_album_name track_album_release_date playlist_name
## Length :32833 Length :32833 Length :32833 Length :32833
## N.unique :22545 N.unique :19743 N.unique : 4530 N.unique : 449
## N.blank : 0 N.blank : 0 N.blank : 0 N.blank : 0
## Min.nchar: 22 Min.nchar: 1 Min.nchar: 4 Min.nchar: 6
## Max.nchar: 22 Max.nchar: 151 Max.nchar: 10 Max.nchar: 122
## NAs : 5
## playlist_id playlist_genre playlist_subgenre danceability
## Length :32833 Length :32833 Length :32833 Min. :0.0000
## N.unique : 471 N.unique : 6 N.unique : 24 1st Qu.:0.5630
## N.blank : 0 N.blank : 0 N.blank : 0 Median :0.6720
## Min.nchar: 22 Min.nchar: 3 Min.nchar: 4 Mean :0.6548
## Max.nchar: 22 Max.nchar: 5 Max.nchar: 25 3rd Qu.:0.7610
## Max. :0.9830
## energy key loudness mode
## Min. :0.000175 Min. : 0.000 Min. :-46.448 Min. :0.0000
## 1st Qu.:0.581000 1st Qu.: 2.000 1st Qu.: -8.171 1st Qu.:0.0000
## Median :0.721000 Median : 6.000 Median : -6.166 Median :1.0000
## Mean :0.698619 Mean : 5.374 Mean : -6.719 Mean :0.5657
## 3rd Qu.:0.840000 3rd Qu.: 9.000 3rd Qu.: -4.645 3rd Qu.:1.0000
## Max. :1.000000 Max. :11.000 Max. : 1.275 Max. :1.0000
## speechiness acousticness instrumentalness liveness
## Min. :0.0000 Min. :0.0000 Min. :0.0000000 Min. :0.0000
## 1st Qu.:0.0410 1st Qu.:0.0151 1st Qu.:0.0000000 1st Qu.:0.0927
## Median :0.0625 Median :0.0804 Median :0.0000161 Median :0.1270
## Mean :0.1071 Mean :0.1753 Mean :0.0847472 Mean :0.1902
## 3rd Qu.:0.1320 3rd Qu.:0.2550 3rd Qu.:0.0048300 3rd Qu.:0.2480
## Max. :0.9180 Max. :0.9940 Max. :0.9940000 Max. :0.9960
## valence tempo duration_ms
## Min. :0.0000 Min. : 0.00 Min. : 4000
## 1st Qu.:0.3310 1st Qu.: 99.96 1st Qu.:187819
## Median :0.5120 Median :121.98 Median :216000
## Mean :0.5106 Mean :120.88 Mean :225800
## 3rd Qu.:0.6930 3rd Qu.:133.92 3rd Qu.:253585
## Max. :0.9910 Max. :239.44 Max. :517810| variable | missing_count |
|---|---|
| track_name | 5 |
| track_artist | 5 |
| track_album_name | 5 |
## [1] 4477## track_id track_name
## 1 6f807x0ima9a1j3VPbc7VN I Don't Care (with Justin Bieber) - Loud Luxury Remix
## 2 0r7CVbZTWZgbTCYdfa2P31 Memories - Dillon Francis Remix
## 3 1z1Hg7Vb0AhHDiEmnDE79l All the Time - Don Diablo Remix
## 4 75FpbthrwQmzHlBJLuGdC7 Call You Mine - Keanu Silva Remix
## 5 1e8PAfcKUYoKkxPhrHqw4x Someone You Loved - Future Humans Remix
## 6 7fvUMiyapMsRRxr07cU8Ef Beautiful People (feat. Khalid) - Jack Wins Remix
## track_artist danceability energy valence tempo acousticness
## 1 Ed Sheeran 0.748 0.916 0.518 122.036 0.1020
## 2 Maroon 5 0.726 0.815 0.693 99.972 0.0724
## 3 Zara Larsson 0.675 0.931 0.613 124.008 0.0794
## 4 The Chainsmokers 0.718 0.930 0.277 121.956 0.0287
## 5 Lewis Capaldi 0.650 0.833 0.725 123.976 0.0803
## 6 Ed Sheeran 0.675 0.919 0.585 124.982 0.0799## [1] "danceability" "energy" "valence" "tempo" "acousticness"## danceability energy valence tempo acousticness
## 0 0 0 0 0## [1] "danceability" "energy" "valence" "acousticness"| danceability | energy | valence | acousticness |
|---|---|---|---|
| 0 | 0 | 0 | 0 |
| track_id | track_name | tempo |
|---|---|---|
| 51w6nRCU68klqNfYaaVP2j | Hi, How’re You Doin’? | 0 |
## track_id track_name
## 1 6f807x0ima9a1j3VPbc7VN I Don't Care (with Justin Bieber) - Loud Luxury Remix
## 2 0r7CVbZTWZgbTCYdfa2P31 Memories - Dillon Francis Remix
## 3 1z1Hg7Vb0AhHDiEmnDE79l All the Time - Don Diablo Remix
## 4 75FpbthrwQmzHlBJLuGdC7 Call You Mine - Keanu Silva Remix
## 5 1e8PAfcKUYoKkxPhrHqw4x Someone You Loved - Future Humans Remix
## 6 7fvUMiyapMsRRxr07cU8Ef Beautiful People (feat. Khalid) - Jack Wins Remix
## track_artist danceability energy valence tempo acousticness
## 1 Ed Sheeran 0.748 0.916 0.518 122.036 0.1020
## 2 Maroon 5 0.726 0.815 0.693 99.972 0.0724
## 3 Zara Larsson 0.675 0.931 0.613 124.008 0.0794
## 4 The Chainsmokers 0.718 0.930 0.277 121.956 0.0287
## 5 Lewis Capaldi 0.650 0.833 0.725 123.976 0.0803
## 6 Ed Sheeran 0.675 0.919 0.585 124.982 0.0799## danceability energy valence tempo acousticness
## [1,] 0.64914972 1.1858761 0.03241421 0.03991597 -0.3374357
## [2,] 0.49819208 0.6354438 0.77924186 -0.77892322 -0.4702877
## [3,] 0.14824483 1.2676235 0.43783493 0.11310084 -0.4388700
## [4,] 0.44329840 1.2621736 -0.99607416 0.03694701 -0.6664239
## [5,] -0.02329795 0.7335407 0.91580463 0.11191325 -0.4348306
## [6,] 0.14824483 1.2022256 0.31834251 0.14924793 -0.4366259| track_name | danceability | energy | valence | tempo | acousticness |
|---|---|---|---|---|---|
| I Don’t Care (with Justin Bieber) - Loud Luxury Remix | 0.75 | 0.92 | 0.52 | 122.04 | 0.10 |
| Memories - Dillon Francis Remix | 0.73 | 0.81 | 0.69 | 99.97 | 0.07 |
| All the Time - Don Diablo Remix | 0.68 | 0.93 | 0.61 | 124.01 | 0.08 |
| Call You Mine - Keanu Silva Remix | 0.72 | 0.93 | 0.28 | 121.96 | 0.03 |
| Someone You Loved - Future Humans Remix | 0.65 | 0.83 | 0.72 | 123.98 | 0.08 |
| Beautiful People (feat. Khalid) - Jack Wins Remix | 0.68 | 0.92 | 0.58 | 124.98 | 0.08 |
| variable | mean | median | sd |
|---|---|---|---|
| danceability | 0.653 | 0.670 | 0.146 |
| energy | 0.698 | 0.722 | 0.183 |
| valence | 0.510 | 0.512 | 0.234 |
| tempo | 120.960 | 121.993 | 26.945 |
| acousticness | 0.177 | 0.080 | 0.223 |
The descriptive statistics indicate that the songs in the dataset generally exhibit moderately high danceability and energy levels.
Across the 28,356 unique tracks, average danceability is approximately 0.653, while average energy is 0.698.
The mean valence score of 0.510 suggests that the data set contains a mixture of songs with positive and negative musical characteristics, with the overall average near the midpoint of the scale.
The average tempo is approximately 121 beats per minute, although considerable variation exists across songs.
Acousticness has a mean of 0.177 and a median of 0.080, indicating that many songs have relatively low acousticness scores, while some songs have substantially higher values.
These initial findings provide a foundation for investigating how musical characteristics differ across genres. In the next stage, we will examine whether combinations of these features reveal distinct groups of songs that may extend beyond traditional genre classifications.
The exploratory data analysis will focus on understanding how Spotify’s audio characteristics vary across different music genres and whether meaningful similarities exist between songs classified under different categories. Let’s begin by examining five audio features: danceability, energy, valence, tempo, and acousticness. These variables were selected because they represent different musical characteristics that can help us compare songs beyond traditional genre labels.
Calculate descriptive statistics for each of the six playlist genres: Pop, Rap, Rock, Latin, R&B, and EDM. Compare the average and distribution of audio features will help identify which genres share similar musical characteristics and which have more distinctive profiles. After let’s examine relationships among the audio features themselves. For example, we want to investigate whether songs with higher danceability also tend to have higher energy or whether acousticness varies systematically across genres. Finally, we will explore whether songs from different genres can be grouped according to their audio characteristics. This will provide a foundation for our proposed machine learning analysis.
The exploratory analysis will address two central questions:
These findings will guide our selection of variables and analytic methods for the clustering stage of the project.
For the visualizations there are several visualizations and summary tables to communicate the results of the exploratory analysis.
Descriptive Statistics by Genre: We will create a table displaying the mean, median, and standard deviation of selected audio features for each genre. This will provide a quantitative comparison of their musical characteristics.
Boxplots: We will use boxplots to compare the distributions of danceability, energy, valence, tempo, and acousticness across the six genres. These visualizations will help identify differences in central tendency, variability, and potential outliers.
Heatmaps: We will create a heatmap showing standardized average audio characteristics by genre. This will allow us to visually compare genre profiles and identify similarities across multiple features.
Scatterplots: We will examine relationships between selected variables, such as danceability and energy, to explore whether songs from different genres occupy similar regions of the feature space.
Together, these visualizations will help us understand the structure of the dataset and determine whether the audio features provide a meaningful basis for machine learning.
As we progress with our analysis, we anticipate several areas where additional research and technical learning will be necessary. Develop a deeper understanding of unsupervised machine learning, particularly K-means clustering. Unlike supervised learning, clustering does not use predefined labels to train a model. Therefore, we will need to learn how to evaluate whether the resulting groups are meaningful.
Investigate methods for determining the appropriate number of clusters, including the elbow method and silhouette analysis. These techniques can help us evaluate alternative clustering solutions.
Explore methods for visualizing multidimensional data. Because our analysis uses five audio features, techniques such as principal component analysis (PCA) may help us represent clustering results in two dimensions.
Interpret the relationship between playlist genre classifications and audio-based clusters. Some songs appear in multiple genre categories, so we must account for overlapping genre membership rather than assume every song belongs to only one genre.
Addressing these challenges will help us develop a more reliable analysis and avoid drawing conclusions that are not supported by the data.
Our primary proposed machine learning technique is K-means clustering, an unsupervised learning algorithm that groups observations based on their numerical similarities. We selected this technique because our research objective is not to predict an existing genre label but to investigate whether songs naturally form groups based on their audio characteristics.
The analysis will use danceability, energy, valence, tempo, and acousticness as input variables. Before applying K-means, we will standardize the features to prevent differences in measurement scales from disproportionately influencing the clustering process. We will compare different numbers of clusters using methods such as the elbow method and silhouette analysis. Once an appropriate clustering solution is selected, we will examine the average audio characteristics of each cluster to understand what distinguishes the resulting groups.
We will then compare cluster membership with the original playlist genre labels. In particular, we want to investigate whether individual clusters contain songs from several different genres, suggesting that some audio characteristics transcend traditional genre boundaries, we will also apply principal component analysis (PCA) to visualize the clusters and investigate whether the selected audio features capture meaningful variation in the dataset.
Although K-means clustering cannot establish whether listeners would enjoy particular songs, it can reveal measurable similarities that may be useful for playlist organization and future music discovery research. Our expected contribution is an exploratory understanding of how machine learning can complement traditional music genre classifications rather than replace them entirely.