Introduction

The aim of this project was to perform a dimension reduction using Principal Component Analysis (PCA) on audio features of songs from Spotify. The data was gather by using Spotipy, which is a Python library for the Spotify Web API. I created a database containing all audio features offered by Spotify of songs from 20 most popular playlist on this platform. Below you can see the code used for gathering the data and data itself.

code data

Load packages needed for the analysis

library(factoextra)
library(flexclust)
library(logspline)
library(NbClust)
library(ggplot2)
library(reshape2)
library(gridExtra)
library(ggfortify)
library(corrplot)
library(knitr)
library(ggbiplot)

Loading the data

After gathering the data from Spotify API, we are left with a dataset containing all possible audio features taken for every song. To perform PCA we do not include “mode” variable and “key” variable in the analysis. In this dataset the variables that were left are:

danceability - describes how suitable a track is for dancing based on a combination of musical elements including tempo, rhythm stability, beat strength, and overall regularity. A value of 0.0 is least danceable and 1.0 is most danceable.

energy - is a measure from 0.0 to 1.0 and represents a perceptual measure of intensity and activity. Typically, energetic tracks feel fast, loud, and noisy. For example, death metal has high energy, while a Bach prelude scores low on the scale. Perceptual features contributing to this attribute include dynamic range, perceived loudness, timbre, onset rate, and general entropy.

loudness - The overall loudness of a track in decibels (dB). Loudness values are averaged across the entire track and are useful for comparing relative loudness of tracks. Loudness is the quality of a sound that is the primary psychological correlate of physical strength (amplitude).

acousticness - A confidence measure from 0.0 to 1.0 of whether the track is acoustic. 1.0 represents high confidence the track is acoustic.

instrumentalness - Predicts whether a track contains no vocals. “Ooh” and “aah” sounds are treated as instrumental in this context. Rap or spoken word tracks are clearly “vocal”. The closer the instrumentalness value is to 1.0, the greater likelihood the track contains no vocal content. Values above 0.5 are intended to represent instrumental tracks, but confidence is higher as the value approaches 1.0.

liveness - detects the presence of an audience in the recording. Higher liveness values represent an increased probability that the track was performed live. A value above 0.8 provides strong likelihood that the track is live.

valence - is a measure from 0.0 to 1.0 describing the musical positiveness conveyed by a track. Tracks with high valence sound more positive (e.g. happy, cheerful, euphoric), while tracks with low valence sound more negative (e.g. sad, depressed, angry).

tempo - The overall estimated tempo of a track in beats per minute (BPM). In musical terminology, tempo is the speed or pace of a given piece and derives directly from the average beat duration.

speechiness - detects the presence of spoken words in a track. The more exclusively speech-like the recording (e.g. talk show, audio book, poetry), the closer to 1.0 the attribute value. Values above 0.66 describe tracks that are probably made entirely of spoken words. Values between 0.33 and 0.66 describe tracks that may contain both music and speech, either in sections or layered, including such cases as rap music. Values below 0.33 most likely represent music and other non-speech-like tracks.

Every variable was scaled to ensure the best results.

data<-read.csv("spotify_database.csv")
features_for_pca <- c("danceability", "energy", "loudness","acousticness","instrumentalness","liveness","valence","tempo")
data1 <- data[, features_for_pca]
data1<-scale(data1)

Checking correlations between the variables

To look for relationships between variables I will start with showing a correlation matrix.

corrplot(cor(data1), type = "lower", order = "hclust", tl.col = "#0041C2", tl.cex = 0.5)

As we can see there are few statistically significant relationships between variables. There is a high positive correlation between loudness and energy, high negative correlation between instrumentalness and loudness and the most noticeable high negative correlation between acousticness and energy.

Prinicpal Component Analysis (PCA)

Principal Component Analysis (PCA) is a statistical method used for dimensionality reduction and data compression. Its primary goal is to transform high-dimensional data into a lower-dimensional representation while retaining as much of the original variability as possible. PCA achieves this by identifying the principal components, which are linear combinations of the original features that capture the maximum variance in the data.

pca <- prcomp(data1, center = TRUE, scale = TRUE)

Next part of our analysis will contain PCA as well as choosing the most appropriate number of newly created variables that will maximize the explained variance and will increase interpretability. To do that I will show few plot that will help us understand choose the right amount.

Eigenvalues

fviz_eig(pca, choice = "eigenvalue", ncp = 22, barfill = "#0041C2", barcolor = "#151B54", linecolor = "black",  addlabels = TRUE)

This plot shows us the eigenvalues for corresponding number of dimensions. I chose 3 components, because the eigenvalue is not below 1.

Plot of the percentage of explained variance

fviz_screeplot(pca, addlabels = TRUE, barfill = "#0041C2", barcolor = "#151B54", linecolor = "black")

By taking 3 components we will have 70.2% of explained variance.

Analysis of components

On this plot we will see how each audio feature impacts an observation as well as its position on the grid. It also shows relations between variables.

fviz_pca_var(pca, col.var = "#0041C2")

autoplot(pca, loadings=TRUE, loadings.colour='#0041C2', loadings.label=TRUE, loadings.label.size=3)

relevant_features<-c("name", "danceability", "energy", "loudness","acousticness","instrumentalness","liveness","valence","tempo")
data<-data[,relevant_features]
num_to_label <- round(0.01 * nrow(data))
sample_indices <- sample(1:nrow(data), num_to_label)
labels <- rep(NA, nrow(data))
labels[sample_indices] <- data$name[sample_indices]
ggbiplot(pca, labels = labels, varname.color = "#0041C2")

This plot shows the contribution of each variable to a certain component.

PC1 <- fviz_contrib(pca, choice = "var", axes = 1,fill = "#1AA7EC",color = "#1AA7EC")
PC2 <- fviz_contrib(pca, choice = "var", axes = 2,fill = "#1AA7EC",color = "#1AA7EC")
PC3 <- fviz_contrib(pca, choice = "var", axes = 3,fill = "#1AA7EC",color = "#1AA7EC")
grid.arrange(PC1, PC2, PC3, ncol=1)

This gives us the information that that PC1 mostly consists of energy, loudness, acousticness and instumentalness, PC2 consists of danceability, valence and tempo and PC3 consists of liveness.

Hierarchial clustering

distance.m<-dist(t(data1))
hc<-hclust(distance.m, method="complete") 
plot(hc, hang=-1)
rect.hclust(hc, k = 3, border='#1AA7EC')

This dendrogram shows us different clusters of variables. It can be explained by acknowledging that this algorithm does not count variables with high negative correlation as similar. In the analysis of the components obtained by PCA the variables that had high negative correlation were a part of the same component.

Conclusion

This project shows how useful dimension reduction can be. By reducing the dataset to just 3 variables we could explain 70.2% of the variance. The could also in a easy way find out which components are made of which variables.