Load the Required Package

library(DT)           #datatables
library(dplyr)        #praprocess data
library(GGally)       #ggcorr
library(factoextra)   #Elbow Method
library(scales)       #number_format

Load the Dataset

spotify_data <- read.csv("data/SpotifyFeatures.csv")

Load 100 samples data

datatable(
  head(spotify_data,100),
  extensions = 'FixedColumns',
  options = list(
    scrollY = "400px",
    scrollX = TRUE,
    fixedColumns = TRUE
  )
)

Check data type

str(spotify_data)

## 'data.frame':    232725 obs. of  18 variables:
##  $ ï..genre        : Factor w/ 27 levels "A Capella","Alternative",..: 16 16 16 16 16 16 16 16 16 16 ...
##  $ artist_name     : Factor w/ 14564 levels "'Til Tuesday",..: 5283 8366 6575 5283 4140 5283 8366 7434 2465 7469 ...
##  $ track_name      : Factor w/ 148615 levels "' Cello Song",..: 20191 96046 34319 33138 93914 71569 99298 72873 52992 72167 ...
##  $ track_id        : Factor w/ 176774 levels "00021Wy6AyMbLP2tqij86e",..: 4971 4768 5608 8233 10160 12589 13641 14649 17254 19465 ...
##  $ popularity      : int  0 1 3 0 4 0 2 15 0 10 ...
##  $ acousticness    : num  0.611 0.246 0.952 0.703 0.95 0.749 0.344 0.939 0.00104 0.319 ...
##  $ danceability    : num  0.389 0.59 0.663 0.24 0.331 0.578 0.703 0.416 0.734 0.598 ...
##  $ duration_ms     : int  99373 137373 170267 152427 82625 160627 212293 240067 226200 152694 ...
##  $ energy          : num  0.91 0.737 0.131 0.326 0.225 0.0948 0.27 0.269 0.481 0.705 ...
##  $ instrumentalness: num  0 0 0 0 0.123 0 0 0 0.00086 0.00125 ...
##  $ key             : Factor w/ 12 levels "A","A#","B","C",..: 5 10 4 5 9 5 5 10 4 11 ...
##  $ liveness        : num  0.346 0.151 0.103 0.0985 0.202 0.107 0.105 0.113 0.0765 0.349 ...
##  $ loudness        : num  -1.83 -5.56 -13.88 -12.18 -21.15 ...
##  $ mode            : Factor w/ 2 levels "Major","Minor": 1 2 2 1 1 1 1 1 1 1 ...
##  $ speechiness     : num  0.0525 0.0868 0.0362 0.0395 0.0456 0.143 0.953 0.0286 0.046 0.0281 ...
##  $ tempo           : num  167 174 99.5 171.8 140.6 ...
##  $ time_signature  : Factor w/ 5 levels "0/4","1/4","3/4",..: 4 4 5 4 4 4 4 4 4 4 ...
##  $ valence         : num  0.814 0.816 0.368 0.227 0.39 0.358 0.533 0.274 0.765 0.718 ...

The data has 232,725 rows and 18 columns.

Variable explanation

• genre : A list of the genres the artist is associated with.
• artist_name : The artists who performed the track.
• track_name : The name of the track.
• track_id : The Spotify ID for the track.
• popularity : The popularity of the track. The popularity of a track is a value between 0 and 100, with 100 being the most popular. The popularity is calculated by algorithm and is based, in the most part, on the total number of plays the track has had and how recent those plays are.
• acousticness : A confidence measure from 0.0 to 1.0 of whether the track is acoustic. 1.0 represents high confidence the track is acoustic.
• danceability : Danceability describes how suitable a track is for dancing based on a combination of musical elements including tempo, rhythm stability, beat strength, and overall regularity. A value of 0.0 is least danceable and 1.0 is most danceable.
• duration_ms : The duration of the track in milliseconds.
• energy : Energy is a measure from 0.0 to 1.0 and represents a perceptual measure of intensity and activity. Typically, energetic tracks feel fast, loud, and noisy. For example, death metal has high energy, while a Bach prelude scores low on the scale. Perceptual features contributing to this attribute include dynamic range, perceived loudness, timbre, onset rate, and general entropy.
• instrumentalness : Predicts whether a track contains no vocals. “Ooh” and “aah” sounds are treated as instrumental in this context. Rap or spoken word tracks are clearly “vocal”. The closer the instrumentalness value is to 1.0, the greater likelihood the track contains no vocal content. Values above 0.5 are intended to represent instrumental tracks, but confidence is higher as the value approaches 1.0.
• key : The estimated overall key of the track. Integers map to pitches using standard Pitch Class notation . E.g. 0 = C, 1 = C♯/D♭, 2 = D, and so on. If no key was detected, the value is -1.
• liveness : Detects the presence of an audience in the recording. Higher liveness values represent an increased probability that the track was performed live. A value above 0.8 provides strong likelihood that the track is live.
• loudness : The overall loudness of a track in decibels (dB). Loudness values are averaged across the entire track and are useful for comparing relative loudness of tracks. Loudness is the quality of a sound that is the primary psychological correlate of physical strength (amplitude). Values typical range between -60 and 0 db.
• mode : Mode indicates the modality (major or minor) of a track, the type of scale from which its melodic content is derived. Major is represented by 1 and minor is 0.
• speechiness : Speechiness detects the presence of spoken words in a track. The more exclusively speech-like the recording (e.g. talk show, audio book, poetry), the closer to 1.0 the attribute value. Values above 0.66 describe tracks that are probably made entirely of spoken words. Values between 0.33 and 0.66 describe tracks that may contain both music and speech, either in sections or layered, including such cases as rap music. Values below 0.33 most likely represent music and other non-speech-like tracks.
• tempo : The overall estimated tempo of a track in beats per minute (BPM). In musical terminology, tempo is the speed or pace of a given piece and derives directly from the average beat duration.
• time_signature : An estimated overall time signature of a track. The time signature (meter) is a notational convention to specify how many beats are in each bar (or measure).
• valence : A measure from 0.0 to 1.0 describing the musical positiveness conveyed by a track. Tracks with high valence sound more positive (e.g. happy, cheerful, euphoric), while tracks with low valence sound more negative (e.g. sad, depressed, angry).

Data Preprocessing

We will use 300 data from Spotify Tracks DB dataset and removed unused column:

set.seed(121)
idx <- sample(nrow(spotify_data), 300)

colnames(spotify_data)[1] <- "genre"
spotify_data <- spotify_data[idx, ] %>% 
  distinct(track_id, .keep_all = TRUE)

datatable(
  spotify_data,
  extensions = 'FixedColumns',
  options = list(
    scrollY = "400px",
    scrollX = TRUE,
    fixedColumns = TRUE
  )
)

We will check if there is any missing values in the data.

colSums(is.na(spotify_data))

##            genre      artist_name       track_name         track_id 
##                0                0                0                0 
##       popularity     acousticness     danceability      duration_ms 
##                0                0                0                0 
##           energy instrumentalness              key         liveness 
##                0                0                0                0 
##         loudness             mode      speechiness            tempo 
##                0                0                0                0 
##   time_signature          valence 
##                0                0

Scaling data

spotify_data_scale <- scale(spotify_data[,-c(1,2,3,4,11,14,17)])

Elbow Method

Before we do cluster analysis, first we need to determine the optimal number of cluster.

set.seed(100)
fviz_nbclust(spotify_data_scale, kmeans, method = "wss") + 
  labs(subtitle = "Elbow method")

Using the elbow method, we know that 3 cluster is good enough since there is no significant decline in total within-cluster sum of squares on higher number of clusters.

K-Means Clustering

Create cluster using K-Means Clustering Method :

set.seed(100)
spotify_cluster <- kmeans(spotify_data_scale, 3)

fviz_cluster(object = spotify_cluster, data = spotify_data_scale)

Cluster profiling :

# data preparation for visualization & profiling
spotify_data$cluster <- as.factor(spotify_cluster$cluster)

datatable(
  spotify_data,
  extensions = 'FixedColumns',
  options = list(
    scrollY = "400px",
    scrollX = TRUE,
    fixedColumns = TRUE
  )
)

# cluster profiling
cluster_profile <- spotify_data %>%
  group_by(cluster) %>% 
  summarise_if(is.numeric, "mean") %>% 
  mutate_if(is.numeric, .funs = "round", digits = 2)

datatable(
  cluster_profile,
  extensions = 'FixedColumns',
  options = list(
    scrollY = "400px",
    scrollX = TRUE,
    fixedColumns = TRUE
  )
)

Evaluation

Within-cluster Sum of Squares (WSS)

spotify_cluster$tot.withinss

## [1] 2097.537

The lower the WSS, the more similar observations are on 1 cluster

Between Sum of Squares (BSS)

spotify_cluster$betweenss

## [1] 1191.463

The higher the BSS, the more different the characteristics between clusters

Total Sum of Squares

spotify_cluster$betweenss / spotify_cluster$totss * 100

## [1] 36.22569

Good cluster has high similarity characteristics in 1 cluster (low WSS) and maximum difference in characteristics between clusters (high BSS). In addition, it can be marked with a BSS / totss ratio that is close to 1 (100%).