Group 1 - Clustering Methods for Spotify Song Data

Jason Caballes

Nick Cruz

Sean Grzenda

Karthik Sri Sakthi Degala

Objective

Our goal is to perform clustering algorithms, i.e. K-means and hierarchical K-means to try and unveil potential similarities among the most popular songs today across the globe. We want to see if favorable music from each country will naturally form clusters.

We may cluster tracks based on their musical attributes to see if tracks within the clusters are more popular in certain countries. Clustering tracks may also allow us to generate playlists, such as Spotify’s own recommendation systems

Methods Used

Created a database file containing the top tracks from 67 countries

we explored the following for analysis:

Hierarchical Clustering and Dendrograms

Elbow Method to find Optimal Clusters

Elbow Method Overview

  1. Identifies optimal clusters in k-means by locating the “elbow” point.
  2. Elbow point: Where WCSS decrease shifts markedly.

Analysis of Spotify Data

  1. No clear elbow observed in the plot.
  2. Implication: Data may not naturally form distinct clusters.
  3. Possible complexity in the clustering structure.

Elbow Method to find Optimal Clusters

Gap statistic to Find optimal Clusters

Gap Statistic Overview

  1. Measures optimal clusters by comparing within-cluster variation to a null reference.
  2. Assesses against expected values under null distribution.

Optimal Clusters Identified

  1. Gap statistic suggests optimal clusters: 2, 4, 9, 16.
  2. Explore each cluster solution’s impact on data insights.
  3. Assess how different cluster counts influence predictions.

Gap Statistic to find Optimal Clusters

Silhouette to find Optimal Clusters

Silhouette Method Overview

  1. Calculates the silhouette score for each data point.
  2. Measures closeness within a cluster versus neighboring clusters.

Optimal Cluster Numbers

  1. Silhouette method identifies optimal clusters: 8, 13, 19.
  2. Higher silhouette scores at these clusters.
  3. Indicates better separation and cohesion among clusters.

Silhouette to find Optimal Clusters

PCA

PCA simplifies data by transforming related variables into uncorrelated ones. Resulting components capture key patterns and variance. Ideal for reducing complexity in large datasets.

PCA

PCA Plot of K-Means Clusters


Distribution of countries Between clusters

Summary

  1. Gap statistics identified optimal clusters as 2, 4, 9, and 16.
  2. Silhouette method suggested optimal clusters as 8, 13, and 19.
  3. Chose to test clusters at 5, 9, 19 and 67 (total countries).
  4. Clusters at 5 and 9 showed decent differentiation on PCA plots.
  5. Cluster at 19 provided good results with visible grouping, though not all clusters were distinct.
  6. Cluster at 67 (equal to the number of countries) performed poorly, mirroring the initial country-based results.

Conclusion

The answer to the original question is unfortunately no, songs do not cluster by country. In fact, the optimal number of clusters is nowhere close to the number of countries.