Question and Background

How well can we build a model that accurately predicts popular songs for 2020 based on data since 2015?

Radio hosts are always looking to optimize their playlists. It is essential for their job to continue to engage listeners and highlight the next best song. Our question centers around the idea of predicting popularity as a measure of success. We want to use data science as a decision making tool for hot new songs. Radio songs are often based on who pays the most and has the best promoter to get played. The goal is to evaluate other methods so that both the hosts and artists will have an idea if their song will be popular enough to be added to radio stations. It will contribute to more viewership the radio station and more exposure for a song deemed hot. The goal is to aid radio hosts in determining which songs to add to their playlists.

The data used for this project is a dataset extracted from a csv file on Kaggle that was taken from the Spotify API. There are almost 170,000 unique songs in this data set from the years 1921 to 2020.

Data Dictionary

acousticness

A measure from 0.0 to 1.0 of whether the track is acoustic.

danceability

How suitable a track is for dancing based on a combination of musical elements including tempo, rhythm stability, beat strength, and overall regularity. A value of 0.0 is least danceable and 1.0 is most danceable.

duration_ms

Duration of track in milliseconds. This value is normalized between 0 and 1 in data cleaning.

energy

A measure from 0.0 to 1.0 and represents a perceptual measure of intensity and activity. Typically, energetic tracks feel fast, loud, and noisy.

explicit

Whether a song is considered explicit (1) or not (2).

instrumentalness

Predicts whether a track contains no vocals. The closer the instrumentalness value is to 1.0, the greater likelihood the track contains no vocal content.

liveness

Detects the presence of an audience in the recording. Higher liveness values represent an increased probability that the track was performed live.

loudness

Overall loudness of a track in decibels (dB). Loudness values are averaged across the entire track. Values typical range between -60 and 0 db. This value is normalized between 0 and 1 in data cleaning.

mode

Indicates the modality (major or minor) of a track, the type of scale from which its melodic content is derived. Major is represented by 1 and minor is 0.

popularity

A 0 to 100 score that ranks how popular an artist is relative to other artists on Spotify. This value was converted to a factor variable where low probability is less than 70 and high probability is more than 70.

speechiness

Detects the presence of spoken words in a track. The more exclusively speech-like the recording (e.g. talk show, audio book, poetry), the closer to 1.0 the attribute value.

tempo

The overall estimated tempo of a track in beats per minute (BPM). In musical terminology, tempo is the speed or pace of a given piece and derives directly from the average beat duration. This value is normalized between 0 and 1 in data cleaning.

valence

A measure from 0.0 to 1.0 describing the musical positiveness conveyed by a track. Tracks with high valence sound more positive (e.g. happy, cheerful, euphoric), while tracks with low valence sound more negative (e.g. sad, depressed, angry).

year

release year of a song

name

song title

EDA

Graph 1

The popularity variable follows a normal distribution with a slight left skew. The mean is graphed in red to indicate the mean popularity of a song to later determine a cut off point for the binary variable that popularity will become.

Graph 2

This is a boxplot of the overall distribution of popularity since 2000. The 3rd quartile for popularity is 62. Since a more popular song is desired, the threshold for high popularity as a binary variable will be changed to 70.

Graph 3

The change in popularity needed to be examined, so popularity was grouped by year to determine how the summary statistics varied over time. Since there appears to be great variation and a slight positive correlation, It was determined that the training dataset should only focus on 2015 and onword.

Graph 4

A correlation plot to determine how numeric variables relate with each other and with popularity as the target variable. The larger and darker the circles the more correlated two variables are to each other.

Graph 5

70 was used as the cutoff point for determining highly popular songs as a binary variable. This was measured against explicit to determine if an explicit song could be a good predictor of popularity. With this variable, it does appear to be the case and could be useful in the final model.

Graph 6

There appears to be a fairly even makeup for different key levels and likely will not play a significant role in the final model.

Methods

Data Cleaning

The target variable of popularity needed to be converted to a binary factor variable because it does not matter how popular a song is predicted to be, rather it matters if a song should be played or not. Therefore, a cutoff threshold for popularity was determined to be 70. Over 70 in popularity is considered high and under is low popularity. The variable key was converted to a factor variable with 4 levels as it was discrete between 1 and 10. Next, years between 2015 and 2019 were determined based on EDA to be the best for the train and tune datasets. The goal is to predict 2020 songs that will be the best to be played on a radio station. All of the numeric variables were normalized between 0 and 1 for interpreting the data. The data was cleaned to minimize unneeded variables that would not add to the model. Year was taken out of the datasets once the model was split into test, train, and tune datasets as year would not help determine the test as it is from a single year. The train set had 90% of the 2015-2019 data and the tune set had 10%. It was important to maintain the prevalence for the target variable among the split datasets. The prevalence for 2020 of high popularity was about 10% higher for test compared to the rest of the data. This could pose some issues in the final testing of the model. Using data partitions, the train and tune data was split to maintain an equal prevalence of popularity. This is imparative for testing the model without chance playing a significant role with skewed results in the tune of the model.

Model Building

The model chosen was a decision tree for classification with the binary variable for popularity. This method was used because it has a high level of visualization and the ability to use all numeric and factor variables without distance measures. Decisions trees allow for simple interpretations and have predictive power that can be easily identified with built in predictive measurements. There is also options for adjusting thresholds and hyperparamters to best tune the model to be the best predictor. Having a visual model was important to the understand of prediction. This type of model is supervised machine learning, which helps give user control of the model depth and gives options for adjusting the model as needed. Decision trees can predict a specific target variable using piecewise functions and non-parametric statistics. It is a simple model to understand with high complexity of information to be gathered. The method for the was rpart which evaluates cp and ROC as the method for determining the best model.

Model Results

The initial model showed large descrepencies in specificity and sensitivity. There was an attempt to adjust the grid as hyperparameters, but this would not allow the model to converge. Next, it was determined to adjust the threshold for the model to make changes to how it determines which category to classify each data point. This was evaluated using the tune set. The optimal threshold was determined to be .5. Three variables (mode, valence, and explicit) did not serve any importance to the model and were therefore removed. Feature engineering, including changing key to include different groupings was attempted, but it did key ended up worked better as is. Other feature engineering did not work and variables were dropped. Therefore, all of these variables were removed. This aided in lessening the complexity of the model.

Result Output

Decision Tree

Variable Importance

## rpart variable importance
## 
##                    Overall
## danceability     100.00000
## instrumentalness  88.14020
## loudness          86.04780
## duration_ms       44.08718
## tempo             43.72299
## acousticness      29.97812
## speechiness       20.39596
## energy             8.18327
## key                0.07482
## liveness           0.00000

Song Recommendations

Confusion Matrix

## Confusion Matrix and Statistics
## 
##           Actual
## Prediction  low high
##       low  1133  613
##       high    0   10
##                                           
##                Accuracy : 0.6509          
##                  95% CI : (0.6281, 0.6732)
##     No Information Rate : 0.6452          
##     P-Value [Acc > NIR] : 0.3185          
##                                           
##                   Kappa : 0.0206          
##                                           
##  Mcnemar's Test P-Value : <2e-16          
##                                           
##             Sensitivity : 0.016051        
##             Specificity : 1.000000        
##          Pos Pred Value : 1.000000        
##          Neg Pred Value : 0.648912        
##               Precision : 1.000000        
##                  Recall : 0.016051        
##                      F1 : 0.031596        
##              Prevalence : 0.354784        
##          Detection Rate : 0.005695        
##    Detection Prevalence : 0.005695        
##       Balanced Accuracy : 0.508026        
##                                           
##        'Positive' Class : high            
##

Evaluation of Model

On our test set, the model performed quite well. The positive predictive value was 1. This means that of the songs predicted to be positive, the model predicted 100% of them right. The sensitivity rate was still quite low at around 0.016, meaning that a large number of false negatives were present in the final results. This means that a DJ using our model would overlook about 90% of popular songs. Nonetheless, 100% of the songs they would use would be popular among their crowd according to our popularity rating. The specificity was also 100%, meaning all songs with low popularity would not be recommended to the radio station. The F1 score is .032 which is extremely low and most recommendations would determine that this is not a “good” model.

Fairness Assessment

This analysis does not include data for any protected classes. The risk for the use of the data is minimal in terms of human confidentiality damage, however there is potential for some financial damage. Artists and publishing groups could potentially lose money if radio stations were to use this model to determine which songs to play and thereby reducing monetary potential for a song that the model deemed unpopular, but really should have been popular. Radio stations could lose viewership and money by using this model and it predicting a song would be popular, but in actuality is not.

Conclusions

Overall, the final model was able to recommend popular songs to radio hosts 100% of the time for 2020 without adding in any low popularity songs. Since our aim is for a radio host and not artists, this model would be helpful in determining a few songs to add to a playlist. It will not be a comprehensive list of all songs, but will be recommendations for adding songs that will most definitely be a hit. The radio stations could benefit greatly from these recommendations. This model may not be comprehensive in weeding out false negatives, it is practically perfect in determining true positives. The final model identified many songs that are considered Latin pop. This indicates a popular rise in this genre of music. This could indicate that this genre should be played more and our model could be helpful at predicting a new trend in music that is not played enough, but should be.

Future Work

Limitations

A significant limitation on our model was the discrepancy between popularity prevalence rates in different years. This limited our model because many of the songs that our model predicted as popular were kept within our test data set. It was necessary to keep them within the test data set because our data sets were divided by year, thus no point could be exchanged between data sets. In the future, we would ideally find a better mechanism for distributing songs across train, tune, and test data sets to contain a similar prevalence rate. Due to the significant increase in performance on the test set, which had a prevalence rate almost double that of the tune data set, it is possible that in the future we would train our model on a data set with a higher prevalence rate.

Further Analysis

Further analysis could be done using a regression based decision tree model. The reason for this is that our cutoff of what was deemed popular and unpopular was somewhat arbitrary. More likely it is the case that some are very highly popular, some highly popular, some moderately popular, and so on. A regression based model would allow us to present songs in order of popularity which would allow DJs using our model to select more songs that are moderately to highly popular that may have barely missed the cutoff on our classification model. Another way in which we could increase the analytical power of our model would be to factor in popularity by genre. This would benefit DJs greatly, as it would enable them to find songs that fit the theme of their radio show or event.