Spotify is one of the largest audio streaming platforms competing with rivals such as Apple Music. In the world of music streaming, these platforms offer virtually the same product and so live and die based on user experience and satisfaction.
Using a dataset of Spotify User Behaviors sourced from Kaggle, I aim to predict user satisfaction with both the music and podcast services of Spotify and identify the most important factors that predict user satisfaction.
This project highlights the insights User Experience Research can uncover and serves as an example of the type of work possible with machine learning techniques.
From an industry perspective, the results of this project can identify the most impactful areas for Spotify to devote resources to in order to improve user satisfaction across their services.
Detailed Explanation of Project
Spotify has two key services that are captured in this dataset:
Music
Podcasts
Outcome Varibles: Satisfaction Variables
Music_Rating: How do you rate the spotify music recommendations? 1 (Dislike a lot) - 5 (Like a lot)
Podcast_Satisfaction: Are you satisfied with the variety and availability of podcasts on Spotify? 1 (Very Dissatisfied) - 5 (Very Satisfied)
Predictors: 3 General Classes
Demographics: Age, Gender
User Behavior: Usage_Period, Listening_Device, Subscription, Subscription_Willingness, Plan_Type Music Specific: Time_Slot, Frequency, Exploration_Method Podcast Specific: Podcast_Frequency
Preferences: Content (preferr music or podcasts) Music Specific: Favorite_Genre, Mood Podcast Specific: Favorite_Podcast_Genre, Pref_Pod_Format,
Pref_Host_Type, Pref_Pod_Duration
High Level Data Analysis
Cleaning the Data
Here I am coverting variables into formats that will be easier to work with later on.
Here I show a simple view of each variable compared to satifaction. If an individual variable shows no variability of satisfaction, then it would not be helpful in using to predict satisfaction. The good news is that this dataset seems to have adequate variability to use.
Bivariate Linear Analysis
Here I try to see if user satisfaction is tied to one obvious variable by using a lot of simple bivariate linear analyses. However, it seems the question of user satisfaction is more complicated than that so I will move forward with more complex analyses using multiple variables at a time.
plots <-lapply(names(data), function(column) { data_to_plot <- data %>%filter(!is.na(.data[[column]])) %>%mutate(across(all_of(column), as.factor),Music_Rating =factor(Music_Rating, levels =1:5, labels =c("1", "2", "3", "4", "5"))) p <-ggplot(data_to_plot, aes_string(x = column, fill ='Music_Rating')) +geom_bar(position ="stack") +labs(title =paste(column, "by Music Satisfaction")) +theme_minimal() +scale_x_discrete(guide =guide_axis(check.overlap =TRUE))})
For Podcasts
Here I do the same as above but with the podcast user behaviors and satifaction and come to the same conclusion. More complex anaylses is needed.
The next level of analyses up is linear regression models which allow the use of multiple variables at a time but constrict each variable to a linear effect on satisfaction.
We can see that linear regression does not predict user satisfaction for either Music or Podcast offerings well so I will try a nonlinear machine learning approach for better modeling.
Methods
This image breaks down the differnt models I will test to see which best predicts User Satisfaction for a total of 8 models. First, the data is broken between Music and Podcasts, then to 2 types of machine learning models (Random Forest and Boosted Trees), and then 2 different versions of input variables included.
Here I create the training and testing datasets that the machine learning algorithms with use to predict User Satisfaction.
Here I create the different machine learning models that will be tested and compared to each other.
Results and Feature Importance
Results in RMSE and Rsq
For Music Satisfaction
Here I list each model tested for predicting Music Satisfaction along with their RMSE and R^2 scores. A lower RMSE is more accurate and a higher R^2 is more accurate.
Random Forest Test Music Behav RMSE: 0.5793257
Random Forest Test Music Behav R^2: 0.6881625
XGBoost Music Behav Test RMSE: 0.615039
XGBoost Music Behav Test R^2: 0.6493327
BEST: Random Forest Test Music Behav Pref RMSE: 0.4696392
BEST: Random Forest Test Music Behav Pref R^2: 0.7948912
XGBoost Music Behav Pref Test RMSE: 0.5209496
XGBoost Music Behav Pref Test R^2: 0.7495241
For Music Satisfaction, a random forest model with Demographic, User Behavior, and Preference predictors scored the best and therefore predicts user satisfaction the best.
For Podcast Satisfaction
Here I do the same for Podcasts.
Random Forest Test Pod Behav RMSE: 0.4142121
Random Forest Test Pod Behav R^2: 0.7276716
XGBoost Pod Behav Test RMSE: 0.4374515
XGBoost Pod Behav Test R^2: 0.6913246
BEST: Random Forest Test Pod Behav Pref RMSE: 0.2356998
BEST: Random Forest Test Pod Behav Pref R^2: 0.9105263
XGBoost Pod Behav Pref Test RMSE: 0.3056584
XGBoost Pod Behav Test R^2: 0.8507245
For Podcast Satisfaction, the same type of Random Forest model with Demographic + User Behaviors + Preferences predictors was the best fitting.
Feature Importance
Music
Actionable Takeaways
GenderMale:
It seems that being male is the best predicts higher user satisfaction with Spotify. However, this result can likely be attributed to the apparant undersampling of males in the data and not something about how males use spotify.
Age 20-35:
Here I isolate the next best predictor, being aged 20-35. It appears those in this age group are likely to have higher satisfaction with Spotify’s Music offerings. The business takeaway here is that there may be more ground to gain with people younger than 20 and those over 35.
Actionable Recommendations:
A kid friendly version of the app with parental controls to allow kids to use the platform safely.
Some more of the marketing budget directed towards older generations as well as more research into why their satisfaction may not be as high.
Subscription Premium:
This was a surprising result as those who had Spotify Premium were more likely to be unsatisfied with the music offerings. This may indicate that users are more critical even when they are paying for a better service.
Actionable Recommendations:
Investigate further with Spotify Premium members about why satisfaction is low. Is it feature related or price related?
Consider a pricing structure change to offer different sets of features at different prices.
Favorite_GenreClassical:
If your favorite genre is classical, it’s likely you are less satisfied with Spotify’s music service.
Actionable Recommendations:
Offer more refined Classical music playlists.
Add more Classical music to Spotify’s library if selection is lacking.
Exploration_MethodOthers
People who perfer to be introduced to new music through others are less satisfied with Spotify’s music service. This makes intuitive sense as you are more likely to turn to others for music recommendations if you do not like Spotify’s algorithm.
Actionable Recommendations:
Is there a common feature between users who do not prefer Spotify’s recommendations and can a different recommendation algorithm be used for those users?
Add more features to make getting music recommendations from others easier, such as friends’ recently liked songs appearing on a playlist in your feed.
Favorite_GenreRap:
If your favorite genre of music is Rap then you are more likely to be satisfied with Spotify.
Actionable Recommendations:
Take lessons from the approach to Rap offerings on Spotify such as the Rap Caviar playlist to apply to other genres.
Communicate Spotify’s other genres to their fans as effectively as they have done for their Rap genre.
Podcasts
Actionable Takeaways
GenderMale:
It seems that being male is the best predicts higher user satisfaction with Spotify. However, this result can likely be attributed to the apparant undersampling of males in the data and not something about how males use spotify.
Age20-35:
Being in the 20-35 demographic seems to predict lower Podcast satisfaction.
Actionable Recommendations:
Investigate this age demographic in more detail in future research.
Use marketing budget to better inform this demographic of Podcast features and offering.
Subscription Premium:
It appears those with premium subscriptions are more satisfied with Spotify’s podcast offerings.
Actionable Recommendations:
Educate and advertise to those with free memberships listening to podcasts the benefits of premium.
Usage_Period More than 2 Years
If one has been using Spotify for more than 2 years, they are likely to be satisfied with Spotify Podcasts. However, this may be explained by surviorship bias, as users who survived to using Spotify for more than 2 years are ones who were satisfied for other reasons.
Actionable takeaways:
Important to offer incentives and discounts from that 0-2 year usage mark to have better retainment of users
Student Plan
library(randomForest)library(pdp)pdp_data <-partial(rf_pod_behav_pref_p, pred.var ="Plan_TypeStudent Plan-Rs 59/month")ggplot(pdp_data, aes(x =`Plan_TypeStudent Plan-Rs 59/month`, y = yhat)) +geom_line() +geom_point() +labs(title ="Partial Dependence of Being a Student on Podcast Satisfaction",x ="Plan_TypeStudent Plan-Rs 59/month",y ="Partial Dependence Effect")
Those on the student plan seem to have higher satisfaction with Spotify Podcasts.
Actionable Recommendation:
Sponsor college events/market to college students.
Offer easy integration with Canvas so that professors who assign podcasts as part of the reading can easily link into Spotify introducing students to the platform early.
Summary of Project
In this analysis, I used Spotify User Behavior survey data to understand what factors contribute the most to user satisfaction, both positively and negatively. This was done by testing many different mathematical models and assessing their accuracy. A Random Forest Machine Learning model that factors in user demographics, behaviors, and preferences was the best at predicting user satisfaction. From there, I analzed the importance of each factor and if it helped or hurt user satisfaction and made actionable recommendations for business strategy based on the result.
These actionable recommendations were only possible through user survey data collected by Spotify User Researchers and provided open source to Kaggle.
