Discussion - Spotify Recommender Systems
Overview
Your task is to analyze an existing recommender system that you find interesting. You should:
Perform a Scenario Design analysis as described below. Consider whether it makes sense for your selected recommender system to perform scenario design twice, once for the organization (e.g. Amazon.com) and once for the organization’s customers. Attempt to reverse engineer what you can about the site, from the site interface and any available information that you can find on the Internet or elsewhere. Include specific recommendations about how to improve the site’s recommendation capabilities going forward. Create your report using an R Markdown file, and create a discussion thread with a link to the GitHub repo where your Markdown file notebook resides. You are not expected to need to write code for this discussion assignment. Here are two examples of the kinds of papers that might be helpful backgrounders for your research in #2 above (if you had chosen amazon.com or nytimes.com as your web site):
IEEE Internet Computing, Greg Linden, Brent Smith, and Jeremy York (2003): Amazon.com Recommendations: Item-to-Item Collaborative Filtering. Alex Spangher (2015): Building the Next New York Times Recommendation Engine.
Spotify’s Recommender System
Spotify is one of the most popular music streaming services according to Statista1. Its popularity as a streaming service and one of its core functionalities, that is recommending new songs and artists to its users, has sparked immense interest in its recommendation system.
There is a plethora of articles attempting to dissect how Spotify’s recommender system works.
Below is a short list of articles I was able to find:
- How Spotify Recommends Your New Favorite Artist
- How Spotify recommender system works
- Recommending music on Spotify with deep learning
- Music To My Ears: De-Blackboxing Spotify’s Recommendation Engine
- How does Spotify’s recommendation system work?
In addition there are a number of articles discussing how to build your own recommender system from your own Spotify listening history such as, Machine learning and recommender systems using your own Spotify data and Spotify Recommendation System with Machine Learning
Based on the abundance of interest in Spotify’s recommender system, it is clear that it is an important part of Spotify as a product and an important part of Spotify’s business model.
Scenario Design: Spotify as a Business
- Who are your target users?
Anyone who listens to music
- What are their key goals?
To listen to the music they already enjoy, but also to discover new music and artists that they have not heard of that they enjoy listening to as well.
- How can you help them accomplish those goals?
Recommend the listener new artists, albums, playlists, and songs that they have or may have not heard before that appeal to their tastes in music.
Reverse Engineering Spotify’s Recommender System
The article, How Spotify Recommends Your New Favorite Artist2, by Clark Boyd, suggests that Spotify is using a combination of three novel techniques in order to generate their recommendations. The author speculates that Spotify is using a combination of “Collaborative Filtering”, “Natural Language Processing”, and “Audio Models” to build their recommendation engine.
At a high level, the author describes Collaborative Filtering, as a method where your preferences are matched to other users with similar preferences, then music that you have not heard from your musically similar neighbors is recommended to you. Natural Language Processing is used to analyze the lyrics, titles of songs, and even playlist titles to find and recommend similar music to that which you already listen to as well. Lastly, the audio signature of songs are analyzed and fed into algorithms which identify songs with similar sound signatures and those are recommended as well. In reality Spotify is likely using a combination of all three of these methods and more in order to generate their recommendations.
The article, How Spotify recommender system works3, by Daniel Roy, dicusses the same techniques outlined by Clark Boyd, but also discusses how Spotify uses multi-armed bandit algorithms to drive their recommendations on their home page. There is a paper published by Spotify researchers called, Explore, Exploit, and Explain: Personalizing Explainable Recommendations with Bandits4. At a high level the multi-armed bandit algorithm is designed to make a choice from a limited pool of resources such that it maximizes gain.
Lastly, the article, Music To My Ears: De-Blackboxing Spotify’s Recommendation Engine5, by Dr. Martin Irvine, provides source material to support Spotify’s use of the collaborative filter and natural language processing techniques that Spotify uses to generate their recommendations. There is a link in this article to a paper published by a lead data scientist from Spotify, Christopher C. Johnson, Logistic Matrix Factorization for Implicit Feedback Data6, which outlines in depth how collaborative filtering actually works.
Improving Spotify’s Recommender System
As a heavy user of Spotify myself I have thought through a few improvements I believe they can make to their recommender system.
Different degrees of ratings
As of today (2021), Spotify only allows you to indicate that you like a song by clicking on a little heart icon which subsequently saves that song to your library. Often times I purposefully avoid doing this because the difference between simply liking a song and loving it so much I want to save it forever in my library is significant. They already have a feature where you can simply give a song a thumbs up or thumbs down but it is only available when using the “Radio” functionality which is a feature that plays songs randomly based on some artist, genre or song.
Similarly, Spotify does not allow you to universally dislike songs or artists either. It does surface that feature in certain areas of the application but I think allowing for explicit dislikes across the entire application could dramatically improve their recommendation system.