1 Assignment Instructions

Your task is to analyze an existing recommender system that you find interesting. You should:

Perform a Scenario Design analysis as described below. Consider whether it makes sense for your selected recommender system to perform scenario design twice, once for the organization (e.g. Amazon.com) and once for the organization’s customers. Attempt to reverse engineer what you can about the site, from the site interface and any available information that you can find on the Internet or elsewhere. Include specific recommendations about how to improve the site’s recommendation capabilities going forward. Create your report using an R Markdown file, and create a discussion thread with a link to the GitHub repo where your Markdown file notebook resides. You are not expected to need to write code for this discussion assignment.

How to Perform a Scenario Analysis (review)

Scenario Design is an easy-to-use “three question framework” that helps you in making sure that the information in your analysis work takes UX (user experience) into account:

Scenario Design

Scenario Design

Source: “Scenario Design: A Disciplined Approach to Customer Experience,” Bruce D. Temkin, Forrester Research, 2004. Temkin notes that before applying Scenario Design, one might ask, “What functionality should we offer?” After applying Scenario Design, one might instead ask, “What user goals should we serve?”

Your task is to:

  • Identify a recommender system web site, then
  • Answer the three scenario design questions for this web site.

This process of guessing/reverse engineering, while inexact, will help you build out your own ability to better account for “user needs” in designing recommender systems going forward. Being able to place the customer first in your mind is a “soft skill” that is highly valued in the technical marketplace.

2 Introduction

Personally, I have experienced very well curated music experience with Spotify and it is my default go-to music radio at my home especially since it is very well integrated with the smart speakers - Google Home and Amazon Alexa.

Simply put, the site will initially predict what one might like based on the taste of similar listeners. It then adjusts its weightings as one listens more according to their feedback and choicesMost recommendations come in at midnight (from playlists) or in the morning (from social media) and are then listened to throughout the day..

Here’s a snapshot of the Spoify Discover Weekly playlist whch recommends various music to listeners:

Spotify Discover Weekly

Spotify Discover Weekly

3 Scenario Analysis

1) Who are the target users?

The target users are music lovers from all age groups, genders using the internet-connected streaming service via various mobile and home electronic sources like Phone, Smart Speakers, Laptops, TV, Tablets, Car Audio etc.

2) What are the key goals?

Provide state-of-the-art uniquely curated music with a mix of variety so as to allow music lovers - Maximize user’s satisfaction - Recommend songs to hit top songs of user’s favourite list - Keep the user engaged in the music app to compete against other similar competitive legacy streaming services like iTunes, Pandora, Soundcloud, Deezer as well as new players like Amazon Music, Google Music

Currently, Spotify achieves a good recommendation that includes Time and Song Attributes - Time - Song Attributes

3) How can you help them accomplish the goals?

  • Uses various sources and ways of recommendation
  • Integrate voice recognition and NLP to ascertain the listeners mood, choice and environment
  • Use AI and ML to gauge the feedback system

4 Reverse Engineering

There are four types of Spotify playlists :

  1. Spotify People Curated - Hand-picked songs by their expert musical staff

  2. Spotify Algorithm Created - Based on what you and other listeners are listening to the most 2. Personally Created -You pick the song

  3. User Created - Other users pick.

Discover Weekly falls in the number two bucket. So what you are presented is based on what you’ve been listening to. Spotify has a very sophisticated algorithm to create everything.

Discover Weekly

Discover Weekly

Looking at the recommender algorithm that Spotify employs, it was interesting to find that Spotify doesn’t actually use a single revolutionary recommendation model. Instead, they mix together some of the best strategies used by other services to create their own uniquely powerful discovery engine.

There are following main types of recommendation models that Spotify employs:~

1. Collaborative Filtering models: Analyzes both user behavior and others’ behaviors (i.e. the ones that Last.fm originally used) - Uses K-nearest neighbour (KNN) technique - Music taste of users calculates distance between different users - Search for neighbour users who share similar interest in music and recommend content - Daily life: friend’s recommendation Collaborative Model

2. Natural Language Processing (NLP) models: Analyzes text within the audio trac

3. Content-based recommendation in Audio models: Analyzes the raw audio tracks themselves with convolutional neural networks! - Without user’s evaluation or ratings - Uses machine language to acquire information - Algorithms: decision trees, neural networks and vector-based methods

Audio Model

Audio Model

4. Knowledge based recommendation in Taste Profile: Spotify predicts what genres and microgenres you like based on songs, artists, and playlists you repeatedly listened to in the past, and recommends new songs from similar categories. On the flip side, Spotify learns which songs, artists, and playlists you don’t like based on the songs you skip often and uses this to improve their model. - Based on demands and preferences of user - Predictions decided by functions and features of objects

Knowledge Model

Knowledge Model

Additionally, Spotify uses traditional models as well a. Memory-based : Predict items based on previous ratings b. Model-based : Uses algorithms and models preferences c. Hybrid : Combines both models and outperforms them

Some basic APIs to get a high level insight of Spotify’s proprietary algorithm:

5 Current Technology behind Recommender System

The recommendation systems that Spotify use collect and process large volumes of data. With over 100 million users and 30 million songs on their database, it’s remarkable that they can maintain data pipelines that allow them to actively personalize recommendations for every user every week. Of course, advancements in big data technologies have helped and made this task easier. Spotify uses Hadoop clusters to distribute the load of managing such voluminous data and discover new songs weekly.

6 Current Issues with Recommender System

  1. Cold-start problem
  2. Unusual taste leads to poor recommendations
  3. Personalization weakened with popular songs recommended
  4. Big amount of data needed

7 Suggestions on Improving the Recommender System

  1. Provide a recommended music as a blend of local weather and user prefences

  2. Include voice based recommendation when using smart speakers to trigger Spotify Discover API. For example, differentiate recommendation between male, female and kid voice

  3. Use a combination of Neural Networks Processing within the device if applicable (similar to Apple Music) or Remote Data Processing (similar to Google Music) to have faster and improved recommendation processing

Source References: