Choose one commercial recommender and describe how you think it works (content-based, collaborative filtering, etc). Does the technique deliver a good experience or are the recommendations off-target?

HULU’s Recommender System

As the Internet continue to evolve, information is very unlimited and people become overwhelmed and confused about what they are looking for. With the help of recommender systems, a lot of useful information can be filtered out to help users discover and select the information that is of interest to them. There are thousands of videos available on Hulu and it may be difficult for users to find videos that best match their interests. Therefore, the first goal of Hulu’s recommendation system is to help users find content which will be of interest to them. A lot of Hulu’s content is based on episodes from shows (which are obviously related since they belong to a show) and so they recommend shows to users instead of single videos. Their content is divided into two sections: on-air and library shows. Majority of the content is on-air shows. Library shows are recommended when summer or weekend is around.

Hulu’s recommendation engine has been built using a collaborative filtering approach, called neural autoregressive distribution estimator for collaborative filtering (CF-NADE). This method is an application of deep learning in recommendation systems, using explicit feedback where users give each item an explicit rating. Collaborative filtering depends on user behavior data so that it can predict user preferences. Specifically, there are two types of collaborative filtering methods: User-based and Item-based. User-based assumes a user will like an item liked by other users with similar preferences. On the other hand, item-based collaborative filtering assumes a user will prefer items similar to ones in their watch or browse history. Hulu uses the Item-based collaborative filtering algorithm.

What drives most recommendation systems is user behavior data. There are two main types: implicit and explicit user feedback data. Explicit user feedback data primarily includes user voting data and can show if the user likes the show. Meanwhile implicit feedback data includes information on users watching, browsing, searching and the like but does not give a clue if user likes the show or not. As the quantity of implicit data at Hulu far outweighs the amount of explicit feedback, Hulu is designed primarily to work with implicit feedback data.

Architecture

There are 5 modules in the development of Hulu’s recommender system.

  1. User Profile Builder - Building the user’s profile based on preferences
  2. Recommendation Core - Generate general recommendations similar so the shows in the user’s history/profile.
  3. Filtering - Shows are filtered to recommedations can be more accurate.
  4. Ranking - Rank shows that closely match user preferences or what they might like.
  5. Explanation - A caption stating why a particular show was recommended. For instance, if you watched “The Rookie” Hulu would say something like “Because you watched The Rookie, we recommend 911”.

Architecture of Hulu

What’s Next?

Hulu’s mobile app give subscribers better control over the way the streaming service presents recommendations and suggested shows. Users are able to tell Hulu to stop suggesting content they have no interest in. They can also remove items from their history. When that happens, Hulu’s recommendation engine will essentially forget that the user ever watched the show or movie in question.

Architecture of Hulu



Does the technique deliver a good experience or are the recommendations off-target?

For me personally, Hulu does really well with recommendations. For example, when I’m watching a movie and it finishes, Hulu will display another movie of similar taste that will show next on autoplay. The recommendations usually what I like. Also, Hulu comes with watch history, so you can easily keep the track of movies and videos you like to watch.



Sources

http://www.josemalvarez.es/web/2012/04/23/hulus-recommendation-system/

https://rethinkresearch.biz/articles/hulu-hints-recommendations-are-dead/

https://www.theverge.com/2018/5/16/17360344/hulu-recommendations-watch-history-new-guide

https://www.cnblogs.com/zhenjing/articles/hulu-recommendation-system.html

LS0tDQp0aXRsZTogIkRBVEE2MTI6IFdlZWsgMiBSZXNlYXJjaCBEaXNjdXNzaW9uIEFzc2lnbm1lbnQgMSINCm5hbWU6ICJKYXZlcm4gV2lsc29uIg0Kb3V0cHV0OiBodG1sX25vdGVib29rDQotLS0NCg0KKipDaG9vc2Ugb25lIGNvbW1lcmNpYWwgcmVjb21tZW5kZXIgYW5kIGRlc2NyaWJlIGhvdyB5b3UgdGhpbmsgaXQgd29ya3MgKGNvbnRlbnQtYmFzZWQsIGNvbGxhYm9yYXRpdmUgZmlsdGVyaW5nLCBldGMpLiBEb2VzIHRoZSB0ZWNobmlxdWUgZGVsaXZlciBhIGdvb2QgZXhwZXJpZW5jZSBvciBhcmUgdGhlIHJlY29tbWVuZGF0aW9ucyBvZmYtdGFyZ2V0PyoqDQoNCiMjIyBIVUxVJ3MgUmVjb21tZW5kZXIgU3lzdGVtDQoNCkFzIHRoZSBJbnRlcm5ldCBjb250aW51ZSB0byBldm9sdmUsIGluZm9ybWF0aW9uIGlzIHZlcnkgdW5saW1pdGVkIGFuZCBwZW9wbGUgYmVjb21lIG92ZXJ3aGVsbWVkIGFuZCBjb25mdXNlZCBhYm91dCB3aGF0IHRoZXkgYXJlIGxvb2tpbmcgZm9yLiBXaXRoIHRoZSBoZWxwIG9mIHJlY29tbWVuZGVyIHN5c3RlbXMsIGEgbG90IG9mIHVzZWZ1bCBpbmZvcm1hdGlvbiBjYW4gYmUgZmlsdGVyZWQgb3V0IHRvIGhlbHAgdXNlcnMgZGlzY292ZXIgYW5kIHNlbGVjdCB0aGUgaW5mb3JtYXRpb24gdGhhdCBpcyBvZiBpbnRlcmVzdCB0byB0aGVtLiBUaGVyZSBhcmUgdGhvdXNhbmRzIG9mIHZpZGVvcyBhdmFpbGFibGUgb24gSHVsdSBhbmQgaXQgbWF5IGJlIGRpZmZpY3VsdCBmb3IgdXNlcnMgdG8gZmluZCB2aWRlb3MgdGhhdCBiZXN0IG1hdGNoIHRoZWlyIGludGVyZXN0cy4gVGhlcmVmb3JlLCB0aGUgZmlyc3QgZ29hbCBvZiBIdWx1J3MgcmVjb21tZW5kYXRpb24gc3lzdGVtIGlzIHRvIGhlbHAgdXNlcnMgZmluZCBjb250ZW50IHdoaWNoIHdpbGwgYmUgb2YgaW50ZXJlc3QgdG8gdGhlbS4gQSBsb3Qgb2YgSHVsdSdzIGNvbnRlbnQgaXMgYmFzZWQgb24gZXBpc29kZXMgZnJvbSBzaG93cyAod2hpY2ggYXJlIG9idmlvdXNseSByZWxhdGVkIHNpbmNlIHRoZXkgYmVsb25nIHRvIGEgc2hvdykgYW5kIHNvIHRoZXkgcmVjb21tZW5kIHNob3dzIHRvIHVzZXJzIGluc3RlYWQgb2Ygc2luZ2xlIHZpZGVvcy4gVGhlaXIgY29udGVudCBpcyBkaXZpZGVkIGludG8gdHdvIHNlY3Rpb25zOiBvbi1haXIgYW5kIGxpYnJhcnkgc2hvd3MuIE1ham9yaXR5IG9mIHRoZSBjb250ZW50IGlzIG9uLWFpciBzaG93cy4gTGlicmFyeSBzaG93cyBhcmUgcmVjb21tZW5kZWQgd2hlbiBzdW1tZXIgb3Igd2Vla2VuZCBpcyBhcm91bmQuDQoNCkh1bHUncyByZWNvbW1lbmRhdGlvbiBlbmdpbmUgaGFzIGJlZW4gYnVpbHQgdXNpbmcgYSAqKmNvbGxhYm9yYXRpdmUgZmlsdGVyaW5nKiogYXBwcm9hY2gsIGNhbGxlZCAqbmV1cmFsIGF1dG9yZWdyZXNzaXZlIGRpc3RyaWJ1dGlvbiBlc3RpbWF0b3IgZm9yIGNvbGxhYm9yYXRpdmUgZmlsdGVyaW5nIChDRi1OQURFKSouIFRoaXMgbWV0aG9kIGlzIGFuIGFwcGxpY2F0aW9uIG9mIGRlZXAgbGVhcm5pbmcgaW4gcmVjb21tZW5kYXRpb24gc3lzdGVtcywgdXNpbmcgZXhwbGljaXQgZmVlZGJhY2sgd2hlcmUgdXNlcnMgZ2l2ZSBlYWNoIGl0ZW0gYW4gZXhwbGljaXQgcmF0aW5nLiBDb2xsYWJvcmF0aXZlIGZpbHRlcmluZyBkZXBlbmRzIG9uIHVzZXIgYmVoYXZpb3IgZGF0YSBzbyB0aGF0IGl0IGNhbiBwcmVkaWN0IHVzZXIgcHJlZmVyZW5jZXMuIFNwZWNpZmljYWxseSwgdGhlcmUgYXJlIHR3byB0eXBlcyBvZiBjb2xsYWJvcmF0aXZlIGZpbHRlcmluZyBtZXRob2RzOiAqVXNlci1iYXNlZCBhbmQgSXRlbS1iYXNlZC4qIFVzZXItYmFzZWQgYXNzdW1lcyBhIHVzZXIgd2lsbCBsaWtlIGFuIGl0ZW0gbGlrZWQgYnkgb3RoZXIgdXNlcnMgd2l0aCBzaW1pbGFyIHByZWZlcmVuY2VzLiBPbiB0aGUgb3RoZXIgaGFuZCwgaXRlbS1iYXNlZCBjb2xsYWJvcmF0aXZlIGZpbHRlcmluZyBhc3N1bWVzIGEgdXNlciB3aWxsIHByZWZlciBpdGVtcyBzaW1pbGFyIHRvIG9uZXMgaW4gdGhlaXIgd2F0Y2ggb3IgYnJvd3NlIGhpc3RvcnkuICoqSHVsdSB1c2VzIHRoZSBJdGVtLWJhc2VkIGNvbGxhYm9yYXRpdmUgZmlsdGVyaW5nIGFsZ29yaXRobSoqLiANCg0KV2hhdCBkcml2ZXMgbW9zdCByZWNvbW1lbmRhdGlvbiBzeXN0ZW1zIGlzIHVzZXIgYmVoYXZpb3IgZGF0YS4gVGhlcmUgYXJlIHR3byBtYWluIHR5cGVzOiAqaW1wbGljaXQgYW5kIGV4cGxpY2l0IHVzZXIgZmVlZGJhY2sgZGF0YSouIEV4cGxpY2l0IHVzZXIgZmVlZGJhY2sgZGF0YSBwcmltYXJpbHkgaW5jbHVkZXMgdXNlciB2b3RpbmcgZGF0YSBhbmQgY2FuIHNob3cgaWYgdGhlIHVzZXIgbGlrZXMgdGhlIHNob3cuIE1lYW53aGlsZSBpbXBsaWNpdCBmZWVkYmFjayBkYXRhIGluY2x1ZGVzIGluZm9ybWF0aW9uIG9uIHVzZXJzIHdhdGNoaW5nLCBicm93c2luZywgc2VhcmNoaW5nIGFuZCB0aGUgbGlrZSBidXQgZG9lcyBub3QgZ2l2ZSBhIGNsdWUgaWYgdXNlciBsaWtlcyB0aGUgc2hvdyBvciBub3QuIEFzIHRoZSBxdWFudGl0eSBvZiBpbXBsaWNpdCBkYXRhIGF0IEh1bHUgZmFyIG91dHdlaWdocyB0aGUgYW1vdW50IG9mIGV4cGxpY2l0IGZlZWRiYWNrLCAqKkh1bHUgaXMgZGVzaWduZWQgcHJpbWFyaWx5IHRvIHdvcmsgd2l0aCBpbXBsaWNpdCBmZWVkYmFjayBkYXRhLioqDQoNCiMjIyBBcmNoaXRlY3R1cmUNCg0KVGhlcmUgYXJlIDUgbW9kdWxlcyBpbiB0aGUgZGV2ZWxvcG1lbnQgb2YgSHVsdSdzIHJlY29tbWVuZGVyIHN5c3RlbS4NCg0KICAxLiBVc2VyIFByb2ZpbGUgQnVpbGRlciAtIEJ1aWxkaW5nIHRoZSB1c2VyJ3MgcHJvZmlsZSBiYXNlZCBvbiBwcmVmZXJlbmNlcw0KICAyLiBSZWNvbW1lbmRhdGlvbiBDb3JlIC0gR2VuZXJhdGUgZ2VuZXJhbCByZWNvbW1lbmRhdGlvbnMgc2ltaWxhciBzbyB0aGUgc2hvd3MgaW4gdGhlIHVzZXIncyBoaXN0b3J5L3Byb2ZpbGUuDQogIDMuIEZpbHRlcmluZyAtIFNob3dzIGFyZSBmaWx0ZXJlZCB0byByZWNvbW1lZGF0aW9ucyBjYW4gYmUgbW9yZSBhY2N1cmF0ZS4NCiAgNC4gUmFua2luZyAtIFJhbmsgc2hvd3MgdGhhdCBjbG9zZWx5IG1hdGNoIHVzZXIgcHJlZmVyZW5jZXMgb3Igd2hhdCB0aGV5IG1pZ2h0IGxpa2UuDQogIDUuIEV4cGxhbmF0aW9uIC0gQSBjYXB0aW9uIHN0YXRpbmcgd2h5IGEgcGFydGljdWxhciBzaG93IHdhcyByZWNvbW1lbmRlZC4gRm9yIGluc3RhbmNlLCBpZiB5b3Ugd2F0Y2hlZCAiVGhlIFJvb2tpZSIgSHVsdSB3b3VsZCBzYXkgc29tZXRoaW5nIGxpa2UgIkJlY2F1c2UgeW91IHdhdGNoZWQgVGhlIFJvb2tpZSwgd2UgcmVjb21tZW5kIDkxMSIuDQogIA0KDQo8aW1nIGNsYXNzPSJpbWcgaW1nLXRodW1ibmFpbCBpbWctcmVzcG9uc2l2ZSIgc3R5bGU9IndpZHRoOjk5JTsiIHNyYz0iaHR0cHM6Ly9naXRodWIuY29tL2phdmVybncvREFUQTYxMi1SZWNvbW1lbmRlci1TeXN0ZW1zL2Jsb2IvbWFzdGVyL2h1bHVfYXJjaGl0ZWN0dXJlLlBORz9yYXc9dHJ1ZSIgYWx0PSJBcmNoaXRlY3R1cmUgb2YgSHVsdSI+DQoNCg0KIyMjIFdoYXQncyBOZXh0Pw0KDQpIdWx1J3MgbW9iaWxlIGFwcCBnaXZlIHN1YnNjcmliZXJzIGJldHRlciBjb250cm9sIG92ZXIgdGhlIHdheSB0aGUgc3RyZWFtaW5nIHNlcnZpY2UgcHJlc2VudHMgcmVjb21tZW5kYXRpb25zIGFuZCBzdWdnZXN0ZWQgc2hvd3MuIFVzZXJzIGFyZSBhYmxlIHRvIHRlbGwgSHVsdSB0byBzdG9wIHN1Z2dlc3RpbmcgY29udGVudCB0aGV5IGhhdmUgbm8gaW50ZXJlc3QgaW4uIFRoZXkgY2FuIGFsc28gcmVtb3ZlIGl0ZW1zIGZyb20gdGhlaXIgaGlzdG9yeS4gV2hlbiB0aGF0IGhhcHBlbnMsIEh1bHUncyByZWNvbW1lbmRhdGlvbiBlbmdpbmUgd2lsbCBlc3NlbnRpYWxseSBmb3JnZXQgdGhhdCB0aGUgdXNlciBldmVyIHdhdGNoZWQgdGhlIHNob3cgb3IgbW92aWUgaW4gcXVlc3Rpb24uIA0KDQo8aW1nIGNsYXNzPSJpbWcgaW1nLXRodW1ibmFpbCBpbWctcmVzcG9uc2l2ZSIgc3R5bGU9IndpZHRoOjk5JTsiIHNyYz0iaHR0cHM6Ly9naXRodWIuY29tL2phdmVybncvREFUQTYxMi1SZWNvbW1lbmRlci1TeXN0ZW1zL2Jsb2IvbWFzdGVyL2h1bHUuUE5HP3Jhdz10cnVlIiBhbHQ9IkFyY2hpdGVjdHVyZSBvZiBIdWx1Ij4NCg0KDQo8YnIvPiA8YnIvPg0KDQoqKkRvZXMgdGhlIHRlY2huaXF1ZSBkZWxpdmVyIGEgZ29vZCBleHBlcmllbmNlIG9yIGFyZSB0aGUgcmVjb21tZW5kYXRpb25zIG9mZi10YXJnZXQ/KioNCg0KRm9yIG1lIHBlcnNvbmFsbHksIEh1bHUgZG9lcyByZWFsbHkgd2VsbCB3aXRoIHJlY29tbWVuZGF0aW9ucy4gRm9yIGV4YW1wbGUsIHdoZW4gSSdtIHdhdGNoaW5nIGEgbW92aWUgYW5kIGl0IGZpbmlzaGVzLCBIdWx1IHdpbGwgZGlzcGxheSBhbm90aGVyIG1vdmllIG9mIHNpbWlsYXIgdGFzdGUgdGhhdCB3aWxsIHNob3cgbmV4dCBvbiBhdXRvcGxheS4gVGhlIHJlY29tbWVuZGF0aW9ucyB1c3VhbGx5IHdoYXQgSSBsaWtlLiBBbHNvLCBIdWx1IGNvbWVzIHdpdGggd2F0Y2ggaGlzdG9yeSwgc28geW91IGNhbiBlYXNpbHkga2VlcCB0aGUgdHJhY2sgb2YgbW92aWVzIGFuZCB2aWRlb3MgeW91IGxpa2UgdG8gd2F0Y2guDQoNCjxici8+IDxici8+DQoNCiMjIyBTb3VyY2VzDQoNCltodHRwOi8vd3d3Lmpvc2VtYWx2YXJlei5lcy93ZWIvMjAxMi8wNC8yMy9odWx1cy1yZWNvbW1lbmRhdGlvbi1zeXN0ZW0vXShodHRwOi8vd3d3Lmpvc2VtYWx2YXJlei5lcy93ZWIvMjAxMi8wNC8yMy9odWx1cy1yZWNvbW1lbmRhdGlvbi1zeXN0ZW0vKQ0KDQpbaHR0cHM6Ly9yZXRoaW5rcmVzZWFyY2guYml6L2FydGljbGVzL2h1bHUtaGludHMtcmVjb21tZW5kYXRpb25zLWFyZS1kZWFkL10oaHR0cHM6Ly9yZXRoaW5rcmVzZWFyY2guYml6L2FydGljbGVzL2h1bHUtaGludHMtcmVjb21tZW5kYXRpb25zLWFyZS1kZWFkLw0KKQ0KDQpbaHR0cHM6Ly93d3cudGhldmVyZ2UuY29tLzIwMTgvNS8xNi8xNzM2MDM0NC9odWx1LXJlY29tbWVuZGF0aW9ucy13YXRjaC1oaXN0b3J5LW5ldy1ndWlkZV0oaHR0cHM6Ly93d3cudGhldmVyZ2UuY29tLzIwMTgvNS8xNi8xNzM2MDM0NC9odWx1LXJlY29tbWVuZGF0aW9ucy13YXRjaC1oaXN0b3J5LW5ldy1ndWlkZSkNCg0KW2h0dHBzOi8vd3d3LmNuYmxvZ3MuY29tL3poZW5qaW5nL2FydGljbGVzL2h1bHUtcmVjb21tZW5kYXRpb24tc3lzdGVtLmh0bWxdKGh0dHBzOi8vd3d3LmNuYmxvZ3MuY29tL3poZW5qaW5nL2FydGljbGVzL2h1bHUtcmVjb21tZW5kYXRpb24tc3lzdGVtLmh0bWwpDQoNCg==