MSDS Spring 2018

DATA 607 Data Aquisition and Management

Jiadi Li

Week 12 Discussion & Assignment:Evaluation of Recommender Systems

Your task is to analyze an existing recommender system that you find interesting.
Quora: A place to share knowledge and better understand the world.
https://www.quora.com/

Quora is a question-and-answer site where questions are asked, answered, edited, and organized by its community of users, its publishers, Quora Inc..

You should:
1.Perform a Scenario Design analysis. Consider whether it makes sense for your selected recommender system to perform scenario design twice, once for the organization (e.g. Amazon.com) and once for the organization’s customers.

Scenario Analysis:

  1. Who are your target users?
    The target users of Quora(’s recommender system) are its community of users.

  2. What are their key goals?
    For those who search and/or ask questions, their key goals are to search for answers, get information, receive feedback or even just to view some interesting contents.
    For those who answer question, their key goals are to express their thoughts, share information, and exchange ideas.
    The mission statement of Quora (listed above) sufficiently include the key goals of Quora’s users.

  3. How can you help them accomplish those goals?
    Improving search engine, recommender system, monitoring and controlling the questions and contents. While Quora isn’t responsible for the accuracy and reliabiligy of the community, extreme expressions (in terms of opinions and contents) should be monitored.

2.Attempt to reverse engineer what you can about the site, from the site interface and any available information that you can find on the Internet or elsewhere.

Quora’s engineer team takes advantage of machine learning for various parts of the product. Features that related to recommender system are:

  1. question-quality classification and question-topic labeling
    This feature builds the foundation for recommending questions and answers.
    To ensure the quality of the questions, after a question is posted, the team classifies the question quality and determines its type by understanding the question with the Machine learning system. Based on a collection of amillion potential topics, the question will be labeled.

  2. A2A (Ask To Answer)
    This feature alone connects to a recommender system which link question to experts.
    The Machine Learning system generates the ranking of questions on an expert’s homepage feed based on user properties and a whole set of other raw and derived features as inputs to the ranking model.

  3. ranking answers on the feed and for questions, Email Digests
    To rank answers, the Machine Learning system uses a different set of features. Multiple models are used to come up with the final ranking to ensure that you see the most relevant questions and answers on the top of the feed or the quetion. “Upvoted” and “downvoted” feature also take part in the evaluation.

  4. Maintain high quality
    Duplicate question detection, abusive content detection and spam detection.

In general, Quora recommender system services well since they apply different models that range from personalized learning-to-rank to collaberative filtering approaches.

3.Include specific recommendations about how to improve the site’s recommendation capabilities going forward.

Quora utilizes recommender systems of various features throughout its application.
Based on my personal experience, Quora did a satisfactory job on recommending answers to my search and questions in email digests.
During my usage of the site, I noticed that they collects my interest first from the register process where I select a list of topics, and from my daily searching terms.
However, when searching for answers on Quora, it still takes some time to go through all relevant answers while seeing duplicated answers under different questions of similar content.
In this case, while Quora did fold some answers under a certain question when it’s somewhow duplicated, it would be more efficient and effective if Quora is able to combine questions in a more concise way.

Reference
https://en.wikipedia.org/wiki/Quora
https://www.forbes.com/sites/quora/2017/04/19/how-does-quora-use-machine-learning-in-2017/#657b3c433f3a
https://engineering.quora.com/Ask-To-Answer-as-a-Machine-Learning-Problem
https://engineering.quora.com/A-Machine-Learning-Approach-to-Ranking-Answers-on-Quora