Mitigating the Harm of Recommender Systems
Read one or more of the articles below and consider how to counter the radicalizing effects of recommender systems or ways to prevent algorithmic discrimination.
Renee Diresta, Wired.com (2018): Up Next: A Better Recommendation System
Zeynep Tufekci, The New York Times (2018): YouTube, the Great Radicalizer
Sanjay Krishnan, Jay Patel, Michael J. Franklin, Ken Goldberg (n/a): Social Influence Bias in Recommender Systems: A Methodology for Learning, Analyzing, and Mitigating Bias in Ratings
Recommendation engines influence the choices we make every day - what book to read next, which song to download, which person to date.
At their best, smart systems serve buyers and sellers alike: Consumers save the time and effort of wading through the vast possibilities of the digital marketplace, and businesses build loyalty and drive sales through differentiated experiences.
But, as with many other new technologies, digital recommendations are also a source of unintended consequences.
Recommendations do more than just reflect consumer preferences - they actually shape them. If this sounds like a subtle distinction, it is not. Recommendation systems have the potential to fuel biases and affect sales in unexpected ways.
Implications for recommendation engine design, not just in the music/e-commerce industry but in any setting where retailers use recommendation algorithms to improve customer experience and drive sales.
E-commerce has dramatically affected consumer choice. Unconstrained by physical limitations of the brick-and-mortar model, businesses can offer virtually unlimited selections of products online, giving consumers access not only to popular items but to obscure, niche ones as well.
There are both more needles and more hay. As consumers face a radically wider set of options, they must exercise greater care in evaluating potential products for purchase or consumption.
Experience-based (or taste-based) goods such as music, books, and movies are particularly complex: Consumers must spend time experiencing them before they know if they like them. Even if the sticker prices for goods aren’t high, or the goods are included as part of subscription services, the time consumers must spend to evaluate each of them is valuable.
Worse, the sunk cost of evaluation time is unrecoverable: Consumers can’t unlisten, unread, or unwatch goods that turn out to be a poor fit.
Music Industry, as in other industries involving experience-based goods, new models (such as Spotify and Apple Music) are disrupting the music industry. Digital distribution channels, including paid subscriptions, on-demand streaming, and digital downloads, are currently about 80% of the U.S. music market.
Regardless of the distribution channel, algorithms and recommendation engines significantly affect the digital consumption of music, as recommendations add value in identifying unknown songs that are more likely to strike a chord with the consumer. Surprisingly, recommendation systems alter how much consumers are willing to pay for a product that they just listened to.
Consumers don’t just prefer what they have experienced and know they enjoy; they prefer what the system said they would like. This is surprising since consumers shouldn’t need a system to tell them how much they enjoyed a song they just heard. The advent of recommendation systems may leave us questioning our own taste. We move from asking ourselves, “Do I like this?” to asking, “Should I like this?”
For consumers, recommendation engines have a potential dark side - they can manipulate preferences in ways consumers don’t realize.
After all, the details underlying recommendation algorithms are far from transparent. Faulty recommendation engines that inaccurately estimate consumers’ true preferences stand to pull down willingness to pay for some items and increase it for others, regardless of the likelihood of actual fit.
This may tempt less ethical organizations to inflate recommendations artificially. Even aside from the disreputable practice of direct manipulation, random error is a real problem for all recommendation systems.
For example, the best-performing recommendation systems in the $1 million Netflix Prize competition, using the latest machine learning developments in recommendation algorithms at the time, were off in their rating predictions on average by 20% of the rating scale (that is, an error of about 0.8 on a scale of 1 to 5 stars).
Both over and under - estimation are problems. Inflated ratings induce consumers to buy products they might not consider otherwise and could leave consumers disappointed from unmet expectations. Deflated ratings potentially turn off consumers from products they may have otherwise purchased. Mistakes hurt in both directions.
And the effects persist beyond dissatisfaction with a single purchase. They compound over time. After consumers experience a product, their feedback (like product ratings or purchases) influences future personalized predictions. Biased feedback can contaminate the system and lead to a vicious cycle of bias - the online retail equivalent of squealing audio feedback. Designers could also get an artificially inflated view of prediction accuracy, compromising their ability to improve systems. Even worse, unscrupulous agents could use such vulnerabilities to manipulate recommendation systems.
Recommendation and personalization are useful technologies which influence more and more our daily decisions. However, the bias that exists in the real world and which is reflected in the training data can be modeled and amplified by recommender systems and in the end returned as biased recommendations to the users.
This feedback process creates a self-perpetuating loop which progressively strengthens the filter bubbles we live in. Biased recommendations can also reinforce stereotypes such as those based on gender or ethnicity, possibly resulting in disparate impact.
If you only give attention to content consumption rates for your recommendation system and use it’s results for your cover and listing pages, you will end up with recommending same content to everybody over and over again.
It will create a chicken-egg problem. People are consuming the content because you promoted them on covers, and because they are consumed more they will be recommended more. You must always add randomness and clever ways to favor new and hidden contents over popular contents.
Which can lead into faulty recommendations (ex: you are labelling nearly all your movies which has a small dose of drama with ‘Drama’ label, so recommender system got confused to figure out right people who are into Drama movies and couldn’t find best matches inside Drama category).
Your content should be clean and good categorized if you are aiming for category/label based recommendation.
It became annoying if you are recommended over and over by the content you have already consumed. If you offer your content with account requirement, you can eliminate this problem most of the cases. But people are becoming less willing to create an account for every content distribution network, so you have to create a way to match same user using different devices to offer already consumed contents
People are using same devices to consume contents, especially families. Which will definitely screw up any recommender system, because you are dealing with multiple profiles without knowing it. Netflix tried to solve this by guessing, but there is no proven or gurantee way to guess how many different users are using the same account and know which one is active now. So they gave up guessing and switched to multiple profiles under one account and you have to select the profile before starting to watch content. It is a bit annoying but the best way to keep recommendation results accurate and on-target.
There is a realistic possibility given that e-commerce sites periodically provide databases to third-party consultants for data mining, intrusion detection, and statistical reporting. In some cases, personalization services are provided entirely by external firms.
(1) Given that perfect prediction is not possible, retailers and managers must be aware of the potential discord from unintended side effects of their recommendations. There is importance of reducing bias in recommendation systems, for example, through innovations in algorithm and user interface design and through human oversight, as an ongoing priority for the future.
(2) Research on the fairness of recommender systems is just getting started, and there are many important questions to explore. There are many more dimensions to the problem, such as the equitable treatment of content producers, as well as the distribution of non-accuracy recommendation value like diversity and serendipity.
(3) No matter what government regulations or private companies with public welfare responsibility do, gengeral public should be empowered through critical thinking and logical reasoning so that they can decide better which content to influence them and what not to. Sometimes, allowing instant gratification of yielding to temptation, people should take a step back and educate them about the pros and cons (SWOT analysis, in management terms) of their choices/decisions towards long-term sustainable welfare.