This article by Sanjay Krishnan, Jay Patel, Michael J. Franklin and Ken Goldberg seems to overpromise and under-deliver.
Increasing attention has been given to the ability of social networks to feed some of our lower instincts. As social media has become ubiquitous in our lives, its ability to feed anger, distrust and provincial attitudes has become an important problem. Social media brought us together. Its promise is high. Its potential for danger is similarly high.
Recommender systems can be especially susceptible to the influence of social networks because they often filter collaboratively to create more full ratings. It’s necessary to use peer ratings to be able to tell how someone who hasn’t experienced something would feel about it. Recommender systems using collaborative filtering, however, may introduce less average content. If the goal is to find what elicited a strong response and deliver more of it, this becomes an efficent way to do that. On some platforms, that can maximize revenue.
The important part of this article is it is approaching a numerical method to find how social factors influence a user’s rating. The authors find a way to reconstruct a pre-influence rating from a rating that may be subjected to bias.
It is unclear why the authors chose to use BIC and maximum likelihood to fit their prediction function. In the end, they fit 4 linear models and 2 quadratic models. They didn’t use standard statistical techniques that we might use to choose between a linear and quadratic model. They weren’t choosing between different types of complicated models where BIC tends to be used.
The authors’ final prediction accounted for the amount of difference in a grade rating. It accounted for the amount of influence in a grade. This was effectively averaged over all graders. Their final prediction didn’t include any way to grade a user’s likelihood of influence. This would have been the more useful numerical technique. Without their method, we could create a rule: don’t suggest controversial topics as often.
It seems that this article dressed standard statistical questions up in flashy-looking techniques. It’s unclear that that improved the depth of their conclusions. It did make it harder for an educated person to read and critique. Sometimes a regression is just a regression.
Few recommender systems make note of changes in reviews over time. It would be possible to use A/B testing to collect the type of data used in this study and apply it more broadly across a wider platform. But this still would not give us any information about a particular users’ potential for influence. It wouldn’t really allow us to predict a particular users’ unbiased rating. It would allow us to find a numerical way to decide how controversial a topic is. In their case, they found out that Obamacare and marriage rights were controversial. It may not be wise for a content producer to limit content based on those topics.
Their ideas may point us in an appropriate direction towards finding numerical techniques to limit the amount of extreme content that could be found in digital content. We would have to go a lot further to find a practical use.