I think that recommenders (and all machine learning algorithms in general) amplify the racial, gender, and other biases that are already present in our day to day culture. As mentioned in Evan Estola’s talk, a very clear example is when Google’s image classifier labeled an African American couple as gorillas. This was clearly because the training data did not have enough instances of African Americans, which is due to an unconscious racial bias either on the part of Google, the data aggregator, or society as a whole. These are the sorts of biases that are propogated by machine learning and recommenders.
I think a good example of this is in online dating which almost always employs a recommendation engine. Certain racial demographics such as black women and Asian men are rated as less attractive than other members of the same gender across the board. This is an example of the existing cultural bias that runs a danger of being amplified by the recommendation algorithm.
The first issue is content-based: is it ethical to allow users to filter potential matches by race? Personally, I think the answer should be no: by allowing users to engage in their preconceived racial biases you perpetuate them further which makes you complicit in spreading racial bias.
The second issue comes about when you begin to think about recommendation algorithms. If we are using a user-based collaborative filtering algorithm but the vast majority of users rate black women and Asian men lower than other members of the same gender, then these groups will also be very unlikely to be recommended. Their overall average rating will be low which will make them already unlikely to be recommended, but on top of that, the nearest neighbors will also likely rate these groups poorly resulting in even lower scores for these groups from the neighborhood similarity metric. This will lead to these groups being severely underrepresented in recommendations which again perpetuates a societal bias which I find unacceptable.
In short, if you have a recommendation algorithm, the onus lies with you to ask yourself if your algorithm is perpetuating pre-existing racial or gender biases and take action to prevent this from having a harmful effect.
Source: https://www.wired.co.uk/article/racial-bias-dating-apps