Survey Review of a Non-personalized recommender system: Metacritics

History

Metacritic was launched in January 2001[1] by Marc Doyle, his sister Julie Doyle Roberts, and a classmate from the University of Southern California law school, Jason Dietz, after two years of developing the site. Rotten Tomatoes was already compiling movie reviews, but Doyle, Roberts and Dietz saw an opportunity to cover a broader range of media. They sold Metacritic to CNET in 2005. CNET and Metacritic were later acquired by the CBS Corporation.[2]

Introduction

Metacritic is basically an aggreater of reviews of films, TV shows, music albmums and video games[3]. Metascores performs a survey of a large group of the world’s most respected critics which they collate and summarize their range of opinions. Mathematically, it then performed a weighted averaged giving more weights to critics that are more famous or respected in their respective communities. For instance, a movie will have an assigned score from a multitude of reviews for which a weighted average was applied to obtain a singular number that captures the essence of all the critical opinions. This is call the Metascore[4].

The website also provides an excerpt from each review and hyperlinks its sources. A colo of green, yellow or red summarizeds the critics recommendation. It’s regarded as the foremost online review aggregation site for the video gamming insdustry [5],[6]

Discussion Section I

Metascores range from 0-100, with higher scores indicating better overall reviews. Metascores are given in three colors so that end-users can instantly compare: green scores for favorable reviews, yellow scores for mixed reviews, and red scores for unfavorable reviews. This is probably one of the sustaining reasons for its popularity; ease of end-user usage experience.

The basis of review summarization & scaling of the scores:

If a critic comes with scale from 0-10, its simply multipied by 100 but other less obvious ones are shown below:

Discussion Section II

One of the ease of usage features in Metacritic is their designator signatures; “Must-See movies and”Must-Play" video games

Metacritic designates a movie as “Must-See” when it achieves a Metascore of 81 or higher and has been reviewed by a minimum of 15 publications. “Must-See” movies are highly acclaimed and have been reviewed by a broad cross-section of the best critics. In total, approximately 5% of movies in Metacritic’s database achieve this elite status.

Similarly, Metacritic designates a game as “Must-Play” when it achieves a Metascore of 90 or higher and has been reviewed by a minimum of 15 publications. Like “Must-See” movies, “Must-Play” games are highly acclaimed and have been reviewed by a broad cross-section of the best critics.

Discussion Section III

Metacritic have been used by businesses to predict future sales and as such yield quite a bit of influence to the point that it can sometimes potentially make or break would-be movies or games coming in. Over the years, Metacritics did find themselves with mixed of “reviews”; meaning there were studies that showed their ratings were correlated to sales in certain genres of movies[7]. While a later study in 2015 indicated there were no impact on video sales due to their overall ratings[8].

But no matter how its being sliced and diced in the ratings industry, the general perception on Metacritic is that its generally influential because of its wide references by columnists and commentators[9].

Conclusion

In general, Metacritic is popular due to its simplicity of usage for its end-users. The simple green, yellow and red buttons is easy for “lazy” viewers. The visuals que the endusers to an action or no action and that is very important in today’s fast-paced world. Metacritic make sure that end-users know that their ratings comes from professionals, not casual viewers. This distinction carries weight with their fan base as they know that the ratings are not skewed by the general public due to fleeting popularity; in other words, its not a popularity contest of the moment. These green, yellow or red lights means something and it comes from a professional opinion.

However, Metaritic does have its share of critques, their simplicity of end-user delivery is also under harsh criticisms. Metacritic has been criticized for converting all scoring systems into a single simple quantitative percentage-based scale. For example, an “A” score equates to the value of 100, an “F” the value of zero, and a “B–” the value of 67. Further criticism was directed to the website’s refusal to publicize how it aggregates scores. This apparant lack of transparency led to further suspcisions that their scoring system (weighted average) may have inadvertenly converted higher reviews to lower percentage scores.

Recommendation

My personal recommendation to Metacritics would be to parsed out the simple color buttons a little more. For example, under those color-coded buttons, proved a little more context of the reviews and scoring. In this business of ratings, once the audience think its “unfair” or inaccurate, they will flock to other sites and leave en-masse. Because Metacritics have a first to market advantage against its peers, it should pay attention on how end-users utilze their tools. There’s a general trend now that viewers or gamers like to have a little more color (no pun intended) associated with their ratings before they commit to purchasing the product.

Reference 1: “Metacritic: The History”, Metacritic.com

Reference 2: “Columbia Journalism Review - CJR’s guide to what the major media companies own”. Archived from the original on April 17, 2016.

Reference 3: https://www.metacritic.com/about-metascores

Reference 4: https://en.wikipedia.org/wiki/Metacritic

Reference 5:Leack, Jonathan (September 25, 2015). “OpenCritic’s Gamer-Centric Style Is Everything Metacritic Should Have Been”. Game Revolution. Retrieved September 12, 2019.

Reference 6:Rose, Mike (July 10, 2012). “Metacritic is here to stay, but can we fix it?”. Gamasutra. Retrieved September 12, 2019.

Reference 7: Stuart, Keith (January 17, 2008). “Interview: the science and art of Metacritic”. The Guardian. Retrieved February 15, 2008.

Reference 8: Khan, Imad (December 11, 2015). “Do Metacritic scores affect game sales?”. The Daily Dot. Retrieved September 12, 2019.

Reference 9: Gilbert, Ben (May 9, 2019). “The 10 best Pokémon games of all time, according to critics”. Business Insider. Retrieved September 8, 2019.