What Makes an Anime Popular? Statistical Analysis Using MAL Data

Results

1. Score (which is user generated) has a strong relationship with the popularity of a show. There appears to be a cubic relationship between these variables.

2. The number of producers on the committee board of a given anime have a quadratic relationship with members (concave). The same can be said about the age of an anime.

3. Rating also has an impact. Anime rated as Children and All Ages are less popular than anime with a PG 13 rating. On the other hand, shows rated as Mild Nudity and R +17 are more popular than PG 13 rated anime.

4. Shows animated by studios: Kyoto Animation, Bones or A-1 Pictures are more popular than shows animated by other studios.

5. Anime genres are very important as well: Harem, Action, Romance, School, Psychological, Supernatural, Shounen, Horror, and Fantasy have a positive impact on popularity. On the other hand, Historical, Music, Space, Mecha, Sports, Demons and Police have a negative impact on popularity.

6. Light Novel adaptations are more popular than original anime, and anime with a different source material (Manga, Visual Novel, Books, etc).

7. Producers Sanrio and Shogakukan have a negative effect on popularity. On the other hand, Shueshia, Toho Animation and Pony Canyon have a positive impact on popularity.

8. Shows with more than 26 episodes are less popular than shows with fewer episodes.

Intro

A few months ago I stumbled upon a tutorial about how to do web scraping in R. So i decided to try on My Anime List. Which is a huge anime and manga database. Some data on the website is user-generated, like the scores and reviews given to an anime. My goal was to discover what makes an anime popular. In terms of methodology I used linear regressions. With that in mind I selected a few features: Genres, Anime Studio, Producers, Rating, how old an anime is, Source Material, Number of producers (as a proxy of budget) and number of episodes.

To measure “popularity” I used the variable “members” which is the sum of users that are currently watching a show, completed a show, have a show on-hold, dropped it or are planning to watch a show in the future.

Selecting Observations

First, I decided to only analyze tv shows, not movies, onas or ovas. Second, I picked shows whose episodes ran longer than 18 minutes. Third, I only considered shows from 2005 to the spring season of 2020. My AnimeList was launched at the end of 2004, so it makes sense to start in 2005. I scraped MAL at the beginning of the 2020 summer season so that’s why the newest shows in the database are from the spring 2020 season.

For feature selection I used my knowledge about the anime industry and I ran a lot of Anova tests to select variables. (I also ran a lasso regression, and picked the variables that were not dropped as a starting point). I had to drop the genre Ecchi because it created multicollinearity with the ratings. Specifically with “Mild-Nudity”.

Methodology

After a lot of trials and errors, this is the model I came out with:

log(Members) = 𝛽0 + 𝛽1 Score + 𝛽2 Score ∧ 2 + 𝛽3 Score ∧ 3 + 𝛽4 Epi + 𝛽5 Producers + 𝛽6 Producers ∧ 2 + 𝛽7 Age+ 𝛽8 Age ∧ 2 + 𝛽9 Children + 𝛽10 All Ages + 𝛽11 R 17 + 𝛽12 Mild Nudity + 𝛽13 Kyoto Animation + 𝛽14 A 1 Pictures + 𝛽15 Bones + 𝛽16 Shin-Ei Animation + 𝛽17 Historical + 𝛽18 Harem + 𝛽19 Music + 𝛽20 Action + 𝛽21 Romance + 𝛽22 School + 𝛽23 Space + 𝛽24 Psychological + 𝛽25 Supernatural+𝛽26 Horror + 𝛽27 Fantasy + 𝛽28 Mecha + 𝛽29 Sports + 𝛽30 Game + 𝛽31 Demons + 𝛽32 Police + 𝛽33 Shounen + 𝛽34 Light.novel + 𝛽35 Sanrio + 𝛽36 Shueisha + 𝛽37 Toho Animation + 𝛽38 TBS + 𝛽39 Pony.Canyon + 𝛽40 Shogakukan + 𝜀

Now, I’ll go over the coefficients. It makes sense to start with the outcome we want to understand: the members variable.

The data is right-skewed; there are more “not that popular” shows than very popular ones. The median was 89418 and the most popular show had 2116123 members. When modeling this outcome, a strong argument can be made that it should be log-transformed.

Here’s the transformed data:

While not perfect (now it looks a bit left-skewed), our model results will probably be better.

Score is a user-generated variable that indicates the evaluation of MAL users towards a particular anime from 1 to 10. The mean and median are 7.2.

Epi is a dummy variable which takes the value 1 for shows with more than 26 episodes and 0 for shows with 26 or fewer episodes. Producers is the number of producers on the committee board of a given anime. Age is how old an anime is (in years). For example, 0 means 2020 and 15 means 2005. Here’s the number of shows animated by year.

Children, All Ages, R 17+ y Mild Nudity are part of the categorical variable rating. The reference level is the PG-13 category. A-1 Pictures, Bones, Kyoto Animation and Shin Ei Animation are dummy variables for those studios. Also, I made dummy variables for the studios:

Toei Animation, Sunrise, J.C Staff, Madhouse, TMS Entertainment, Production.I.G, Studio.Deen, Studio.Pierrot, OLM, Nippon.Animation, DLE, Tatsunoko.Production, Gonzo, Xebec, AIC, Shaft, Brain.Base, ufotable, Gainax and feel.

They were not considered in the final model, because the anova tests that I carried out reported that these studios did not contribute statistically to the model.

The coefficients from 𝛽17 to 𝛽33 are dummy variables for anime genres. It is necessary to mention that as with anime studios, I created variables for many genres that were not considered in the final model because they did not contribute statistically to our model: Among the genres dropped are:

Comedy, Adventure, Slice of Life, Drama, Sci Fi, Mystery, Seinen, Martial Arts, Samurai, Shoujo, Shoujo Ai, Shounen Ai, Josei, Vampire, Military, Demons, Magic and Thriller

Light Novel is a variable which takes the value 1, when the source material of a show is a light novel and 0 for anime with a different source. Finally, from 𝛽35 to 𝛽40 correspond to dummy variables for different anime production companies. As with genres and studios, there were many producers who were initially considered but were dropped for the same reasons. Among them were:

Kodansha, Bandai, Square Enix, Kadokawa, Aniplex, NHK, TV Tokyo, TV Asahi, Fuji TV, Gakken, Dentsu, Media Factory, Lantis, KlockWorx, Frontier Works, flying DOG, Nippon Television Network, Showgate, Starchild Records, Sony Music y VAP.

There were some influential observations so I decided to eliminate them one by one until there were no more shows with high leverage. The anime involved in the process were: “Naikaku Kenryoku Hanzai Kyousei Torishimarikan Zaizen Joutarou”, “Death Note”, ”Kaitou Joker 4th Season”, “Active Raid: Kidou Kyoushuushitsu Dai Hachi Gakari” and “Tsurune: Kazemai Koukou Kyuudoubu”.

After doing this it is possible to point out that the most important change is that the Shin Ei animation coefficient is no longer statistically significant. In addition, there are significant increases in the impact of certain variables such as “Police” and “Kyoto Animation”

I ran a Shapiro test to assess normality. And there wasn’t normality. So I decided to winsorize the data. There were 24 anime involved in the process, all of them had less than 1700 members. Lastly, I decided to run the model with robust standard errors because I detected heteroscedasticity.

Understanding Anime Popularity

	Dependent Variable: Log (Members)
	Model 1	Model 2	Model 3
Score	-13.849***	-14.570***	-14.570***
	(3.396)	(3.374)	(3.555)
Score^2	1.970***	2.074***	2.074***
	(0.482)	(0.479)	(0.503)
Score^3	-0.086***	-0.091***	-0.091***
	(0.023)	(0.022)	(0.024)
Episode	-0.282***	-0.328***	-0.328***
	(0.085)	(0.085)	(0.101)
Producers	0.210***	0.210***	0.210***
	(0.026)	(0.026)	(0.030)
Producers^2	-0.013***	-0.013***	-0.013***
	(0.002)	(0.002)	(0.003)
Anime Age	0.113***	0.111***	0.111***
	(0.019)	(0.019)	(0.019)
Anime Age^2	-0.009***	-0.009***	-0.009***
	(0.001)	(0.001)	(0.001)
Children	-0.940***	-0.878***	-0.878***
	(0.108)	(0.109)	(0.125)
All Ages	-0.993***	-0.974***	-0.974***
	(0.116)	(0.115)	(0.143)
R 17+	0.393***	0.402***	0.402***
	(0.060)	(0.060)	(0.059)
Mild Nudity	0.369***	0.369***	0.369***
	(0.079)	(0.078)	(0.080)
Kyoto Animation	0.792***	0.864***	0.864***
	(0.175)	(0.177)	(0.129)
A1 Pictures	0.432***	0.435***	0.435***
	(0.093)	(0.092)	(0.096)
Bones	0.276**	0.279**	0.279**
	(0.128)	(0.127)	(0.122)
Shin Ei Animation	-0.559**	-0.289	-0.289
	(0.228)	(0.243)	(0.258)
Historical	-0.594***	-0.592***	-0.592***
	(0.080)	(0.080)	(0.083)
Harem	0.442***	0.447***	0.447***
	(0.083)	(0.083)	(0.074)
Music	-0.592***	-0.599***	-0.599***
	(0.094)	(0.093)	(0.095)
Action	0.372***	0.382***	0.382***
	(0.047)	(0.047)	(0.048)
Romance	0.363***	0.360***	0.360***
	(0.052)	(0.051)	(0.051)
School	0.316***	0.315***	0.315***
	(0.050)	(0.049)	(0.047)
Space	-0.487***	-0.479***	-0.479***
	(0.140)	(0.139)	(0.132)
Psychological	0.390***	0.364***	0.364***
	(0.101)	(0.101)	(0.094)
Supernatural	0.249***	0.235***	0.235***
	(0.054)	(0.054)	(0.052)
Horror	0.314***	0.322***	0.322***
	(0.120)	(0.119)	(0.116)
Fantasy	0.234***	0.226***	0.226***
	(0.050)	(0.049)	(0.049)
Mecha	-0.224***	-0.250***	-0.250***
	(0.086)	(0.085)	(0.087)
Sports	-0.252***	-0.239***	-0.239***
	(0.086)	(0.086)	(0.090)
Game	-0.206**	-0.208**	-0.208*
	(0.097)	(0.096)	(0.116)
Demons	-0.307***	-0.299***	-0.299***
	(0.093)	(0.092)	(0.099)
Police	-0.393**	-0.453**	-0.453**
	(0.185)	(0.199)	(0.185)
Shounen	0.117**	0.122**	0.122**
	(0.055)	(0.055)	(0.061)
Light Novel	0.416***	0.420***	0.420***
	(0.059)	(0.059)	(0.055)
Sanrio	-0.813***	-0.819***	-0.819***
	(0.265)	(0.263)	(0.266)
Shueisha	0.224**	0.207**	0.207**
	(0.088)	(0.088)	(0.081)
Toho Animation	0.280**	0.272**	0.272**
	(0.125)	(0.124)	(0.111)
TBS	0.175**	0.166**	0.166*
	(0.084)	(0.084)	(0.085)
Pony Canyon	0.152**	0.158**	0.158**
	(0.075)	(0.074)	(0.078)
Shogakukan	-0.362***	-0.347***	-0.347**
	(0.114)	(0.113)	(0.142)
Constant	39.778***	41.433***	41.433***
	(7.918)	(7.867)	(8.322)
Observations	1,718	1,712
R2	0.710	0.712
Adjusted R2	0.703	0.705
Residual Std. Error	0.797 (df = 1677)	0.791 (df = 1671)
F Statistic	102.633*** (df = 40; 1677)	103.043*** (df = 40; 1671)
Note:	p<0.1; p<0.05; p<0.01

Model 1 is the first model I came up with. Model 2 shows the results after I dropped the high leverage observations and fixed the normality issues. Model 3 is Model 2 with robust standard errors. All 3 models are present in the table so you can see how the results change. The most important model is number 3 and therefore, when I refer to the results from now on, I will be talking about the coefficients of this model. Finally, it is necessary to point out that when I interpret a coefficient in the way: “this or that variable has an effect of X% on popularity”. These effects are always on average and keeping the rest of the variables constant. I find it a bit repetitive to have to write it every time I interpret a coefficient, so I prefer to make it clear here.

Findings

Score

It is possible to say that, in general terms, shows with a high score tend to be more popular. That being said, the data show us that there is a cubic relationship between the variables members and score. As you can see on the graph. The curve (in orange) goes down, back up, then back down again. Which means that for shows with very low scores one more point in the score has a negative effect on popularity. Then, for shows over 5.5 an increase in the score has a positive effect on popularity. Finally, for shows with a high score (over 8.4 or 8.5) the marginal effect of a point seems to flatten out.

Age

There is a quadratic relationship between the age of an anime and its popularity. Specifically, an additional year has a positive effect until 2014 (6 years), which makes sense, since shows need time to gain members. After 6 years the effect of an additional year becomes negative, which also makes sense, since intuition would tell us that old shows are less popular.

Producers

When it comes to the number of producers, the relationship with the variable members is also quadratic. Because the coefficient on producers is positive and the coefficient on producers ^2 is negative, this equation literally implies that, at low numbers of “producers”, an additional producer has a positive effect on log(members). At 8 producers, the effect becomes negative. The quadratic shape means that the semi-elasticity of producers with respect to members is decreasing as the number of producers increases.

Number of Episodes

While I was exploring the database looking for a relationship between the number of episodes and the member variable. I noticed that shows with more than 26 episodes seemed to be less popular. Therefore, I decided to create a dummy variable, which takes the value 1, for shows with more than 26 episodes and 0 for shows with 26 or fewer episodes. The results indicate that shows with more than 26 episodes are on average 33% less popular than shows with fewer episodes.

Rating

Anime under the categories “Children” and “All Ages” are on average 85% and 92% less popular than shows listed as PG-13. This perhaps has to do with the variable “members” being generated by users. Children, who are the target audience for these shows, probably don’t have MAL accounts, hence why they are less popular. In relation to the above, anime under the categories “Mild Nudity” and “R +17” are 40% and 36% more popular than those in the PG-13 category.

Anime Studios

There are 3 studios that have a positive impact on the popularity of a show. The first one is Kyoto Animation, on average, the shows animated by this studio are 86% more popular than the rest of the anime (without considering the anime produced by Bones, A1 Pictures and Shin Ei Animation, it is necessary to clarify this, since the anime studios in this model are a categorical variable and not a mere dummy variable). On the other hand, Bones has a positive effect of 28% while A.1 Pictures has one of 44%. Shin Ei Animation doesn’t seem to have a statistically significant effect.

These results make sense to me. Kyoto Animation is probably one of the best studios in terms of animation quality. Despite not animating many shows, it must be considered that during the past decade each anime they made was a commercial success, (Air, Kanon, Clannad, Lucky Star, Suzumiya Haruhi, K-On!).

Top 10 Most Popular Anime Made by Kyoto Animation .

Anime	Members	Year
Clannad	965840	2007
Clannad: After Story	806320	2008
Chuunibyou demo Koi ga Shitai!	791911	2012
Hyouka	778324	2012
Violet Evergarden	776039	2018
Kyoukai no Kanata	725282	2013
Suzumiya Haruhi no Yuuutsu	681322	2006
K-On!	671464	2009
Kobayashi-san Chi no Maid Dragon	619021	2017
Nichijou	557696	2011

On the other hand, Bones is perhaps one of the most important anime studios ever. Many of their “old” shows like Eureka Seven, Darker than Black, Ouran Koukou Host Club or Soul Eater, have many members. Also, as you can see in the table, they are animating Boku No Hero Academia, which is one of the most popular anime shows at the moment.

Top 10 Most Popular Anime Made by Bones.

Anime	Members	Year
Fullmetal Alchemist: Brotherhood	1844434	2009
Boku no Hero Academia	1465688	2016
Noragami	1270201	2014
Boku no Hero Academia 2nd Season	1236998	2017
Soul Eater	1067021	2008
Boku no Hero Academia 3rd Season	986716	2018
Mob Psycho 100	965277	2016
Noragami Aragoto	858029	2015
Ouran Koukou Host Club	707578	2006
Darker than Black: Kuro no Keiyakusha	696411	2007

Finally, A-1 Pictures is the one that surprises me the least. In the past decade they became the powerhouse of anime production (Sword Art Online, Ao no Exorcist, Shigatsu wa Kimi no Uso, Fairy Tail, Nanatsu no Taizai). Between 2014 and 2016 they produced 10 or more television shows per year. Which is a very high figure comparatively. The only studio that has done something similar is JC Staff in 2012 and 2017.

Top 10 Most Popular Anime Made by A-1 Pictures

Anime	Members	Year
Sword Art Online	1864435	2012
Ao no Exorcist	1210116	2011
Sword Art Online II	1202263	2014
Shigatsu wa Kimi no Uso	1146702	2014
Fairy Tail	1137434	2009
Nanatsu no Taizai	1106646	2014
Boku dake ga Inai Machi	1087139	2016
Ano Hi Mita Hana no Namae wo Bokutachi wa Mada Shiranai.	963519	2011
Kuroshitsuji	755402	2008
Darling in the FranXX	709474	2018

As you can see on the boxplot behind, It is clear that these 3 studios animate shows that are more popular than the shows animated by the rest of the anime studios.

Genres

The genre of an anime is perhaps one of the most important elements in determining the popularity of a show. Surprisingly, the genre that has the highest positive effect on the member variable is Harem, on average a show of this type is 45% more popular than a show that is not listed as Harem. Other genres that have a positive effect on popularity are: Action, Romance, School, Psychological, Supernatural, Horror, Shounen and Fantasy.

The genres with the biggest negative effect are “Music” and “Historical”. A show with these genres is, on average, 59% less popular than a show that’s not listed as “Music” or “Historical”. The other genres that have a negative effect are Space, Mecha, Sports, Demons and Police. In the graph behind you can look at the popularity density distribution of some of the most influential genres. I picked the two genres with the highest negative and positive effect on popularity.

Source Material

Of all the sources, the only one that has a significant effect is light novel. If you don’t know what a light novel is read this. Light novels adaptations are on average 42% more popular than anime whose source material comes from another medium. I think this is partly due to the rise in popularity of Isekai in the last decade, as Isekai anime are predominantly adaptations of light novels. If you don’t know what an isekai anime is read this

Most Popular Light Novel Adaptations

Anime	Members	Year
Sword Art Online	1864435	2012
No Game No Life	1459578	2014
Toradora!	1287796	2008
Sword Art Online II	1202263	2014
Re:Zero kara Hajimeru Isekai Seikatsu	1177143	2016
Another	1049527	2012
Kono Subarashii Sekai ni Shukufuku wo!	1017264	2016
Durarara!!	983181	2010
Fate/Zero	938115	2011
Bakemonogatari	921784	2009

Producers

There are two producers that have a negative effect on popularity. The first one is Sanrio. Shows Sanrio is part of the production committee are on average 82% less popular than shows Sanrio is not part of the committee. From wikipedia: “Sanrio is a Japanese company that designs, licenses and produces products focusing on the kawaii (‘cute’) segment of Japanese popular culture”. The other producer that has a negative effect is the publisher Shogakukan, the shows in which Shogakukan is involved are on average 35% less popular than the shows in which this publisher is not on the production committee.

The producers Shueisha, Toho Animation, and Pony Canyon have a positive effect on popularity. Shows in which Shūeisha is involved are on average 21% more popular than the anime in which this publisher is not involved. This makes sense since this publisher publishes the Weekly Shōnen Jump magazine.

Top 10 Most Popular Anime Produced by Shueisha

Anime	Members	Year
Death Note	2116123	2006
One Punch Man	1702381	2015
Tokyo Ghoul	1526793	2014
Boku no Hero Academia	1465688	2016
Hunter x Hunter (2011)	1264941	2011
Boku no Hero Academia 2nd Season	1236998	2017
Naruto: Shippuuden	1222099	2007
Ao no Exorcist	1210116	2011
Boku no Hero Academia 3rd Season	986716	2018
Ansatsu Kyoushitsu	970173	2015

Here’s a boxplot with the most important manga/light novel publishers (Kadokawa,Kodansha,Shogakukan and Shueisha). As can be seen, Shueisha has the highest log(members) median. On the other hand Shogakukan has the lowest one.

The other two producers that have a positive effect are 1) Toho Animation, which is an animation studio and film distributor. 2) Pony Canyon, which according to wikipedia is: “a Japanese company, which publishes music, DVD and VHS videos, movies, and video games”. Toho Animation has a positive effect of 27% while Pony Canyon has an effect of 16%. Finally, the television company TBS does not seem to have a statistically significant effect on the popularity of an anime.

Final Thoughts

Our adjusted r squared tells us that the variables included in the model are explaining about 70% of the variability of the members variable. However there are certain things to keep in mind when analyzing these results. First, I don’t know the geographic distribution of MAL users. I suspect that they are mostly from English-speaking countries. Therefore, one must be careful when making generalizations regarding these results.

Second, I couldn’t find a way to control the effect prequels have on sequels. If the first season of a show is popular, this will make the sequels to that show very popular as well. With the data I have there is no way to control this effect. Third, I cannot control the effect that certain streaming platforms such as Netflix, Amazon or Crunchyroll may have. Fourth, the data may show numbers that are not very favorable towards shows whose target audience is children since children probably do not have MAL accounts.

Despite all of the above, I think this first approach sheds some light on what elements contribute to the popularity of anime. In the future I would like to create machine learning models that predict the popularity of anime.