1.0 Introduction

The ranking of an application reflects both public preferences of using it and the market position of a product, which will not only affect users’ preferences of whether to install the app, but also affect its financing capacity as well as market penetration. As for low ranking applications, it makes sense to investigate influential factors on the rankings of them because this could help analyze reasons contributing to low rankings, as well as exploring out how to improve and popularize the applications. Thus, this report is aimed at analyzing which factors affect low-ranking applications. Applications ranked below 600 in the Apple’s App store were selected as the sample for conducting the regression analysis.

2.0 Methods and data

2.1 Data Description

Our regression seeks to find factors which are correlated with low app rankings in the App store. Our regression model covers the rankings in November 2019 and we selected apps ranking behind 600 in overall rankings. Below is the basic form of our fitted model:

\[ Y_i=\beta_1sub-ranking_i + \beta_2 X_i + \epsilon_i\] where \(Y_i\) equals to the rank of an app in App store in November of 2019. \(sub-ranking_i\) refers to the rank of an app in its sub-category rankings. For example, the “Facebook” app has an overall ranking of all apps in the Apple store and an additional ranking in the Social category. For both variables, top-ranking apps would have smaller values. \(X_i\) contains a set of control variables, including dummies that we will be using for further exploration of our initial model. All data are collected from Kaggle, with more details here.

2.2 Methods

The real algorithm of rankings is unknown. Hence our goal is to seek key factors which would have a significant impact on low rankings. We initially make an OLS regression of Rankings on Sub-ranking to check for the significance of its coefficient. However, we appreciate that we may overestimate the coefficient of Sub-ranking, causing an upward bias in our model. To mitigate our bias problem, we will begin introducing our control variables price and comment-rating.price refers to the payment for downloading the app, and comment-rating means the rating provided by user’s comments. Moreover, we intend to generate interaction variables consisting of a dummy feed and one other control variable to extend our model and make in-depth analysis. feed has three categories: ‘free’ (no payment for downloading and in-app charges), ‘paid’ (only one payment for downloading) and ‘grossing’ (sum payment of downloading and in-app charges).

3.0 Result of analysis

3.1 Confounding factors

To begin with, we make a scatter plot of ranking and sub-ranking:

The graph above implies a positive correlation. Higher sub-ranking is positively related with higher overall ranking.

We could make a regression to test our expectation:

## 
## Call:
## lm(formula = ranking ~ sub_ranking, data = data)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -305.28 -147.12  -20.64  139.65  410.42 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 748.4266    10.7405   69.68   <2e-16 ***
## sub_ranking   1.4797     0.1373   10.78   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 171.5 on 1063 degrees of freedom
## Multiple R-squared:  0.0985, Adjusted R-squared:  0.09765 
## F-statistic: 116.1 on 1 and 1063 DF,  p-value: < 2.2e-16

The regression result shows a highly significant and positive relationship between ranking and sub-ranking. It suggests that whenever an app moves up by one position in its industry ranking, it will correspondingly react by moving 1.48 places ahead on average in the whole app store ranking. However, unbiasedness is a pivotal characteristic that we need to ensure before we eventually draw a conclusion.

Thus, we test the independence between sub-ranking and the error term of the model:

## [1] 0.3328362

The above result showed that sub_ranking is correlated with the error terms of the model, meaning that the model might have endogeneity problem or omitted variables.

On realizing that price and the comment rating of an app may affect both minor and overall ranking simultaneously, we introduced price and comment-rating in succession, with a hope to mitigate the endogeneity concern caused by any confounding factors:

## 
## Call:
## lm(formula = ranking ~ sub_ranking + price, data = data)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -325.26 -147.97   -9.41  126.29  392.07 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 766.4593    10.3187   74.28   <2e-16 ***
## sub_ranking   1.4981     0.1302   11.50   <2e-16 ***
## price        -5.8889     0.5380  -10.95   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 162.7 on 1062 degrees of freedom
## Multiple R-squared:  0.1899, Adjusted R-squared:  0.1884 
## F-statistic: 124.5 on 2 and 1062 DF,  p-value: < 2.2e-16

It is noteworthy in this regression analysis that a larger number of rankings reflects a worse performance. By looking at the first result, we witness a significantly negative relationship between price and an application’s ranking: an app’s position in the whole app store market will favorably move up by around 5.89 places for every incremental increase in its price. Meanwhile, the estimated coefficient of sub-ranking remains significant but increases from 1.48 to 1.50, suggesting that the previous estimate of sub-ranking was downward biased. This indicates that the variable, price, is such a confounding factor positively correlating with sub-ranking (variable of interest) whilst negatively correlating with ranking (dependent variable). On one hand within a certain category, the users will be less willing to pay for the higher priced application, thus switching to other substitutes. The declining demand leads to a lower market position within the industry accordingly, reflected by the descending sub-ranking. On the other hand, though unexpected, the negative relationship between price and ranking actually makes sense. It could be argued that most higher-priced applications are those who operate in popular industries with a large installed user base, because such a pricing strategy will be hard to sustain without a solid market demand otherwise. When competing with this kind of app, the ones in the industry with a sluggish growth rate can be disadvantaged, even if they are market leaders who charge nothing. Consequently, the ranking of those higher priced applications tend to gain more benefits from the industry popularity, thus winning a higher ranking result (a smaller value in the variable, ranking).

Therefore, we introduce comment-rating as a control variable, and make the regression:

## 
## Call:
## lm(formula = ranking ~ sub_ranking + price + comment_rating, 
##     data = data)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -335.33 -145.18   -8.88  121.35  406.76 
## 
## Coefficients:
##                Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     675.695     31.168  21.679  < 2e-16 ***
## sub_ranking       1.523      0.130  11.721  < 2e-16 ***
## price            -5.857      0.536 -10.928  < 2e-16 ***
## comment_rating   20.451      6.630   3.085  0.00209 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 162 on 1061 degrees of freedom
## Multiple R-squared:  0.1971, Adjusted R-squared:  0.1948 
## F-statistic: 86.81 on 3 and 1061 DF,  p-value: < 2.2e-16

After comment-rating is added, the result shows that rating is significantly positively related to ranking: for every extra score in comment ratings, the application’s overall ranking will drop by around 20 places. The estimated coefficient of sub-ranking remains significant but is now much higher, suggesting a downward bias in the previous result. It is normal that within a category, an application’s sub ranking will be improved with an increase in users’ scoring and an enhanced satisfaction. However, the relationship between an application’s rating and its overall ranking is another story. We infer that most apps with a high commentary score are those who are serving a niche market in which users’ demand is extremely special. The applications operating in this kind of environment tend to gain higher marks in comment ratings because they act as problem solvers of these specific users’ demand in most cases. However, obtaining an advanced ranking performance will be more challenging for them since their market sizes are usually small and restricted. In contrast, popular applications such as Facebook are more likely to obtain lower ratings in comment, since this kind of company mainly serves a broader range of market, which indicates that not everyone’s expectation could be met. Therefore, they tend to absorb a larger number of negative comments, in spite of the high overall rank in the app store.

3.2 Regression with Dummy variable

The dummy variables fall into 3 different categories:

  • Free (no charge for download and in-app charges)

  • Paid (charging for download and no in-app charges)

  • Grossing (charging for download and in-app charges)

The first regression shows the results below:

## 
## Call:
## lm(formula = ranking ~ sub_ranking + price + comment_rating + 
##     feed, data = data)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -300.12 -104.57    0.48  100.83  357.89 
## 
## Coefficients:
##                 Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     739.1889    28.1043  26.302   <2e-16 ***
## sub_ranking       1.4086     0.1167  12.071   <2e-16 ***
## price            -0.1217     0.6432  -0.189   0.8499    
## comment_rating   15.3107     5.9453   2.575   0.0102 *  
## feedgrossing   -257.4253    25.5051 -10.093   <2e-16 ***
## feedpaid       -202.3025    14.5521 -13.902   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 144.4 on 1059 degrees of freedom
## Multiple R-squared:  0.3635, Adjusted R-squared:  0.3605 
## F-statistic:   121 on 5 and 1059 DF,  p-value: < 2.2e-16

When looking at the results of this regression, we find that when adding feed, price becomes insignificant. Its p-value is very high and thus not statistically significant at any level. Indeed, in the results above, price has a negative effect on rankings, which makes sense but it is reduced compared to previous regression (price effect was approximately -5.9). This shows that the dummy variables includes the effect of price, meaning that price is a subset of feed. Thus, we decide to exclude price , otherwise, multicollinearity would be the case and thus lower the significance further, and we find :

Thus, we decide to exclude price and we find : 

## 
## Call:
## lm(formula = ranking ~ sub_ranking + comment_rating + feed, data = data)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -300.08 -103.13    1.08  100.86  357.78 
## 
## Coefficients:
##                 Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     739.4470    28.0584   26.35   <2e-16 ***
## sub_ranking       1.4087     0.1166   12.08   <2e-16 ***
## comment_rating   15.2509     5.9342    2.57   0.0103 *  
## feedgrossing   -257.4531    25.4931  -10.10   <2e-16 ***
## feedpaid       -204.1432    10.8185  -18.87   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 144.3 on 1060 degrees of freedom
## Multiple R-squared:  0.3635, Adjusted R-squared:  0.3611 
## F-statistic: 151.3 on 4 and 1060 DF,  p-value: < 2.2e-16

Here, the dummy variables become statistically significant at the 5% level as their p-values are extremely small. The results show that when an app charges for download and include in-app charges, the ranking will be higher (739-257=482) than if the app is free to download and include in-app charges (739-204=535). As a result, if an app is free to download and does not include in-app charges, the app will be more likely to have a low ranking.

An explanation might be that when an app is free and does not include extra charges, more people will download it and therefore more people will have to rank it and that leaves more room for unsatisfied users. On the other hand, when an app is charging you for download and includes more in-app charges, if a user decides to purchase it, it is because he/she really needs it, or he/she really knows that the app is really worth it. A grossing app generally indicates quality content and therefore, users are more inclined to be satisfied and thus the app should have a higher ranking.

3.3 Regression with interaction variables

In first situation, we made an interaction with feed and comment-rating:

## 
## Call:
## lm(formula = ranking ~ sub_ranking + comment_rating + feed + 
##     feed * comment_rating, data = data)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -303.21 -100.18    2.36  101.47  367.48 
## 
## Coefficients:
##                              Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                  717.8234    31.0446  23.122  < 2e-16 ***
## sub_ranking                    1.4054     0.1167  12.045  < 2e-16 ***
## comment_rating                20.2449     6.6828   3.029  0.00251 ** 
## feedgrossing                -113.2711   261.3397  -0.433  0.66479    
## feedpaid                    -106.1657    63.7851  -1.664  0.09632 .  
## comment_rating:feedgrossing  -31.5733    56.4929  -0.559  0.57636    
## comment_rating:feedpaid      -23.0259    14.7800  -1.558  0.11955    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 144.3 on 1058 degrees of freedom
## Multiple R-squared:  0.3651, Adjusted R-squared:  0.3615 
## F-statistic: 101.4 on 6 and 1058 DF,  p-value: < 2.2e-16

When regressing the ranks of poorly performing applications on their different charging methods and comment ratings, we tend to be interested in how the importance of comment ratings vary between applications with different charges. Omitting this factor may potentially cause the endogeneity problem with some bias in the regression result. Therefore, we make the variable of comment rating interact with both feedpaid and feedgrossing.

From the regression result presented above, apart from the downward bias, we can also find that the importance of comment rating differs between different types of applications, after we include the two binary-continuous interaction terms. The coefficient of comment rating has increased from 15.25 to 20.25, indicating that an additional score point in comments would lead to 20.25 rankings worse for those totally free applications on average, significant at 5% level. However, a conspicuous finding is that, for those who charge installation fees only, comment rating has a negative correlation with the dependent variable, though insignificant. Whenever a comment rating increases by an extra score, the application of this kind will get around 2.78 rankings upward (20.25-23.03) than before. Likewise, for those who charge for both installation and usage, an additional improvement in commentary score is expected to lead to 11.32 rankings higher (20.25-31.57) than before, a larger improvement. Therefore, it could be concluded that the improvement in ranking for a given change in comment rating is the largest for the more expensive app groups.

This outcome could be attributed to the fact that the main commentators of the extra fee charging apps are those who have already experienced and paid for this application. Payment relating to installation will trigger a higher user expectation. They are more likely to write comments in an objective way to share with others about their using experiences once realizing that downloading this app is not worthy at all. As such, the comment rating of this kind of application can truly reflect the product quality, positively correlating with the overall rankings. However, the users of free apps are more likely to be those who do not take the comment rating very seriously because everyone can freely download the app. This means that even when it is performing very poorly, many of them still prefer to simply click onto the five-star rating to save time without giving real feedback in detail. After all, they will not lose anything even if the application does not meet their expectations.

In second case, we made an interaction with feed and sub-ranking:

## 
## Call:
## lm(formula = ranking ~ sub_ranking + comment_rating + feed + 
##     feed * sub_ranking, data = data)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -302.81  -87.80   -0.64   84.32  385.46 
## 
## Coefficients:
##                           Estimate Std. Error t value Pr(>|t|)    
## (Intercept)               700.6126    27.9263  25.088  < 2e-16 ***
## sub_ranking                 1.9810     0.1386  14.294  < 2e-16 ***
## comment_rating             15.1282     5.7986   2.609 0.009210 ** 
## feedgrossing             -153.4159    42.3391  -3.624 0.000304 ***
## feedpaid                  -83.0586    20.2834  -4.095 4.55e-05 ***
## sub_ranking:feedgrossing   -2.1157     0.8013  -2.640 0.008408 ** 
## sub_ranking:feedpaid       -1.7378     0.2485  -6.994 4.72e-12 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 141 on 1058 degrees of freedom
## Multiple R-squared:  0.3938, Adjusted R-squared:  0.3903 
## F-statistic: 114.5 on 6 and 1058 DF,  p-value: < 2.2e-16

In this case, we can find that there is a downward bias in sub ranking and that the importance of sub ranking varies between different types of applications. For those free apps, each additional improvement in sub ranking is expected to enhance the overall ranking by 2.12 places (the coefficient of sub_ranking), which is highly significant. However, the improvement in an app’s overall ranking for a given change in its sub-ranking will be larger for free apps than the ones requiring somewhat payments. This was probably caused by the fact that potential users are more likely to be attracted by those free applications because they can have a try without any investments once witnessing an improvement in the app’s sub ranking.

4.0 Conclusion

With the aim of finding influential factors on low ranking applications in apple store, we carried out an OLS regression of rankings on sub-rankings and further introduce price and comment ratings as control variables to mitigate endogeneity problems. Also, a dummy variable was introduced and regression on interaction terms was performed. It can be witnessed that correlation between the variables and results of analyses are out of our initial expectation. By regression analysis, ranking is positively associated with sub rankings and comment ratings, while negatively related to price. As was pointed out above, higher ranking numbers mean low rankings, it can be concluded higher rankings are associated with higher price, higher sub rankings and lower ratings.

Correlation between price and rankings are attributed to industry popularity (explained in 3.1) that higher-priced applications in popular industries tend to have solid and larger market demand, which maintain their pricing advantages and result in higher rankings. Also, by conducting a dummy variable analysis, we found that grossing apps rank higher than free apps because grossing apps are more likely to attract users who recognize them and have a higher willingness to pay. Possible reasons for the negative relationship between rankings and ratings can be summarized that ratings of applications are associated with comments from consumers. And the larger consumer group of an application, the higher possibility of achieving low ratings, which can explain why high-ranking applications have low ratings. For the sub ranking variable, their positive relationship is sensible because applications which are outstanding in their subcategories also are more likely to perform well in a larger market.

Based on our analysis in 3.1, a higher-priced application tends to have a larger installed user base. Thus, a suggestion for developers of low-ranking applications could be not to focus on lowering price, however even making apps free in order to penetrate the market. Additionally, developers should not necessarily rely solely on ratings as a measure, as our regression results confirm that higher-ranking apps tend not to have higher ratings.