Analysis of PodCruncher App Ratings and Reviews on iTunes
April 16, 2017
PodCruncher Podcast Player
Podcasts changed my life. Literally. Radio in my car doesn’t have AM or FM presets. Long before I paid my last TV bill (some 3 years ago) I forgot how to tune to KERA station. All thanks to the podcast player PodCruncher that was around at least since 2014 (disclosure: creator of PodCruncher is a friend of mine who helped me with researching and assembling iTunes RSS feed URLs):
Figure 1. Screenshot of PodCruncher podcast app in app store.
The problem is Obsessive Coders that makes PodCruncher released its last update back in 2014! With every iOS update and every iPhone release iPhone Apps get a bit more outdated or introduce new bugs mostly because Apple never takes complete responsibility for its changes to be backward compatible or because they themselves introduce bugs into iOS or just break backward compatibility outright. Little by little any App look and feel and performance will degrade with time even if they were almost perfect like PodCruncher was at one point. Don’t get me wrong: PodCruncher is still a superb podcast player that consistently ranks in the top of its group (news apps) but time is the enemy without new release.
Indeed, if you look at how ratings changed since last release (in orange circle above) you find that they moved from 4 1/2 in all versions to 4 stars in current version. Such dynamics clearly indicates deteriorating quality and lower satisfaction. Without new release ratings will slowly but inevitably decline which almost always leads to diminishing market share by the App.
That is why if you ask an App developer what AppStore metric they care the most universally App’s ranking comes up first together wiht App’s rating based on user reviews. To demonstrate how lack of new release affected ratings and user sentiment for PodCruncher I decided to analyze its users’ ratings found in AppStore.
iTunes RSS Feed
But first I’d need to obtain ratings and reviews for PodCruncher. There is iTunes RSS feed that offers information about Apple products and content, e.g. using iTunes RSS Feed Generator
that helps construct custom RSS feed URL to fetch various information about its applications, podcasts, audiobooks, etc. by location and media types:
Figure 2. Screenshot of the iTunes RSS Feed Generator.
After playing and researching a bit more the way to retrieve information about single iOS App that includes its ratings and reviews was discovered. The one for PodCruncher returns xml with 50 most recent reviews like this: 
Retrieving All App Reviews with RSS Feed
The workflow of getting and parsing RSS feed of an AppStore app may look as following:
Using R packages RCurl to download RSS feed XMLs and XML to parse them into R dataframe this is one of possible workflow implementations:itunesPodcruncherUrlXml = "http://itunes.apple.com/us/rss/customerreviews/id=421894356/sortBy=mostRecent/xml"
podcruncherReviews = parseiTunesXML(itunesPodcruncherUrlXml)
lastiTunesURL = findPageLink(itunesPodcruncherUrlXml, "last")
firstiTunesURL = findPageLink(itunesPodcruncherUrlXml, "first")
totalPages = as.integer(str_extract(str_extract(lastiTunesURL, "/page=[0-9]+"), "[0-9]+"))
podcruncherReviews = podcruncherReviews[c(),]
for(page in 1:totalPages) {
pageURL = sub("/page=1", paste0("/page=", page), firstiTunesURL)
podcruncherReviews = rbind(podcruncherReviews, parseiTunesXML(pageURL))
}Preparing for Analysis
The resulting dataframe with application reviews contains following data:
- review id
- review title
- review content
- review rating
- review timestamp
| id | title | rating | date | content |
|---|---|---|---|---|
| 1585440549 | This app is dead | 1 | 2017-04-10 12:26:36 | This once great app is now a ghost town…. |
| 1584881163 | A.Rocker | 1 | 2017-04-09 18:52:57 | (Update) |
| With the rele | ase of 10.3 – th… | |||
| 1582069719 | Horrible app | 1 | 2017-04-06 12:10:41 | Doesn’t let you search for podcasts. You… |
| 1581551401 | Too many glitches | 3 | 2017-04-05 21:47:05 | Although this podcast app looks pretty g… |
| 1580769019 | Loading Error | 1 | 2017-04-04 21:59:38 | Waste of money. So far I’ve been able to… |
| 1580579494 | Compared to iTunes it’s outstandingly great | 5 | 2017-04-04 14:28:26 | They fixed the crashing problem. This i… |
| 1578933913 | AWESOME APP | 5 | 2017-04-02 14:22:21 | I love this app makes life so easy and e… |
| 1576013058 | Better than the other podcast app | 5 | 2017-03-30 06:22:21 | The reviewer below is mistaken. You can … |
| 1573349301 | Massively outdated | 2 | 2017-03-26 19:38:38 | It hasn’t been updated since before iPho… |
| 1572718076 | UPDATE!!!!! | 1 | 2017-03-26 02:11:00 | It’s time to update this neglected app…. |
To improve readability and robustness of analysis let’s derive following features:
- from rating:
- feeling: if rating is 4 or 5 then feeling is POSITIVE, otherwise it’s NEGATIVE
- from reviews:
- nchar: review length in characters
- from review timestamp:
- wday: day of the week as ordered factor
- yearmonth: date truncted to first day of the month
- hour: day hour from 0 to 23
- dayPart: day part for night, morning, day, and evening as ordered factor
podcruncherReviews$feeling = factor(ifelse(podcruncherReviews$rating > 3,
"POSITIVE (4 or 5)", "NEGATIVE (1,2, or 3)"))
podcruncherReviews$rating = factor(podcruncherReviews$rating,
labels = c('*','**','***','****','*****'), ordered = TRUE)
podcruncherReviews$nchar = nchar(podcruncherReviews$content)
podcruncherReviews$wday = wday(podcruncherReviews$date, label = TRUE)
podcruncherReviews$year = factor(year(podcruncherReviews$date))
podcruncherReviews$month = month(podcruncherReviews$date, label = TRUE)
podcruncherReviews$yearmonth = as.Date(floor_date(podcruncherReviews$date, unit="month"))
podcruncherReviews$hour = hour(podcruncherReviews$date)
podcruncherReviews$dayPart = factor(floor(podcruncherReviews$hour/6),
labels = c("Night","Morning","Day","Evening"), ordered = TRUE)Exploratory Analysis
Review Ratings
Figure 6. Overall Reviews by Ratings.
Overall PodCruncher still maintains very favorable review ratings. But serious shift towards negative ratings is discovered when yearly distributions revealed:
Figure 7. Each Year Reviews by Ratings.
Review Timelines
Figure 8. Review Ratings Monthly Timeline in Percents.
Remember that we defined user feeling as an aggregate of review ratings: NEGATIVE for ratings 3 or below and POSITIVE otherwise. This lets us represent review counts instead of percentages to track absolute number of reviews monthly by user feeling:
Figure 9. User Review Feeling Monthly Timeline with Absolute Counts.
Number of Reviews Monthly Trends
In case you didn’t notice the overall trend of number of reviews over time is not stationary, which is expected. But non-stationary time series often hide dominant trends. Simple way to uncover it is to divide seasonal data into separate time series, e.g. time series for each month separately.
Figure 10. User Review Feeling Monthly Timeline with Absolute Counts.
Unfortunately for PodCruncher all months with 20 or more reivews a year demonstrate both decrease in review activity and increas in negative feeling. Finally, we conclude timeline analysis with overall trends for both negative and positive feel side by side starting with January 2015 to exclude strong response to last ast release in August 2014:
Figure 11. Monthly Timeline of Review Counts by Feeling.
Review Lengths
By review length we simply mean a number of characters in each review. This usually indicates user’s involvement or commitment to provide information about the produc and thus longer reviews indicate stronger intent and/or interest.
Figure 12. Review Length Distributions by Rating
The effect when negative reviews are longer with fatter tail is not uncommon. I observed more drastic differences between negative and positive reviews in that respect before, which indicates that AppCenter reviews are more involved by both negative and positive reviewers.
How does user involvement trend in time? As before we look at monthly aggregates, i.e. average review sizes:
Figure 13. Average Review Length Monthly Trends.
Reviews by Submission Time
By Day of Week
We switch our focus from trends and distributions to analysis when reviews are submitted. This includes both days of week and time of day such as night, moring, day or evening.
Figure 14. Reviews Submitted by Day of Week.
Figure 14. Reviews Submitted by Rating and Day of Week.
Clearly even though Saturdays feature the least number of reviews submitted they are the best day of week to ask for one. And Tuesday and Wednesday should be avoided as they feature the largest share of negative reviews (3 stars or less).
By Time of Day
We simply divide 24 hours into 4 day parts:
- Night: from midnight to 6
- Morning: from 6 to noon
- Day: from noon to 18
- Evening: from 18 to midnight
Figure 15. Reviews Submitted by Time of Day.
Figure 16. Reviews Submitted by Rating and Time of Day.
The best time to ask for review is at night on Saturday but some more detailed analysis on other Apps is highly desirable.
Text Analytics
Term Frequency Analysis
Term frequency looks at most frequent terms overall or in certain subsets of reviews. Thus, when considering most frequent term for negative or positive reviews there is no adjustment for how common term across all reviews. Later we will revisit top words using TF-IDF metric to account for how unique each term within its class.
Figure 17. Top Overall 50 Review Words by Term Frequency.
Figure 18. Top 50 Negative and Positive Review Words by Frequency.
TF-IDF Word Analysis
If instead of simple term frequency we use term frequency - inverse document frequency (TF-IDF) then top lists change because TF-IDF reflects term importance to a document (negative or positive review sets) in a collection of all reviews:
Figure 19. Top 50 Negative and Positive Review Words by TF-IDF.