Call of Duty: Gulf War - analysis on player churn

library(tidyverse)
library(gganimate) #For animated line graph
library(gifski) #For animated line graph
library(ggplot2)
library(dplyr)
library(magick) #For animated line graph
library(ggiraph) #For interactive scatterplot
game_churn <- read_csv("game_churn.csv")

Due to the success of our new mobile game “Call of Duty: Gulf War”, we wanted to run an analysis on what makes people stay, and what makes people leave.

The importance of player retention, especially in a mobile game where players receive a lot of distractions through the likes of mobile notifications is a huge issue and needs to be addressed.

Players want to feel like they can compete, whether they purchase in-game transactions or not.

The following is an analysis into:

Analysis

Average Play Time by Day of the Week

Average time players spend in the game on each day of the week, which could reveal patterns in player engagement based on day.

play_time_by_day <- game_churn %>%
  mutate(day = factor(day, 
                      levels = c("Monday", "Tuesday", "Wednesday", 
                                 "Thursday", "Friday", "Saturday", "Sunday"))) %>%
  group_by(day) %>%
  summarise(average_play_time = round(mean(play_time_sec, na.rm = TRUE), 0), total_players = n()) %>%
#Renaming the columns
  rename(Days = day,
         'Play Time (Sec)' = average_play_time,
         'Total Players' = total_players)

knitr::kable(play_time_by_day)
Days Play Time (Sec) Total Players
Monday 123 1850
Tuesday 127 2069
Wednesday 120 1938
Thursday 134 2149
Friday 140 1959
Saturday 127 1913
Sunday 129 1670

From the above data we can gather that people generally play for a longer period of time on Fridays (140 seconds), contrasted by a low of 120 seconds on a Wednesday. It should be of priority to be adding content that retains players during the week.

The total player count reaches a high of 2,149 on Thursday with a low of 1,670 on Sunday. This does not reach above 2,000 again until Tuesday so it may be worth sending push-notifications on Sundays & Mondays to get the player count up on these days.

Heatmap of Player Count by Day and Hour

Visualises the number of players playing at each time.

player_count_by_day_hour <- game_churn %>%
  group_by(day, hour) %>%
  summarise(player_count = n())

player_count_by_day_hour <- player_count_by_day_hour %>%
  mutate(day = factor(day, levels = c("Monday", "Tuesday", "Wednesday", 
                                      "Thursday", "Friday", "Saturday", "Sunday")))

max_value <- max(player_count_by_day_hour$player_count)
min_value <- min(player_count_by_day_hour$player_count)

highlight_points <- player_count_by_day_hour %>%
  filter(player_count == max_value | player_count == min_value)

ggplot(player_count_by_day_hour, aes(x = hour, y = day, fill = player_count)) +
  geom_tile() +
  scale_fill_gradient(low = "white", high = "deepskyblue") +
  geom_point(data = highlight_points, aes(x = hour, y = day), 
        color = "black", size = 4, shape = 21, fill = "white") +
  geom_text(data = highlight_points, aes(x = hour, y = day, label = player_count), 
        color = "black", size = 3, vjust = -1) +
  labs(title = "Player Count Heatmap by Day and Hour (Highlighting Min and Max Values)",
       x = "Hour of the Day",
       y = "Day of the Week",
       fill = "Player Count") +
  theme_minimal()

By analysing this graph, we can see that the lowest player count is at 7am on a Sunday, and the highest is at 7pm on a Wednesday.

There are two suggestions that we can make from this data:

  • Target players through push-notifications & incentives (e.g. daily log-in rewards) during quieter periods - This will lead to an increase in player count.

  • Release content around 6-7pm - This will lead to a higher player retention as they have more to do when on the app.

Distribution of Player Levels by Churn Status

Visualising the distribution of player levels among churned and non-churned players. This could help to see if players at certain levels tend to churn more frequently.

#Grouping levels
game_churn <- mutate(game_churn, levels_per_1500 = case_when(
  level >= 6000 ~ "4500+",
  level >= 3000 ~ "3000 - 4499",
  level >= 1500 ~ "1500 - 2999",
  level >= 0 ~ "0 - 1499"
))

ggplot(data = game_churn, aes(x = levels_per_1500, fill = churn)) +
  geom_bar(position = "dodge") +
  scale_fill_manual(values = c("deepskyblue", "goldenrod1")) +
  labs(title = "Distribution of Player Levels by Churn Status",
       x = "Level",
       y = "Player Count") +
  theme_light()

From the above graph, we can look compare those who churned vs didn’t churn based on player levels. Of casual players in the level 0-1499 range, over 30% of players end up churning. There is a dramatic drop off of those who churn in the rest of the level ranges, from level 1500+.

This may be combated by either making earlier stages of the game easier, or allowing players to earn more XP which makes them feel more rewarded for their work.

Churn Rate by Day of the Week (Comparing Extra Moves Buyers vs Non-Buyers)

Shows the average churn rates for those who bought extra moves compared to those who didn’t.

game_churn <- game_churn %>%
  mutate(bought_extra_moves = ifelse(extra_moves > 0, "Yes", "No"))

game_churn$day <- factor(game_churn$day, 
                        levels = c("Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"))

churn_by_day_and_moves <- game_churn %>%
  group_by(day, bought_extra_moves) %>%
  summarise(churn_rate = mean(churn == "Yes"))

ggplot(churn_by_day_and_moves, aes(x = day, y = churn_rate, color = bought_extra_moves, group = bought_extra_moves)) +
  geom_line(size = 1.2) +
  geom_point(size = 2) +
  labs(title = "Churn Rate by Day of the Week (Comparing Extra Moves Buyers vs Non-Buyers)",
       x = "Day of the Week",
       y = "Churn Rate",
       color = "Bought Extra Moves") +
  scale_colour_manual(values = c("deepskyblue", "goldenrod1")) +
theme_light()

We can see from the above graph that there is a dramatic difference in churn rate for those who have bought extra moves compared to those who didn’t. Those who buy more moves are more engaged players. This can be improved by doing any of the following:

  • Lowering the price of moves to buy

  • Allowing players to spend real money to buy coins

  • Altering the UI of the app to push more players to the store

Rolling Losses vs Scores by Churn Status

Visualising the relationship between the number of rolling losses & scores and whether a player churned.

ggplot(data = game_churn) +
  geom_point(mapping = aes(x = scores, y = rolling_losses, colour = churn), size = 0.8) +
  scale_colour_manual(values = c("deepskyblue", "goldenrod1")) +
  facet_wrap(~ churn) +
  labs(title = "Rolling Losses vs Scores by Churn Status",
       x = "Scores",
       y = "Rolling Losses") +
  theme_light()

On each of these scatter-plots, we can see that both plots are positively skewed (right skewed). We can see that the average scores of both churn statuses are very similar, although those who do not churn have a slightly higher max score (10,000).

The rolling losses are also much higher, this may be because it is more of a ‘hardcore’ player-base and they do not mind losing. We may be able to use Skill Based Matchmaking to make the levels slightly easier for players of a lower skill level.

Player moves by levels

Visualising the moves used by players as they progress through different levels. This can help show the difficulty/challenge on some levels.

#Sampling first 100 levels & between 10-200 moves
game_churn <- data.frame(
  level = sample(1:100, 1000, replace = TRUE),
  used_moves = sample(10:200, 1000, replace = TRUE)
)

moves_by_level <- game_churn %>%
  group_by(level) %>%
  summarise(average_moves = mean(used_moves))

moves_plot <- ggplot(moves_by_level, aes(x = level, y = average_moves)) +
  geom_line(color = "deepskyblue") +
  geom_point(color = "goldenrod1") +
  labs(title = "Average Moves by Level",
       x = "Level",
       y = "Average Moves") +
  theme_minimal()

#Animation using gganimate function
moves_animation <- moves_plot +
  transition_reveal(level)

#Animation output file settings
output_file <- "gamechurn_animation.gif"
animate(moves_animation, nframes = 200, fps = 10, end_pause = 50, renderer = gifski_renderer(output_file))

This graph outlines the difference of difficulty between levels. The first 100 levels are used as a sample. On level 34, the difficulty drops massively which is followed by a dramatic increase on the next level. This may cause the player to be poorly prepared for the more difficult level as there is such a high jump.

This may be causing players to drop off and as the game is not linear, it could be a fix as simple as switching the levels around, to where the more difficult one is first, as level 33 is a closer difficulty to level 35.

End Type Distribution by Churn

Helps to see if churned players are more likely to quit or lose levels compared to non-churned players.

game_churn <- read_csv("game_churn.csv")

end_type_by_churn <- game_churn %>%
  group_by(churn, end_type) %>%
  summarise(count = n()) %>%
  spread(key = end_type, value = count, fill = 0) %>%
  rename(Churn = churn) %>%
  select(Churn, Win, Lose, Restart, Quit)

knitr::kable(end_type_by_churn)
Churn Win Lose Restart Quit
No 2936 5837 41 82
Yes 1235 3306 19 92

Roughly 50% of those who do not churn, actually win while playing their levels. As seen earlier, there is a number of ‘hardcore’ players which stick around no matter how their game goes. We must turn these churning players into this ‘hardcore’ segment by nurturing them.

This can be done by providing tips & tricks based on how they lost the level.

Used Moves vs Scores

Visualises the relationship between the score and the number of moves used by players.

scatter_data <- game_churn %>%
  select(id, used_moves, scores, level)

# Identify outliers based on 1.5 * IQR rule
Q1 <- quantile(scatter_data$scores, 0.25, na.rm = TRUE)
Q3 <- quantile(scatter_data$scores, 0.75, na.rm = TRUE)
IQR <- Q3 - Q1

# Define outliers: scores outside of 1.5 * IQR from Q1 and Q3
outliers <- scatter_data %>%
  filter(scores < (Q1 - 1.5 * IQR) | scores > (Q3 + 1.5 * IQR))

ggplot(scatter_data, aes(x = used_moves, y = scores)) +
  geom_point(color = "deepskyblue", size = 2, alpha = 0.7) +
  geom_point(data = outliers, aes(x = used_moves, y = scores), 
             size = 2.5, shape = 21, fill = "goldenrod1") +
  labs(title = "Used Moves vs. Scores (Outliers Highlighted)",
       x = "Used Moves",
       y = "Scores") +
  theme_minimal()

Here we can see that there are a number of outliers where the majority are over the scores of ~6,000. Scores in relation to Used Moves are linear for the most part but these begin to disperse once players reach the 6,000 score threshold. It is interesting to see the difference between some who used 8 moves getting 8,400 score, while some scored zero.

Player skill seems well balanced for the most part but if we can aid players to hitting that 6,000 score threshold, we may be able to nurture them and provide players with a better experience and a lower churn rate.

This may be done through providing video walk-throughs of our levels online.

Suggestions

The following is a summary for all of the recommendations made above:

  • Sending push-notifications on quiet days
  • Provide players with incentives to log in
  • Release daily content at 6-7pm
  • Making high-churn levels easier
  • Rewarding players with more XP
  • Lowering the price of moves to buy
  • Allowing players to spend real money to buy coins
  • Altering the UI of the app to push more players to the store
  • Use Skill Based Matchmaking to make the levels slightly easier for players of a lower skill level
  • Switching the levels around, to where the more difficult one is first, as level 33 is a closer difficulty to level 35
  • Providing tips & tricks based on how they lost the level
  • Uploading video walk-throughs of our levels online