Investigation
The objective is to evaluate the launch of the new route with the
wise_funnel_events_regional
data. The table provided
contains the transfer events triggered by each user, allowing us to
evaluate based on: volume of transactions, and funnel conversion.
The overall funnel shows that from 43.1k (100%) transfers created,
only 19.4k (45%) were funded, and 11.1k (26%) were transferred. It is
not trivial to evaluate if the funnel is performing as expected without
a benchmark of other currency routes or more granular data on why the
drops happened. However, it’s important to monitor the launch
performance over time during the two months of data from
2024-01-01
to 2024-03-01
.
If we look at the end-to-end conversion, from transfer
created-to-transferred, it maintained stable through the whole period at
around 26%. However, if we breakdown the funnel into two steps: (1)
created-to-funded and (2) funded-to-transferred, there is a sudden
change in the first days of February. Created-to-funded had a drop and
stayed in a lower baseline for the rest of the period; while
funded-to-transferred had a bump and stayed in a higher baseline,
compensating the former drop to keep the created-to-transferred stable
through the whole period.
Looking at the volume of transfers in the period, we see a fast
ramp-up until Jan 10th where the three event types maintained stable
with a slight upwards trend. However, the number of transfers created
had a sudden increase in Feb 1st and maintained stable in a higher
baseline. This behavior matches the dates in which there were the funnel
conversion anomalies, likely being associated with the disturbance.
Breaking down the volume of transfer events by region, there is a
clear sudden increase in events from Europe and a sudden decrease in
Other and NorthAm (to a smaller degree). This raises the questions: (1)
Are these movements expected?; (2) Could it be the effect from marketing
campaigns; (3) Could it be a misattribution of regions?
That’s definitely worth investigating, but we can go further. Even if
the volume/mix of regions is changing, it doesn’t necessarily cause
changes in the funnel conversion. For example, if the conversion rates
were similar between regions, the funnel conversion would have remained
stable through the whole period even if the mix changed. This raises two
hypothesis for the funnel behavior: (1) the mix of regions change caused
the conversion rates to change because of unequal performance across
regions; (2) the conversion rates from the regions changed in the
beginning of Feb.
Breaking down the conversion rates for each step of the funnel by
region, we see that the two hypothesis were true. (1) Users from Europe
have a worse created-to-funded rate and a better funded-to-transferred
rate than the other regions; (2) The created-to-funded conversion rate
in Europe had the biggest change in February going from ~38% to
~30%.
Complete Code Investigation
Setup and Data Loading
Loading the libraries and the dataset.
library(tidyverse)
library(plotly)
df <- read_csv("wise_funnel_events_regional - wise_funnel_events_regional.csv")
Data Inspection
Inspecting the dataframe to check the possible values in each
column.
- 73,440 rows
- 0
NULL
values
event_name
: Transfer Created
,
Transfer Funded
, Transfer Transferred
region
: Europe
, NorthAm
,
Other
platform
: Android
, iOS
,
Web
dt
: date from 2024-01-01
to
2024-03-01
head(df)
## # A tibble: 6 × 6
## event_name dt user_id region platform experience
## <chr> <date> <dbl> <chr> <chr> <chr>
## 1 Transfer Funded 2024-02-15 1513280 NorthAm iOS Existing
## 2 Transfer Funded 2024-02-29 1270052 NorthAm iOS Existing
## 3 Transfer Transferred 2024-02-29 1270052 NorthAm iOS Existing
## 4 Transfer Funded 2024-02-25 1399835 NorthAm iOS Existing
## 5 Transfer Transferred 2024-02-25 1399835 NorthAm iOS Existing
## 6 Transfer Funded 2024-02-01 1972860 NorthAm iOS Existing
dim(df)
## [1] 73440 6
sum(is.na(df))
## [1] 0
df %>%
group_by(event_name) %>%
summarise(event_count = n(),
unique_users = n_distinct(user_id))
## # A tibble: 3 × 3
## event_name event_count unique_users
## <chr> <int> <int>
## 1 Transfer Created 43070 40223
## 2 Transfer Funded 19290 18195
## 3 Transfer Transferred 11080 10557
funnel_order <- c("Transfer Created", "Transfer Funded", "Transfer Transferred")
df %>%
group_by(region) %>%
summarise(event_count = n(),
unique_users = n_distinct(user_id))
## # A tibble: 3 × 3
## region event_count unique_users
## <chr> <int> <int>
## 1 Europe 25665 15695
## 2 NorthAm 19468 9393
## 3 Other 28307 15704
df %>%
group_by(platform) %>%
summarise(event_count = n(),
unique_users = n_distinct(user_id))
## # A tibble: 3 × 3
## platform event_count unique_users
## <chr> <int> <int>
## 1 Android 27593 15955
## 2 Web 20243 11681
## 3 iOS 25604 13172
df %>%
group_by(experience) %>%
summarise(event_count = n(),
unique_users = n_distinct(user_id))
## # A tibble: 2 × 3
## experience event_count unique_users
## <chr> <int> <int>
## 1 Existing 33628 15808
## 2 New 39812 25234
df %>%
summarise(first_dt = min(dt),
last_dt = max(dt),
unique_users = n_distinct(user_id))
## # A tibble: 1 × 3
## first_dt last_dt unique_users
## <date> <date> <int>
## 1 2024-01-01 2024-03-01 40223
Funnel Overall
overall_funnel_data <- df %>%
group_by(event_name) %>%
summarise(event_count = n())
overall_funnel_plot <- plot_ly(
overall_funnel_data,
x = ~event_count,
y = ~event_name,
type = 'funnel',
textposition = 'inside',
textinfo = 'value+percent initial+percent previous'
) %>%
layout(
title = "Overall Conversion Funnel by Event Count",
yaxis = list(title = "Funnel Step"),
xaxis = list(title = "Number of Unique Users")
)
overall_funnel_plot
overall_funnel_data <- df %>%
group_by(event_name) %>%
summarise(event_count = n_distinct(user_id))
overall_funnel_plot <- plot_ly(
overall_funnel_data,
x = ~event_count,
y = ~event_name,
type = 'funnel',
textposition = 'inside',
textinfo = 'value+percent initial+percent previous'
) %>%
layout(
title = "Overall Conversion Funnel by Unique Users",
yaxis = list(title = "Funnel Step"),
xaxis = list(title = "Number of Unique Users")
)
overall_funnel_plot
Funnel Over Time
daily_funnel <- df %>%
mutate(event_name = str_to_lower(str_remove(event_name, "Transfer "))) %>%
group_by(dt, event_name) %>%
summarise(distinct_users = n_distinct(user_id), .groups = 'drop') %>%
pivot_wider(
names_from = event_name,
values_from = distinct_users,
values_fill = 0 # Fills in 0 if an event type didn't occur on a given day
) %>%
# Calculate conversion rates for each step
mutate(
created_to_funded_rate = funded / created,
funded_to_transferred_rate = transferred / funded,
overall_completion_rate = transferred / created
) %>%
# Reshape data for plotting
pivot_longer(
cols = c(created_to_funded_rate, funded_to_transferred_rate, overall_completion_rate),
names_to = "metric",
values_to = "rate"
)
# Create the time series plot
p_time_series <- ggplot(daily_funnel, aes(x = dt, y = rate, color = metric, group = metric)) +
geom_line() +
labs(
title = "Daily Funnel Conversion Rates Over Time",
x = "Date of Transfer Event",
y = "Conversion Rate",
color = "Conversion Metric"
) +
scale_y_continuous(labels = scales::percent, limits = c(0, 1)) +
scale_x_date(date_breaks = "1 week", date_labels = "%b %d") +
theme_minimal()
ggplotly(p_time_series)
p_time_series <- df %>%
group_by(dt, event_name) %>%
summarise(event_count = n(),
unique_users = n_distinct(user_id)) %>%
ggplot(aes(x = dt, y = unique_users, color = event_name)) +
geom_line() +
labs(
title = "Daily Unique User Over Time",
x = "Date of Transfer Event",
y = "Unique Users",
color = "Conversion Metric"
) +
scale_x_date(date_breaks = "1 week", date_labels = "%b %d") +
theme_minimal()
ggplotly(p_time_series)
plot_daily_funnel_by_group <- function(data, grouping_var) {
# Convert the string grouping_var to a symbol for use in dplyr
grouping_sym <- sym(grouping_var)
daily_funnel_by_group <- df %>%
mutate(event_name = str_to_lower(str_remove(event_name, "Transfer "))) %>%
group_by(dt, event_name, !!grouping_sym) %>%
summarise(distinct_users = n_distinct(user_id), .groups = 'drop') %>%
pivot_wider(
names_from = event_name,
values_from = distinct_users,
values_fill = 0 # Fills in 0 if an event type didn't occur on a given day
) %>%
# Calculate conversion rates for each step
mutate(
created_to_funded_rate = funded / created,
funded_to_transferred_rate = transferred / funded,
overall_completion_rate = transferred / created
) %>%
# Reshape data for plotting
pivot_longer(
cols = c(created_to_funded_rate, funded_to_transferred_rate, overall_completion_rate),
names_to = "metric",
values_to = "rate"
)
plot_title <- paste("Daily Funnel Conversion Rates Over Time by", tools::toTitleCase(grouping_var))
p <- ggplot(daily_funnel_by_group, aes(x = dt, y = rate, color = !!grouping_sym, group = !!grouping_sym)) +
geom_line() +
labs(
title = plot_title,
x = "Date of Transfer Event",
y = "Conversion Rate",
color = tools::toTitleCase(grouping_var) # Label color by the grouping variable
) +
scale_y_continuous(labels = scales::percent, limits = c(0, 1)) +
scale_x_date(date_breaks = "2 weeks", date_labels = "%b %d") +
theme_minimal() +
facet_wrap(~ metric, ncol = 1) # Facet by the conversion metric now
return(ggplotly(p))
}
plot_daily_funnel_by_group(user_summary, "region")
plot_daily_funnel_by_group(user_summary, "experience")
plot_daily_funnel_by_group(user_summary, "platform")
User Volume Over Time
plot_daily_counts_by_group <- function(data, grouping_var) {
# Convert the string grouping_var to a symbol for use in dplyr
grouping_sym <- sym(grouping_var)
daily_counts_by_group <- df %>%
mutate(event_name = str_to_lower(str_remove(event_name, "Transfer "))) %>%
group_by(dt, event_name, !!grouping_sym) %>%
summarise(distinct_users = n_distinct(user_id), .groups = 'drop')
plot_title <- paste("Daily Unique Users Over Time by", tools::toTitleCase(grouping_var))
p <- ggplot(daily_counts_by_group, aes(x = dt, y = distinct_users, color = !!grouping_sym, group = !!grouping_sym)) +
geom_line() +
labs(
title = plot_title,
x = "Date of Transfer Event",
y = "Number of Unique Users",
color = tools::toTitleCase(grouping_var)
) +
scale_x_date(date_breaks = "2 weeks", date_labels = "%b %d") +
theme_minimal() +
# Facet by funnel step, allow y-axis to scale independently for clarity
facet_wrap(~ event_name, ncol = 1, scales = "free_y")
return(ggplotly(p))
}
plot_daily_counts_by_group(user_summary, "region")
plot_daily_counts_by_group(user_summary, "experience")
plot_daily_counts_by_group(user_summary, "platform")
User Mix Over Time
plot_daily_counts_by_group <- function(data, grouping_var) {
# Convert the string grouping_var to a symbol for use in dplyr
grouping_sym <- sym(grouping_var)
daily_counts_by_group <- df %>%
mutate(event_name = str_to_lower(str_remove(event_name, "Transfer "))) %>%
group_by(dt, event_name, !!grouping_sym) %>%
summarise(distinct_users = n_distinct(user_id), .groups = 'drop') %>%
group_by(dt, event_name) %>%
mutate(percentage = distinct_users / sum(distinct_users)) %>%
ungroup
plot_title <- paste("Share of Unique Users Over Time by", tools::toTitleCase(grouping_var))
p <- ggplot(daily_counts_by_group, aes(x = dt, y = percentage, fill = !!grouping_sym)) +
geom_area(alpha = 0.5) +
labs(
title = plot_title,
x = "Date of Transfer Event",
y = "Share of Unique Users",
color = tools::toTitleCase(grouping_var)
) +
scale_y_continuous(labels = scales::percent, limits = c(0, 1)) +
scale_x_date(date_breaks = "2 weeks", date_labels = "%b %d") +
theme_minimal() +
# Facet by funnel step, allow y-axis to scale independently for clarity
facet_wrap(~ event_name, ncol = 1, scales = "free_y")
return(ggplotly(p))
}
plot_daily_counts_by_group(user_summary, "region")
plot_daily_counts_by_group(user_summary, "experience")
plot_daily_counts_by_group(user_summary, "platform")