INFO H-510 Project
Sashank
2024-04-21
Understanding and predicting customer churn is of paramount importance for businesses in the banking industry. By analyzing a comprehensive dataset containing information on customer demographics, financial behavior, and tenure, the aim of the project is to uncover key insights into the factors driving churn. This project focuses on applying statistical techniques to infer patterns within the data, ultimately providing actionable insights to mitigate churn and enhance customer retention strategies. Through exploratory data analysis and statistical modeling, we seek to identify correlations, trends, and potential predictors of churn, thereby empowering banks to proactively address customer attrition and optimize their operations.
library(tidyverse)## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.4 ✔ readr 2.1.4
## ✔ forcats 1.0.0 ✔ stringr 1.5.1
## ✔ ggplot2 3.5.0 ✔ tibble 3.2.1
## ✔ lubridate 1.9.3 ✔ tidyr 1.3.0
## ✔ purrr 1.0.2
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errorslibrary(dplyr)
data = read_delim("Churn_Modelling.csv", delim=",")## Rows: 10002 Columns: 14
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (3): Surname, Geography, Gender
## dbl (11): RowNumber, CustomerId, CreditScore, Age, Tenure, Balance, NumOfPro...
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.head(data)## # A tibble: 6 × 14
## RowNumber CustomerId Surname CreditScore Geography Gender Age Tenure Balance
## <dbl> <dbl> <chr> <dbl> <chr> <chr> <dbl> <dbl> <dbl>
## 1 1 15634602 Hargra… 619 France Female 42 2 0
## 2 2 15647311 Hill 608 Spain Female 41 1 83808.
## 3 3 15619304 Onio 502 France Female 42 8 159661.
## 4 4 15701354 Boni 699 France Female 39 1 0
## 5 5 15737888 Mitche… 850 Spain Female 43 2 125511.
## 6 6 15574012 Chu 645 Spain Male 44 8 113756.
## # ℹ 5 more variables: NumOfProducts <dbl>, HasCrCard <dbl>,
## # IsActiveMember <dbl>, EstimatedSalary <dbl>, Exited <dbl>str(data)## spc_tbl_ [10,002 × 14] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
## $ RowNumber : num [1:10002] 1 2 3 4 5 6 7 8 9 10 ...
## $ CustomerId : num [1:10002] 15634602 15647311 15619304 15701354 15737888 ...
## $ Surname : chr [1:10002] "Hargrave" "Hill" "Onio" "Boni" ...
## $ CreditScore : num [1:10002] 619 608 502 699 850 645 822 376 501 684 ...
## $ Geography : chr [1:10002] "France" "Spain" "France" "France" ...
## $ Gender : chr [1:10002] "Female" "Female" "Female" "Female" ...
## $ Age : num [1:10002] 42 41 42 39 43 44 50 29 44 NA ...
## $ Tenure : num [1:10002] 2 1 8 1 2 8 7 4 4 2 ...
## $ Balance : num [1:10002] 0 83808 159661 0 125511 ...
## $ NumOfProducts : num [1:10002] 1 1 3 2 1 2 2 4 2 1 ...
## $ HasCrCard : num [1:10002] 1 0 1 0 NA 1 1 1 0 1 ...
## $ IsActiveMember : num [1:10002] 1 1 0 0 1 0 1 0 NA 1 ...
## $ EstimatedSalary: num [1:10002] 101349 112543 113932 93827 79084 ...
## $ Exited : num [1:10002] 1 0 1 0 0 1 0 1 0 0 ...
## - attr(*, "spec")=
## .. cols(
## .. RowNumber = col_double(),
## .. CustomerId = col_double(),
## .. Surname = col_character(),
## .. CreditScore = col_double(),
## .. Geography = col_character(),
## .. Gender = col_character(),
## .. Age = col_double(),
## .. Tenure = col_double(),
## .. Balance = col_double(),
## .. NumOfProducts = col_double(),
## .. HasCrCard = col_double(),
## .. IsActiveMember = col_double(),
## .. EstimatedSalary = col_double(),
## .. Exited = col_double()
## .. )
## - attr(*, "problems")=<externalptr>Variables Overview
Continuous Variables Overview
continuous_variables <- c("CreditScore", "Balance", "NumOfProducts", "EstimatedSalary", "Age", "Tenure")
summary(data[continuous_variables])## CreditScore Balance NumOfProducts EstimatedSalary
## Min. :350.0 Min. : 0 Min. :1.00 Min. : 11.58
## 1st Qu.:584.0 1st Qu.: 0 1st Qu.:1.00 1st Qu.: 50983.75
## Median :652.0 Median : 97199 Median :1.00 Median :100185.24
## Mean :650.6 Mean : 76491 Mean :1.53 Mean :100083.33
## 3rd Qu.:718.0 3rd Qu.:127648 3rd Qu.:2.00 3rd Qu.:149383.65
## Max. :850.0 Max. :250898 Max. :4.00 Max. :199992.48
##
## Age Tenure
## Min. :18.00 Min. : 0.000
## 1st Qu.:32.00 1st Qu.: 3.000
## Median :37.00 Median : 5.000
## Mean :38.92 Mean : 5.012
## 3rd Qu.:44.00 3rd Qu.: 7.000
## Max. :92.00 Max. :10.000
## NA's :1Insights
From the above summary it can be noted that the Minimum value of the Balance variable, which is
0, indicating that there is at least one customer with zero-balance account. And the 1st Quantile value being0as well, indicating that there are a significant number of customers with zero-account balances.Further, the Minimum value of the EstimatedSalary is
11.58, while the 1st Quantile value if above50,000indicating there could only a fewer number of customers with a low (estimated) Salary.
Categorical Variables Overview
categorical_variables <- c("Geography", "Gender", "HasCrCard", "IsActiveMember", "Exited")
cat_counts <- lapply(data[categorical_variables], table)
names(cat_counts) <- categorical_variables
cat_counts## $Geography
##
## France Germany Spain
## 5014 2510 2477
##
## $Gender
##
## Female Male
## 4544 5458
##
## $HasCrCard
##
## 0 1
## 2945 7056
##
## $IsActiveMember
##
## 0 1
## 4851 5150
##
## $Exited
##
## 0 1
## 7964 2038Insights
- The variables
IsActiveMemberandGenderhave a fairly even proportionate counts compared to the remaining variables.
Visualizations
Bar Chart of Average Balance by Geography
library(ggplot2)
na.omit(data) |>
ggplot(aes(x = Geography, y = Balance)) +
geom_bar(stat = "summary", fun = "mean", fill = "skyblue", color = "black") +
labs(title = "Average Balance by Geography",
x = "Geography",
y = "Average Balance") +
theme_minimal()
The bar chart illustrates distinct average balance levels across
geographic regions. Customers from Germany exhibit the highest average
balance, while both France and Spain show similar lower averages around
62,500 each. These disparities highlight potential
opportunities for tailored banking services and marketing strategies to
address varying customer needs across regions.
Bar Chart of Amount of Churned Customers by Geography
na.omit(data) |>
ggplot(aes(x = Geography, fill = factor(Exited))) +
geom_bar(position = "dodge") +
labs(title = "Number of People Exited by Geography",
x = "Geography",
y = "Number of People Exited",
fill = "Exited") +
scale_fill_manual(values = c("0" = "skyblue", "1" = "salmon")) +
theme_minimal()
exit_prop <- data |>
na.omit() |>
group_by(Geography, Exited) |>
summarize(count = n()) |>
mutate(proportion = count / sum(count)) |>
filter(Exited == 1) |>
select(Geography, proportion)## `summarise()` has grouped output by 'Geography'. You can override using the
## `.groups` argument.ggplot(exit_prop, aes(x = Geography, y = proportion, fill = Geography)) +
geom_bar(stat = "identity") +
labs(title = "Proportion of Customers Exited by Geography",
x = "Geography",
y = "Proportion of Customers Exited",
fill = "Geography") +
theme_minimal()
The bar chart demonstrates varying proportions of exited customers
across different geographical regions. Notably, Germany exhibits the
highest proportion of exited customers at approximately
32%, followed by Spain with around 16%, and
France with approximately 16%. This suggests that there may
be distinct factors influencing customer churn rates in each region,
potentially related to economic conditions, cultural differences, or the
effectiveness of retention strategies.
Interestingly, despite boasting the highest average balance, Germany also displays the highest churn rate. This suggests that factors beyond financial standing may strongly influence customer retention in the region, urging banks to explore nuanced strategies for maintaining long-term relationships despite initial financial advantages.
Distribution of Customer Account Balance
data |>
na.omit() |>
ggplot(aes(x = Balance)) +
geom_histogram(fill = "#1f77b4", color = "white") +
labs(title = "Histogram of Balance Variable",
x = "Balance",
y = "Frequency") +
theme_minimal()## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
The prevalence of customers with zero balances suggests a spectrum of circumstances. It could signify dormant or recently opened accounts. Further, the distribution is highly skewed due zero-balance accounts. Moreover, focusing on customer segmentation or targeted marketing strategies tailored to different balance brackets can aid in effectively managing customer relationships and optimizing business outcomes.
Bar Chart of Number of Products by Churn Status
count_data <- data |>
group_by(NumOfProducts, Exited) |>
summarise(count = n())## `summarise()` has grouped output by 'NumOfProducts'. You can override using the
## `.groups` argument.ggplot(count_data, aes(x = factor(NumOfProducts), y = count, fill = factor(Exited))) +
geom_bar(stat = "identity", position = "dodge", color = "white") +
geom_text(aes(label = count), position = position_dodge(width = 0.9), vjust = -0.5) +
labs(title = "Distribution of Number of Products by Churn Status",
x = "Number of Products",
y = "Count",
fill = "Exited") +
scale_fill_manual(values = c("#1f77b4", "#ff7f0e"),
labels = c("Not Churned", "Churned")) +
theme_minimal()
The data reveals a clear trend: as customers hold more products, churn rates rise. While those with one or two products largely stay, a significant portion with three or four products churn, with all customers holding four products leaving. This suggests that managing multiple products may increase churn risk. Understanding and addressing this trend are vital for improving retention strategies and ensuring customer satisfaction across product lines.
Tests
Age Vs Churn Test
Using this test the aim to investigate whether there’s a statistically significant difference in the average age between churned and retained bank customers.
Hypothesis
Null Hypothesis (H0) - There is no significant difference in the average age between churned and retained bank customers.
Alternate Hypothesis (H1) - There is a significant difference in the average age between churned and retained bank customers.
data |> na.omit() |> ggplot(aes(x = factor(Exited), y = Age, fill = factor(Exited))) + geom_boxplot() + labs(title = "Age Distribution by Churn Status", x = "Exited", y = "Age", fill = "Exited") + scale_fill_manual(values = c("#1f77b4", "#ff7f0e"), labels = c("Not Churned", "Churned")) + theme_minimal()
t_test_age_churn <- t.test(Age ~ Exited, data = data)
t_test_age_churn##
## Welch Two Sample t-test
##
## data: Age by Exited
## t = -30.424, df = 3250.8, p-value < 2.2e-16
## alternative hypothesis: true difference in means between group 0 and group 1 is not equal to 0
## 95 percent confidence interval:
## -7.906664 -6.949257
## sample estimates:
## mean in group 0 mean in group 1
## 37.40864 44.83660From the above hypothesis testing it is clear that there is a notable
age discrepancy between the two groups. The 95% confidence
interval for the difference in means ranges from -7.906 to
-6.949. The sample mean age for churned customers is
approximately 44.84 years, while for retained customers, it
is about 37.41 years.
This insight underscores the importance of considering age demographics in the development of targeted retention strategies and customer engagement initiatives to mitigate churn and enhance customer loyalty.
Balance Vs Churn Test
Using this analysis, the aim is to explore if there exists a statistically significant distinction in the account balances between churned and retained bank customers.
Hypothesis
Null Hypothesis (H0) - There is no significant difference in the average account balances between churned and retained bank customers.
Alternate Hypothesis (H1) - There is a significant difference in the average account balances between churned and retained bank customers.
From the distribution of the Balances variable, there are a significant number of customers with zero-balance, hence it would be logical to perform the test on customers with non-zero account balances.
data |>
filter(Balance != 0) |>
ggplot(aes(x = Exited, y = Balance, fill = factor(Exited))) +
geom_boxplot() +
labs(x = "Churn Status", y = "Account Balance", fill = "Exited") +
theme_minimal()
data_filtered <- data |>
filter(Balance != 0)
t_test_balance_churn <- t.test(Balance ~ Exited, data = data_filtered)
t_test_balance_churn##
## Welch Two Sample t-test
##
## data: Balance by Exited
## t = -1.3246, df = 2540.7, p-value = 0.1854
## alternative hypothesis: true difference in means between group 0 and group 1 is not equal to 0
## 95 percent confidence interval:
## -2924.851 566.405
## sample estimates:
## mean in group 0 mean in group 1
## 119538.1 120717.3Based on the p-value of 0.1854, we fail to reject the
null hypothesis at the typical significance level of 0.05.
This suggests that there is no significant difference in the mean
account balances between churned and retained customers.
One insight from this analysis is that, there appears to be no substantial difference in the average account balances between customers who churn and those who remain with the bank. This suggests that factors other than account balance may be more influential in determining customer churn within this banking dataset.
Active Member Vs Churn Test
Through this analysis, our goal is to assess if there’s a statistically significant difference in activity levels between active members and churned bank customers.
Hypothesis
Null Hypothesis (H0) - There is no statistically significant difference in activity levels between active members and churned bank customers.
Alternate Hypothesis (H1) - There is a statistically significant difference in activity levels between active members and churned bank customers.
data |>
na.omit() |>
mutate(Exited = factor(Exited)) |>
ggplot(aes(x = Exited, fill = factor(IsActiveMember))) +
geom_bar(position = "fill", color = "black") +
labs(x = "Exited", y = "Proportion", fill = "IsActiveMember") +
scale_fill_manual(values = c("skyblue", "salmon")) +
theme_minimal()
t_test_active_exited <- t.test(IsActiveMember ~ Exited, data = data)
t_test_active_exited##
## Welch Two Sample t-test
##
## data: IsActiveMember by Exited
## t = 16.137, df = 3244.4, p-value < 2.2e-16
## alternative hypothesis: true difference in means between group 0 and group 1 is not equal to 0
## 95 percent confidence interval:
## 0.1702450 0.2173382
## sample estimates:
## mean in group 0 mean in group 1
## 0.5544393 0.3606477The analysis revealed a significant difference in activity levels
between the two groups. Active members exhibited a higher mean activity
level (0.5544) compared to churned bank customers
(0.3606). This suggests that there is a notable disparity
in engagement between retained customers and those who have exited the
bank.
Therefore customers who are more actively engaged with the bank are less likely to churn. Moreover, implementing strategies to increase customer engagement, such as personalized services, targeted promotions, and proactive communication, could help reduce churn rates.
Modelling
In this section, we utilize logistic regression to analyze the bank customer churn dataset, focusing on the ‘Exited’ variable as the output. By exploring factors such as credit score, geography, age, tenure, and product usage.
Let, the response variable be Exited, and the explanatory variables be, Credit Score, Geography, Gender, Age, Tenure, Balance, Num. Of Products, HasCrCard, IsActiveMember and EstimatedSalary (Every column except Customer ID, RowNumber and Surname).
data_processed <- data |>
na.omit() |>
filter(Balance != 0) |>
select(-`CustomerId`, -Surname, -RowNumber) |>
mutate(
Geography = as.factor(Geography),
Gender = as.factor(Gender),
HasCrCard = as.integer(HasCrCard),
IsActiveMember = as.integer(IsActiveMember),
Exited = as.integer(Exited)
)
logit_model <- glm(Exited ~ ., data = data_processed, family = "binomial")
summary(logit_model)##
## Call:
## glm(formula = Exited ~ ., family = "binomial", data = data_processed)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -3.851e+00 3.135e-01 -12.285 < 2e-16 ***
## CreditScore -6.090e-04 3.298e-04 -1.846 0.0648 .
## GeographyGermany 7.205e-01 7.260e-02 9.925 < 2e-16 ***
## GeographySpain 9.098e-02 9.335e-02 0.975 0.3297
## GenderMale -5.040e-01 6.437e-02 -7.830 4.89e-15 ***
## Age 7.403e-02 3.123e-03 23.706 < 2e-16 ***
## Tenure -1.184e-02 1.102e-02 -1.075 0.2824
## Balance 1.885e-06 1.080e-06 1.745 0.0810 .
## NumOfProducts 2.241e-01 5.317e-02 4.214 2.51e-05 ***
## HasCrCard -2.738e-02 7.003e-02 -0.391 0.6958
## IsActiveMember -1.058e+00 6.750e-02 -15.672 < 2e-16 ***
## EstimatedSalary 1.468e-07 5.609e-07 0.262 0.7936
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 7048.5 on 6381 degrees of freedom
## Residual deviance: 5987.6 on 6370 degrees of freedom
## AIC: 6011.6
##
## Number of Fisher Scoring iterations: 5The summary of the Logistic Regression Model above reveals that certain explanatory variables make a notably more significant contribution compared to others.
Constructing a model offers distinct advantages -
Make predictions (Not the point of this course)
Enhanced Decision-Making for Business Success
Notably, customers from Germany exhibit a higher likelihood of churning. Developing tailored business strategies specifically targeted for the German market could potentially mitigate churn rates and improve customer retention.
Moreover, older customers are more inclined to churn, suggesting a noteworthy correlation between age and churn rates. Implementing personalized communication channels, offering exclusive loyalty rewards, and providing specialized customer support tailored to the unique needs of older clientele can significantly enhance retention efforts.
Model analysis reveals a significant gender-based discrepancy in churn likelihood among bank customers. This underscores the relevance of gender as a predictive factor in customer retention strategies within the banking sector. To address this observed disparity, banks can deploy tailored retention initiatives.
The analysis underscores a significant relationship between the number of bank products utilized by customers and churn rate, indicating that each additional product correlates with an increased likelihood of churn.
Additionally, there is a strong statistically significant relationship between the Active Member variable and churning. To leverage this insight and mitigate churn, banks can focus on strategies to enhance customer engagement and foster active participation.
Overall Conclusions
The summary highlights variations in average balance levels and customer churn rates across different regions. Germany stands out with the highest average balance, but also exhibits the highest churn rate, suggesting factors beyond finances influence retention. Additionally, the prevalence of zero-balance accounts underscores the need for targeted marketing strategies. Notably, customers with multiple products are more likely to churn, emphasizing the importance of understanding and addressing this trend for better retention strategies.
Age demographics, particularly older customers, influence churn rates, highlighting the importance of tailored retention strategies. Active engagement is linked to lower churn, advocating for personalized approaches. German customers exhibit higher churn, urging targeted interventions. Gender disparities in churn emphasize the need for tailored initiatives. Moreover, increased product usage correlates with higher churn, emphasizing the need for strategic management.