Introduction

Regork is entering the telecommunication service area and wants to develop an understanding of customer behavior in order to retain customers and grow their business. One of the key aspects of this behavior is customer churn. Maintaining existing customers is critical to Regork’s strategy as it improves operational efficiency by eliminating the need to spend additional resources on new customer acquisition.

To assist them with this task, we analyzed the data from existing customers and visualized trends. Using this information, we built, trained, and tested three different types of machine learning models. The machine learning models that we chose were logistic regression, multivariate adaptive regression splines (MARS), and a decision tree model. Each of these models used all 19 predictors. Performance of each model was measured using a standard metric area under the curve.

After running these models and analyzing the data, we determined that the model that produced the best results was the logistic regression model. We then developed a list of recommendations based on these predictive models to decrease customer churn. Our conclusions and recommendations can be found under the summary tab.


Data Preparation

In order to reproduce the results contained within this report, the following packages must be installed and loaded:

library(tidymodels)
library(tidyverse)
library(vip)
library(pdp)
library(ggplot2)
library(rpart)
library(rpart.plot)
library(caret)
library(pROC)
library(dials)

As part of our data preprocessing, we converted our response variable, Status, to be a factor and removed 11 null values existed in the data set.

churn <- read_csv("Data/customer_retention.csv")
churn <- churn %>% 
  mutate(Status = as.factor(Status))

churn <- drop_na(churn)
sum(is.na(churn))
## [1] 0

In order to prepare our data for machine learning, we split the data set into a training set and a test set. The test set accounted for 30% of our data while the training set accounted for 70% of our data. The data split would ensure that we could get a good assessment of our model parameters and that our model was generalizable.

set.seed(123)
split <- initial_split(churn, prop = 0.7, strata = Status)
churn_train <- training(split)
churn_test <- testing(split)

Next, we constructed a recipe that stored standardized and normalized data. This process was done to ensure that the data is appropriately prepared for analysis.

churn_recipe <- recipe(Status ~ ., data = churn_train) %>%
  step_normalize(all_numeric_predictors()) %>%
  step_dummy(all_nominal_predictors())


Exploratory Data Analysis

To begin our analysis, we started by identifying some trends in the data by using graphs to visualize these points.

Graph 1

These plots show the relationship between customer tenure and whether the customer has dependents. The analysis is further broken down by Month-to-Month, One-year, and Two-year Contracts.

churn %>%
  ggplot(aes(Tenure, fill = Contract)) + 
  geom_bar() +
  facet_wrap(~Dependents + Contract) +
  ggtitle("Customer Tenure vs. Dependents") +
  labs(y = "Number of Customers", x = "Length of Stay (Months)") +
    theme(legend.position = "bottom")

The data shows that around five times as many customers are those without dependents. It also indicates that most of Regork’s customers, both those without and with dependents, initially chose month-to-month contracts. However, the month-to-month customers without dependents sharply declines by around 10 months into their contract. Both types of customers switch to two-year contracts around 70 months after they initially begin service. For those customers with dependents, there is a sharp decline of month-to-month customers by 5 months from their service start date.

Summary:

Regork’s target customers are those without dependents who initially choose month-to-month contracts. However, by 70 months, these customers prefer 2-year contracts.

Graph 2

These plots show the relationship between internet service customers and whether those customers have dependents. The analysis is further broken down by the type of service they have (DSL, fiber optic, or no internet service) along with whether they are current customers or are no longer a Regork customer.

# Average monthly price by internet services
churn %>%
  filter(Dependents == "No") %>%
  group_by(InternetService) %>%
  summarize(average = mean(MonthlyCharges))
## # A tibble: 3 × 2
##   InternetService average
##   <chr>             <dbl>
## 1 DSL                56.4
## 2 Fiber optic        90.7
## 3 No                 21.0
churn %>%
  group_by(InternetService, Status, Dependents) %>%
  summarize(count = n()) %>%
  ggplot(aes(x = InternetService, y = count, fill = Status)) +
  geom_bar(position = "dodge", stat = "identity") +
  facet_wrap(~Dependents) +
  scale_y_continuous(name = "Number of Customers", labels = comma_format()) +
  xlab("Internet Service") +
  ggtitle("Internet Service vs. Dependents") +
  scale_fill_manual(values = c("#AFCBFF", "#FF9CEE")) +
  theme(legend.position = "bottom")

Analysis of customers without dependents:

The data indicates that this group comprises most of Regork’s customers. The majority of these customers choose either DSL or fiber optic internet service. The internet service retention rate for DSL customers is about 70% while the retention rate for fiber optic customers is less than 20%. The retention rate for Regork customers that don’t use internet service is around 80%.

Analysis of customers with dependents:

These customers are fairly evenly distributed between DSL and those that do not use internet service, followed by fiber optic customers. The retention rate for DSL customers is about 80%, about 55% for fiber optic ones, and 90% for non-internet customers.

Summary:

The majority of Regork’s customers, those without dependents, choose either DSL or fiber optic internet service. However, the retention rate for fiber optic customers is very low and is around 70% for DSL customers. For those Regork customers that do not use their internet services, the retention rate is high.

Graph 3

This plot shows the relationship between the customer’s monthly charges and the probability of churn (the likelihood that customers will stop doing business with the company). The analysis is further broken down by customers with month-to-month contracts, one-year contracts, and two-year contracts.

lr_mod <- logistic_reg()

lr_fit <- lr_mod %>% 
  fit(Status ~ MonthlyCharges + Contract, data = churn_train)

tidy(lr_fit)
## # A tibble: 4 × 5
##   term             estimate std.error statistic  p.value
##   <chr>               <dbl>     <dbl>     <dbl>    <dbl>
## 1 (Intercept)       -1.59     0.104       -15.4 3.35e-53
## 2 MonthlyCharges     0.0193   0.00139      13.9 4.42e-44
## 3 ContractOne year  -1.92     0.111       -17.2 1.13e-66
## 4 ContractTwo year  -3.19     0.172       -18.6 2.58e-77
lr_fit %>% 
  predict(churn_train, type = "prob") %>%
  mutate(
    MonthlyCharges = churn_train$MonthlyCharges,
    Contract = churn_train$Contract
  ) %>%
  ggplot(aes(MonthlyCharges, .pred_Left, color = Contract)) +
  geom_point(alpha = 0.5, size = 0.8) +
  scale_y_continuous("Probability of Churn", limits = c(0, 1)) +
  scale_x_continuous("Monthly Charges", labels = dollar_format(prefix = "$")) +
  ggtitle("Predicted probabilities for Monthly Charges and Contract")

This analysis shows that month-to-month customers by far have the highest probability of churn, especially as their monthly charges increase. At a $100 price point, their probability rate exceeds 60%. Customers with two-year contracts have the lowest, fairly flat churn rate probability even as monthly charges increase. At $100 monthly price point, their probability of churn is around 6%, One-year contract customers have a higher churn rate probability than those with two-year contracts and are somewhat more sensitive to monthly charge increases. At a $100 monthly charge price point, their probability of churn is around 15%.

Summary:

Month-to-month customers overall have a high probability of churn rate which is very price sensitive. Two-year contract customers have a low probability of churn rate that is not very sensitive to price, while one-year contract customers have a somewhat higher probability that increases with price.

Overall Data Takeaways:

The target Regork customers are those without dependents that use DSL or fiber optic services. Although these customers initially choose one-year contracts, the ones that are retained are the two-year contract holders who are much less price sensitive. Overall, Regork has a major problem retaining fiber optic customers and should focus on what would entice these customers to remain. However, there is also room for improvement with DSL customers and Regork should consider the factors that make non-internet customers so loyal.


Machine Learning

Logistic Model

A logistic regression model was used to predict churn as our first machine learning model. It produces probability values that are between 0 and 1 and will serve as our baseline to compare our other models against.

log_mod <- logistic_reg()

set.seed(123)
kfolds <- vfold_cv(churn_train, v = 5, strata = Status)

log_results <- log_mod %>% 
  fit_resamples(Status ~ ., kfolds)

collect_metrics(log_results) %>% filter(.metric == "roc_auc")
## # A tibble: 1 × 6
##   .metric .estimator  mean     n std_err .config             
##   <chr>   <chr>      <dbl> <int>   <dbl> <chr>               
## 1 roc_auc binary     0.845     5 0.00521 Preprocessor1_Model1

The logistic regression model produces a ROC-AUC variable of 0.845. This indicates that the model is a good predictor of churn factors.

final_fit <- log_mod %>%
  fit(Status ~ ., data = churn_train)

final_fit %>%
   predict(churn_test) %>%
   bind_cols(churn_test %>% select(Status)) %>%
   conf_mat(truth = Status, estimate = .pred_class)
##           Truth
## Prediction Current Left
##    Current    1362  225
##    Left        178  332

We can see that the matrix correctly predicts 1362 people to be current customers. 225 are predicted to be current customers, but they left. This is a false positive. 178 customers are predicted to have left, but they are actually current customers. This is a false negative. The matrix correctly predicts 332 customers to have left.


MARS

The next model we ran was a MARS (Multivariate Adaptive Regression Splines) algorithm to train a classification model on the customer retention data set. The MARS model uses automatic variable interaction to determine interactions between predictor variables.

mars_mod <- mars(
  mode = "classification", 
  num_terms = tune(),
  prod_degree = tune()
)

# create k-fold cross validation object
set.seed(123)
kfolds <- vfold_cv(churn_train, v = 5, strata = Status)

# create a hyper parameter tuning grid
hyper_grid <- grid_regular(
  num_terms(range = c(1, 40)),
  prod_degree(),
  levels = 50
)

# train our model across the hyper parameter grid
mars_results <- tune_grid(mars_mod, churn_recipe, resamples = kfolds, grid = hyper_grid)

# get best results
show_best(mars_results, metric = "roc_auc")
## # A tibble: 5 × 8
##   num_terms prod_degree .metric .estimator  mean     n std_err .config          
##       <int>       <int> <chr>   <chr>      <dbl> <int>   <dbl> <chr>            
## 1        20           1 roc_auc binary     0.850     5 0.00486 Preprocessor1_Mo…
## 2        19           1 roc_auc binary     0.849     5 0.00486 Preprocessor1_Mo…
## 3        18           1 roc_auc binary     0.849     5 0.00486 Preprocessor1_Mo…
## 4        21           1 roc_auc binary     0.849     5 0.00486 Preprocessor1_Mo…
## 5        22           1 roc_auc binary     0.849     5 0.00486 Preprocessor1_Mo…
autoplot(mars_results)

The MARS model produces good accuracy and ROC_AUC mean values. From this, we selected the best hyperparameter values, final workflow, and plotted the ten most important inputs into the model.

mars_best <- select_best(mars_results, metric = "roc_auc")

mars_final <- workflow() %>%
  add_model(mars_mod) %>%
  add_formula(Status ~ .) %>%
  finalize_workflow(mars_best)

We can see that the matrix correctly predicts 1382 people to be current customers. 242 are predicted to be current customers, but they left. This is a false positive. 158 customers are predicted to have left, but they are actually current customers. This is a false negative. The matrix correctly predicts 315 customers to have left.

mars_final_fit <- mars_final %>%
  fit(data = churn_train)

mars_final_fit %>%
   predict(churn_test) %>%
   bind_cols(churn_test %>% select(Status)) %>%
   conf_mat(truth = Status, estimate = .pred_class)
##           Truth
## Prediction Current Left
##    Current    1383  242
##    Left        157  315


Decision Tree

Our final model was a decision tree model. We split the data into a training and testing set. We then created a graph of the results of the decision tree analysis to visualize the rank order of the variables relative to each other.

dt_mod <- decision_tree(mode = "classification") %>%
  set_engine("rpart")

dt_fit <- workflow() %>%
  add_recipe(churn_recipe) %>%
  add_model(dt_mod) %>%
  fit(data = churn_train)

dt_results <- fit_resamples(dt_mod, churn_recipe, kfolds)

collect_metrics(dt_results)
## # A tibble: 2 × 6
##   .metric  .estimator  mean     n std_err .config             
##   <chr>    <chr>      <dbl> <int>   <dbl> <chr>               
## 1 accuracy binary     0.788     5 0.00399 Preprocessor1_Model1
## 2 roc_auc  binary     0.710     5 0.00529 Preprocessor1_Model1
rpart.plot::rpart.plot(dt_fit$fit$fit$fit)

The model was first tuned and then trained with hyperparameter values.

# create model object with tuning options
dt_mod <- decision_tree(
  mode = "classification",
  cost_complexity = tune(),
  tree_depth = tune(),
  min_n = tune()
) %>% 
  set_engine("rpart")

# create the hyperparameter grid
hyper_grid <- grid_regular(
  cost_complexity(),
  tree_depth(),
  min_n()
)

# create k-fold cross validation object
set.seed(123)
kfolds <- vfold_cv(churn_train, v = 5, strata = Status)

# train our model across the hyper parameter grid
dt_results <- tune_grid(dt_mod, churn_recipe, resamples = kfolds, grid = hyper_grid)

# get best results
show_best(dt_results, metric = "roc_auc", n = 10)
## # A tibble: 10 × 9
##    cost_complexity tree_depth min_n .metric .estimator  mean     n std_err
##              <dbl>      <int> <int> <chr>   <chr>      <dbl> <int>   <dbl>
##  1    0.0000000001          8    40 roc_auc binary     0.815     5 0.00420
##  2    0.00000316            8    40 roc_auc binary     0.815     5 0.00420
##  3    0.0000000001         15    40 roc_auc binary     0.815     5 0.00683
##  4    0.00000316           15    40 roc_auc binary     0.815     5 0.00683
##  5    0.0000000001          8    21 roc_auc binary     0.813     5 0.00570
##  6    0.00000316            8    21 roc_auc binary     0.813     5 0.00570
##  7    0.0000000001         15    21 roc_auc binary     0.800     5 0.00438
##  8    0.00000316           15    21 roc_auc binary     0.800     5 0.00438
##  9    0.0000000001          8     2 roc_auc binary     0.784     5 0.00645
## 10    0.00000316            8     2 roc_auc binary     0.784     5 0.00645
## # ℹ 1 more variable: .config <chr>

From this we selected the best hyperparameter values, final workflow, and fit final workflow across the entire training data.

# get best hyperparameter values
dt_best <- select_best(dt_results, 'roc_auc')

# put together final workflow
dt_final <- workflow() %>%
  add_recipe(churn_recipe) %>%
  add_model(dt_mod) %>%
  finalize_workflow(dt_best)

# fit final workflow across entire training data
dt_final_fit <- dt_final %>%
  fit(data = churn_train)

We can see that the matrix correctly predicts 1331 people to be current customers. 248 are predicted to be current customers, but they left. This is a false positive. 209 customers are predicted to have left, but they are actually current customers. This is a false negative. The matrix correctly predicts 309 customers to have left.

dt_final_fit %>%
   predict(churn_test) %>%
   bind_cols(churn_test %>% select(Status)) %>%
   conf_mat(truth = Status, estimate = .pred_class)
##           Truth
## Prediction Current Left
##    Current    1331  248
##    Left        209  309


Optimal Model: MARS

To determine accuracy, we used the ROC_AUC. ROC is a probability curve and AUC represents the degree of separation. It shows how much the model is capable of distinguishing between classes. The higher the AUC, the better the model is at differentiating between customers that leave Regork and those that are retained. Excellent models have AUC near 1, which means it has a good measure of separability. We determined that the optimal model was the MARS because its ROC_AUC was 0.850. In comparison, the logistic regression model has a 0.845 ROC_AUC, and the decision tree model has a 0.815 ROC_AUC. Additionally, we saw that the generalization error is slightly less than the cross validation error that we see in earlier results.

mars_final %>%
  fit(data = churn_train) %>%
  extract_fit_parsnip() %>%
  vip(10)

The model indicates the top ten factors that are important in determining customer churn. The top five are: tenure, total charges, monthly charges, payment method/electronic check, and online security. However, the relative importance drops off significantly after the first three indicators. Therefore, the most meaningful predictors of Regork customer churn using the MARS model are tenure, total charges, and monthly charges.

Key Takeaways:

Machine learning models are important tools that can be used to guide important business decisions. Although the predictions are not perfect, the MARS model provided a very good estimate for customer churn. This information is useful in developing business strategies for customer retention. However, the model should continue to be run when additional customer information datasets are available to continue refining estimates on customer retention and customer behaviors.


Summary

In terms of relative importance how would you rate the predictors in your model

From our optimal models, we determined that Tenure, Total Charges and Contract Length were consistently the most important factors in building models for datasets that predict customer retention.

Tenure is the most important predictor variable. The data shows that the longer a person stays with a company, the more likely he or she will remain a customer. Conversely, customers with shorter tenures have a much higher churn rate.

Contract length is another important factor. Our analysis indicated that customers with two-year contracts are much more likely to remain customers than those with month-to-month contracts. This relationship is understandable as the longer the period of time someone is locked into a contract, the more likely they are to remain with that company.

Another key factor is total monthly charges. As the total price of internet services increases, so does customer churn. The impact of monthly charges is especially evident for fiber optic customers. This group of customers has the highest churn rate. However, customers with a two-year contract appear to be less sensitive to price.

Collect all the customers from the test dataset that you predict are going to leave

The demographics of customers who will be leaving are those with dependents that have chosen month-to-month contracts. We also understand that customer churn is tied to monthly charges These conclusions appear to be valid as it is understandable that customers with dependents (families) have less money available to spend because they are spending more money supporting their dependents. It also makes sense that they would not wish to be tied to longer-term contracts as their financial situation may not be as stable as customers without dependents. We determined that Regork’s customer churn rate will be approximately 25%.

What is the predicted loss in revenue per month if no action is taken?

Under Regork’s current method of doing business, the company is projected to lose approximately $460,000. Unfortunately, there is no way to predict with certainty that this will actually be the amount lost due to customer churn. However, we do know that Regork will lose a significant percentage of customers based on current data and that this will result in a loss of revenue.

Propose an incentive scheme to your manager to retain these customers

Regork can capitalize on the direct relationship between tenure and retention by offering incentives to customers who sign up for two-year contracts. By making the price point of two- year contracts much lower than shorter length contracts, customers will be much more likely to choose the longer contract. This will also result in lower monthly costs, which is another key indicator for customer retention. Regork can also offer incentives to those month-to-month customers who switch to a two-year contract. Incentives could be in the form of a gift card or offering their first month of service for free.

As Regork’s churn rate is highest in the fiber optic space, it should evaluate the current pricing levels and adjust it downward if possible. As customer churn is directly related to higher monthly bills, offering competitive pricing is critical to customer retention. If lowering the price is not feasible, Regork should look at offering other types of incentives or promotions to retain these customers.

Regork’s retention rate for non-internet customers is extremely high. Therefore, they should evaluate these services and determine what makes their customers so loyal in this space. Any insight from this area might be applied to internet services.

Conclusion

Regork can improve its market share in the telecommunication space by understanding the needs and behaviors of its customers and potential customers. Through the use of machine learning models, we were able to identify that the key factors that drive customer churn are tenure, contract length, and monthly costs. We were also able to identify target customer characteristics and the types of services that have the highest churn rate. By using these predictive models, Regork can develop data-driven strategies to retain existing customers and attract new ones.

Limitations of The Analysis

The main limitation of our analysis is that we could improve the hyperparameter tuning for future predictive models. We could also improve our understanding of our target customer by further evaluation of customer demographics.