Assignment 9

Objective

Using the random forest algorithm from the randomForest package, the model was created using all possible predictors, except for CLIENTNUM, predicting Attrition_Flag.

Data Prep

• 70/30 data split data (10127 observations) into training (70) and validation (30) datasets

• The training data has 7137 observations

• The validation data has 2990 observations

The Model

## 
## Call:
##  randomForest(formula = Attrition_Flag ~ ., data = train, type = "classification",      importance = T) 
##                Type of random forest: classification
##                      Number of trees: 500
## No. of variables tried at each split: 4
## 
##         OOB estimate of  error rate: 3.98%
## Confusion matrix:
##                   Attrited Customer Existing Customer class.error
## Attrited Customer               939               206  0.17991266
## Existing Customer                78              5914  0.01301736

Variable Importance

We see that the variables of most importance are involved with the frequency and magnitude of transactions between each customer.

Model Accuracy

Using the validation data, it is observed that the model is able to:

• Predict attrited customers correctly 93.6% of the time.

• Misclassify existing customers as attrited 6.4% of the time.

## 
##  
##    Cell Contents
## |-------------------------|
## |                       N |
## | Chi-square contribution |
## |           N / Row Total |
## |           N / Col Total |
## |         N / Table Total |
## |-------------------------|
## 
##  
## Total Observations in Table:  2990 
## 
##  
##                   | val$Attrition_Flag 
##  val$predicted_rf | Attrited Customer | Existing Customer |         Row Total | 
## ------------------|-------------------|-------------------|-------------------|
## Attrited Customer |               392 |                27 |               419 | 
##                   |          1558.549 |           299.530 |                   | 
##                   |             0.936 |             0.064 |             0.140 | 
##                   |             0.813 |             0.011 |                   | 
##                   |             0.131 |             0.009 |                   | 
## ------------------|-------------------|-------------------|-------------------|
## Existing Customer |                90 |              2481 |              2571 | 
##                   |           253.999 |            48.815 |                   | 
##                   |             0.035 |             0.965 |             0.860 | 
##                   |             0.187 |             0.989 |                   | 
##                   |             0.030 |             0.830 |                   | 
## ------------------|-------------------|-------------------|-------------------|
##      Column Total |               482 |              2508 |              2990 | 
##                   |             0.161 |             0.839 |                   | 
## ------------------|-------------------|-------------------|-------------------|
## 
## 

Financial Impact

Based on the model and the following assumptions:

• the customer retention program consists of forgiving the annual fee of $100 (on average)

• the value of a customer is $2,300 per year (on average)

• customer retention programs have a 30% success rate

Therefore:

• (2300-100)*.30*392= $258,720 of customer value is retained, due to the 392 correct predictions.

• (2300-100)*.30*27= $17,820 of customer value is missed, due to the 27 incorrect predictions.