Evaluating the Logistic Regression, CART, and Random Forest Models

Srishti Puri

5/4/2021

Question: Evaluate the Logistic Regression, CART, and Random Forest Models to determine which is the best. This referring to the three previous assignments using these models. Be sure to explain why you believe the model you selected as the best is in fact the best.

The three models were separately employed to the same HR dataset to predict loyal and leaving employees.
Individually, each model has been employed and analyzed in detail separately on this Rpubs profile. Besides, the analysis has been run again for this presentation.

Assumptions

To accurately predict loyal employees.
To accurately predict those to leave.
To maintain the lowest error in prediction of loyal as well as of those to leave.
To develop a model which is overall significant.

The Best Model: Random Forest

After considering all the assumptions and results, it is concluded that the Random Forest model is the most suitable as:
- it expresses the highest model-wise accuracy.
- it expresses the highest overall accuracy.
- on comparison, shows highest accuracy in predicting loyal employees as well as in predicting those to leave.
- on comparison, commits the lowest amount of error in predicting loyal employees as well as those to leave.

Overall Accuracy of the Models

Comparision

Employees Loyal - Correctly Predicted

Employees Left - Correctly Predicted

Employees Loyal - Incorrectly Predicted

Employees Left - Incorrectly Predicted

Confusion Matrix: Logistic Regression

87.2% of the employees were accurately predicted as loyal.
12.8% of the loyal employees were incorrectly predicted to leave.
57.8% of the employees were accurately predicted to leave.
42.2% of those of left were inaccurately predicted as loyal.

CrossTable(valid_lr$left , valid_lr$predicted_left01)

## 
##  
##    Cell Contents
## |-------------------------|
## |                       N |
## | Chi-square contribution |
## |           N / Row Total |
## |           N / Col Total |
## |         N / Table Total |
## |-------------------------|
## 
##  
## Total Observations in Table:  4453 
## 
##  
##               | valid_lr$predicted_left01 
## valid_lr$left |         0 |         1 | Row Total | 
## --------------|-----------|-----------|-----------|
##             0 |      2915 |       468 |      3383 | 
##               |    46.282 |   146.330 |           | 
##               |     0.862 |     0.138 |     0.760 | 
##               |     0.862 |     0.437 |           | 
##               |     0.655 |     0.105 |           | 
## --------------|-----------|-----------|-----------|
##             1 |       468 |       602 |      1070 | 
##               |   146.330 |   462.650 |           | 
##               |     0.437 |     0.563 |     0.240 | 
##               |     0.138 |     0.563 |           | 
##               |     0.105 |     0.135 |           | 
## --------------|-----------|-----------|-----------|
##  Column Total |      3383 |      1070 |      4453 | 
##               |     0.760 |     0.240 |           | 
## --------------|-----------|-----------|-----------|
## 
##

Confusion Matrix: Classification and Regression Trees (CART)

98.9% of the loyal employees were accurately predicted.
The model inaccurately predicted 1.1% of the loyal employees to leave.
The model accurately predicted 91.2% of the employee to leave.
Yet, it incorrectly predicted 8.8% of the employees who left as loyal.
The model is 96.9% (74.8+22.1) of the times correct.
The model is 3% (2.1+0.9) of the times incorrect.

## 
##  
##    Cell Contents
## |-------------------------|
## |                       N |
## | Chi-square contribution |
## |           N / Row Total |
## |           N / Col Total |
## |         N / Table Total |
## |-------------------------|
## 
##  
## Total Observations in Table:  4482 
## 
##  
##              | val_ca$left_predicted 
##  val_ca$left |         0 |         1 | Row Total | 
## -------------|-----------|-----------|-----------|
##            0 |      3368 |        38 |      3406 | 
##              |   207.503 |   702.506 |           | 
##              |     0.989 |     0.011 |     0.760 | 
##              |     0.973 |     0.037 |           | 
##              |     0.751 |     0.008 |           | 
## -------------|-----------|-----------|-----------|
##            1 |        92 |       984 |      1076 | 
##              |   656.837 |  2223.733 |           | 
##              |     0.086 |     0.914 |     0.240 | 
##              |     0.027 |     0.963 |           | 
##              |     0.021 |     0.220 |           | 
## -------------|-----------|-----------|-----------|
## Column Total |      3460 |      1022 |      4482 | 
##              |     0.772 |     0.228 |           | 
## -------------|-----------|-----------|-----------|
## 
##

##           CP nsplit rel error    xerror        xstd
## 1 0.25651303      0 1.0000000 1.0000000 0.017484778
## 2 0.18056112      1 0.7434870 0.7434870 0.015666232
## 3 0.07314629      3 0.3823647 0.3823647 0.011804695
## 4 0.05170341      5 0.2360721 0.2372745 0.009473482
## 5 0.03486974      6 0.1843687 0.1871743 0.008466911
## 6 0.01883768      7 0.1494990 0.1523046 0.007670612
## 7 0.01082164      8 0.1306613 0.1358717 0.007259632
## 8 0.01000000      9 0.1198397 0.1298597 0.007102431

Variable importance:

##                       ct1.variable.importance
## satisfaction_level                2140.452565
## number_project                    1068.545707
## average_montly_hours              1044.670670
## last_evaluation                   1039.025614
## time_spend_company                 806.168440
## Work_accident                       24.640791
## promotion_last_5years                7.878501
## sales                                1.673746
## salary                               1.575700

Summary of the validation dataset:

##  satisfaction_level last_evaluation  number_project  average_montly_hours
##  Min.   :0.0900     Min.   :0.3600   Min.   :2.000   Min.   : 96.0       
##  1st Qu.:0.4400     1st Qu.:0.5600   1st Qu.:3.000   1st Qu.:156.0       
##  Median :0.6400     Median :0.7200   Median :4.000   Median :201.0       
##  Mean   :0.6103     Mean   :0.7157   Mean   :3.799   Mean   :201.7       
##  3rd Qu.:0.8200     3rd Qu.:0.8700   3rd Qu.:5.000   3rd Qu.:246.0       
##  Max.   :1.0000     Max.   :1.0000   Max.   :7.000   Max.   :310.0       
##  time_spend_company Work_accident    left     promotion_last_5years
##  Min.   : 2.000     Min.   :0.0000   0:3406   Min.   :0.00000      
##  1st Qu.: 3.000     1st Qu.:0.0000   1:1076   1st Qu.:0.00000      
##  Median : 3.000     Median :0.0000            Median :0.00000      
##  Mean   : 3.547     Mean   :0.1439            Mean   :0.02253      
##  3rd Qu.: 4.000     3rd Qu.:0.0000            3rd Qu.:0.00000      
##  Max.   :10.000     Max.   :1.0000            Max.   :1.00000      
##     sales              salary          left_predicted
##  Length:4482        Length:4482        0:3460        
##  Class :character   Class :character   1:1022        
##  Mode  :character   Mode  :character                 
##                                                      
##                                                      
##

Confusion Matrix: Random Forest

99.9% of the loyal employees were predicted correctly.
0.1% of the loyal employees were inaccurately predicted to leave.
96.6% of the employees were correctly predicted to leave.
3.4% of those to leave were inaccurately predicted as loyal.
The model is 99.1% of the times correct.
The model 0.9% of the times incorrect.

## 
##  
##    Cell Contents
## |-------------------------|
## |                       N |
## | Chi-square contribution |
## |           N / Row Total |
## |           N / Col Total |
## |         N / Table Total |
## |-------------------------|
## 
##  
## Total Observations in Table:  4438 
## 
##  
##               | valid_rf$predicted_left 
## valid_rf$left |         0 |         1 | Row Total | 
## --------------|-----------|-----------|-----------|
##             0 |      3407 |         4 |      3411 | 
##               |   223.279 |   763.685 |           | 
##               |     0.999 |     0.001 |     0.769 | 
##               |     0.992 |     0.004 |           | 
##               |     0.768 |     0.001 |           | 
## --------------|-----------|-----------|-----------|
##             1 |        27 |      1000 |      1027 | 
##               |   741.581 |  2536.444 |           | 
##               |     0.026 |     0.974 |     0.231 | 
##               |     0.008 |     0.996 |           | 
##               |     0.006 |     0.225 |           | 
## --------------|-----------|-----------|-----------|
##  Column Total |      3434 |      1004 |      4438 | 
##               |     0.774 |     0.226 |           | 
## --------------|-----------|-----------|-----------|
## 
##

Variable importance:

The MeanDecreaseGini coefficient measures of how each variable contributes to the homogeneity of the nodes and leaves in the resulting random forest.
Higher the value of the mean decrease Gini score, higher the importance of the variable in the model.

##                                  0           1 MeanDecreaseAccuracy
## satisfaction_level    5.089311e-02 0.626768281         0.1892740331
## last_evaluation       3.848014e-03 0.445871093         0.1100990715
## number_project        1.568435e-02 0.464747244         0.1236116423
## average_montly_hours  1.662821e-02 0.417991437         0.1130440948
## time_spend_company    1.148732e-02 0.361266592         0.0955090368
## Work_accident         4.441186e-04 0.007008677         0.0020235652
## promotion_last_5years 7.065715e-05 0.001245845         0.0003530784
## sales                 1.751556e-03 0.015747150         0.0051128282
## salary                1.419513e-03 0.011228851         0.0037755517
##                       MeanDecreaseGini
## satisfaction_level         1352.210213
## last_evaluation             446.219785
## number_project              684.555539
## average_montly_hours        552.098742
## time_spend_company          694.736443
## Work_accident                19.090172
## promotion_last_5years         2.552582
## sales                        63.867698
## salary                       30.124421

Summary of left in validation dataset:

##    0    1 
## 3411 1027

0 represents loyal.
1 represents left.