Logistic Regression and Random Forest to Determine Credit Card Default
Travis Gubbe
July 17, 2023
In this session, I examine the Credit Card Clients data set found on the UCI Machine Learning Repository website to determine if a person will default on their credit card using logistic regression and random forest.
About the Data Set
The data set can be found on the UCI Machine Learning Repository site at the following link: https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients.
The data contains 24 variables and a total of 30,000 individual instances. The variables are:
- Default Payment (0 = No, 1 = Yes)
- Amount of Given Credit
- Gender (1 = Male, 2 = Female)
- Education (1 = Graduate School, 2 = University, 3 = High School, 4 = Other, 5 = Unknown)
- Marital Status (1 = Married, 2 = Single, 3 = Others)
- Age
- History of Past Payment from April 2005 to September 2005 where -2=no consumption, -1=pay duly, 0=the use of revolving credit, 1=payment delay for one month, 2=payment delay for two months, … 9=payment delay for nine months and above
- Amount of Bill Statement from April 2005 to September 2005
- Amount of Previous Payment April 2005 to September 2005
There is a separate variable for each past payment, bill statement, and previous payment from April to September.
Exploratory Analysis
First, the data set is loaded into a data frame named “credit”. The data frame is also viewed to see the columns and class types.
data <- read.csv("~/R datasets/Credit_Card.csv")
credit = subset(data, select = c("Limit_Bal", "Sex", "Education", "Marriage", "Age", "Pay_Sep",
"Pay_Aug", "Pay_July", "Pay_June", "Pay_May", "Pay_April",
"Bill_Amt_Sep", "Bill_Amt_Aug", "Bill_Amt_July",
"Bill_Amt_June", "Bill_Amt_May", "Bill_Amt_April",
"Pay_Amt_Sep", "Pay_Amt_Aug", "Pay_Amt_July","Pay_Amt_June",
"Pay_Amt_May", "Pay_Amt_April", "Default"))
#Inspect the data frame
head(credit)## Limit_Bal Sex Education Marriage Age Pay_Sep Pay_Aug Pay_July Pay_June
## 1 20000 2 2 1 24 2 2 -1 -1
## 2 120000 2 2 2 26 -1 2 0 0
## 3 90000 2 2 2 34 0 0 0 0
## 4 50000 2 2 1 37 0 0 0 0
## 5 50000 1 2 1 57 -1 0 -1 0
## 6 50000 1 1 2 37 0 0 0 0
## Pay_May Pay_April Bill_Amt_Sep Bill_Amt_Aug Bill_Amt_July Bill_Amt_June
## 1 -2 -2 3913 3102 689 0
## 2 0 2 2682 1725 2682 3272
## 3 0 0 29239 14027 13559 14331
## 4 0 0 46990 48233 49291 28314
## 5 0 0 8617 5670 35835 20940
## 6 0 0 64400 57069 57608 19394
## Bill_Amt_May Bill_Amt_April Pay_Amt_Sep Pay_Amt_Aug Pay_Amt_July Pay_Amt_June
## 1 0 0 0 689 0 0
## 2 3455 3261 0 1000 1000 1000
## 3 14948 15549 1518 1500 1000 1000
## 4 28959 29547 2000 2019 1200 1100
## 5 19146 19131 2000 36681 10000 9000
## 6 19619 20024 2500 1815 657 1000
## Pay_Amt_May Pay_Amt_April Default
## 1 0 0 1
## 2 0 2000 1
## 3 1000 5000 0
## 4 1069 1000 0
## 5 689 679 0
## 6 1000 800 0#Inspect the classes of the data frame
sapply(credit, class)## Limit_Bal Sex Education Marriage Age
## "integer" "integer" "integer" "integer" "integer"
## Pay_Sep Pay_Aug Pay_July Pay_June Pay_May
## "integer" "integer" "integer" "integer" "integer"
## Pay_April Bill_Amt_Sep Bill_Amt_Aug Bill_Amt_July Bill_Amt_June
## "integer" "integer" "integer" "integer" "integer"
## Bill_Amt_May Bill_Amt_April Pay_Amt_Sep Pay_Amt_Aug Pay_Amt_July
## "integer" "integer" "integer" "integer" "integer"
## Pay_Amt_June Pay_Amt_May Pay_Amt_April Default
## "integer" "integer" "integer" "integer"Next, I want to see the amount of missing values and duplicates in the data frame.
sum(is.na(credit))## [1] 0duplicates <- credit%>%duplicated()
duplicates_amount <- duplicates%>%(table)
duplicates_amount## .
## FALSE TRUE
## 29965 35Since there are 35 duplicates in the data, the data frame is filtered to remove the duplicates.
credit <- credit%>%distinct()
#Displays how many duplicates are present in the updated data frame.
duplicates_counts_unique <- credit%>%duplicated()%>%table()
duplicates_counts_unique## .
## FALSE
## 29965Next, the factor variables are converted from their numeric values to their actual names.
credit1 <- data.frame(credit)
head(credit1)## Limit_Bal Sex Education Marriage Age Pay_Sep Pay_Aug Pay_July Pay_June
## 1 20000 2 2 1 24 2 2 -1 -1
## 2 120000 2 2 2 26 -1 2 0 0
## 3 90000 2 2 2 34 0 0 0 0
## 4 50000 2 2 1 37 0 0 0 0
## 5 50000 1 2 1 57 -1 0 -1 0
## 6 50000 1 1 2 37 0 0 0 0
## Pay_May Pay_April Bill_Amt_Sep Bill_Amt_Aug Bill_Amt_July Bill_Amt_June
## 1 -2 -2 3913 3102 689 0
## 2 0 2 2682 1725 2682 3272
## 3 0 0 29239 14027 13559 14331
## 4 0 0 46990 48233 49291 28314
## 5 0 0 8617 5670 35835 20940
## 6 0 0 64400 57069 57608 19394
## Bill_Amt_May Bill_Amt_April Pay_Amt_Sep Pay_Amt_Aug Pay_Amt_July Pay_Amt_June
## 1 0 0 0 689 0 0
## 2 3455 3261 0 1000 1000 1000
## 3 14948 15549 1518 1500 1000 1000
## 4 28959 29547 2000 2019 1200 1100
## 5 19146 19131 2000 36681 10000 9000
## 6 19619 20024 2500 1815 657 1000
## Pay_Amt_May Pay_Amt_April Default
## 1 0 0 1
## 2 0 2000 1
## 3 1000 5000 0
## 4 1069 1000 0
## 5 689 679 0
## 6 1000 800 0#Rename factor variables to their appropriate settings
credit$Sex[credit$Sex %in% "1"] = "Male"
credit$Sex[credit$Sex %in% "2"] = "Female"
credit$Education[credit$Education %in% "1"] = "Grad School"
credit$Education[credit$Education %in% "2"] = "College"
credit$Education[credit$Education %in% "3"] = "High School"
credit$Education[credit$Education %in% "4"] = "Other"
credit$Education[credit$Education %in% "5"] = "Unknown"
credit$Marriage[credit$Marriage %in% "0"] = "Unknown"
credit$Marriage[credit$Marriage %in% "1"] = "Married"
credit$Marriage[credit$Marriage %in% "2"] = "Single"
credit$Marriage[credit$Marriage %in% "3"] = "Other"
credit$Default[credit$Default %in% "0"] = "No"
credit$Default[credit$Default %in% "1"] = "Yes"
#See the change in the variable names
head(credit)## Limit_Bal Sex Education Marriage Age Pay_Sep Pay_Aug Pay_July Pay_June
## 1 20000 Female College Married 24 2 2 -1 -1
## 2 120000 Female College Single 26 -1 2 0 0
## 3 90000 Female College Single 34 0 0 0 0
## 4 50000 Female College Married 37 0 0 0 0
## 5 50000 Male College Married 57 -1 0 -1 0
## 6 50000 Male Grad School Single 37 0 0 0 0
## Pay_May Pay_April Bill_Amt_Sep Bill_Amt_Aug Bill_Amt_July Bill_Amt_June
## 1 -2 -2 3913 3102 689 0
## 2 0 2 2682 1725 2682 3272
## 3 0 0 29239 14027 13559 14331
## 4 0 0 46990 48233 49291 28314
## 5 0 0 8617 5670 35835 20940
## 6 0 0 64400 57069 57608 19394
## Bill_Amt_May Bill_Amt_April Pay_Amt_Sep Pay_Amt_Aug Pay_Amt_July Pay_Amt_June
## 1 0 0 0 689 0 0
## 2 3455 3261 0 1000 1000 1000
## 3 14948 15549 1518 1500 1000 1000
## 4 28959 29547 2000 2019 1200 1100
## 5 19146 19131 2000 36681 10000 9000
## 6 19619 20024 2500 1815 657 1000
## Pay_Amt_May Pay_Amt_April Default
## 1 0 0 Yes
## 2 0 2000 Yes
## 3 1000 5000 No
## 4 1069 1000 No
## 5 689 679 No
## 6 1000 800 NoNext, exploratory tables are made to view the distribution of the data set.
Data Distribution
Next, bar plots and distribution tables are created to see the proportion of the variables. This is done to see if the data is normally distributed. If the data is not normally distributed, it’s advantageous to see how the data is skewed.
#View the bar plots for the amount for each categorical variable
counts_Sex <- table(credit$Sex)
barplot(counts_Sex, col = c("royalblue", "darkorange1"))
#Basic table view of the amount of males and females
table(credit$Sex)##
## Female Male
## 18091 11874#Proportion of each gender in table
prop.table(counts_Sex)##
## Female Male
## 0.6037377 0.3962623counts_Education <- table(credit$Education)
barplot(counts_Education, col = c("brown4", "green3", "mediumpurple2", "slategray3",
"darkgoldenrod2"))
table(credit$Education)##
## College Grad School High School Other Unknown
## 14019 10563 4915 123 345#Proportion of each education level in table
prop.table(counts_Education)##
## College Grad School High School Other Unknown
## 0.467845820 0.352511263 0.164024695 0.004104789 0.011513432counts_Marriage <- table(credit$Marriage)
barplot(counts_Marriage, col = c("magenta2", "Cyan3", "goldenrod"))
table(credit$Marriage)##
## Married Other Single Unknown
## 13643 323 15945 54#Proportion of each marriage status in table
prop.table(counts_Marriage)##
## Married Other Single Unknown
## 0.455297847 0.010779242 0.532120808 0.001802102counts_Default <- table(credit$Default)
barplot(counts_Default, col = c("turquoise2", "sienna1"))
table(credit$Default)##
## No Yes
## 23335 6630prop.table(counts_Default)##
## No Yes
## 0.7787419 0.2212581table.default_gender <- table(credit$Default, credit$Sex)
prop.table(table.default_gender, 2)##
## Female Male
## No 0.7921066 0.7583797
## Yes 0.2078934 0.2416203prop.table(table.default_gender, 1)##
## Female Male
## No 0.614099 0.385901
## Yes 0.567270 0.432730barplot(table.default_gender, col = c("sienna1", "royalblue"), beside = T,
names.arg = c("Female", "Male"))
legend("topright", legend = c("No", "Yes"), fill = c("sienna1", "royalblue"))
ggplot(data = credit, aes(x = Age)) + geom_histogram(fill = "Blue", col = "Grey", bins = 30)
ggplot(data = credit, aes(x = Age)) + geom_histogram(aes(y = ..density..), fill = "Blue", col = "Grey", binwidth = 5)+geom_density(alpha = 0.2, color = "black", fill = "blue")## Warning: The dot-dot notation (`..density..`) was deprecated in ggplot2 3.4.0.
## ℹ Please use `after_stat(density)` instead.
mean(credit$Age)## [1] 35.48797Looking at the created charts and tables, the data has more females than males. In addition, the age distribution is skewed to the right, meaning the data is represented by younger participants. As such, it may be easier to predict credit card default for females or for younger participants compared to males or older participants.
Scaling the Data
Before setting up the prediction model, all variables except for Default (the variable we are trying to predict) are scaled so the data is standardized.
credit1 = credit1 %>% mutate_at(c(0:23), funs(c(scale(.))))## Warning: `funs()` was deprecated in dplyr 0.8.0.
## ℹ Please use a list of either functions or lambdas:
##
## # Simple named list: list(mean = mean, median = median)
##
## # Auto named with `tibble::lst()`: tibble::lst(mean, median)
##
## # Using lambdas list(~ mean(., trim = .2), ~ median(., na.rm = TRUE))head(credit1)## Limit_Bal Sex Education Marriage Age Pay_Sep Pay_Aug
## 1 -1.1362658 0.8101398 0.1857827 -1.0572427 -1.2460567 1.79507537 1.782007
## 2 -0.3656131 0.8101398 0.1857827 0.8584783 -1.0291243 -0.87517053 1.782007
## 3 -0.5968089 0.8101398 0.1857827 0.8584783 -0.1613944 0.01491143 0.110216
## 4 -0.9050700 0.8101398 0.1857827 -1.0572427 0.1640043 0.01491143 0.110216
## 5 -0.9050700 -1.2343136 0.1857827 -1.0572427 2.3333289 -0.87517053 0.110216
## 6 -0.9050700 -1.2343136 -1.0890004 0.8584783 0.1640043 0.01491143 0.110216
## Pay_July Pay_June Pay_May Pay_April Bill_Amt_Sep Bill_Amt_Aug
## 1 -0.6987406 -0.6686308 -1.5328219 -1.4886233 -0.6431063 -0.6479949
## 2 0.1374654 0.1874052 0.2336195 1.9923918 -0.6598187 -0.6673360
## 3 0.1374654 0.1874052 0.2336195 0.2518842 -0.2992746 -0.4945444
## 4 0.1374654 0.1874052 0.2336195 0.2518842 -0.0582829 -0.0140931
## 5 -0.6987406 0.1874052 0.2336195 0.2518842 -0.5792437 -0.6119253
## 6 0.1374654 0.1874052 0.2336195 0.2518842 0.1780793 0.1100157
## Bill_Amt_July Bill_Amt_June Bill_Amt_May Bill_Amt_April Pay_Amt_Sep
## 1 -0.66856007 -0.6730531 -0.6636014 -0.6532534 -0.3421525
## 2 -0.63983063 -0.6222089 -0.6067918 -0.5985149 -0.3421525
## 3 -0.48303680 -0.4503613 -0.4178154 -0.3922509 -0.2505514
## 4 0.03204614 -0.2330771 -0.1874362 -0.1572832 -0.2214659
## 5 -0.16192442 -0.3476629 -0.3487888 -0.3321241 -0.2214659
## 6 0.15193713 -0.3716865 -0.3410114 -0.3171344 -0.1912942
## Pay_Amt_Aug Pay_Amt_July Pay_Amt_June Pay_Amt_May Pay_Amt_April Default
## 1 -0.2272537 -0.2969790 -0.3082477 -0.3143255 -0.29355736 1
## 2 -0.2137633 -0.2402136 -0.2444497 -0.3143255 -0.18111555 1
## 3 -0.1920746 -0.2402136 -0.2444497 -0.2489078 -0.01245282 0
## 4 -0.1695617 -0.2288605 -0.2380699 -0.2443940 -0.23733645 0
## 5 1.3339872 0.2706751 0.2659346 -0.2692527 -0.25538337 0
## 6 -0.1784107 -0.2596841 -0.2444497 -0.2489078 -0.24858064 0Train and Test Sets
Before creating prediction models, training and testing data sets are created. A training data set is a subset of examples used to train the model, while the testing data set is a subset used to test the training model.
#Initializes number generator.
set.seed(123)
#New sample created for the training and testing data sets. The data is split with 75% in training and 25% in testing.
sample <- sample(c(TRUE, FALSE), nrow(credit1), replace = TRUE, prob = c(0.75, 0.25))
train_set <- credit1[sample, ]
test_set <- credit1[!sample, ]Sampling the Data
From the bar plots, it is clear there is an imbalance between those who default and those who did not in the data. This could cause issues in creating a prediction model, which would most likely skew towards predicting much more “No” answers since there are more within the sampled data. To solve this issue, oversampling and undersampling the training set data can be done. Oversampling duplicates random samples from the minority class, while undersampling randomly reduces samples from the majority class. Doing both helps to “even out” the bias and possibly improve the model’s overall performance.
The random oversampling and undersampling is performed below:
credit_balance <- ovun.sample(Default ~., data = train_set, N = nrow(train_set),
p = 0.5, seed = 1, method = "both")$dataNow that the training and testing data sets are created and have been randomly sampled, prediction analysis methods such as logistic regression and random forest can be completed.
Logistic Regression
First, logistic regression is done to find the probability of default for an individual. Logistic regression models the probability that a response variable (Y) belongs to a particular category. This method uses maximum likelihood to fit the model in the range between 0 and 1.
Logistic regression is a classification method great for a yes/no response. A number closer to 1 represents “Yes”, while a number closer to 0 represents “No”.
A logistic regression model is created below, which is then used to predict the probabilities of credit card default for three individuals:
# With Training Set
fit_glm <- glm(Default ~ ., data = credit_balance, family = binomial())
#Displays summary of the logistic regression model.
summary(fit_glm)##
## Call:
## glm(formula = Default ~ ., family = binomial(), data = credit_balance)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -3.374 -1.069 -0.290 1.058 3.373
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -0.21360 0.01524 -14.019 < 2e-16 ***
## Limit_Bal -0.07719 0.01931 -3.998 6.40e-05 ***
## Sex -0.01751 0.01455 -1.204 0.228712
## Education -0.10015 0.01583 -6.327 2.50e-10 ***
## Marriage -0.11434 0.01588 -7.199 6.05e-13 ***
## Age 0.08549 0.01603 5.334 9.62e-08 ***
## Pay_Sep 0.57998 0.01864 31.116 < 2e-16 ***
## Pay_Aug 0.10019 0.02342 4.278 1.88e-05 ***
## Pay_July 0.09912 0.02573 3.853 0.000117 ***
## Pay_June -0.03820 0.02815 -1.357 0.174724
## Pay_May 0.07881 0.02942 2.679 0.007394 **
## Pay_April -0.01194 0.02413 -0.495 0.620786
## Bill_Amt_Sep -0.27420 0.06829 -4.015 5.95e-05 ***
## Bill_Amt_Aug -0.00415 0.08933 -0.046 0.962947
## Bill_Amt_July 0.15211 0.08369 1.818 0.069138 .
## Bill_Amt_June 0.05001 0.08019 0.624 0.532842
## Bill_Amt_May 0.04072 0.07756 0.525 0.599581
## Bill_Amt_April -0.04903 0.05878 -0.834 0.404190
## Pay_Amt_Sep -0.23726 0.03325 -7.135 9.67e-13 ***
## Pay_Amt_Aug -0.32177 0.04357 -7.385 1.52e-13 ***
## Pay_Amt_July -0.03188 0.02520 -1.265 0.205806
## Pay_Amt_June -0.03050 0.02259 -1.350 0.176926
## Pay_Amt_May -0.02117 0.02107 -1.005 0.315120
## Pay_Amt_April -0.01832 0.01930 -0.950 0.342360
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 31159 on 22476 degrees of freedom
## Residual deviance: 27542 on 22453 degrees of freedom
## AIC: 27590
##
## Number of Fisher Scoring iterations: 5#Predicts values in the test set.
pred_probs <- predict.glm(fit_glm, newdata = test_set, type = "response")
#Displays the predictions for a few values.
head(pred_probs)## 2 4 5 8 11 16
## 0.3781745 0.5350573 0.3453484 0.4157203 0.4751291 0.5263390#Sorts predictions into their respective class (0 or 1) depending on their value.
pred <- ifelse(pred_probs<0.5, 0,1)
#Creates and displays the confusion matrix table based on the actual and predicted values.
confusion_table <- table(test_set$Default, pred)
confusion_table## pred
## 0 1
## 0 4264 1584
## 1 605 1035#Creates the confusion matrix statistics for the logistic regression model.
cm_log <- confusionMatrix(confusion_table, mode = "everything")
#Saves the accuracy, precision, and recall values.
log_accuracy = accuracy(test_set$Default, pred)
log_precision = cm_log$byClass['Precision']
log_recall = cm_log$byClass['Recall']
log_pos_precision = cm_log$byClass['Neg Pred Value']
#Prints the accuracy, precision, and recall values.
print(paste("Accuracy: ", round(log_accuracy,3)))## [1] "Accuracy: 0.708"print(paste("Precision: ", round(log_precision,3)))## [1] "Precision: 0.729"print(paste("Recall: ", round(log_recall,3)))## [1] "Recall: 0.876"print(paste("Default Precision: ", round(log_pos_precision,3)))## [1] "Default Precision: 0.631"Once the model was run, the accuracy, precision, and recall were found for the prediction model. Accuracy describes how often the model is correct in its overall prediction. Precision identifies how often the model identifies those who default on their credit card out of all who do so, while recall identifies how often the model correctly identifies those who default on their credit card.Another way of describing precision and recall is precision is a measure of quality, while recall is a measure of quantity.
In the case of the logistic regression model, the accuracy was 70.8%, precision was 72.9%, and recall was 87.6%. The precision for predicting actual default cases correctly was 63.1%. Overall, the logistic regression model was fairly decent at its predicting whether a client would default.
Random Forest
Another prediction model used is random forest. Random forest is a classifying method consisting of many decision trees. By creating a “forest” of decision trees, the classifying model hopes to select it’s best model by running many different decision trees and “takes the majority” to determine classification. To do so, random forest uses out-of-bag sampling.
A random forest model is created to determine the probability of credit card default:
set.seed(123)
#Random Forest for variables. mtry = 5 since there are 24 variables (square root of 24 is close to 5).
fit_rf <- randomForest(factor(Default) ~., mtry = 5, data = credit_balance)
varImpPlot(fit_rf)
#Predicts values in the test set.
predict_rf <- predict(fit_rf, test_set)
#Creates the confusion matrix table for the random forest model.
confusion_table_rf <- table(test_set$Default, predict_rf)
#Creates and displays the confusion matrix statistics for the random forest model.
cm_rf <- confusionMatrix(confusion_table_rf, mode = "everything")
cm_rf## Confusion Matrix and Statistics
##
## predict_rf
## 0 1
## 0 5077 771
## 1 737 903
##
## Accuracy : 0.7986
## 95% CI : (0.7893, 0.8076)
## No Information Rate : 0.7764
## P-Value [Acc > NIR] : 1.704e-06
##
## Kappa : 0.4157
##
## Mcnemar's Test P-Value : 0.3954
##
## Sensitivity : 0.8732
## Specificity : 0.5394
## Pos Pred Value : 0.8682
## Neg Pred Value : 0.5506
## Precision : 0.8682
## Recall : 0.8732
## F1 : 0.8707
## Prevalence : 0.7764
## Detection Rate : 0.6780
## Detection Prevalence : 0.7810
## Balanced Accuracy : 0.7063
##
## 'Positive' Class : 0
## #Saves the accuracy, precision, and recall values.
rf_accuracy = accuracy(test_set$Default, predict_rf)
rf_precision = cm_rf$byClass['Precision']
rf_recall = cm_rf$byClass['Recall']
rf_pos_precision = cm_rf$byClass['Neg Pred Value']
#Prints the accuracy, total precision, recall, and default precision values.
print(paste("Accuracy: ", round(rf_accuracy,3)))## [1] "Accuracy: 0.799"print(paste("Precision: ", round(rf_precision,3)))## [1] "Precision: 0.868"print(paste("Recall: ", round(rf_recall,3)))## [1] "Recall: 0.873"print(paste("Default Precision: ", round(rf_pos_precision,3)))## [1] "Default Precision: 0.551"From the input printed and the plot provided, it is seen that the pay amount and bill amount in September, as well as limit balance are important variables in determining credit card default. It can also be argued age, bill amount in August and bill amount in July are important variables in determining credit card default.
Looking at the confusion matrix, the random forest model’s accuracy was 79.9%, precision was 86.8%, and recall was 87.3%. The precision for predicting actual default cases correctly was 55.1%. Overall, the random forest model was better in its prediction. However, the logistic regression model performed better than the random forest model when predicting actual default cases.
Conclusion
In this project, logistic regression and random forest models were created to predict if an individual would default on their credit card.
First, the data was cleaned for accuracy and manipulated to view distributions and trends in the data. From the tables and plots created, the data had more females than males and was skewed in age, with participants below the age of 40 much more prevalent than participants over the age of 40.
Next, prediction models were created to predict an individual’s chances of defaulting on their credit card. The first model used was a logistic regression model. This model was used to predict if an individual would default on their credit card based on their information. This model is great for predicting a Yes/No classification for individuals. From the model created, three individuals were created with their unique information. In the example above, all three individuals created had a good chance of not defaulting on their credit card.
A random forest model was also created to determine the most important variables in a prediction model, as well as to see the accuracy of the created model. From the results, the random forest model could accurately predict someone not defaulting on their credit card, but had a more difficult time accurately predicting when someone would default on their credit card.
When comparing the two models, the following table was created:
set.caption("Performance for Logistic Regression and Random Forest Models")
data.table = rbind(c(log_accuracy, log_precision, log_recall, log_pos_precision), c(rf_accuracy, rf_precision, rf_recall, rf_pos_precision))
colnames(data.table) = c("Accuracy", "Precision", "Recall", "Default Precision")
rownames(data.table) = c("Logistic Regression", "Random Forest")
pander(data.table)| Accuracy | Precision | Recall | Default Precision | |
|---|---|---|---|---|
| Logistic Regression | 0.7077 | 0.7291 | 0.8757 | 0.6311 |
| Random Forest | 0.7986 | 0.8682 | 0.8732 | 0.5506 |
Overall, it seems the logistic regression model has a higher recall and default precision rate than the random forest model, but does worse than the random forest model in overall precision. This means the logistic regression model incorrectly labels those who end up not defaulting on their credit card more often than the random forest model. Though the accuracy, precision, and recall values seem fairly high for the random forest prediction model, I am concerned with the results for accurately predicting an individual who does default on their credit card.
Though these prediction models are acceptable, there is room for improvement, particularly in accurately predicting client that will default. I believe adding certain variables such as credit score, credit age, and credit card utilization can help improve the prediction models.
Thank you for viewing my project.