Data Set Summary:

The data set Credit Approval is used in credit risk analysis to develop models that can accurately predict credit card approval or denial based on the applicant’s characteristics and financial history. The data set is also used to evaluate the performance of various predictive models and to identify factors that are the most predictive of credit approval.There are a total of 690 observations in the data set.

The data set contains the following variables: A1: Categorical variable Gender (0, 1). A2: Numerical variable Age. A3: Numerical variable Debt. A4: Categorical variable indicating Marital Status (0,1). A5: Categorical variable Bank Customer (0,1). A6: Categorical variable Occupation. A7: Categorical variable Indicating Ethnicity. A8: Numerical variable Years Employed. A9: Categorical variable Prior Default (0,1) A10: Categorical variable Employed (0,1) A11: Numerical variable Credit Score. A12: Categorical variable Drivers License. A13: Categorical variable Citizenship. A14: Numerical variable Zip Code. A15: Numerical variable Income. A16: Binary response variable indicating whether the credit card application was Approved (0,1).

Research Question:

Can a multiple logistic regression model be used to predict whether a credit card application will be approved or not, based on the applicant’s demographic and financial information?

Exploratory Analysis:

First pairwise scatter plots are created to find potential issues with normality in numerical predictor variables. All four of the scatter plots show issues, so a closer look is needed.

Because there appears to be issues with the scatter plots, histograms are created for each of the variables. Age, YearsEmployed, and Debt, all show varying amounts of skewness, whereas Income seems to be suffering from a grouping problem.

To fix the issues with the histograms, the cut function is used to group the variables. This will help normalize the histograms distributions.

Building the Multiple Logistic Regression Models:

Starting off, a full model is built with some non-significant variables removed. The model gives the summary of the inferential statistics and shows the p-values of each variable. This can be used to find variables that are not statistically significant.

Summary of inferential statistics of the full model
Estimate Std. Error z value Pr(>|z|)
(Intercept) -1.3451043 1.2590670 -1.0683342 0.2853698
grp.Debt0.1-0.6 -2.0457678 0.8428722 -2.4271388 0.0152184
grp.Debt0.61-1.2 -3.3892800 0.9200526 -3.6837893 0.0002298
grp.Debt1.21-5.0 -3.1604617 0.8380558 -3.7711831 0.0001625
grp.Debt5.1-10.0 -2.5166373 0.8679872 -2.8993943 0.0037388
grp.Debt10.1-40 -3.0749208 0.8553437 -3.5949533 0.0003244
grp.Age17-21 0.2710334 0.9598219 0.2823789 0.7776530
grp.Age21-26 0.2339748 0.9414384 0.2485290 0.8037251
grp.Age26-35 -0.3188429 0.9510212 -0.3352637 0.7374262
grp.Age35-50 0.2373826 0.9625368 0.2466218 0.8052009
grp.Age50-100 -0.4443699 1.0449929 -0.4252373 0.6706637
grp.Income0.1-5 -2.1327787 0.7929849 -2.6895580 0.0071547
grp.Income5-10 0.7716000 0.8142668 0.9476010 0.3433326
grp.Income10-300 -0.1033976 0.3791983 -0.2726743 0.7851036
grp.Income300-5000 0.9837330 0.3782887 2.6004818 0.0093093
grp.Income5000+ 3.0404111 1.0500793 2.8954111 0.0037866
Gender 0.0081213 0.2975490 0.0272939 0.9782253
BankCustomer 0.8959578 0.3289897 2.7233608 0.0064621
EthnicityBlack 0.8362797 0.5464358 1.5304261 0.1259113
EthnicityLatino -0.8261380 0.8676858 -0.9521166 0.3410379
EthnicityOther 1.0941315 0.8719407 1.2548233 0.2095429
EthnicityWhite 0.4714134 0.4707245 1.0014636 0.3166027
grp.YearsEmployed0.1-1.0 -0.8413427 0.6809067 -1.2356210 0.2165995
grp.YearsEmployed3.0-5.0 0.2886461 0.7262822 0.3974297 0.6910506
grp.YearsEmployed5.0-10.0 0.3594689 0.7789135 0.4615004 0.6444396
grp.YearsEmployed10.1-50.0 0.8506472 0.8222453 1.0345419 0.3008829
grp.YearsEmployed50+ -0.2292317 0.8587276 -0.2669434 0.7895127
PriorDefault 3.5527489 0.3549968 10.0078345 0.0000000
Employed 0.7351539 0.3736206 1.9676484 0.0491085
CreditScore 0.1022253 0.0557785 1.8327008 0.0668471
DriversLicense 0.0176826 0.2804494 0.0630510 0.9497259

A reduced model is now created with non-significant variables taken out manually.

Summary of inferential statistics of the reduced model
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.4433921 0.5461622 -0.8118323 0.4168879
grp.Income0.1-5 -3.3044573 0.6708738 -4.9256020 0.0000008
grp.Income5-10 -1.5894303 0.6120379 -2.5969476 0.0094056
grp.Income10-300 -0.9974284 0.2928437 -3.4060097 0.0006592
grp.Income300-5000 0.3354653 0.2649959 1.2659263 0.2055395
grp.Income5000+ 2.5665203 0.7929171 3.2368077 0.0012087
grp.Debt0.1-0.6 -1.0233252 0.5792296 -1.7667003 0.0772784
grp.Debt0.61-1.2 -1.4984775 0.6135070 -2.4424783 0.0145868
grp.Debt1.21-5.0 -1.1917078 0.5449017 -2.1870142 0.0287415
grp.Debt5.1-10.0 -0.4199398 0.5722382 -0.7338549 0.4630372
grp.Debt10.1-40 -0.6843711 0.5778427 -1.1843554 0.2362724
CreditScore 0.1956628 0.0512145 3.8204552 0.0001332
Employed 1.4801347 0.2998444 4.9363430 0.0000008
BankCustomer 0.5344925 0.2433127 2.1967305 0.0280397

Next a model using automatic variable selection is performed to to select the most relevant variables from the set of predictor variables.

Summary of inferential statistics of the final model
Estimate Std. Error z value Pr(>|z|)
(Intercept) -1.3172370 0.8513798 -1.5471790 0.1218201
grp.Income0.1-5 -1.9875260 0.7764240 -2.5598461 0.0104719
grp.Income5-10 0.8072470 0.8024122 1.0060253 0.3144034
grp.Income10-300 -0.1113254 0.3772895 -0.2950662 0.7679433
grp.Income300-5000 0.9935654 0.3731707 2.6624959 0.0077564
grp.Income5000+ 2.9841562 1.0523408 2.8357317 0.0045721
grp.Debt0.1-0.6 -2.1228736 0.8259741 -2.5701455 0.0101656
grp.Debt0.61-1.2 -3.4516563 0.8932555 -3.8641311 0.0001115
grp.Debt1.21-5.0 -3.2520806 0.8090601 -4.0195785 0.0000583
grp.Debt5.1-10.0 -2.5254756 0.8460892 -2.9848811 0.0028369
grp.Debt10.1-40 -3.0766593 0.8400838 -3.6623244 0.0002499
CreditScore 0.0972506 0.0553596 1.7567058 0.0789680
Employed 0.7095433 0.3725014 1.9048073 0.0568051
BankCustomer 0.8509058 0.3222964 2.6401342 0.0082873
PriorDefault 3.5192042 0.3451889 10.1950089 0.0000000
grp.YearsEmployed0.1-1.0 -0.7288612 0.6790085 -1.0734198 0.2830828
grp.YearsEmployed3.0-5.0 0.3838400 0.7230751 0.5308439 0.5955270
grp.YearsEmployed5.0-10.0 0.3731764 0.7763964 0.4806520 0.6307639
grp.YearsEmployed10.1-50.0 0.7432616 0.8207288 0.9056117 0.3651414
grp.YearsEmployed50+ -0.2614599 0.8467595 -0.3087771 0.7574911
EthnicityBlack 0.9229021 0.5337120 1.7292135 0.0837709
EthnicityLatino -0.8316552 0.8527065 -0.9753124 0.3294053
EthnicityOther 1.2276316 0.8707288 1.4098897 0.1585723
EthnicityWhite 0.5613195 0.4641797 1.2092719 0.2265584

The table below compares the global goodness-of-fit statistics for the three models. The final model has a slightly higher Deviance and lower AIC than the full model, indicating that it has a slightly worse fit but better predictive performance. But, since the automatic selection has a much lower AIC and only a slightly higher deviance than the others, it is chosen to be the final model for the regression.

Comparison of global goodness-of-fit statistics
Deviance.residual Null.Deviance.Residual AIC
full.model 397.5970 948.1551 459.5970
reduced.model 635.0433 948.1551 663.0433
final.model 401.8565 948.1551 449.8565

Odds Ratio:

Below an odds ratio is performed for a more practical interpretation. the group variable Income has 6 categories with the base being 0. grp.Income 0.1-5 has an odds ratio of 0.1388257, which means that the odds of being approved for this income group are about 86% lower than the baseline group. While grp.Income5000+ has an odds ratio of 19.7698140, which means that the odds of being approved for a credit card are about 20 times higher than the base group. The same general principal can be applied to the rest of the variables listed in the summary, with debt resulting in a decrease in credit card approval while the variables Credit Score and Employed positively increase approval rates. However, the most influential variable is PriorDefault with people having no prior default being 34 times more likely to be approved for a credit card than those who have defaulted.

Summary Stats with Odds Ratios
Estimate Std. Error z value Pr(>|z|) odds.ratio
(Intercept) -1.3172370 0.8513798 -1.5471790 0.1218201 0.2678744
grp.Income0.1-5 -1.9875260 0.7764240 -2.5598461 0.0104719 0.1370340
grp.Income5-10 0.8072470 0.8024122 1.0060253 0.3144034 2.2417279
grp.Income10-300 -0.1113254 0.3772895 -0.2950662 0.7679433 0.8946476
grp.Income300-5000 0.9935654 0.3731707 2.6624959 0.0077564 2.7008469
grp.Income5000+ 2.9841562 1.0523408 2.8357317 0.0045721 19.7698140
grp.Debt0.1-0.6 -2.1228736 0.8259741 -2.5701455 0.0101656 0.1196872
grp.Debt0.61-1.2 -3.4516563 0.8932555 -3.8641311 0.0001115 0.0316931
grp.Debt1.21-5.0 -3.2520806 0.8090601 -4.0195785 0.0000583 0.0386936
grp.Debt5.1-10.0 -2.5254756 0.8460892 -2.9848811 0.0028369 0.0800202
grp.Debt10.1-40 -3.0766593 0.8400838 -3.6623244 0.0002499 0.0461131
CreditScore 0.0972506 0.0553596 1.7567058 0.0789680 1.1021365
Employed 0.7095433 0.3725014 1.9048073 0.0568051 2.0330626
BankCustomer 0.8509058 0.3222964 2.6401342 0.0082873 2.3417670
PriorDefault 3.5192042 0.3451889 10.1950089 0.0000000 33.7575520
grp.YearsEmployed0.1-1.0 -0.7288612 0.6790085 -1.0734198 0.2830828 0.4824581
grp.YearsEmployed3.0-5.0 0.3838400 0.7230751 0.5308439 0.5955270 1.4679106
grp.YearsEmployed5.0-10.0 0.3731764 0.7763964 0.4806520 0.6307639 1.4523405
grp.YearsEmployed10.1-50.0 0.7432616 0.8207288 0.9056117 0.3651414 2.1027827
grp.YearsEmployed50+ -0.2614599 0.8467595 -0.3087771 0.7574911 0.7699267
EthnicityBlack 0.9229021 0.5337120 1.7292135 0.0837709 2.5165832
EthnicityLatino -0.8316552 0.8527065 -0.9753124 0.3294053 0.4353281
EthnicityOther 1.2276316 0.8707288 1.4098897 0.1585723 3.4131362
EthnicityWhite 0.5613195 0.4641797 1.2092719 0.2265584 1.7529840

Conclusion:

Performing a multiple logistic regression on the data set Credit does provide important information on the relationship between credit card approval and multiple predictor variables. Despite the initial histograms being skewed, regrouping each variable allowed for a more normal distribution. The automatic variable selection performed the best in the global goodness-of-fit test, making it the final model used. Additionally, the odds ratio provided very useful information in approval prediction, with PriorDefault being the best predictor of credit card approval. Overall, multiple logistic regression is a valuable tool for analyzing complex data sets and can provide insights into the factors that influence categorical outcomes.