The data set Credit Approval is used in credit risk analysis to develop models that can accurately predict credit card approval or denial based on the applicant’s characteristics and financial history. The data set is also used to evaluate the performance of various predictive models and to identify factors that are the most predictive of credit approval.There are a total of 690 observations in the data set.
The data set contains the following variables: A1: Categorical variable Gender (0, 1). A2: Numerical variable Age. A3: Numerical variable Debt. A4: Categorical variable indicating Marital Status (0,1). A5: Categorical variable Bank Customer (0,1). A6: Categorical variable Occupation. A7: Categorical variable Indicating Ethnicity. A8: Numerical variable Years Employed. A9: Categorical variable Prior Default (0,1) A10: Categorical variable Employed (0,1) A11: Numerical variable Credit Score. A12: Categorical variable Drivers License. A13: Categorical variable Citizenship. A14: Numerical variable Zip Code. A15: Numerical variable Income. A16: Binary response variable indicating whether the credit card application was Approved (0,1).
Can a multiple logistic regression model be used to predict whether a credit card application will be approved or not, based on the applicant’s demographic and financial information?
First pairwise scatter plots are created to find potential issues with normality in numerical predictor variables. All four of the scatter plots show issues, so a closer look is needed.
Because there appears to be issues with the scatter plots, histograms
are created for each of the variables. Age, YearsEmployed, and Debt, all
show varying amounts of skewness, whereas Income seems to be suffering
from a grouping problem.
To fix the issues with the histograms, the cut function is used to group the variables. This will help normalize the histograms distributions.
Starting off, a full model is built with some non-significant variables removed. The model gives the summary of the inferential statistics and shows the p-values of each variable. This can be used to find variables that are not statistically significant.
| Estimate | Std. Error | z value | Pr(>|z|) | |
|---|---|---|---|---|
| (Intercept) | -1.3451043 | 1.2590670 | -1.0683342 | 0.2853698 |
| grp.Debt0.1-0.6 | -2.0457678 | 0.8428722 | -2.4271388 | 0.0152184 |
| grp.Debt0.61-1.2 | -3.3892800 | 0.9200526 | -3.6837893 | 0.0002298 |
| grp.Debt1.21-5.0 | -3.1604617 | 0.8380558 | -3.7711831 | 0.0001625 |
| grp.Debt5.1-10.0 | -2.5166373 | 0.8679872 | -2.8993943 | 0.0037388 |
| grp.Debt10.1-40 | -3.0749208 | 0.8553437 | -3.5949533 | 0.0003244 |
| grp.Age17-21 | 0.2710334 | 0.9598219 | 0.2823789 | 0.7776530 |
| grp.Age21-26 | 0.2339748 | 0.9414384 | 0.2485290 | 0.8037251 |
| grp.Age26-35 | -0.3188429 | 0.9510212 | -0.3352637 | 0.7374262 |
| grp.Age35-50 | 0.2373826 | 0.9625368 | 0.2466218 | 0.8052009 |
| grp.Age50-100 | -0.4443699 | 1.0449929 | -0.4252373 | 0.6706637 |
| grp.Income0.1-5 | -2.1327787 | 0.7929849 | -2.6895580 | 0.0071547 |
| grp.Income5-10 | 0.7716000 | 0.8142668 | 0.9476010 | 0.3433326 |
| grp.Income10-300 | -0.1033976 | 0.3791983 | -0.2726743 | 0.7851036 |
| grp.Income300-5000 | 0.9837330 | 0.3782887 | 2.6004818 | 0.0093093 |
| grp.Income5000+ | 3.0404111 | 1.0500793 | 2.8954111 | 0.0037866 |
| Gender | 0.0081213 | 0.2975490 | 0.0272939 | 0.9782253 |
| BankCustomer | 0.8959578 | 0.3289897 | 2.7233608 | 0.0064621 |
| EthnicityBlack | 0.8362797 | 0.5464358 | 1.5304261 | 0.1259113 |
| EthnicityLatino | -0.8261380 | 0.8676858 | -0.9521166 | 0.3410379 |
| EthnicityOther | 1.0941315 | 0.8719407 | 1.2548233 | 0.2095429 |
| EthnicityWhite | 0.4714134 | 0.4707245 | 1.0014636 | 0.3166027 |
| grp.YearsEmployed0.1-1.0 | -0.8413427 | 0.6809067 | -1.2356210 | 0.2165995 |
| grp.YearsEmployed3.0-5.0 | 0.2886461 | 0.7262822 | 0.3974297 | 0.6910506 |
| grp.YearsEmployed5.0-10.0 | 0.3594689 | 0.7789135 | 0.4615004 | 0.6444396 |
| grp.YearsEmployed10.1-50.0 | 0.8506472 | 0.8222453 | 1.0345419 | 0.3008829 |
| grp.YearsEmployed50+ | -0.2292317 | 0.8587276 | -0.2669434 | 0.7895127 |
| PriorDefault | 3.5527489 | 0.3549968 | 10.0078345 | 0.0000000 |
| Employed | 0.7351539 | 0.3736206 | 1.9676484 | 0.0491085 |
| CreditScore | 0.1022253 | 0.0557785 | 1.8327008 | 0.0668471 |
| DriversLicense | 0.0176826 | 0.2804494 | 0.0630510 | 0.9497259 |
A reduced model is now created with non-significant variables taken out manually.
| Estimate | Std. Error | z value | Pr(>|z|) | |
|---|---|---|---|---|
| (Intercept) | -0.4433921 | 0.5461622 | -0.8118323 | 0.4168879 |
| grp.Income0.1-5 | -3.3044573 | 0.6708738 | -4.9256020 | 0.0000008 |
| grp.Income5-10 | -1.5894303 | 0.6120379 | -2.5969476 | 0.0094056 |
| grp.Income10-300 | -0.9974284 | 0.2928437 | -3.4060097 | 0.0006592 |
| grp.Income300-5000 | 0.3354653 | 0.2649959 | 1.2659263 | 0.2055395 |
| grp.Income5000+ | 2.5665203 | 0.7929171 | 3.2368077 | 0.0012087 |
| grp.Debt0.1-0.6 | -1.0233252 | 0.5792296 | -1.7667003 | 0.0772784 |
| grp.Debt0.61-1.2 | -1.4984775 | 0.6135070 | -2.4424783 | 0.0145868 |
| grp.Debt1.21-5.0 | -1.1917078 | 0.5449017 | -2.1870142 | 0.0287415 |
| grp.Debt5.1-10.0 | -0.4199398 | 0.5722382 | -0.7338549 | 0.4630372 |
| grp.Debt10.1-40 | -0.6843711 | 0.5778427 | -1.1843554 | 0.2362724 |
| CreditScore | 0.1956628 | 0.0512145 | 3.8204552 | 0.0001332 |
| Employed | 1.4801347 | 0.2998444 | 4.9363430 | 0.0000008 |
| BankCustomer | 0.5344925 | 0.2433127 | 2.1967305 | 0.0280397 |
Next a model using automatic variable selection is performed to to select the most relevant variables from the set of predictor variables.
| Estimate | Std. Error | z value | Pr(>|z|) | |
|---|---|---|---|---|
| (Intercept) | -1.3172370 | 0.8513798 | -1.5471790 | 0.1218201 |
| grp.Income0.1-5 | -1.9875260 | 0.7764240 | -2.5598461 | 0.0104719 |
| grp.Income5-10 | 0.8072470 | 0.8024122 | 1.0060253 | 0.3144034 |
| grp.Income10-300 | -0.1113254 | 0.3772895 | -0.2950662 | 0.7679433 |
| grp.Income300-5000 | 0.9935654 | 0.3731707 | 2.6624959 | 0.0077564 |
| grp.Income5000+ | 2.9841562 | 1.0523408 | 2.8357317 | 0.0045721 |
| grp.Debt0.1-0.6 | -2.1228736 | 0.8259741 | -2.5701455 | 0.0101656 |
| grp.Debt0.61-1.2 | -3.4516563 | 0.8932555 | -3.8641311 | 0.0001115 |
| grp.Debt1.21-5.0 | -3.2520806 | 0.8090601 | -4.0195785 | 0.0000583 |
| grp.Debt5.1-10.0 | -2.5254756 | 0.8460892 | -2.9848811 | 0.0028369 |
| grp.Debt10.1-40 | -3.0766593 | 0.8400838 | -3.6623244 | 0.0002499 |
| CreditScore | 0.0972506 | 0.0553596 | 1.7567058 | 0.0789680 |
| Employed | 0.7095433 | 0.3725014 | 1.9048073 | 0.0568051 |
| BankCustomer | 0.8509058 | 0.3222964 | 2.6401342 | 0.0082873 |
| PriorDefault | 3.5192042 | 0.3451889 | 10.1950089 | 0.0000000 |
| grp.YearsEmployed0.1-1.0 | -0.7288612 | 0.6790085 | -1.0734198 | 0.2830828 |
| grp.YearsEmployed3.0-5.0 | 0.3838400 | 0.7230751 | 0.5308439 | 0.5955270 |
| grp.YearsEmployed5.0-10.0 | 0.3731764 | 0.7763964 | 0.4806520 | 0.6307639 |
| grp.YearsEmployed10.1-50.0 | 0.7432616 | 0.8207288 | 0.9056117 | 0.3651414 |
| grp.YearsEmployed50+ | -0.2614599 | 0.8467595 | -0.3087771 | 0.7574911 |
| EthnicityBlack | 0.9229021 | 0.5337120 | 1.7292135 | 0.0837709 |
| EthnicityLatino | -0.8316552 | 0.8527065 | -0.9753124 | 0.3294053 |
| EthnicityOther | 1.2276316 | 0.8707288 | 1.4098897 | 0.1585723 |
| EthnicityWhite | 0.5613195 | 0.4641797 | 1.2092719 | 0.2265584 |
The table below compares the global goodness-of-fit statistics for the three models. The final model has a slightly higher Deviance and lower AIC than the full model, indicating that it has a slightly worse fit but better predictive performance. But, since the automatic selection has a much lower AIC and only a slightly higher deviance than the others, it is chosen to be the final model for the regression.
| Deviance.residual | Null.Deviance.Residual | AIC | |
|---|---|---|---|
| full.model | 397.5970 | 948.1551 | 459.5970 |
| reduced.model | 635.0433 | 948.1551 | 663.0433 |
| final.model | 401.8565 | 948.1551 | 449.8565 |
Below an odds ratio is performed for a more practical interpretation. the group variable Income has 6 categories with the base being 0. grp.Income 0.1-5 has an odds ratio of 0.1388257, which means that the odds of being approved for this income group are about 86% lower than the baseline group. While grp.Income5000+ has an odds ratio of 19.7698140, which means that the odds of being approved for a credit card are about 20 times higher than the base group. The same general principal can be applied to the rest of the variables listed in the summary, with debt resulting in a decrease in credit card approval while the variables Credit Score and Employed positively increase approval rates. However, the most influential variable is PriorDefault with people having no prior default being 34 times more likely to be approved for a credit card than those who have defaulted.
| Estimate | Std. Error | z value | Pr(>|z|) | odds.ratio | |
|---|---|---|---|---|---|
| (Intercept) | -1.3172370 | 0.8513798 | -1.5471790 | 0.1218201 | 0.2678744 |
| grp.Income0.1-5 | -1.9875260 | 0.7764240 | -2.5598461 | 0.0104719 | 0.1370340 |
| grp.Income5-10 | 0.8072470 | 0.8024122 | 1.0060253 | 0.3144034 | 2.2417279 |
| grp.Income10-300 | -0.1113254 | 0.3772895 | -0.2950662 | 0.7679433 | 0.8946476 |
| grp.Income300-5000 | 0.9935654 | 0.3731707 | 2.6624959 | 0.0077564 | 2.7008469 |
| grp.Income5000+ | 2.9841562 | 1.0523408 | 2.8357317 | 0.0045721 | 19.7698140 |
| grp.Debt0.1-0.6 | -2.1228736 | 0.8259741 | -2.5701455 | 0.0101656 | 0.1196872 |
| grp.Debt0.61-1.2 | -3.4516563 | 0.8932555 | -3.8641311 | 0.0001115 | 0.0316931 |
| grp.Debt1.21-5.0 | -3.2520806 | 0.8090601 | -4.0195785 | 0.0000583 | 0.0386936 |
| grp.Debt5.1-10.0 | -2.5254756 | 0.8460892 | -2.9848811 | 0.0028369 | 0.0800202 |
| grp.Debt10.1-40 | -3.0766593 | 0.8400838 | -3.6623244 | 0.0002499 | 0.0461131 |
| CreditScore | 0.0972506 | 0.0553596 | 1.7567058 | 0.0789680 | 1.1021365 |
| Employed | 0.7095433 | 0.3725014 | 1.9048073 | 0.0568051 | 2.0330626 |
| BankCustomer | 0.8509058 | 0.3222964 | 2.6401342 | 0.0082873 | 2.3417670 |
| PriorDefault | 3.5192042 | 0.3451889 | 10.1950089 | 0.0000000 | 33.7575520 |
| grp.YearsEmployed0.1-1.0 | -0.7288612 | 0.6790085 | -1.0734198 | 0.2830828 | 0.4824581 |
| grp.YearsEmployed3.0-5.0 | 0.3838400 | 0.7230751 | 0.5308439 | 0.5955270 | 1.4679106 |
| grp.YearsEmployed5.0-10.0 | 0.3731764 | 0.7763964 | 0.4806520 | 0.6307639 | 1.4523405 |
| grp.YearsEmployed10.1-50.0 | 0.7432616 | 0.8207288 | 0.9056117 | 0.3651414 | 2.1027827 |
| grp.YearsEmployed50+ | -0.2614599 | 0.8467595 | -0.3087771 | 0.7574911 | 0.7699267 |
| EthnicityBlack | 0.9229021 | 0.5337120 | 1.7292135 | 0.0837709 | 2.5165832 |
| EthnicityLatino | -0.8316552 | 0.8527065 | -0.9753124 | 0.3294053 | 0.4353281 |
| EthnicityOther | 1.2276316 | 0.8707288 | 1.4098897 | 0.1585723 | 3.4131362 |
| EthnicityWhite | 0.5613195 | 0.4641797 | 1.2092719 | 0.2265584 | 1.7529840 |
Performing a multiple logistic regression on the data set Credit does provide important information on the relationship between credit card approval and multiple predictor variables. Despite the initial histograms being skewed, regrouping each variable allowed for a more normal distribution. The automatic variable selection performed the best in the global goodness-of-fit test, making it the final model used. Additionally, the odds ratio provided very useful information in approval prediction, with PriorDefault being the best predictor of credit card approval. Overall, multiple logistic regression is a valuable tool for analyzing complex data sets and can provide insights into the factors that influence categorical outcomes.