Agenda

• Title: Exploring Predictive Models
• Introduction
• Data Set Description
• Research Questions
• Exploratory Data Analysis
• Model Selection: Multiple Logistic Regression
• Data Split
• k-Fold Cross Validation
• Discussion
• Conclusion
• REFs

Introduction

• Big Data has become an inescapable fixture of life.
• It is ubiquitous.
• It is valuable.
• According to Segal(2022), big data may take multiple forms, however, generally it may be encapsulated by what are called the “Three V’s”. These V’s refer to velocity, volume, and variety.

Introduction: Three V’s

• Typically, this data source seems vast, it may have no end to its growth, and the information amounts it produces, is where the volume V comes from.
• The velocity V may refer to that data’s creation an collection speed.
• And, the Variety V may refer to the diversity of data points represented in it.

Introduction: Banking

• As on might imagine, then, big data, may be intertwined with and a product of daily human interactions. Interactions like those associated with banking. This has not gone unnoticed by the banking industry.
• According to Garadis( n.d.), the banking industry is rife with data. Daily transactions may be a prime example of that. The value of that data may be vast and the insights that data may provide may be under-leveraged.
• Generally, the data generated and collected with in the industry, may help institutions improve costumer experience through the understandings that it can provide, help them better position their internal interest, and much more. Ultimately, these things may work to increase revenue and/or limit losses.

Introduction: Loan default

• One may imagine, then, that data concerning loans, a source of revenue for many banking institutions, is highly valued and thoroughly analysed. Especially, information on defaulting.
• According to Brozic(2023), Loan default may be considered a label applied by a lender to a borrower, when that borrower, missed or neglected to make payments on a loan provided by that lender within a time period.
• Defaulting on a loan is a terrible experience, one that most wish to avoid. It may leave long lasting effects on ones mental health and have consequences that may limit ones quality of life, earning potential, and much more. Consequently, the information gleaned from loan default data may be useful.

Data Set Description

• The data set use is called the “BankLoanDefaultDataset”. The Structure of the data set follows.
• It is taken from the book “Applied Analytics through Case Studies Using SAS and R, Deepti Gupta by APress, ISBN - 978-1-4842-3525-6”. It was described where it was posted as, “…a subset of a large o data set.
• The structure of the data set is simple.
• It can be used for logistic and binary classification / predictive models and algorithms”.
• Defaut is used as its response variable. There are 1000 observations and 15 variables.

Research Questions

Exploratory Data Analysis

• Exploratory Data Analysis was conducted.
  • Before the models could be explored, they needed to be built.
  • Before they could be built, some data set exploration was necessary.
• The following is the structure of the data set.
  • At the top it may be clearly seen that there were 1000 observations, and 16 variables.
  • Default, the response variable, is characterized as an integer object.
  • One may also note that several of the variables that were characterized as integers, seem to only posses values 0 and 1.
  • Further there were also variables characterized as characters, which seemed to posses at least two categories.
  • Both of those variables types were assumed to be factors after exploring summary statistics for each of the variables and the data set description.

Exploratory Data Analysis: Data Structure Table

'data.frame':   1000 obs. of  16 variables:
 $ Default         : int  0 0 0 1 1 0 0 0 0 1 ...
 $ Checking_amount : int  988 458 158 300 63 1071 -192 172 585 189 ...
 $ Term            : int  15 15 14 25 24 20 13 16 20 19 ...
 $ Credit_score    : int  796 813 756 737 662 828 856 763 778 649 ...
 $ Gender          : chr  "Female" "Female" "Female" "Female" ...
 $ Marital_status  : chr  "Single" "Single" "Single" "Single" ...
 $ Car_loan        : int  1 1 0 0 0 1 1 1 1 1 ...
 $ Personal_loan   : int  0 0 1 0 0 0 0 0 0 0 ...
 $ Home_loan       : int  0 0 0 0 0 0 0 0 0 0 ...
 $ Education_loan  : int  0 0 0 1 1 0 0 0 0 0 ...
 $ Emp_status      : chr  "employed" "employed" "employed" "employed" ...
 $ Amount          : int  1536 947 1678 1804 1184 475 626 1224 1162 786 ...
 $ Saving_amount   : int  3455 3600 3093 2449 2867 3282 3398 3022 3475 2711 ...
 $ Emp_duration    : int  12 25 43 0 4 12 11 12 12 0 ...
 $ Age             : int  38 36 34 29 30 32 38 36 36 29 ...
 $ No_of_credit_acc: int  1 1 1 1 1 2 1 1 1 1 ...

Exploratory Data Analysis: Summary Statistics

• The following represents summary statistics for each of the variables.
• as well as a table for the variable No_of_credit_acc bellow the summary output.
• Note, that that this variable had only 9 values and was also characterized as an integer.
• For those reasons it was also assumed to be a factor variable, and was converted into one.
• Lastly, from the summary output it may be noticed that there were no missing values in the data set.

Exploratory Data Analysis: Data

Data Summary

FREQ Table: No_of_credit_acc

Exploratory Data Analysis: Response Variable

• Next the Response variable was explored.
• From the previous information it may be clear that it is a binary variable which used the values 0 and 1 to encode a burrowers loan default status.
• A proportion table of the frequency for this two levels follows.
• From it, it may be clear that there are 7/3 more borrowers that did not default than that did.
• Note that the response variable is a binary factor variable that was stored as an integer, with the integer 1 associated bank loan default, so \(P(Y=1|X)=P(Default=1|X)\).

Exploratory Data Analysis: Response Variable Proportion Table

Exploratory Data Analysis: Correlations

• Next correlations between the numeric variables that were not turned into factors was assessed.
• For most of these variables, it seems that there were low amounts of correlation detected and that no variable had a correlation higher than .35.
• In fact, it seems that Age and Checking_amount, Age and Term, Age and Credit_score, and Age and Saving_amount, accounted for the highest amounts of correlation for each of these variables.

Exploratory Data Analysis: Correlations Matrix

Exploratory Data Analysis: Variables Transformations

'data.frame':   1000 obs. of  16 variables:
 $ Default         : Factor w/ 2 levels "0","1": 1 1 1 2 2 1 1 1 1 2 ...
 $ Checking_amount : int  988 458 158 300 63 1071 -192 172 585 189 ...
 $ Term            : int  15 15 14 25 24 20 13 16 20 19 ...
 $ Credit_score    : int  796 813 756 737 662 828 856 763 778 649 ...
 $ Gender          : Factor w/ 2 levels "Female","Male": 1 1 1 1 1 2 2 1 1 2 ...
 $ Marital_status  : Factor w/ 2 levels "Married","Single": 2 2 2 2 2 1 2 2 2 1 ...
 $ Car_loan        : Factor w/ 2 levels "0","1": 2 2 1 1 1 2 2 2 2 2 ...
 $ Personal_loan   : Factor w/ 2 levels "0","1": 1 1 2 1 1 1 1 1 1 1 ...
 $ Home_loan       : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
 $ Education_loan  : Factor w/ 2 levels "0","1": 1 1 1 2 2 1 1 1 1 1 ...
 $ Emp_status      : Factor w/ 2 levels "employed","unemployed": 1 1 1 1 2 1 1 1 2 1 ...
 $ Amount          : int  1536 947 1678 1804 1184 475 626 1224 1162 786 ...
 $ Saving_amount   : int  3455 3600 3093 2449 2867 3282 3398 3022 3475 2711 ...
 $ Emp_duration    : int  12 25 43 0 4 12 11 12 12 0 ...
 $ Age             : int  38 36 34 29 30 32 38 36 36 29 ...
 $ No_of_credit_acc: Factor w/ 9 levels "1","2","3","4",..: 1 1 1 1 1 2 1 1 1 1 ...

Model Selection: Multiple Logistic Regression

• Before training the predictive models and exploring their performance, the models had to be built.
• Simple criteria was used to do that.
  • The first model contains all variables from the data set.
  • The Second model contains only those variables which were found to be statistically significant in the first model.
  • The third model was produced using an automated process. Specifically, an automatic variable selection procedure, its choice of final model is based on an AIC value that has been minimized the most by a particular model.

Model Selection: Multicolinearity

• After the models were built multicolinearity was explored within each model using variable inflation factors.
• The following output displays that information for each model. For models 2 and 3, none of the variables crawled much from one.
• The same may be said for model 1.
• However, for each of the loan types, even after adjustment, these values were all greater than three some were even greater than 9.
• Why this is the case is unknown.
• However, after exploring the raw data, it was found that all observations had at most one type of loan.
• Output of the model summarry is withheld; the models were not used to make inferences about Default.

Model Selection: GVIF Model 1

Model Selection: GVIF Model 2

Model Selection: GVIF Model 3

Data Split

• After MLR was conducted, The three models were cross-validated using k Fold Cross Validation.
• Their predictive error and and accuracy were stored after each was fit.
  • This was done five times.
  • The average of these five iterations for the predictive error and accuracy were stored and then output to tables.

The following is tabular output of the quantities of observations in each non overlapping partition employed in the k Fold Cross Validation procedure.

Fold Quantities

k-Fold Cross Validation

• Tabular output from the 5-Fold Cross Validation procedure follows.
  • the first table should contain the average predictive errors from each model.
  • Note, PE1, PE2, and PE3 should represent the full, reduced, and auto models respectively.
  • Likewise ACC1, ACC2, and ACC3, which should represent the average accuracy of each model, follows that same order.
• The final model was chosen based on greatest minimized predictive error.
• The cut-off probability used to dichotomize each models predictions for later assessment was .5.

Average prediction errors

Average prediction accuracy

Discussion: Results

• The predictive abilities of three models was explored.
• Ultimately, these abilities were quantified in the predictive error of each model and then these errors were compared.
• It was found that the predictive error of model three was the lowest.
• However, each models predictive error did not differ by much.

Discussion: Process

• To get these predicitive errors, 5-Fold Cross Validation was used each model fitted with training data.
  • At each iteration, test data with held from the model training, was used to compare predictions made by each model.
  • The cut-off probability used to dichotomize each models predictions for later assessment was .5.
• Exploratory data analysis was conducted before each models predictive abilities were explored.
  • Durring that exploration, categorical and pseudo-categorical variables were transformed into factors.
  • Multicolinearity within each model was also explored. For models 2 and 3, none was found.
  • The same may be said for model 1.
    • However, for each of the loan types, even after adjustment, these variables had high GVIF values. Why this was the case was unknown. - However, note that all observations had at most one type of loan.
  • Correlations between the numeric variables that were not turned into factors was assessed as well. low amounts were found.

Discussion: Models

The models were not used to make inferences, so no output of the model summary or interpretations were given.

• The first model contained all variables from the data set.
• The Second model contained only those variables which were found to be statistically significant in the first model.
• The third model was produced using automatic variable selection, its choice of final model is based on an AIC value that has been minimized the most by a particular model.

Note that both the automatic model and 2nd model might have differed with each iteration of k-Fold Cross Validation, based on their selection criteria. That is, if the full model was fitted with training data from differing folds, it may have produced a reduced model that could be entirely different from model 2. The same can be said for model 3.

Conclusion

Predictive Error among the three models did not differ by much. However, if one was to pick a model for prediction based on lowest predictive error, one might choose the third model. Further research might be directed to exploring the cause of the high GVIF values in model 1.

REFs

• Garadis, P. (n.d.). Modern Data Analytics in Banking: Benefits, Outlook & More. Retrieved from Hitachi Solutions: https://global.hitachi-solutions.com/blog/big-data-banking/
• Qureshi, A. (n.d.). Data Retention Policy 101: Best Practices, Examples & More [with Template]. Retrieved August 07, 2021, from https://www.intradyn.com/data-retention-policy/
• Segal, T. (2022, November 29). What Is Big Data? Definition, How It Works, and Uses. Retrieved from investopedia: https://www.investopedia.com/terms/b/big-data.asp