On this occasion, I will try to predict the probability that the customer will fail to pay the loan (loan default), to indicate whether the customer is good or not to be given a loan based on the category of several supporting variables. The algorithm that I will use is to use logistic regression and k-nearest neighbor which is included in supervised learning.
Load required library.
library(dplyr)
library(gtools)
library(gmodels)
library(ggplot2)
library(class)
library(tidyr)
library(caret)Load data to perform regression model
loans <- read.csv(file="data_input/loan2017Q4.csv",stringsAsFactors = F)after we have successfully imported our data, we will do a data inspection to find out contents our data, actually we can use the view() function to view the contents of the data but it will take time to see the whole data so we use a function that sees the head() only.
head(loans)Descriptions:
initial_list_status: Either w (whole) or
f (fractional). This variable indicates if the loan was a
whole loan or fractional loan. For background: Some institutional
investors have a preference to purchase loans in their entirety to
obtain legal and accounting treatment specific to their situation - with
the added benefit of “instant funding” to borrowerspurpose: Simplified from the original data; One of:
credit_card, debt_consolidation,
home_improvement, major_purchase and
small_businessint_rate: Interest rate in percentagesinstallment: Monthly payment owed by the borrowerannual_inc: Self-reported annual income provided by the
borrower / co-borrowers during applicationdti: A ratio of the borrower’s total monthly debt
payments on his/her total obligations to the self-reported monthly
incomeverification_status: is the reported income verified,
not verified, or if the income source was verifiedgrade: software-assigned loan graderevol_bal: total credit revolving balance (in the case
of credit card, it refers to the portion of credit card spending that
goes unpaid at the end of a billing cycle)inq_last_12m: number of credit inquiries in the last 12
monthsdelinq_2yrs: number of 30+ days past-due incidences of
delinquency in the borrower’s credit file for the past 2 yearshome_ownership: one of MORTGAGE,
OWN and RENTnot_paid: 0 for fully-paid loans, 1 for charged-off,
past-due / grace period or defaultedlog_inc: log of annual_incverified: 0 for “Not verified” under
verification_status, 1 otherwisegrdCtoA: 1 for a grade of A, B or C, 0
otherwiseCheck the structure of the data copier. Are there any columns whose data types do not match?
str(loans)#> 'data.frame': 1556 obs. of 16 variables:
#> $ initial_list_status: chr "w" "f" "w" "w" ...
#> $ purpose : chr "debt_consolidation" "debt_consolidation" "debt_consolidation" "debt_consolidation" ...
#> $ int_rate : num 14.08 9.44 28.72 13.59 15.05 ...
#> $ installment : num 676 480 1010 484 476 ...
#> $ annual_inc : num 156700 50000 25000 175000 109992 ...
#> $ dti : num 19.1 19.4 65.6 12.6 10 ...
#> $ verification_status: chr "Source Verified" "Not Verified" "Verified" "Not Verified" ...
#> $ grade : chr "C" "B" "F" "C" ...
#> $ revol_bal : int 21936 5457 23453 31740 2284 2016 14330 27588 27024 11719 ...
#> $ inq_last_12m : int 3 1 0 0 3 5 0 1 8 1 ...
#> $ delinq_2yrs : int 0 1 0 0 0 0 0 0 0 0 ...
#> $ home_ownership : chr "MORTGAGE" "RENT" "OWN" "MORTGAGE" ...
#> $ not_paid : int 0 1 1 1 0 1 0 1 1 0 ...
#> $ log_inc : num 12 10.8 10.1 12.1 11.6 ...
#> $ verified : int 1 0 1 0 0 0 0 0 1 1 ...
#> $ grdCtoA : int 0 1 0 0 0 1 0 1 0 0 ...Target: not_paid (paid = 0, not_paid = 1)
Columns whose data types do not match:
not_paid -> factorverified -> factorgrade -> factorhome_ownership -> factorpurpose-> factorinitial_list_status -> factorAre any of the variables below that can be removed?
grade -> we will not omit grades, because they are
more precise. Value to takeout value grdCtoAlog_inc -> can be discarded because it is the log
value of annual incverified_status -> discard because it is already
verifiedloans_clean <- loans %>%
mutate(not_paid = as.factor(not_paid),
verified = as.factor(verified),
grade = as.factor(grade),
home_ownership= as.factor(home_ownership),
purpose=as.factor(purpose),
initial_list_status=as.factor(initial_list_status))%>%
select(-c(verification_status,grdCtoA,log_inc))
glimpse(loans_clean)#> Rows: 1,556
#> Columns: 13
#> $ initial_list_status <fct> w, f, w, w, w, w, w, w, w, w, w, w, w, f, w, w, w,~
#> $ purpose <fct> debt_consolidation, debt_consolidation, debt_conso~
#> $ int_rate <dbl> 14.08, 9.44, 28.72, 13.59, 15.05, 10.91, 15.05, 10~
#> $ installment <dbl> 675.99, 480.08, 1010.30, 484.19, 476.33, 130.79, 3~
#> $ annual_inc <dbl> 156700, 50000, 25000, 175000, 109992, 49000, 65000~
#> $ dti <dbl> 19.11, 19.35, 65.58, 12.60, 10.00, 5.12, 22.38, 33~
#> $ grade <fct> C, B, F, C, C, B, C, B, D, D, F, C, C, E, B, C, C,~
#> $ revol_bal <int> 21936, 5457, 23453, 31740, 2284, 2016, 14330, 2758~
#> $ inq_last_12m <int> 3, 1, 0, 0, 3, 5, 0, 1, 8, 1, 0, 12, 4, 8, 1, 3, 0~
#> $ delinq_2yrs <int> 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,~
#> $ home_ownership <fct> MORTGAGE, RENT, OWN, MORTGAGE, MORTGAGE, MORTGAGE,~
#> $ not_paid <fct> 0, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1,~
#> $ verified <fct> 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1,~Check blank data in our dataset
colSums(is.na(loans))#> initial_list_status purpose int_rate installment
#> 0 0 0 0
#> annual_inc dti verification_status grade
#> 0 0 0 0
#> revol_bal inq_last_12m delinq_2yrs home_ownership
#> 0 0 0 0
#> not_paid log_inc verified grdCtoA
#> 0 0 0 0from the results of our exploratory data, we get the result that there is no NA or balnk data.
Before doing the modeling, we need to first see the proportion of the target variable that we have in the target column.
prop.table(table(loans$not_paid))#>
#> 0 1
#> 0.5 0.5When viewed from the proportion of the two classes, it is already balanced, so we don’t really need additional pre-processing to balance the proportion between the two target classes of variables.
Cross Validation is a stage where we will divide the data into 2 parts, namely data train and data test.
The purpose of cross validation is to find out how well the model predicts unseen data.
RNGkind(sample.kind = "Rounding")
set.seed(123)
index <- sample(x = nrow(loans_clean),
size = nrow(loans_clean)*0.8)
# splitting
loans_train <- loans_clean[index, ]
loans_test <- loans_clean[-index, ]Doing modeling using logistic regression. Modeling using glm() function in modeling using logistic regression. The variables used are several variables that we think affect the target variable, where the target variable is the response variable.
# Model with all Prediction
model_all <- glm(formula = not_paid ~ ., data = loans_train, family = binomial)summary(model_all)#>
#> Call:
#> glm(formula = not_paid ~ ., family = binomial, data = loans_train)
#>
#> Deviance Residuals:
#> Min 1Q Median 3Q Max
#> -1.919 -1.100 0.554 1.107 1.677
#>
#> Coefficients:
#> Estimate Std. Error z value Pr(>|z|)
#> (Intercept) -0.895253816 0.467909249 -1.913 0.0557 .
#> initial_list_statusw -0.048747890 0.148032695 -0.329 0.7419
#> purposedebt_consolidation 0.115336589 0.155024560 0.744 0.4569
#> purposehome_improvement 0.204093183 0.230295537 0.886 0.3755
#> purposemajor_purchase 0.737075738 0.355047479 2.076 0.0379 *
#> purposesmall_business 0.938694313 0.526603436 1.783 0.0747 .
#> int_rate -0.018378629 0.051945000 -0.354 0.7235
#> installment 0.001110541 0.000228727 4.855 0.0000012 ***
#> annual_inc -0.000003243 0.000001321 -2.456 0.0141 *
#> dti 0.000021291 0.004807931 0.004 0.9965
#> gradeB 0.182773836 0.269467189 0.678 0.4976
#> gradeC 0.605594745 0.422222941 1.434 0.1515
#> gradeD 0.993606286 0.657471666 1.511 0.1307
#> gradeE 1.034959679 0.951289595 1.088 0.2766
#> gradeF 1.080915152 1.240401733 0.871 0.3835
#> gradeG 0.150604983 1.381648411 0.109 0.9132
#> revol_bal 0.000001796 0.000003377 0.532 0.5948
#> inq_last_12m -0.031628334 0.024697577 -1.281 0.2003
#> delinq_2yrs 0.166826833 0.083625900 1.995 0.0461 *
#> home_ownershipOWN 0.439661888 0.185875847 2.365 0.0180 *
#> home_ownershipRENT 0.237032187 0.137641085 1.722 0.0851 .
#> verified1 0.227585954 0.125577982 1.812 0.0699 .
#> ---
#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> (Dispersion parameter for binomial family taken to be 1)
#>
#> Null deviance: 1724.5 on 1243 degrees of freedom
#> Residual deviance: 1630.4 on 1222 degrees of freedom
#> AIC: 1674.4
#>
#> Number of Fisher Scoring iterations: 4In the first modeling, there are still many predictor variables that are not significant to the target variable, therefore we will try to do a model fitting using the stepwise method.
model_step <- step(object = model_all,
direction = "backward",
trace = F)
summary(model_step)#>
#> Call:
#> glm(formula = not_paid ~ installment + annual_inc + grade + delinq_2yrs +
#> home_ownership + verified, family = binomial, data = loans_train)
#>
#> Deviance Residuals:
#> Min 1Q Median 3Q Max
#> -1.9162 -1.1142 0.6502 1.1104 1.6614
#>
#> Coefficients:
#> Estimate Std. Error z value Pr(>|z|)
#> (Intercept) -1.016867074 0.211758025 -4.802 0.000001571 ***
#> installment 0.001129976 0.000225011 5.022 0.000000512 ***
#> annual_inc -0.000003002 0.000001133 -2.651 0.008029 **
#> gradeB 0.120955536 0.203135292 0.595 0.551547
#> gradeC 0.449697161 0.199287045 2.257 0.024037 *
#> gradeD 0.757091610 0.213986088 3.538 0.000403 ***
#> gradeE 0.698809943 0.275943528 2.532 0.011327 *
#> gradeF 0.650505301 0.424713120 1.532 0.125613
#> gradeG -0.280828387 0.635944022 -0.442 0.658784
#> delinq_2yrs 0.162157394 0.082759891 1.959 0.050069 .
#> home_ownershipOWN 0.472482689 0.183936211 2.569 0.010207 *
#> home_ownershipRENT 0.278843394 0.130350327 2.139 0.032421 *
#> verified1 0.235884735 0.124633872 1.893 0.058408 .
#> ---
#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> (Dispersion parameter for binomial family taken to be 1)
#>
#> Null deviance: 1724.5 on 1243 degrees of freedom
#> Residual deviance: 1639.3 on 1231 degrees of freedom
#> AIC: 1665.3
#>
#> Number of Fisher Scoring iterations: 4by using the stepwise method we get a model of several significant variables, which are:
When we have succeeded in making the model, we will try to predict the test data that we have prepared at the cross validation stage.
we will see the results of our model using the data load test and will be entered into the form of probabilities.
loans_test$prediction <- predict(object = model_step,
newdata = loans_test,
type = "response")loans_testAfter getting the prediction results that still have a probability
value, we must change the value to a label that matches the label on the
target variable. In this case we will change the probability value to 1
or 0. To perform the transformation we will use the
ifelse() function, this function can help us to create
simple if-else logic functions.
The determining condition when we want to transform our prediction results into a class, the condition is often called the Threshold.
The initial threshold that can be used is 0.5. When the predicted probability > 0.5 -> 1/ not paid When the predicted probability <=0.5 -> 0/paid
loans_test$pred_label <- ifelse(test = loans_test$prediction > 0.5,
yes = "1",
no = "0")
loans_testprop.table(table(loans_test$pred_label))#>
#> 0 1
#> 0.4711538 0.5288462From the results of the prediction proportions, we get the results that there are still many customers who have the opportunity not to pay the loans they did.
After making predictions using the model, there are still predictions that are wrong. In classification, we evaluate the model based on the confusion matrix, To perform the confusion matrix calculation we will use the confusionMatrix() function from the library(caret). In this function there are 3 parameters that we can use, which is:
before we evaluate the model we have to change pred_label to factor.
loans_test$pred_label <- as.factor(loans_test$pred_label)# confusion matrix
confusionMatrix(data = loans_test$pred_label,
reference = loans_test$not_paid,
positive = "1")#> Confusion Matrix and Statistics
#>
#> Reference
#> Prediction 0 1
#> 0 90 57
#> 1 69 96
#>
#> Accuracy : 0.5962
#> 95% CI : (0.5394, 0.6511)
#> No Information Rate : 0.5096
#> P-Value [Acc > NIR] : 0.001304
#>
#> Kappa : 0.1932
#>
#> Mcnemar's Test P-Value : 0.327107
#>
#> Sensitivity : 0.6275
#> Specificity : 0.5660
#> Pos Pred Value : 0.5818
#> Neg Pred Value : 0.6122
#> Prevalence : 0.4904
#> Detection Rate : 0.3077
#> Detection Prevalence : 0.5288
#> Balanced Accuracy : 0.5967
#>
#> 'Positive' Class : 1
#> From the evaluation results we get the value:
After the team considered the results, we decided to focus on the recall value in our model, because we want to create a predictive model that is more focused on customers who do not pay (False Negative) because it will harm the company if the customer does not pay the loan, if we focus on customers who pay (False Positive) will not affect our company.
because we want to focus our model on Recall we will do tuning by shifting the threshold closer to 0 (Paid), from the previous 0.5 we will change it to 0.45 then we will see the confusion matrix.
loans_test$pred_label_tuning <- ifelse(test = loans_test$prediction > 0.45,
yes = "1",
no = "0")
head(loans_test)loans_test$pred_label_tuning <- as.factor(loans_test$pred_label_tuning)
confusionMatrix(data = loans_test$pred_label_tuning,
reference = loans_test$not_paid,
positive = "1")#> Confusion Matrix and Statistics
#>
#> Reference
#> Prediction 0 1
#> 0 73 31
#> 1 86 122
#>
#> Accuracy : 0.625
#> 95% CI : (0.5687, 0.6789)
#> No Information Rate : 0.5096
#> P-Value [Acc > NIR] : 0.0000264303
#>
#> Kappa : 0.2548
#>
#> Mcnemar's Test P-Value : 0.0000005966
#>
#> Sensitivity : 0.7974
#> Specificity : 0.4591
#> Pos Pred Value : 0.5865
#> Neg Pred Value : 0.7019
#> Prevalence : 0.4904
#> Detection Rate : 0.3910
#> Detection Prevalence : 0.6667
#> Balanced Accuracy : 0.6283
#>
#> 'Positive' Class : 1
#> Recall Before Tuning: 0.627451
FN Before Tunning: 57
Recall After Tuning: 0.7974
FN After Tunning: 31
from the previous prediction where the threshold is 0.5, the recall value is 62%. After we try to tune it by lowering the threshold value to 0.45 and the recall value to 79%. So the model that we have made can be said to be quite good/accurate in categorizing people who fail to pay.
k-NN is K-nearest neighbor. This method will classify the new data by comparing the characteristics of the new data (test data) with existing data (train data). The closeness of these characteristics is measured by Euclidean Distance to obtain the distance. Then the k closest neighbors will be selected from the new data, then the class will be determined using majority voting.
For k-NN, it is separated between predictor and label (the target variable).
Data -> Predictors + Targets . separate . New Data -> New Data Predictor -> Target
# predictor train
loans_train_predictor <- loans_train %>%
select(-c(not_paid,initial_list_status,purpose,grade,home_ownership,verified))
# target train
loans_train_target <- loans_train %>%
pull(not_paid)
# prediktor test
loans_test_predictor <- loans_test %>%
select(-c(not_paid,initial_list_status,purpose,grade,home_ownership,verified,pred_label,pred_label_tuning,prediction))
# taget test
loans_test_target <- loans_test %>%
pull(not_paid)Predictor data will be scaled using z-score standardization. The test data must also be scaled using the parameters from the train data (because it assumes the test data is unseen data).
# scaling train predictor
loans_train_predictor_scale <- loans_train_predictor %>%
scale()# scaling test predictor
loans_test_predictor_scale <- loans_test_predictor %>%
scale(center = attr(loans_train_predictor_scale,"scaled:center"),
scale = attr(loans_train_predictor_scale,"scaled:scale"))
loans_test_predictor_scale#> int_rate installment annual_inc dti revol_bal
#> 8 -0.66022148 1.4948365921 0.223067258 1.056897913 0.579389743
#> 13 0.05857979 -1.2582895691 -0.808844492 -0.489770533 -0.657478652
#> 16 -0.10983500 -0.0687352331 -0.189038073 -0.202852329 -0.418671784
#> 24 1.16977016 -0.3126986793 -0.765985537 -0.391889583 -0.282564001
#> 44 2.60042776 1.4980588762 -0.189038073 0.200626396 -0.212853673
#> 45 0.74960130 0.6463792965 -0.601143405 0.849928271 -0.336083243
#> 49 0.58118651 1.3383398880 -0.024195941 -0.013067891 -0.350454582
#> 52 -1.27832113 -0.5565956866 -0.518722338 1.795114543 -0.144589811
#> 62 0.05857979 0.8131075831 -0.574949990 -0.175206564 -0.285316953
#> 63 -0.66022148 -0.4602261390 -0.189038073 -0.811059147 -0.150655636
#> 68 -0.36332531 -0.9229727083 -0.189038073 0.738598030 -0.294648991
#> 72 -0.66022148 -0.5688536539 -0.782469750 1.204840112 -0.230164605
#> 80 0.22699458 0.2059561798 -0.551690765 -0.950035153 -0.377144212
#> 98 -1.63077682 -1.2462641378 -0.980280309 0.058661660 -0.486002443
#> 100 2.79141360 -0.1718151042 -1.046217162 3.051128870 -0.168246529
#> 103 -0.10983500 0.7766326561 -0.452785485 -0.344069882 0.635148674
#> 107 -0.10983500 1.8635721929 -0.090132794 1.145065486 1.316574132
#> 110 0.05857979 0.1821710726 -0.172553860 0.042970821 -0.283263904
#> 113 0.91801609 0.1820049755 -0.683564471 1.409568206 -0.020660338
#> 114 -1.27832113 0.0001950707 -0.568174978 0.220053150 -0.472097705
#> 119 0.58118651 1.4098613068 1.953909649 0.464381933 7.639730180
#> 120 1.16977016 -0.6995388659 -0.024195941 -0.453905757 -0.102875598
#> 124 -0.47270811 0.3005650979 0.305488324 -0.211818522 0.129305522
#> 126 -0.19491052 -0.9820036239 -0.930827670 0.016072239 -0.307060603
#> 132 1.59167525 1.2641941347 -0.551690765 0.917921908 7.054004778
#> 135 -0.19491052 -0.0145543534 1.113214773 -0.990383025 -0.460805938
#> 141 -1.32693571 -0.9303142009 -0.749501324 1.117419722 0.416452349
#> 143 -0.66022148 2.7982006730 0.091193552 -0.500978275 1.395523178
#> 148 0.05857979 0.0359391703 -0.156069647 0.416562233 0.353321108
#> 160 -0.36332531 0.2521643979 -0.386848632 -0.724385940 -0.707731680
#> 162 -1.27832113 -1.0308693959 -0.419817059 -1.087516792 -0.580302693
#> 176 -0.10983500 1.2412395131 1.129698986 0.602610756 0.952344665
#> 177 0.41277172 -0.7755781263 0.800014721 0.912691628 4.020672294
#> 180 0.74960130 -0.5116498066 -0.518722338 0.050442649 -0.163720490
#> 190 1.16977016 0.3424547909 -0.518722338 -0.455400123 -0.010721717
#> 193 0.74960130 0.2449890024 -0.411574952 -0.026517182 0.353414429
#> 194 -0.74529700 0.0713178564 -0.287943353 -0.544314879 -0.017020843
#> 195 -0.10983500 0.1585852819 -0.353880206 0.441966449 0.090250940
#> 199 -0.19491052 2.9686495350 0.058225125 -0.296997364 -0.061627988
#> 205 0.74960130 0.6955440433 -0.024195941 -0.282800891 -0.596727080
#> 212 -1.17067456 1.1069333839 -0.601143405 1.189896456 1.267627589
#> 214 -0.10983500 -1.2054042469 -0.930827670 -0.255155126 -0.610631818
#> 215 1.16977016 -0.2871529426 -0.716532897 -0.683290884 -0.237490255
#> 228 -1.30262842 -0.4763043400 -0.914343456 -0.601847957 -0.311913263
#> 230 -0.83037251 -1.0109709612 -0.189038073 -0.329126226 0.302694799
#> 233 -0.47270811 -0.7741496911 -0.601143405 -0.936585862 -0.707871660
#> 234 -0.10983500 -0.4097326153 -0.683564471 -0.601100774 -0.495987724
#> 236 1.16977016 -0.4578343405 0.569235736 -0.760997898 -0.231564410
#> 237 0.41277172 -0.7192047646 -0.848406603 0.359776338 -0.480216579
#> 238 0.41277172 0.1138054990 -0.353880206 1.706199787 -0.137264160
#> 263 -1.17067456 2.6156931603 3.602330973 -0.343322699 0.739154244
#> 265 0.05857979 2.2564583146 2.118751781 -1.260863207 -0.605639177
#> 267 0.22699458 -0.5534066219 -0.601143405 -0.899226721 -0.428890366
#> 284 1.16977016 -0.4124898275 -0.930827670 -0.638459915 -0.579462809
#> 285 -0.83037251 -0.3685073108 0.882435788 -0.672083142 -0.027332746
#> 288 -0.74529700 1.0193337641 0.235875491 -0.947793604 -0.561638615
#> 289 -0.10983500 1.5083568974 -0.123101220 0.184935557 0.181751579
#> 292 0.22699458 1.2822655011 -0.024195941 -0.219290351 -0.566211314
#> 294 2.68376703 1.0413582419 -0.765985537 1.727120906 -0.157887966
#> 297 -0.19491052 0.3725183692 -0.683564471 0.985915545 0.609018966
#> 300 0.41277172 0.5216071419 0.235597073 0.024291250 1.202723261
#> 301 -0.66022148 0.2783413036 -0.185362094 0.645200177 -0.607878867
#> 307 -0.10983500 -0.5784540673 -0.683564471 0.266378485 -0.399914387
#> 310 -1.17067456 0.2224994527 -0.022547520 0.002622948 -0.171979344
#> 321 -0.91544802 -0.9510099018 -0.650596044 -0.938827410 -0.596587100
#> 322 -1.27832113 0.0001950707 -0.353880206 -0.953771067 -0.626729585
#> 328 -1.38770393 -0.8824117923 -0.353880206 -0.201357963 -0.571763877
#> 334 -0.74529700 -1.0071507275 -0.254974926 0.181946826 -0.550346849
#> 335 1.16977016 -0.9167606762 -0.724775004 -0.711683832 -0.219992683
#> 339 0.91801609 1.9740599955 0.058225125 0.435241803 0.865043444
#> 344 0.91801609 -1.2994816543 -1.046217162 1.795861726 -0.313266408
#> 353 0.91801609 -0.8056417046 -0.090132794 -0.899226721 -0.027706027
#> 362 -0.91544802 -0.6958515099 -0.716532897 0.427022792 -0.377190873
#> 370 -0.47270811 -1.3698736127 -0.518722338 -0.794621125 -0.443168385
#> 373 0.05857979 -0.3479112682 0.173614618 0.107975726 -0.141603558
#> 381 0.41277172 -0.8348747972 -0.436301272 -0.538337417 0.218659791
#> 386 -0.10983500 1.1815774285 0.882435788 1.135352109 2.769665861
#> 391 0.74960130 -0.4012948817 -0.683564471 -0.397119863 -0.455113395
#> 395 0.05857979 3.0631255754 1.772583303 0.820040958 3.257638159
#> 398 -1.17067456 0.0924121903 -0.106617007 0.069869402 -0.081831851
#> 399 -0.74529700 -0.6836267620 -0.271459140 -0.716166929 -0.328011029
#> 400 1.92850483 0.9833239091 -0.174350639 0.476336859 -0.178745072
#> 403 -0.10983500 0.7269031791 0.305488324 -0.028758731 0.489848833
#> 405 -0.74529700 0.3948418218 -0.106617007 -1.256380111 -0.573910246
#> 408 0.22699458 -1.4149523703 -0.930827670 -0.232739642 -0.629762497
#> 411 2.81745712 0.0946378917 -0.436301272 1.897478590 0.989439519
#> 415 -1.38770393 -1.2399524474 -0.189038073 -0.873822505 -0.594767352
#> 420 0.58118651 -0.3444232287 -0.551690765 1.440949884 0.846239387
#> 422 0.05857979 -0.1222849438 0.058225125 -0.492759264 0.089177756
#> 423 -0.10983500 -0.6791421399 -0.403332846 0.549560775 -0.187517188
#> 427 -1.27832113 1.1067340674 0.832983148 -0.323148763 0.350334856
#> 437 -0.36332531 1.3925207677 1.723130663 0.003370131 1.828063167
#> 440 -0.74529700 -0.4060784787 -0.353880206 -0.375451561 -0.412792600
#> 446 2.43201297 2.1243778868 0.717593655 -0.089280539 0.830981504
#> 453 2.68376703 2.2274245385 0.272519897 0.883551498 -0.170252917
#> 460 -0.19491052 -0.3209370963 -0.601143405 0.268620033 -0.192463169
#> 461 -0.74529700 -0.0497005034 0.008772486 0.246951732 -0.438315725
#> 474 -0.83037251 0.5676160434 0.371425177 -0.638459915 -0.044923639
#> 477 -1.30262842 -1.2377267460 0.223067258 -0.556269804 -0.568357683
#> 488 0.05857979 0.5278856129 -0.601143405 -0.099741099 -0.613291449
#> 497 -0.19491052 1.6141275419 0.316466810 -0.094510819 -0.233337498
#> 500 0.58118651 1.3383398880 0.635172589 0.667615662 0.448041300
#> 510 2.43201297 1.5999760674 0.214825151 -0.326137495 0.006635875
#> 512 -1.63077682 2.0550489499 11.020226934 -0.964231626 -0.237350274
#> 516 -0.83037251 -0.8610849224 -0.353880206 -0.246936115 -0.565091470
#> 521 -0.47270811 -0.4431845748 -0.518722338 -0.484540253 -0.430243512
#> 533 0.22699458 -0.9622712863 0.635172589 0.091537704 -0.185557460
#> 542 -0.66022148 -0.7535204291 -0.189038073 -0.122903766 0.908857366
#> 553 0.22699458 0.8781844337 0.272519897 -0.205841060 -0.532849276
#> 566 -0.91544802 -0.4832139801 0.058225125 0.237985538 -0.221999071
#> 567 -1.27832113 -0.7215633437 -0.057164367 0.340349585 0.841340066
#> 583 1.59167525 1.0560080077 -0.588120876 -0.457641672 -0.121493015
#> 584 -1.27832113 -0.9354964310 -0.297833881 -1.339317404 -0.622763468
#> 586 -1.27832113 -1.0308693959 -0.271459140 -0.978428100 -0.599480032
#> 591 0.05857979 0.7583619731 0.058225125 0.217811601 0.183991268
#> 592 -1.38770393 -0.8926433747 0.173614618 -1.275806864 -0.624349915
#> 601 0.05857979 0.3523873986 0.074709339 0.359029155 -0.092610356
#> 607 0.74960130 1.2129033449 1.706646450 -0.169976284 0.903491443
#> 619 1.16977016 1.3564112544 -0.848406603 3.828199007 0.026839738
#> 625 0.58118651 -0.0439203238 0.223067258 -0.446433929 -0.214253479
#> 628 0.22699458 1.6107391607 1.953909649 -0.259638223 -0.232777575
#> 630 1.59167525 -0.1163718864 -0.603286352 1.630734322 -0.543907742
#> 632 -0.36332531 -0.6471186156 -0.353880206 -0.562247267 -0.391422231
#> 642 0.22699458 0.1023115785 -0.320911779 0.317186917 -0.427490561
#> 650 -0.47270811 -1.2154365128 -1.046217162 -0.146813617 -0.452500424
#> 651 -1.32693571 -1.1870006863 -0.502238125 -0.263374137 -0.506019665
#> 653 -0.74529700 -0.4679330453 1.376962185 -1.109932277 0.575283646
#> 654 -1.38770393 -1.3318705922 -0.601143405 -0.550292342 -0.677122593
#> 659 -1.27832113 0.0001950707 -0.551690765 0.555538238 -0.587348382
#> 663 2.43201297 -0.3717295948 -0.568174978 -0.585409935 -0.698679602
#> 664 0.41277172 -1.0720282616 0.078764455 0.620543144 1.158256097
#> 671 1.16977016 1.6285447717 -0.024195941 -0.206588243 0.753292282
#> 672 0.05857979 -0.2014136106 -0.213827033 -0.598859225 -0.475690540
#> 676 -1.27832113 0.5156940845 1.047277920 0.176716546 -0.352740932
#> 680 -0.19491052 1.1343061889 -0.601143405 0.236491172 -0.367812174
#> 684 0.74960130 0.2808991992 -0.864890817 0.352304510 -0.188217091
#> 686 -0.19491052 -0.4176388381 0.223067258 -0.635471184 -0.430710114
#> 691 -0.36332531 2.0159164691 -0.238490713 -1.209307593 -0.273885205
#> 695 -1.17067456 -0.9220757839 -0.106617007 -0.293261450 0.020727253
#> 698 0.05857979 0.7583619731 0.470330456 0.740839578 0.072426747
#> 708 -0.66022148 -0.2430043288 -0.024195941 -0.708695100 -0.605405876
#> 714 -0.74529700 -0.4679330453 -0.024195941 0.442713631 -0.338929514
#> 715 1.59167525 1.1229783654 -0.485753912 0.149070782 -0.366785650
#> 719 1.42326046 0.1338368114 0.302850850 0.440472083 0.183571326
#> 721 -0.47270811 -0.7189390092 0.289004111 0.576459357 0.014008185
#> 733 0.41277172 0.1079920999 0.310301714 -0.731857768 0.135791289
#> 735 0.41277172 -0.1233479653 1.212120053 -0.158021359 0.794306592
#> 737 1.92850483 -0.2530365947 -0.823680283 1.449168896 -0.218452896
#> 739 0.58118651 -0.9453958192 -0.287943353 0.148323599 -0.700312709
#> 741 -1.38770393 -1.0356529929 -0.683564471 0.219305967 -0.544560985
#> 744 -0.19491052 -0.8239788263 -0.733017111 -0.290272719 -0.552119936
#> 747 1.42326046 -1.1257440694 -0.864890817 -0.059393226 -0.395015066
#> 749 0.05857979 0.8270929604 18.438122895 -1.125623116 -0.312986447
#> 750 -0.36332531 -0.4331523089 0.305488324 -0.116926304 -0.305847438
#> 768 -1.30262842 -0.9290518628 -0.353880206 -0.955265432 -0.641894147
#> 770 0.22699458 -1.0791039988 -0.353880206 -0.539831782 -0.532522655
#> 772 -0.19491052 1.1343061889 0.552751523 -0.537590234 -0.523143956
#> 780 -0.66022148 -0.8078009671 -0.526964445 -0.216301619 -0.398234620
#> 792 -1.63077682 -0.0457806115 -0.271459140 1.553774491 -0.156348179
#> 796 0.41277172 -1.0720282616 -0.337395993 -1.052399200 -0.463838851
#> 799 -0.19491052 -0.0145543534 0.272519897 -0.338092420 0.463299183
#> 806 -0.83037251 -1.4071790253 -0.403332846 0.506971354 -0.680902069
#> 811 -1.17067456 0.9508685323 -0.353880206 -0.450169843 -0.024113193
#> 812 0.91801609 0.5658886334 -0.601143405 0.563757249 -0.290822855
#> 821 -0.91544802 -0.7103351785 0.635172589 -0.409821971 -0.136750898
#> 823 -1.27832113 0.5156940845 -0.485753912 -0.127386863 -0.323111709
#> 825 0.74960130 -0.7180088654 -0.650596044 -0.392636766 -0.676609331
#> 827 0.22699458 1.6863465685 1.953909649 -0.533107137 0.383976855
#> 835 0.74960130 1.2129033449 -0.024195941 -1.358744157 -0.696486573
#> 840 1.16977016 0.4855640674 -0.353880206 -0.601100774 -0.518757898
#> 849 0.22699458 0.5564210977 1.129698986 -0.946299239 -0.324978117
#> 856 0.22699458 1.6863465685 -0.518722338 -0.374704378 -0.144076549
#> 866 0.74960130 -0.6936590279 -0.927712153 0.974707802 -0.095503288
#> 867 0.05857979 0.0359391703 -0.518623433 1.381175259 2.265455816
#> 868 0.58118651 0.7510204805 0.012893539 0.895506423 -0.405093668
#> 870 -0.19491052 -0.6433648208 -0.370364419 0.596633293 0.234524257
#> 879 -0.36332531 1.2366884521 0.602204162 -0.039966473 -0.052809211
#> 885 2.01184411 -1.1768687621 -0.848406603 1.311687256 0.219219714
#> 886 -0.83037251 -0.8503550486 -0.930827670 0.268620033 -0.688087738
#> 887 2.01184411 -0.0456145144 0.429119923 -0.418040982 -0.229604682
#> 889 0.22699458 0.3932472895 0.404393603 0.304484809 0.507673027
#> 897 0.41277172 -0.7162814553 -0.419817059 -0.698234541 -0.300901457
#> 905 1.59167525 0.7416526031 0.069203611 0.141598953 0.273018916
#> 908 0.41277172 -0.5537388162 -0.353880206 -1.379665277 -0.607085643
#> 910 0.22699458 1.3741836459 1.123303112 -0.854395751 -0.349054776
#> 911 -0.66022148 2.2551295377 0.882435788 -0.256649492 0.182871423
#> 913 0.74960130 -1.1078388001 -0.419817059 -0.512933201 -0.600973158
#> 916 -0.10983500 2.4318900901 1.986878075 -0.083303077 2.150905042
#> 918 -0.83037251 0.0598239358 -0.815438177 0.412826318 -0.377330853
#> 920 -0.74529700 0.3409266975 -0.057164367 0.064639122 0.250948645
#> 922 0.58118651 -0.3924917345 -0.551690765 1.333355558 -0.464678734
#> 923 -0.19491052 -0.8268024773 -0.667080258 -0.559258536 -0.225405265
#> 927 -0.36332531 -0.7114646389 -0.077110265 -0.109454476 -0.258254041
#> 929 -0.36332531 -1.1567710109 0.008772486 0.390410834 -0.183924353
#> 943 0.05857979 0.7583619731 0.058225125 -0.423271262 -0.453013686
#> 944 -1.17067456 -1.2550340656 0.190098831 -0.013067891 -0.676655991
#> 948 0.41277172 1.9518694206 -0.223902184 -0.497989544 0.643267547
#> 954 1.76009004 -0.2632017383 0.536267309 -0.027264365 1.899546582
#> 955 -0.36332531 -1.2124135453 -0.271459140 0.161025707 -0.197129188
#> 956 0.41277172 0.1079920999 -0.518722338 -0.264121320 0.057308844
#> 965 -0.83037251 -0.2614078894 -0.238490713 0.289541153 1.734789441
#> 966 2.01184411 -0.0456145144 -0.848406603 1.024769051 -0.669423661
#> 967 -0.36332531 0.0677965975 0.058225125 -0.579432472 -0.453480288
#> 971 1.59167525 -0.9218100285 0.206583044 0.240227086 -0.098862822
#> 992 -0.47270811 -0.9506444881 0.552751523 -0.388900852 -0.542741237
#> 1004 -1.17067456 -1.2862603237 -0.024195941 0.068375037 -0.053415794
#> 1011 0.91801609 0.5658886334 0.964856854 0.007106045 -0.507746093
#> 1014 0.05857979 -0.6244961882 2.448436046 0.285805238 1.294783822
#> 1016 -1.38770393 -1.0560829383 0.272519897 -1.222009701 -0.550720130
#> 1025 -0.74529700 0.0500574253 0.305488324 1.214553489 0.521251143
#> 1028 1.16977016 0.2678771852 -0.320911779 1.083796495 1.100024176
#> 1032 -0.36332531 0.7018225143 -0.320911779 -0.054910129 -0.223865479
#> 1038 0.05857979 0.0359391703 0.223067258 0.424781244 -0.260400410
#> 1039 0.41277172 0.6533553755 -0.647299202 -0.473332511 -0.396088251
#> 1041 -0.91544802 -1.0147911949 -0.815438177 -0.094510819 -0.496547646
#> 1044 1.59167525 -0.0210321410 0.272519897 -0.594376128 4.309685529
#> 1048 -1.17067456 0.0143797645 0.124161978 -0.575696558 -0.046370105
#> 1050 -1.63077682 -1.3713020479 1.261572692 -0.577938106 -0.523890519
#> 1052 0.22699458 -0.9622712863 -0.436301272 1.240704888 0.036778359
#> 1062 -0.19491052 0.7111239529 0.305488324 0.447943911 -0.123266103
#> 1065 -0.66022148 -1.3834603569 -0.848406603 -0.943310507 -0.619263954
#> 1070 2.43201297 0.0268038289 -0.518722338 0.594391745 -0.544374344
#> 1075 -0.36332531 -0.7003361321 0.008772486 -0.430743090 -0.203288333
#> 1082 0.74960130 -0.3392077792 -0.518722338 1.050920450 -0.611051760
#> 1088 0.22699458 -1.0206710329 -0.601143405 -0.169976284 -0.421238095
#> 1090 0.74960130 0.6463792965 -0.518722338 -0.561500084 -0.506439607
#> 1095 0.22699458 -0.9622712863 -0.947311883 -1.021017521 -0.499627219
#> 1100 1.92850483 1.4268032128 1.294541119 -0.033989010 1.788262022
#> 1101 -0.83037251 -1.2251365845 -0.024195941 -1.328109662 -0.543207839
#> 1103 -0.19491052 2.9686495350 0.964856854 -0.086291808 0.794213272
#> 1104 -0.83037251 -1.1180703826 -0.024195941 -0.988141477 -0.572697081
#> 1108 -1.27832113 -0.5153371626 0.420877817 -0.088533356 -0.332397087
#> 1111 -1.30262842 -0.5174632057 -0.156069647 0.292529884 -0.456793162
#> 1114 -0.91544802 1.6431613180 -0.436301272 1.428247776 0.361579962
#> 1115 0.05857979 -0.8088639886 -0.469269699 -0.298491730 -0.625423099
#> 1128 0.74960130 -0.1820134672 0.234276523 -0.485287436 -0.308133787
#> 1129 0.41277172 2.0111660915 -0.765985537 -1.417024418 -0.707871660
#> 1133 -0.10983500 0.0022878943 -0.512128653 -0.554028256 0.095196921
#> 1137 -0.66022148 0.0828449963 -0.601143405 -0.748295790 -0.375977708
#> 1145 0.91801609 -1.0525616794 -0.683564471 -0.092269271 -0.623929973
#> 1148 2.01184411 -0.4881968936 -0.642353938 0.959016963 -0.326377923
#> 1159 -0.47270811 1.2240318517 -0.106617007 0.614565681 0.091650746
#> 1163 0.74960130 -0.1667325324 -0.601143405 0.292529884 -0.420771493
#> 1168 2.80009478 0.8235717015 0.305488324 -0.717661294 -0.096949754
#> 1170 -0.19491052 1.8398867439 -0.024195941 -0.290272719 0.778395466
#> 1177 -1.27832113 -0.8167702114 -0.568174978 -0.820025341 -0.707871660
#> 1187 -1.17067456 0.9508685323 1.080246347 0.007853228 1.557434048
#> 1188 -0.36332531 -0.8784586809 -0.106617007 0.663132565 0.169386628
#> 1189 0.74960130 0.7817816667 0.305488324 -0.698981724 -0.366925630
#> 1201 -0.74529700 1.3044228566 0.635172589 -0.375451561 0.111154707
#> 1209 -0.19491052 -0.4176388381 -0.024195941 -0.476321242 -0.446947861
#> 1215 -0.91544802 -0.6958515099 -0.568174978 0.170739083 -0.074832822
#> 1217 -0.47270811 0.6600324796 1.541804317 -0.892502075 0.159774628
#> 1223 -1.32693571 -1.2383579150 -0.848406603 1.254901361 -0.592947605
#> 1224 -0.19491052 -0.0145543534 -0.353880206 0.836478980 0.193090006
#> 1237 -0.19491052 -0.8126842223 -0.436301272 0.140104588 -0.348028252
#> 1239 0.22699458 -1.0791039988 -0.518722338 -0.036230559 -0.323158369
#> 1241 -1.30262842 -0.7232575343 0.305488324 0.110964458 0.204615073
#> 1244 -0.47270811 -0.4431845748 -0.765985537 0.453174191 -0.364172679
#> 1248 2.01184411 -0.4458089092 -0.815438177 0.802855753 -0.295395554
#> 1252 -0.10983500 1.8635721929 1.294541119 -0.950035153 0.556199627
#> 1257 -0.91544802 -0.4832139801 -0.123101220 -0.353036076 1.309295142
#> 1262 1.16977016 2.8609189446 0.239551471 -0.494253630 3.173463170
#> 1272 -0.10983500 -0.6539618169 -0.024195941 0.169991901 -0.147576063
#> 1281 -0.66022148 -0.4602261390 -0.304427566 -0.221531899 -0.342802310
#> 1291 0.05857979 -1.1430513890 -0.930827670 -1.156257612 -0.480263239
#> 1294 -0.36332531 -0.4331523089 -0.312669673 -0.475574059 -0.174872276
#> 1298 0.22699458 1.2822655011 0.321972537 -1.406563858 -0.707311738
#> 1305 -0.66022148 0.0882597623 -0.622902566 -0.452411392 -0.542087995
#> 1310 2.01184411 1.9554238989 -1.293480361 6.622662770 -0.440088813
#> 1315 0.91801609 -0.4902564979 -0.535206552 0.998617653 -0.231517750
#> 1319 0.41277172 -1.1876650748 -0.996764523 -1.250402648 -0.707871660
#> 1320 0.58118651 -0.4905886921 -0.320911779 -0.755767618 -0.456606521
#> 1324 -1.17067456 2.6156931603 4.426541636 -0.826749987 0.619844130
#> 1327 -0.19491052 -0.0145543534 -0.024195941 -0.338839603 -0.334916738
#> 1334 -0.83037251 -0.4755735127 -0.353880206 -0.432984638 -0.340889243
#> 1342 -0.91544802 -0.1004597825 0.140646191 0.406848856 -0.239123362
#> 1346 -0.47270811 -0.2165948871 -1.145122442 -0.669841594 -0.622810128
#> 1349 0.41277172 -0.3055897226 -0.650596044 0.792395193 1.013469518
#> 1350 0.74960130 -0.5718434020 -0.024212425 -0.653403571 -0.069140279
#> 1357 -0.91544802 2.7063489670 1.953909649 -0.794621125 1.750467266
#> 1358 1.16977016 0.2669138219 -0.601143405 -0.695992992 -0.032698668
#> 1362 0.05857979 0.0359391703 -0.551690765 1.452157627 -0.329877437
#> 1366 -0.66022148 -0.4602261390 -0.551690765 -0.516669115 -0.509845801
#> 1376 -1.63077682 -0.5459986905 -0.156069647 -0.078072797 0.341282778
#> 1380 -0.83037251 -0.3685073108 -0.419817059 0.096020801 -0.112767559
#> 1381 -0.83037251 2.7368443978 0.734077868 -0.047438301 1.774217304
#> 1384 2.43201297 0.5512056482 -0.586307613 0.373972812 -0.097509676
#> 1396 -0.47270811 -0.5535062802 -0.963796096 -0.246936115 -0.324371534
#> 1402 0.58118651 -1.1858047871 -0.106617007 -0.057151678 -0.143376646
#> 1406 0.58118651 -0.5117162454 -0.189038073 0.642958628 -0.202028508
#> 1426 0.22699458 -0.6585460973 -0.584659191 -0.489023350 0.283004197
#> 1437 -0.36332531 0.3271074173 -0.518722338 0.688536781 0.187444122
#> 1446 -0.91544802 -0.8978588242 -0.189038073 -0.940321776 -0.626402963
#> 1453 -1.27832113 0.1032749418 0.470330456 -0.860373214 -0.173285829
#> 1461 -0.36332531 -1.0120339827 -0.765985537 -0.197622049 -0.577783042
#> 1463 -0.83037251 0.5952213843 -0.106617007 0.096767984 0.159727968
#> 1466 0.58118651 -1.3720993140 -0.551690765 0.093032070 0.160941133
#> 1471 -0.83037251 1.2722996741 1.624225384 -0.577938106 1.421046304
#> 1473 0.58118651 -0.3444232287 -0.765985537 -0.107960110 -0.296468739
#> 1478 -1.17067456 0.0143797645 -0.271459140 -0.613055699 -0.145429694
#> 1496 0.22699458 -0.3781741630 0.008772486 -0.057898861 0.948891811
#> 1498 -0.83037251 -0.1370343678 0.058225125 -0.774447189 -0.258534002
#> 1502 0.58118651 -0.3444232287 -0.518722338 -0.512186018 -0.145569675
#> 1503 0.22699458 1.3741836459 -0.765985537 -0.601847957 -0.202961712
#> 1504 0.05857979 0.0359391703 0.404393603 -0.433731821 0.129258862
#> 1507 -0.36332531 0.4574272157 -0.353880206 0.032510261 -0.461039239
#> 1508 -0.74529700 -0.4679330453 0.173614618 -0.673577508 -0.498320734
#> 1518 -1.17067456 0.5346623753 1.047277920 -0.410569154 -0.175898800
#> 1519 -0.47270811 0.1084239524 -0.436301272 -0.855142934 -0.589261450
#> 1531 0.05857979 -0.9702107286 0.668141015 0.385180554 0.204521753
#> 1533 0.58118651 -0.7050200708 0.800014721 -0.217795985 0.814650436
#> inq_last_12m delinq_2yrs
#> 8 -0.5626946 -0.3897707
#> 13 0.6304737 -0.3897707
#> 16 0.2327509 -0.3897707
#> 24 -0.9604173 0.9608554
#> 44 0.2327509 -0.3897707
#> 45 -0.1649718 -0.3897707
#> 49 1.8236420 -0.3897707
#> 52 0.6304737 -0.3897707
#> 62 0.2327509 -0.3897707
#> 63 0.2327509 -0.3897707
#> 68 -0.9604173 -0.3897707
#> 72 -0.5626946 -0.3897707
#> 80 -0.5626946 3.6621078
#> 98 -0.9604173 -0.3897707
#> 100 -0.1649718 -0.3897707
#> 103 0.6304737 -0.3897707
#> 107 -0.5626946 -0.3897707
#> 110 -0.5626946 -0.3897707
#> 113 -0.5626946 -0.3897707
#> 114 -0.9604173 -0.3897707
#> 119 0.2327509 -0.3897707
#> 120 -0.5626946 2.3114816
#> 124 -0.1649718 -0.3897707
#> 126 0.2327509 -0.3897707
#> 132 0.2327509 -0.3897707
#> 135 -0.9604173 0.9608554
#> 141 1.8236420 -0.3897707
#> 143 -0.1649718 0.9608554
#> 148 -0.9604173 -0.3897707
#> 160 -0.9604173 -0.3897707
#> 162 -0.5626946 -0.3897707
#> 176 -0.9604173 -0.3897707
#> 177 0.6304737 -0.3897707
#> 180 2.2213647 -0.3897707
#> 190 -0.1649718 -0.3897707
#> 193 -0.9604173 -0.3897707
#> 194 -0.5626946 -0.3897707
#> 195 -0.5626946 0.9608554
#> 199 -0.9604173 -0.3897707
#> 205 -0.1649718 -0.3897707
#> 212 -0.9604173 -0.3897707
#> 214 1.4259192 -0.3897707
#> 215 -0.5626946 -0.3897707
#> 228 -0.9604173 -0.3897707
#> 230 0.2327509 -0.3897707
#> 233 0.2327509 -0.3897707
#> 234 -0.5626946 -0.3897707
#> 236 0.6304737 -0.3897707
#> 237 1.0281965 -0.3897707
#> 238 -0.1649718 -0.3897707
#> 263 0.2327509 -0.3897707
#> 265 1.0281965 -0.3897707
#> 267 -0.9604173 0.9608554
#> 284 -0.9604173 0.9608554
#> 285 -0.9604173 -0.3897707
#> 288 -0.5626946 -0.3897707
#> 289 -0.9604173 -0.3897707
#> 292 -0.5626946 -0.3897707
#> 294 -0.1649718 -0.3897707
#> 297 -0.1649718 -0.3897707
#> 300 -0.9604173 -0.3897707
#> 301 -0.1649718 -0.3897707
#> 307 -0.9604173 -0.3897707
#> 310 -0.9604173 -0.3897707
#> 321 0.6304737 -0.3897707
#> 322 -0.9604173 -0.3897707
#> 328 -0.9604173 -0.3897707
#> 334 -0.9604173 -0.3897707
#> 335 2.6190875 0.9608554
#> 339 -0.9604173 -0.3897707
#> 344 -0.9604173 -0.3897707
#> 353 -0.9604173 -0.3897707
#> 362 0.2327509 0.9608554
#> 370 -0.1649718 -0.3897707
#> 373 1.0281965 -0.3897707
#> 381 -0.5626946 -0.3897707
#> 386 -0.5626946 -0.3897707
#> 391 -0.5626946 -0.3897707
#> 395 1.0281965 -0.3897707
#> 398 -0.1649718 0.9608554
#> 399 0.2327509 -0.3897707
#> 400 1.0281965 -0.3897707
#> 403 -0.9604173 0.9608554
#> 405 -0.9604173 0.9608554
#> 408 -0.9604173 9.0646126
#> 411 1.0281965 -0.3897707
#> 415 -0.5626946 -0.3897707
#> 420 -0.9604173 -0.3897707
#> 422 -0.5626946 -0.3897707
#> 423 -0.1649718 -0.3897707
#> 427 2.6190875 -0.3897707
#> 437 1.0281965 -0.3897707
#> 440 -0.9604173 -0.3897707
#> 446 0.2327509 -0.3897707
#> 453 -0.1649718 -0.3897707
#> 460 -0.9604173 -0.3897707
#> 461 -0.9604173 -0.3897707
#> 474 -0.9604173 -0.3897707
#> 477 -0.9604173 -0.3897707
#> 488 1.0281965 -0.3897707
#> 497 0.6304737 -0.3897707
#> 500 -0.9604173 -0.3897707
#> 510 -0.9604173 -0.3897707
#> 512 4.6077013 -0.3897707
#> 516 -0.1649718 -0.3897707
#> 521 -0.9604173 -0.3897707
#> 533 -0.5626946 0.9608554
#> 542 -0.5626946 0.9608554
#> 553 1.0281965 -0.3897707
#> 566 0.2327509 -0.3897707
#> 567 0.6304737 -0.3897707
#> 583 0.6304737 0.9608554
#> 584 -0.9604173 -0.3897707
#> 586 -0.1649718 -0.3897707
#> 591 -0.5626946 5.0127340
#> 592 -0.5626946 0.9608554
#> 601 -0.5626946 -0.3897707
#> 607 -0.9604173 0.9608554
#> 619 0.2327509 -0.3897707
#> 625 -0.1649718 0.9608554
#> 628 -0.9604173 -0.3897707
#> 630 -0.1649718 -0.3897707
#> 632 -0.9604173 -0.3897707
#> 642 0.6304737 -0.3897707
#> 650 -0.5626946 -0.3897707
#> 651 -0.9604173 -0.3897707
#> 653 0.6304737 -0.3897707
#> 654 -0.5626946 -0.3897707
#> 659 -0.9604173 -0.3897707
#> 663 0.6304737 -0.3897707
#> 664 -0.1649718 -0.3897707
#> 671 -0.5626946 -0.3897707
#> 672 -0.1649718 -0.3897707
#> 676 -0.1649718 0.9608554
#> 680 -0.5626946 0.9608554
#> 684 -0.1649718 -0.3897707
#> 686 -0.9604173 3.6621078
#> 691 -0.1649718 -0.3897707
#> 695 -0.9604173 -0.3897707
#> 698 0.2327509 -0.3897707
#> 708 -0.9604173 -0.3897707
#> 714 -0.9604173 0.9608554
#> 715 -0.9604173 0.9608554
#> 719 -0.1649718 -0.3897707
#> 721 -0.5626946 -0.3897707
#> 733 -0.9604173 -0.3897707
#> 735 1.0281965 -0.3897707
#> 737 0.6304737 -0.3897707
#> 739 -0.5626946 3.6621078
#> 741 -0.9604173 -0.3897707
#> 744 0.2327509 -0.3897707
#> 747 -0.1649718 9.0646126
#> 749 -0.5626946 -0.3897707
#> 750 -0.9604173 -0.3897707
#> 768 -0.9604173 -0.3897707
#> 770 -0.1649718 -0.3897707
#> 772 0.2327509 -0.3897707
#> 780 -0.9604173 -0.3897707
#> 792 -0.1649718 -0.3897707
#> 796 -0.5626946 -0.3897707
#> 799 0.2327509 -0.3897707
#> 806 0.2327509 -0.3897707
#> 811 -0.1649718 -0.3897707
#> 812 3.0168102 -0.3897707
#> 821 -0.9604173 -0.3897707
#> 823 -0.1649718 -0.3897707
#> 825 1.0281965 -0.3897707
#> 827 -0.5626946 2.3114816
#> 835 0.2327509 3.6621078
#> 840 -0.5626946 -0.3897707
#> 849 0.2327509 -0.3897707
#> 856 -0.1649718 -0.3897707
#> 866 -0.1649718 -0.3897707
#> 867 -0.1649718 2.3114816
#> 868 -0.9604173 -0.3897707
#> 870 -0.9604173 -0.3897707
#> 879 1.4259192 -0.3897707
#> 885 -0.5626946 -0.3897707
#> 886 0.6304737 -0.3897707
#> 887 0.2327509 -0.3897707
#> 889 -0.5626946 -0.3897707
#> 897 -0.9604173 -0.3897707
#> 905 0.2327509 -0.3897707
#> 908 -0.9604173 0.9608554
#> 910 -0.5626946 5.0127340
#> 911 -0.5626946 -0.3897707
#> 913 1.4259192 -0.3897707
#> 916 0.2327509 -0.3897707
#> 918 -0.1649718 0.9608554
#> 920 -0.9604173 -0.3897707
#> 922 1.4259192 -0.3897707
#> 923 -0.5626946 2.3114816
#> 927 0.2327509 -0.3897707
#> 929 6.1985923 -0.3897707
#> 943 0.2327509 -0.3897707
#> 944 2.2213647 -0.3897707
#> 948 -0.5626946 0.9608554
#> 954 -0.1649718 -0.3897707
#> 955 -0.1649718 -0.3897707
#> 956 -0.9604173 -0.3897707
#> 965 -0.5626946 0.9608554
#> 966 -0.1649718 -0.3897707
#> 967 -0.1649718 -0.3897707
#> 971 -0.1649718 -0.3897707
#> 992 2.2213647 0.9608554
#> 1004 -0.5626946 -0.3897707
#> 1011 -0.9604173 -0.3897707
#> 1014 -0.1649718 0.9608554
#> 1016 0.2327509 -0.3897707
#> 1025 1.4259192 -0.3897707
#> 1028 -0.9604173 -0.3897707
#> 1032 1.8236420 -0.3897707
#> 1038 -0.5626946 3.6621078
#> 1039 -0.5626946 -0.3897707
#> 1041 -0.9604173 -0.3897707
#> 1044 0.2327509 2.3114816
#> 1048 3.4145330 -0.3897707
#> 1050 0.2327509 -0.3897707
#> 1052 -0.9604173 -0.3897707
#> 1062 -0.9604173 -0.3897707
#> 1065 -0.1649718 -0.3897707
#> 1070 1.0281965 -0.3897707
#> 1075 -0.9604173 -0.3897707
#> 1082 -0.5626946 -0.3897707
#> 1088 1.8236420 -0.3897707
#> 1090 -0.9604173 -0.3897707
#> 1095 -0.5626946 -0.3897707
#> 1100 1.0281965 -0.3897707
#> 1101 0.2327509 -0.3897707
#> 1103 -0.5626946 -0.3897707
#> 1104 1.8236420 2.3114816
#> 1108 -0.1649718 -0.3897707
#> 1111 0.2327509 -0.3897707
#> 1114 -0.9604173 3.6621078
#> 1115 -0.1649718 -0.3897707
#> 1128 3.0168102 0.9608554
#> 1129 0.2327509 -0.3897707
#> 1133 -0.9604173 -0.3897707
#> 1137 -0.9604173 2.3114816
#> 1145 -0.9604173 0.9608554
#> 1148 1.0281965 -0.3897707
#> 1159 -0.9604173 -0.3897707
#> 1163 0.2327509 -0.3897707
#> 1168 0.2327509 -0.3897707
#> 1170 -0.5626946 3.6621078
#> 1177 -0.5626946 -0.3897707
#> 1187 0.6304737 0.9608554
#> 1188 -0.5626946 -0.3897707
#> 1189 -0.1649718 -0.3897707
#> 1201 -0.9604173 -0.3897707
#> 1209 -0.5626946 -0.3897707
#> 1215 -0.9604173 -0.3897707
#> 1217 -0.5626946 6.3633602
#> 1223 -0.5626946 -0.3897707
#> 1224 -0.9604173 -0.3897707
#> 1237 0.6304737 -0.3897707
#> 1239 -0.1649718 -0.3897707
#> 1241 -0.5626946 -0.3897707
#> 1244 -0.9604173 -0.3897707
#> 1248 3.8122557 -0.3897707
#> 1252 -0.9604173 2.3114816
#> 1257 0.6304737 0.9608554
#> 1262 -0.1649718 2.3114816
#> 1272 0.6304737 -0.3897707
#> 1281 5.4031468 -0.3897707
#> 1291 -0.9604173 -0.3897707
#> 1294 0.2327509 -0.3897707
#> 1298 -0.1649718 -0.3897707
#> 1305 -0.5626946 2.3114816
#> 1310 1.4259192 -0.3897707
#> 1315 -0.1649718 -0.3897707
#> 1319 -0.5626946 2.3114816
#> 1320 -0.9604173 0.9608554
#> 1324 -0.5626946 -0.3897707
#> 1327 -0.5626946 -0.3897707
#> 1334 -0.9604173 -0.3897707
#> 1342 1.4259192 -0.3897707
#> 1346 -0.1649718 -0.3897707
#> 1349 -0.5626946 -0.3897707
#> 1350 0.6304737 0.9608554
#> 1357 -0.9604173 -0.3897707
#> 1358 -0.5626946 0.9608554
#> 1362 -0.9604173 -0.3897707
#> 1366 -0.5626946 -0.3897707
#> 1376 -0.9604173 -0.3897707
#> 1380 0.6304737 -0.3897707
#> 1381 -0.5626946 -0.3897707
#> 1384 -0.5626946 -0.3897707
#> 1396 -0.9604173 -0.3897707
#> 1402 0.2327509 0.9608554
#> 1406 -0.5626946 0.9608554
#> 1426 -0.9604173 -0.3897707
#> 1437 -0.9604173 -0.3897707
#> 1446 0.6304737 -0.3897707
#> 1453 -0.1649718 -0.3897707
#> 1461 -0.1649718 -0.3897707
#> 1463 -0.9604173 2.3114816
#> 1466 -0.9604173 -0.3897707
#> 1471 -0.1649718 -0.3897707
#> 1473 0.6304737 -0.3897707
#> 1478 -0.1649718 -0.3897707
#> 1496 -0.1649718 -0.3897707
#> 1498 -0.9604173 -0.3897707
#> 1502 0.6304737 0.9608554
#> 1503 -0.5626946 -0.3897707
#> 1504 -0.5626946 -0.3897707
#> 1507 -0.9604173 5.0127340
#> 1508 -0.5626946 0.9608554
#> 1518 -0.1649718 -0.3897707
#> 1519 0.2327509 -0.3897707
#> 1531 1.0281965 -0.3897707
#> 1533 -0.5626946 3.6621078
#> attr(,"scaled:center")
#> int_rate installment annual_inc dti revol_bal
#> 14.7126045 465.5112781 81467.8250402 18.9648955 15170.7829582
#> inq_last_12m delinq_2yrs
#> 2.4147910 0.2885852
#> attr(,"scaled:scale")
#> int_rate installment annual_inc dti revol_bal
#> 5.7595892 301.0287039 60664.1023773 13.3836053 21431.5444635
#> inq_last_12m delinq_2yrs
#> 2.5143143 0.7403973Step 1: We will calculate what is the most optimum value of k, using the sqrt() function of our total data.
# find optimum k
sqrt(nrow(loans_train_predictor))#> [1] 35.27038K -> 35
Step 2: We will predict by using knn() function from library(class).
loans_pred <- knn(train = loans_train_predictor_scale,
test = loans_test_predictor_scale,
cl = loans_train_target,
k = 35)
loans_pred#> [1] 1 0 0 1 1 1 1 0 1 0 1 1 1 0 1 1 0 0 1 0 0 1 0 1 1 0 0 1 0 1 0 1 0 0 1 1 0
#> [38] 1 1 1 1 0 1 0 0 0 1 1 0 1 0 1 1 1 0 0 1 1 1 1 1 0 1 0 0 1 0 0 0 1 1 1 1 0
#> [75] 0 0 0 1 0 1 0 1 1 0 1 1 0 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 0 0 0 1 1 1 0 0
#> [112] 1 0 0 1 0 0 1 1 1 1 1 0 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 0 0 0 1 1 1 0 1 0 1
#> [149] 0 0 0 1 0 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 1 1 1 1 1 1 0 0 0 1 1 1 0 1 0 1 1
#> [186] 1 1 1 1 0 0 1 0 0 1 0 1 1 0 1 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 0 0 0 1 0 0 1
#> [223] 0 1 0 1 0 0 0 1 0 0 0 1 0 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 0 0 1 0 1 0 0 0 0
#> [260] 1 1 0 0 0 0 0 0 1 1 1 1 0 1 1 0 0 0 1 1 1 1 1 1 0 0 0 1 1 0 1 1 0 1 0 0 0
#> [297] 1 1 1 0 0 0 0 1 1 0 1 0 0 1 0 1
#> Levels: 0 1Because we want to focus on the recall value like the model we created earlier, we will look at the sensitivity value.
confusionMatrix(data = loans_pred,
reference = loans_test_target,
positive = "1")#> Confusion Matrix and Statistics
#>
#> Reference
#> Prediction 0 1
#> 0 89 63
#> 1 70 90
#>
#> Accuracy : 0.5737
#> 95% CI : (0.5168, 0.6293)
#> No Information Rate : 0.5096
#> P-Value [Acc > NIR] : 0.01348
#>
#> Kappa : 0.1479
#>
#> Mcnemar's Test P-Value : 0.60288
#>
#> Sensitivity : 0.5882
#> Specificity : 0.5597
#> Pos Pred Value : 0.5625
#> Neg Pred Value : 0.5855
#> Prevalence : 0.4904
#> Detection Rate : 0.2885
#> Detection Prevalence : 0.5128
#> Balanced Accuracy : 0.5740
#>
#> 'Positive' Class : 1
#> from the results of the evaluation of the model that we have made, the recall value that we get from the KNN model is still not good for classifying customers who will pay or fail to pay because they only get a sensitivity value of 0.5882, not much different from the model value before tunning.
from the results of the prediction model that we have made using the
step() function we get:
these results are still not good if we set a target based on the
business value that is obtained must be at least 0.75, for that we are
tuning our model where we want to focus our prediction model on recall
so that the model does not predict wrongly for customers who fail to
pay(False Negative), for that we perform tuning by shifting the
threshold to 0 in order to reduce the FN value and increase the FP
value, we get a better sensitivity result, which is
0.7974. We also make a prediction model with K-NN, only the results
obtained are still not good when compared to the model using
step() which has been tuned, in which the recall results
(because our focus is on FN) obtained 0.5882 is still below the minimum
standard set. we made, this happened because of the omitted predictors
because the knn function only allowed numeric predictors,
for that we had to have further discussion about converting non-numeric
predictors to numeric based on advice from business expertise.