Objective

The data-set provides details from Thera Bank about a Personal Loan Campaign that was executed by the bank. 4982 customers were targeted with an offer of personal loan, out of which 480 customers responded positively. The data needs to be used to create classification model(s) in order to predict the response of new set of customers in the future, depending on the attributes available in the data. Classification Models using following Supervised Machine Learning Techniques: 1. Exploratory Analytics 2. Clustering 3. CART & Random Forest Once the Classification Models are built, after pruning a recommendation system to be provided.

#install.packages("caret, repos = http://cran.us.r-project.org")
#install.packages("rpart, repos = http://cran.us.r-project.org")
#install.packages("rpart.plot, repos = http://cran.us.r-project.org")
#install.packages("randomForest, repos = http://cran.us.r-project.org")
library(caret)

Loading required package: lattice
Loading required package: ggplot2
Want to understand how all the pieces fit together? Buy the ggplot2 book: http://ggplot2.org/book/

library(rpart)
library(rpart.plot)
library(ggplot2)
library(randomForest)

randomForest 4.6-14
Type rfNews() to see new features/changes/bug fixes.

Attaching package: 㤼㸱randomForest㤼㸲

The following object is masked from 㤼㸱package:ggplot2㤼㸲:

    margin

Reading the input data

base_data <-read.csv('D:/Freelancer_questions/Thera_bank/Thera Bank_Personal_Loan_Modelling.csv')

1)Exploratory Data Analysis and Descriptive Statistics

# Find out Total Number of Rows and Columns 
dim(base_data)

[1] 5000   14

Find out Class of each Feature, along with internal structure

str(base_data)

'data.frame':   5000 obs. of  14 variables:
 $ ï..ID                : int  1 2 3 4 5 6 7 8 9 10 ...
 $ Age..in.years.       : int  25 45 39 35 35 37 53 50 35 34 ...
 $ Experience..in.years.: int  1 19 15 9 8 13 27 24 10 9 ...
 $ Income..in.K.month.  : int  49 34 11 100 45 29 72 22 81 180 ...
 $ ZIP.Code             : int  91107 90089 94720 94112 91330 92121 91711 93943 90089 93023 ...
 $ Family.members       : int  4 3 1 1 4 4 2 1 3 1 ...
 $ CCAvg                : num  1.6 1.5 1 2.7 1 0.4 1.5 0.3 0.6 8.9 ...
 $ Education            : int  1 1 1 2 2 2 2 3 2 3 ...
 $ Mortgage             : int  0 0 0 0 0 155 0 0 104 0 ...
 $ Personal.Loan        : int  0 0 0 0 0 0 0 0 0 1 ...
 $ Securities.Account   : int  1 1 0 0 0 0 0 0 0 0 ...
 $ CD.Account           : int  0 0 0 0 0 0 0 0 0 0 ...
 $ Online               : int  0 0 0 0 0 1 1 0 1 0 ...
 $ CreditCard           : int  0 0 0 0 1 0 0 1 0 0 ...

colnames(base_data)<-c('ID','Age_in_years','Experience(years)','Income_Monthly','Zip_code','Family_members','CCAvg','Education','Mortgage','Personal_loan','Securities_Account','CD_Account','Online','CreditCard')
head(base_data)

  ID Age_in_years Experience(years) Income_Monthly Zip_code Family_members CCAvg Education Mortgage Personal_loan Securities_Account
1  1           25                 1             49    91107              4   1.6         1        0             0                  1
2  2           45                19             34    90089              3   1.5         1        0             0                  1
3  3           39                15             11    94720              1   1.0         1        0             0                  0
4  4           35                 9            100    94112              1   2.7         2        0             0                  0
5  5           35                 8             45    91330              4   1.0         2        0             0                  0
6  6           37                13             29    92121              4   0.4         2      155             0                  0
  CD_Account Online CreditCard
1          0      0          0
2          0      0          0
3          0      0          0
4          0      0          0
5          0      0          1
6          0      1          0

prop.table(table(base_data$Personal_loan))*100


   0    1 
90.4  9.6

the customers who took personal loan vs no personal loan was 90.4% and 9.6% respectively

## Univariate analysis
hist(base_data$Age_in_years,
     main = "Histogram of Age",
     xlab = "Age in Years")

Inference :we can observe that the Age is very close to the normal distribution

Converting Zipcodes/Familymembers/Education/Mortgage/PersonalLoan/SecuritiesAccount/CD Account/Online/Creditcard to factors

base_data$Zip_code<-as.factor(base_data$Zip_code)
base_data$Family_members<-as.factor(base_data$Family_members)
base_data$Education<-as.factor(base_data$Education)
base_data$Personal_loan<-as.factor(base_data$Personal_loan)
base_data$Securities_Account<-as.factor(base_data$Securities_Account)
base_data$CD_Account<-as.factor(base_data$CD_Account)
base_data$Online<-as.factor(base_data$Online)
base_data$CreditCard<-as.factor(base_data$CreditCard)

Barplot of multiple dimensions

# Grouped Bar Plot
counts <- table(base_data$Family_members, base_data$Personal_loan)
barplot(counts, main="Family members vs Personal Loan",
  xlab="Personal Loan No vs Yes", col=c("darkblue","red","green","yellow"),
  legend = rownames(counts), beside=TRUE)

Inference : We can clearly see that those people having more family members have higher liklihood to take loan

counts <- table(base_data$Education, base_data$Personal_loan)
barplot(counts, main="Education Category vs Personal Loan",
  xlab="Personal Loan No vs Yes", col=c("darkblue","red","green"),
  legend = c("1 Undergrad", "2 Graduate","3 Advanced/Professional"), beside=TRUE)

Inference : Hypothesis : Advanced/Professional require loan for higher studies

Boxplot for numerical data

boxplot(base_data$Age_in_years,
        main = toupper("Boxplot of Age"),
        ylab = "Age in years",
        col = "blue")

Inference : Not much outlier in Age column

boxplot(base_data$`Experience(years)`,
        main = toupper("Boxplot of Experience"),
        ylab = "Experience in years",
        col = "purple")

Inference : Not much outlier in Experience column

boxplot(base_data$Income_Monthly,
        main = toupper("Boxplot of Monthly Income"),
        ylab = "Monthly Income",
        col = "pink")

Inference : There are lots of outliers in the monthly income

boxplot(base_data$CCAvg,
        main = toupper("Boxplot of Average Spending of credit card per month"),
        ylab = "Average Spending",
        col = "maroon")

Inference : Here too in the average spending of credit card per month there are lots of outliers

boxplot(base_data$Mortgage,
        main = toupper("Boxplot of House Mortgage if any"),
        ylab = "House Mortgag",
        col = "maroon")

Inference : Here too in there are lots of outliers

Correlation between the numeric features

my_data <- base_data[, c(2,3,4,7,9)]
res <- cor(my_data)
round(res, 2)

                  Age_in_years Experience(years) Income_Monthly CCAvg Mortgage
Age_in_years              1.00              0.99          -0.06 -0.05    -0.01
Experience(years)         0.99              1.00          -0.05 -0.05    -0.01
Income_Monthly           -0.06             -0.05           1.00  0.65     0.21
CCAvg                    -0.05             -0.05           0.65  1.00     0.11
Mortgage                 -0.01             -0.01           0.21  0.11     1.00

Inference

Age in Years and Experience are highly positively correlated
Monthly Income and Average credit card spend is also highly positively correlated

2) Cluster Analysis

Clustering features are only numberical

All the categorical features have not been considered as they do not make much sense when we do clustering

Age in Years
Experience
Monthly Income
CCAvg
Mortgage

wss <- (nrow(my_data)-1)*sum(apply(my_data,2,var))
for(i in 2:15)wss[i]<- sum(fit=kmeans(my_data,centers=i,15)$withinss)
plot(1:15,wss,type="b",main="15 clusters",xlab="no. of cluster",ylab="with clsuter sum of squares")

A fundamental step for any unsupervised algorithm is to determine the optimal number of clusters into which the data may be clustered. The Elbow Method is one of the most popular methods to determine this optimal value of k.

We now demonstrate the given method using the K-Means clustering technique

Inference : Based on the elbow curve we can see 4 clusters formed

fit <- kmeans(my_data,4)
library(cluster)
library(fpc)

package 㤼㸱fpc㤼㸲 was built under R version 3.5.3

plotcluster(my_data,fit$cluster)
points(fit$centers,col=1:8,pch=16)

getting the cluster means

mydata <- data.frame(my_data,fit$cluster)
cluster_mean <- aggregate(mydata,by = list(fit$cluster),FUN = mean)
cluster_mean

  Group.1 Age_in_years Experience.years. Income_Monthly    CCAvg   Mortgage fit.cluster
1       1     45.73072          20.40566       51.66331 1.387648   0.000000           1
2       2     44.62385          19.47706      129.17431 3.044434 343.116208           2
3       3     44.44778          19.44667      139.28778 3.605644   1.925556           3
4       4     45.35738          20.12164       56.98742 1.566745 141.411074           4

“It is important to remember that Data Analytics Projects require a delicate balance between experimentation, intuition, but also following (once a while) a process to avoid getting fooled by randomness and “finding results and patterns” that are mainly driven by our own biases and not by the facts/data themselves"-https://inseaddataanalytics.github.io/INSEADAnalytics/CourseSessions/Sessions45/ClusterAnalysisReading.html

As Kmeans is prone to outliers lets recluster them after outlier removal

my_data2<-my_data
outliers3 <- boxplot(my_data2$Income_Monthly, plot=FALSE)
outliers3<-outliers3$out
my_data2 <- my_data2[-which(my_data2$Income_Monthly %in% outliers3),]
outliers4 <- boxplot(my_data2$CCAvg, plot=FALSE)
outliers4<-outliers4$out
my_data2 <- my_data2[-which(my_data2$CCAvg %in% outliers4),]
outliers5 <- boxplot(my_data2$Mortgage, plot=FALSE)
outliers5<-outliers5$out
my_data2 <- my_data2[-which(my_data2$Mortgage %in% outliers5),]
nrow(my_data2)

[1] 4374

Inference : Outliers have been successfully removed

plotting elbow curve for the outlier removed data

wss <- (nrow(my_data2)-1)*sum(apply(my_data2,2,var))
for(i in 2:15)wss[i]<- sum(fit2=kmeans(my_data2,centers=i,15)$withinss)
plot(1:15,wss,type="b",main="15 clusters",xlab="no. of cluster",ylab="with clsuter sum of squares")

Inference : 5 clusters make sense here

fit2<-kmeans(my_data2,5)
my_data3 <- data.frame(my_data2)
cluster_mean_2 <- aggregate(my_data3,by = list(fit2$cluster),FUN = mean)
cluster_mean_2

  Group.1 Age_in_years Experience.years. Income_Monthly     CCAvg   Mortgage
1       1     55.67742          30.26100       75.12023 1.8905572   0.111437
2       2     44.76296          19.78889      136.06481 2.5069074   1.851852
3       3     46.07710          20.75944       31.42791 0.9644025   0.000000
4       4     45.42485          20.18332       54.30061 1.4017289 140.939183
5       5     35.54119          10.31960       73.52983 1.7996591   0.000000

Inference : The 5 clusters make much more sense after outlier removal

my_data2$cluster<-fit2$cluster
library(dplyr)

package 㤼㸱dplyr㤼㸲 was built under R version 3.5.1
Attaching package: 㤼㸱dplyr㤼㸲

The following object is masked from 㤼㸱package:randomForest㤼㸲:

    combine

The following objects are masked from 㤼㸱package:stats㤼㸲:

    filter, lag

The following objects are masked from 㤼㸱package:base㤼㸲:

    intersect, setdiff, setequal, union

head(my_data2)

  Age_in_years Experience(years) Income_Monthly CCAvg Mortgage cluster
1           25                 1             49   1.6        0       5
2           45                19             34   1.5        0       3
3           39                15             11   1.0        0       3
4           35                 9            100   2.7        0       5
5           35                 8             45   1.0        0       3
6           37                13             29   0.4      155       4

index<-as.integer(row.names.data.frame(my_data2))
Personal_loan<-base_data[index,10]
my_data2$Personal_loan<-Personal_loan
head(my_data2)

  Age_in_years Experience(years) Income_Monthly CCAvg Mortgage cluster Personal_loan
1           25                 1             49   1.6        0       5             0
2           45                19             34   1.5        0       3             0
3           39                15             11   1.0        0       3             0
4           35                 9            100   2.7        0       5             0
5           35                 8             45   1.0        0       3             0
6           37                13             29   0.4      155       4             0

Inference : We have got the two cluster

To check the Personal_loan vs Cluster barchart

# Grouped Bar Plot
counts <- table( my_data2$Personal_loan,my_data2$cluster)
barplot(counts, main="Family members vs Personal Loan",
  xlab="Personal Loan No vs Yes", col=c("red","green"),
  legend = c("Personal_Loan_No","Personal_Loan_Yes"), beside=TRUE)

Inference : Targeting the cluster 4 segment would be the best option for conversion rate to be higher, also the population is close to 500

This would help the company to spend the marketing money on the correct customers rather than waste it on all customers

Cart and RandomForest algorithm

Creating Training and Testing Dataset The given data set is divided into Training and Testing data set, with 70:30 proportion. The distribution of Responder and Non Responder Class is verified in both the data sets, and ensured it’s close to equal.

set.seed(111)
trainIndex <- createDataPartition(Personal_loan,
                                  p=0.7,
                                  list = FALSE,
                                  times = 1)
train.data <- base_data[trainIndex,2:length(base_data) ]
test.data  <- base_data[-trainIndex,2:length(base_data) ]

Model Building - CART (Unbalanced Dataset) Setting the control parameter inputs for rpart

r.ctrl <- rpart.control(minsplit = 100,
                        minbucket = 10,
                        cp = 0,
                        xval = 10
                        )
#Exclude columns - "Customer ID" and "Acct Opening Date"
cart.train <- train.data
m1 <- rpart(formula = Personal_loan~.,
            data = cart.train,
            method = "class",
            control = r.ctrl
            )
#install.packages("rattle") 
#install.packages("RColorBrewer") 
library(rattle)

Rattle: A free graphical interface for data science with R.
Version 5.1.0 Copyright (c) 2006-2017 Togaware Pty Ltd.
Type 'rattle()' to shake, rattle, and roll your data.

Attaching package: 㤼㸱rattle㤼㸲

The following object is masked from 㤼㸱package:randomForest㤼㸲:

    importance

library(RColorBrewer) 
fancyRpartPlot(m1)

Variables used in the tree construction

printcp(m1)


Classification tree:
rpart(formula = Personal_loan ~ ., data = cart.train, method = "class", 
    control = r.ctrl)

Variables actually used in tree construction:
[1] CCAvg          Education      Family_members Income_Monthly Zip_code      

Root node error: 308/3062 = 0.10059

n= 3062 

         CP nsplit rel error  xerror     xstd
1 0.3214286      0  1.000000 1.00000 0.054039
2 0.1525974      2  0.357143 0.36688 0.033871
3 0.0487013      3  0.204545 0.21753 0.026283
4 0.0097403      5  0.107143 0.23052 0.027039
5 0.0000000      6  0.097403 0.23377 0.027224

Pruning the cart tree to ensure that the model is not overfitting

“Overfitting happens when a model learns the detail and noise in the training data to the extent that it negatively impacts the performance of the model on new data. This means that the noise or random fluctuations in the training data is picked up and learned as concepts by the model. The problem is that these concepts do not apply to new data and negatively impact the models ability to generalize” -https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/

plotcp(m1)

Pruning the tree has started

We are considering 0.045 as the pruned parameter and rebuild the tree

ptree<- prune(m1, cp= 0.045 ,"CP") 
printcp(ptree)


Classification tree:
rpart(formula = Personal_loan ~ ., data = cart.train, method = "class", 
    control = r.ctrl)

Variables actually used in tree construction:
[1] CCAvg          Education      Family_members Income_Monthly Zip_code      

Root node error: 308/3062 = 0.10059

n= 3062 

        CP nsplit rel error  xerror     xstd
1 0.321429      0   1.00000 1.00000 0.054039
2 0.152597      2   0.35714 0.36688 0.033871
3 0.048701      3   0.20455 0.21753 0.026283
4 0.045000      5   0.10714 0.23052 0.027039

Plotting of pruned tree

fancyRpartPlot(ptree, 
               uniform = TRUE, 
               main = "Final Tree", 
               palettes = c("Blues", "Oranges")
               )

predicting on the test set

## Scoring Holdout sample 
cart.test <- test.data
cart.test$predict.class = predict(ptree, cart.test,type = "class")
x<-cart.test$Personal_loan
cart.test$predict.score = predict(ptree, cart.test, type = "prob")
library(caret)
confusionMatrix(table(as.factor(x),cart.test$predict.class ))

Confusion Matrix and Statistics

   
       0    1
  0 1744   22
  1   25  147
                                          
               Accuracy : 0.9757          
                 95% CI : (0.9679, 0.9821)
    No Information Rate : 0.9128          
    P-Value [Acc > NIR] : <2e-16          
                                          
                  Kappa : 0.8489          
 Mcnemar's Test P-Value : 0.7705          
                                          
            Sensitivity : 0.9859          
            Specificity : 0.8698          
         Pos Pred Value : 0.9875          
         Neg Pred Value : 0.8547          
             Prevalence : 0.9128          
         Detection Rate : 0.8999          
   Detection Prevalence : 0.9112          
      Balanced Accuracy : 0.9278          
                                          
       'Positive' Class : 0

AUC/ROC performance metrics

ROC cure for pruned tree

library("ROCR")
Pred.cart = predict(ptree, newdata = cart.test, type = "prob")[,2] 
Pred2 = prediction(Pred.cart, cart.test$Personal_loan) 
plot(performance(Pred2, "tpr", "fpr"))
abline(0, 1, lty = 2)

 
#######################################
##

plotting auc

auc.tmp <- performance(Pred2,"auc")
auc <- as.numeric(auc.tmp@y.values)
print(auc)

[1] 0.973238

Inference : The area under the curve is close to 0.97

Result : The Cart model has given close to 97.5 % accurcy in predicting the people who will take personal loan on the test data

Random Forest model

library(randomForest)
library(caret)
library(e1071)
trainIndex <- createDataPartition(Personal_loan,
                                  p=0.7,
                                  list = FALSE,
                                  times = 1)
base_data_2<-base_data[,-5]
train.data <- base_data_2[trainIndex,2:length(base_data_2) ]
colnames(train.data)<-c('Age_in_years','Experience_years','Income_Monthly','Family_members','CCAvg','Education','Mortgage',
                        'Personal_loan','Securities_Account','CD_Account','Online','CreditCard')
train.data$Personal_loan<-as.factor(train.data$Personal_loan)
train.data<-na.omit(train.data)
test.data  <- base_data_2[-trainIndex,2:length(base_data_2) ]
colnames(test.data)<-c('Age_in_years','Experience_years','Income_Monthly','Family_members','CCAvg','Education','Mortgage',
                        'Personal_loan','Securities_Account','CD_Account','Online','CreditCard')
test.data<-na.omit(test.data)
test.data$Personal_loan<-as.factor(test.data$Personal_loan)
model1 <- randomForest(Personal_loan ~ ., ntree = 100,data = train.data, importance = TRUE)
model1


Call:
 randomForest(formula = Personal_loan ~ ., data = train.data,      ntree = 100, importance = TRUE) 
               Type of random forest: classification
                     Number of trees: 100
No. of variables tried at each split: 3

        OOB estimate of  error rate: 1.41%
Confusion matrix:
     0   1 class.error
0 2730   6 0.002192982
1   37 277 0.117834395

Pred_rf <- predict(model1, test.data, type = 'class')
confusionMatrix(test.data$Personal_loan, Pred_rf)

Confusion Matrix and Statistics

          Reference
Prediction    0    1
         0 1766    2
         1   25  139
                                          
               Accuracy : 0.986           
                 95% CI : (0.9797, 0.9908)
    No Information Rate : 0.927           
    P-Value [Acc > NIR] : < 2.2e-16       
                                          
                  Kappa : 0.9039          
 Mcnemar's Test P-Value : 2.297e-05       
                                          
            Sensitivity : 0.9860          
            Specificity : 0.9858          
         Pos Pred Value : 0.9989          
         Neg Pred Value : 0.8476          
             Prevalence : 0.9270          
         Detection Rate : 0.9141          
   Detection Prevalence : 0.9151          
      Balanced Accuracy : 0.9859          
                                          
       'Positive' Class : 0

Result : Random forest has perfomed very well with 98.9% accuracy on the test data

ROC curve for random forest

library("ROCR")
Pred_rf <- predict(model1, test.data, type = 'prob')[,2]
require(pROC)
rf.roc<-roc(test.data$Personal_loan,Pred_rf)
plot(rf.roc)

#######################################
##

Inference : The ROC is very close to ideal

auc(rf.roc)

Area under the curve: 0.9975

varImpPlot(model1,  
           sort = T,
           n.var=10,
           main="Top 10 - Variable Importance")

Inference

Monthly Income and Education is the most significant factor that decides personal loan

LS0tDQp0aXRsZTogIlRoZXJhIEJhbmsgUGVyc29uYWwgTG9hbiBNb2RlbGxpbmciDQpvdXRwdXQ6DQogIGh0bWxfbm90ZWJvb2s6IGRlZmF1bHQNCiAgaHRtbF9kb2N1bWVudDoNCiAgICBkZl9wcmludDogcGFnZWQNCiAgd29yZF9kb2N1bWVudDogZGVmYXVsdA0KLS0tDQojIE9iamVjdGl2ZQ0KDQpUaGUgZGF0YS1zZXQgcHJvdmlkZXMgZGV0YWlscyBmcm9tIFRoZXJhIEJhbmsgYWJvdXQgYSBQZXJzb25hbCBMb2FuIENhbXBhaWduIHRoYXQgd2FzIGV4ZWN1dGVkIGJ5IHRoZSBiYW5rLiANCjQ5ODIgY3VzdG9tZXJzIHdlcmUgdGFyZ2V0ZWQgd2l0aCBhbiBvZmZlciBvZiBwZXJzb25hbCBsb2FuLCBvdXQgb2Ygd2hpY2ggNDgwIGN1c3RvbWVycyByZXNwb25kZWQgcG9zaXRpdmVseS4gDQpUaGUgZGF0YSBuZWVkcyB0byBiZSB1c2VkIHRvIGNyZWF0ZSBjbGFzc2lmaWNhdGlvbiBtb2RlbChzKSBpbiBvcmRlciB0byBwcmVkaWN0IHRoZSByZXNwb25zZSBvZiBuZXcgc2V0IG9mIGN1c3RvbWVycyBpbiB0aGUgZnV0dXJlLCBkZXBlbmRpbmcgb24gdGhlIGF0dHJpYnV0ZXMgYXZhaWxhYmxlIGluIHRoZSBkYXRhLg0KQ2xhc3NpZmljYXRpb24gTW9kZWxzIHVzaW5nIGZvbGxvd2luZyBTdXBlcnZpc2VkIE1hY2hpbmUgTGVhcm5pbmcgVGVjaG5pcXVlczoNCjEuIEV4cGxvcmF0b3J5IEFuYWx5dGljcw0KMi4gQ2x1c3RlcmluZw0KMy4gQ0FSVCAmIFJhbmRvbSBGb3Jlc3QNCk9uY2UgdGhlIENsYXNzaWZpY2F0aW9uIE1vZGVscyBhcmUgYnVpbHQsIGFmdGVyIHBydW5pbmcgYSByZWNvbW1lbmRhdGlvbiBzeXN0ZW0gdG8gYmUgcHJvdmlkZWQuDQoNCg0KYGBge3J9DQojaW5zdGFsbC5wYWNrYWdlcygiY2FyZXQsIHJlcG9zID0gaHR0cDovL2NyYW4udXMuci1wcm9qZWN0Lm9yZyIpDQojaW5zdGFsbC5wYWNrYWdlcygicnBhcnQsIHJlcG9zID0gaHR0cDovL2NyYW4udXMuci1wcm9qZWN0Lm9yZyIpDQojaW5zdGFsbC5wYWNrYWdlcygicnBhcnQucGxvdCwgcmVwb3MgPSBodHRwOi8vY3Jhbi51cy5yLXByb2plY3Qub3JnIikNCiNpbnN0YWxsLnBhY2thZ2VzKCJyYW5kb21Gb3Jlc3QsIHJlcG9zID0gaHR0cDovL2NyYW4udXMuci1wcm9qZWN0Lm9yZyIpDQoNCmxpYnJhcnkoY2FyZXQpDQpsaWJyYXJ5KHJwYXJ0KQ0KbGlicmFyeShycGFydC5wbG90KQ0KbGlicmFyeShnZ3Bsb3QyKQ0KbGlicmFyeShyYW5kb21Gb3Jlc3QpDQpgYGANCg0KIyMgUmVhZGluZyB0aGUgaW5wdXQgZGF0YQ0KDQpgYGB7cn0NCmJhc2VfZGF0YSA8LXJlYWQuY3N2KCdEOi9GcmVlbGFuY2VyX3F1ZXN0aW9ucy9UaGVyYV9iYW5rL1RoZXJhIEJhbmtfUGVyc29uYWxfTG9hbl9Nb2RlbGxpbmcuY3N2JykNCg0KYGBgDQojIDEpRXhwbG9yYXRvcnkgRGF0YSBBbmFseXNpcyBhbmQgRGVzY3JpcHRpdmUgU3RhdGlzdGljcw0KDQoNCmBgYHtyfQ0KIyBGaW5kIG91dCBUb3RhbCBOdW1iZXIgb2YgUm93cyBhbmQgQ29sdW1ucyANCmRpbShiYXNlX2RhdGEpDQpgYGANCg0KIyMgRmluZCBvdXQgQ2xhc3Mgb2YgZWFjaCBGZWF0dXJlLCBhbG9uZyB3aXRoIGludGVybmFsIHN0cnVjdHVyZSANCg0KYGBge3J9DQoNCnN0cihiYXNlX2RhdGEpIA0KY29sbmFtZXMoYmFzZV9kYXRhKTwtYygnSUQnLCdBZ2VfaW5feWVhcnMnLCdFeHBlcmllbmNlKHllYXJzKScsJ0luY29tZV9Nb250aGx5JywnWmlwX2NvZGUnLCdGYW1pbHlfbWVtYmVycycsJ0NDQXZnJywnRWR1Y2F0aW9uJywnTW9ydGdhZ2UnLCdQZXJzb25hbF9sb2FuJywnU2VjdXJpdGllc19BY2NvdW50JywnQ0RfQWNjb3VudCcsJ09ubGluZScsJ0NyZWRpdENhcmQnKQ0KaGVhZChiYXNlX2RhdGEpDQpgYGANCg0KYGBge3J9DQpwcm9wLnRhYmxlKHRhYmxlKGJhc2VfZGF0YSRQZXJzb25hbF9sb2FuKSkqMTAwDQpgYGANCiMjIHRoZSBjdXN0b21lcnMgd2hvIHRvb2sgcGVyc29uYWwgbG9hbiB2cyBubyBwZXJzb25hbCBsb2FuIHdhcyA5MC40JSBhbmQgOS42JSByZXNwZWN0aXZlbHkNCg0KYGBge3J9DQojIyBVbml2YXJpYXRlIGFuYWx5c2lzDQoNCmhpc3QoYmFzZV9kYXRhJEFnZV9pbl95ZWFycywNCiAgICAgbWFpbiA9ICJIaXN0b2dyYW0gb2YgQWdlIiwNCiAgICAgeGxhYiA9ICJBZ2UgaW4gWWVhcnMiKQ0KDQpgYGANCkluZmVyZW5jZSA6d2UgY2FuIG9ic2VydmUgdGhhdCB0aGUgQWdlIGlzIHZlcnkgY2xvc2UgdG8gdGhlIG5vcm1hbCBkaXN0cmlidXRpb24NCg0KIyMjIENvbnZlcnRpbmcgWmlwY29kZXMvRmFtaWx5bWVtYmVycy9FZHVjYXRpb24vTW9ydGdhZ2UvUGVyc29uYWxMb2FuL1NlY3VyaXRpZXNBY2NvdW50L0NEIEFjY291bnQvT25saW5lL0NyZWRpdGNhcmQgdG8gZmFjdG9ycw0KDQpgYGB7cn0NCmJhc2VfZGF0YSRaaXBfY29kZTwtYXMuZmFjdG9yKGJhc2VfZGF0YSRaaXBfY29kZSkNCmJhc2VfZGF0YSRGYW1pbHlfbWVtYmVyczwtYXMuZmFjdG9yKGJhc2VfZGF0YSRGYW1pbHlfbWVtYmVycykNCmJhc2VfZGF0YSRFZHVjYXRpb248LWFzLmZhY3RvcihiYXNlX2RhdGEkRWR1Y2F0aW9uKQ0KYmFzZV9kYXRhJFBlcnNvbmFsX2xvYW48LWFzLmZhY3RvcihiYXNlX2RhdGEkUGVyc29uYWxfbG9hbikNCmJhc2VfZGF0YSRTZWN1cml0aWVzX0FjY291bnQ8LWFzLmZhY3RvcihiYXNlX2RhdGEkU2VjdXJpdGllc19BY2NvdW50KQ0KYmFzZV9kYXRhJENEX0FjY291bnQ8LWFzLmZhY3RvcihiYXNlX2RhdGEkQ0RfQWNjb3VudCkNCmJhc2VfZGF0YSRPbmxpbmU8LWFzLmZhY3RvcihiYXNlX2RhdGEkT25saW5lKQ0KYmFzZV9kYXRhJENyZWRpdENhcmQ8LWFzLmZhY3RvcihiYXNlX2RhdGEkQ3JlZGl0Q2FyZCkNCg0KYGBgDQoNCiMjIyBCYXJwbG90IG9mIG11bHRpcGxlIGRpbWVuc2lvbnMNCg0KYGBge3J9DQojIEdyb3VwZWQgQmFyIFBsb3QNCmNvdW50cyA8LSB0YWJsZShiYXNlX2RhdGEkRmFtaWx5X21lbWJlcnMsIGJhc2VfZGF0YSRQZXJzb25hbF9sb2FuKQ0KYmFycGxvdChjb3VudHMsIG1haW49IkZhbWlseSBtZW1iZXJzIHZzIFBlcnNvbmFsIExvYW4iLA0KICB4bGFiPSJQZXJzb25hbCBMb2FuIE5vIHZzIFllcyIsIGNvbD1jKCJkYXJrYmx1ZSIsInJlZCIsImdyZWVuIiwieWVsbG93IiksDQogIGxlZ2VuZCA9IHJvd25hbWVzKGNvdW50cyksIGJlc2lkZT1UUlVFKQ0KYGBgDQoNCkluZmVyZW5jZSA6IFdlIGNhbiBjbGVhcmx5IHNlZSB0aGF0IHRob3NlIHBlb3BsZSBoYXZpbmcgbW9yZSBmYW1pbHkgbWVtYmVycyBoYXZlIGhpZ2hlciBsaWtsaWhvb2QgdG8gdGFrZSBsb2FuDQoNCg0KYGBge3J9DQpjb3VudHMgPC0gdGFibGUoYmFzZV9kYXRhJEVkdWNhdGlvbiwgYmFzZV9kYXRhJFBlcnNvbmFsX2xvYW4pDQpiYXJwbG90KGNvdW50cywgbWFpbj0iRWR1Y2F0aW9uIENhdGVnb3J5IHZzIFBlcnNvbmFsIExvYW4iLA0KICB4bGFiPSJQZXJzb25hbCBMb2FuIE5vIHZzIFllcyIsIGNvbD1jKCJkYXJrYmx1ZSIsInJlZCIsImdyZWVuIiksDQogIGxlZ2VuZCA9IGMoIjEgVW5kZXJncmFkIiwgIjIgR3JhZHVhdGUiLCIzIEFkdmFuY2VkL1Byb2Zlc3Npb25hbCIpLCBiZXNpZGU9VFJVRSkNCmBgYA0KDQpJbmZlcmVuY2UgOiBIeXBvdGhlc2lzIDogQWR2YW5jZWQvUHJvZmVzc2lvbmFsIHJlcXVpcmUgbG9hbiBmb3IgaGlnaGVyIHN0dWRpZXMNCg0KIyMjIEJveHBsb3QgZm9yIG51bWVyaWNhbCBkYXRhDQoNCmBgYHtyfQ0KYm94cGxvdChiYXNlX2RhdGEkQWdlX2luX3llYXJzLA0KICAgICAgICBtYWluID0gdG91cHBlcigiQm94cGxvdCBvZiBBZ2UiKSwNCiAgICAgICAgeWxhYiA9ICJBZ2UgaW4geWVhcnMiLA0KICAgICAgICBjb2wgPSAiYmx1ZSIpDQpgYGANCg0KSW5mZXJlbmNlIDogTm90IG11Y2ggb3V0bGllciBpbiBBZ2UgY29sdW1uDQoNCmBgYHtyfQ0KYm94cGxvdChiYXNlX2RhdGEkYEV4cGVyaWVuY2UoeWVhcnMpYCwNCiAgICAgICAgbWFpbiA9IHRvdXBwZXIoIkJveHBsb3Qgb2YgRXhwZXJpZW5jZSIpLA0KICAgICAgICB5bGFiID0gIkV4cGVyaWVuY2UgaW4geWVhcnMiLA0KICAgICAgICBjb2wgPSAicHVycGxlIikNCmBgYA0KSW5mZXJlbmNlIDogTm90IG11Y2ggb3V0bGllciBpbiBFeHBlcmllbmNlIGNvbHVtbg0KDQoNCmBgYHtyfQ0KDQpib3hwbG90KGJhc2VfZGF0YSRJbmNvbWVfTW9udGhseSwNCiAgICAgICAgbWFpbiA9IHRvdXBwZXIoIkJveHBsb3Qgb2YgTW9udGhseSBJbmNvbWUiKSwNCiAgICAgICAgeWxhYiA9ICJNb250aGx5IEluY29tZSIsDQogICAgICAgIGNvbCA9ICJwaW5rIikNCg0KYGBgDQoNCkluZmVyZW5jZSA6IFRoZXJlIGFyZSBsb3RzIG9mIG91dGxpZXJzIGluIHRoZSBtb250aGx5IGluY29tZQ0KDQpgYGB7cn0NCg0KYm94cGxvdChiYXNlX2RhdGEkQ0NBdmcsDQogICAgICAgIG1haW4gPSB0b3VwcGVyKCJCb3hwbG90IG9mIEF2ZXJhZ2UgU3BlbmRpbmcgb2YgY3JlZGl0IGNhcmQgcGVyIG1vbnRoIiksDQogICAgICAgIHlsYWIgPSAiQXZlcmFnZSBTcGVuZGluZyIsDQogICAgICAgIGNvbCA9ICJtYXJvb24iKQ0KDQpgYGANCkluZmVyZW5jZSA6IEhlcmUgdG9vIGluIHRoZSBhdmVyYWdlIHNwZW5kaW5nIG9mIGNyZWRpdCBjYXJkIHBlciBtb250aCB0aGVyZSBhcmUgbG90cyBvZiBvdXRsaWVycw0KDQpgYGB7cn0NCg0KYm94cGxvdChiYXNlX2RhdGEkTW9ydGdhZ2UsDQogICAgICAgIG1haW4gPSB0b3VwcGVyKCJCb3hwbG90IG9mIEhvdXNlIE1vcnRnYWdlIGlmIGFueSIpLA0KICAgICAgICB5bGFiID0gIkhvdXNlIE1vcnRnYWciLA0KICAgICAgICBjb2wgPSAibWFyb29uIikNCg0KYGBgDQpJbmZlcmVuY2UgOiBIZXJlIHRvbyBpbiB0aGVyZSBhcmUgbG90cyBvZiBvdXRsaWVycw0KDQojIyMgQ29ycmVsYXRpb24gYmV0d2VlbiB0aGUgbnVtZXJpYyBmZWF0dXJlcw0KDQpgYGB7cn0NCg0KbXlfZGF0YSA8LSBiYXNlX2RhdGFbLCBjKDIsMyw0LDcsOSldDQpyZXMgPC0gY29yKG15X2RhdGEpDQpyb3VuZChyZXMsIDIpDQpgYGANCg0KSW5mZXJlbmNlDQoNCjEpIEFnZSBpbiBZZWFycyBhbmQgRXhwZXJpZW5jZSBhcmUgaGlnaGx5IHBvc2l0aXZlbHkgY29ycmVsYXRlZA0KMikgTW9udGhseSBJbmNvbWUgYW5kIEF2ZXJhZ2UgY3JlZGl0IGNhcmQgc3BlbmQgaXMgYWxzbyBoaWdobHkgcG9zaXRpdmVseSBjb3JyZWxhdGVkDQoNCiMgMikgQ2x1c3RlciBBbmFseXNpcw0KDQpDbHVzdGVyaW5nIGZlYXR1cmVzIGFyZSBvbmx5IG51bWJlcmljYWwNCg0KQWxsIHRoZSBjYXRlZ29yaWNhbCBmZWF0dXJlcyBoYXZlIG5vdCBiZWVuIGNvbnNpZGVyZWQgYXMgdGhleSBkbyBub3QgbWFrZSBtdWNoIHNlbnNlIHdoZW4gd2UgZG8gY2x1c3RlcmluZw0KDQoxKSBBZ2UgaW4gWWVhcnMNCjIpIEV4cGVyaWVuY2UNCjMpIE1vbnRobHkgSW5jb21lDQo0KSBDQ0F2Zw0KNSkgTW9ydGdhZ2UNCg0KYGBge3J9DQp3c3MgPC0gKG5yb3cobXlfZGF0YSktMSkqc3VtKGFwcGx5KG15X2RhdGEsMix2YXIpKQ0KDQpmb3IoaSBpbiAyOjE1KXdzc1tpXTwtIHN1bShmaXQ9a21lYW5zKG15X2RhdGEsY2VudGVycz1pLDE1KSR3aXRoaW5zcykNCg0KcGxvdCgxOjE1LHdzcyx0eXBlPSJiIixtYWluPSIxNSBjbHVzdGVycyIseGxhYj0ibm8uIG9mIGNsdXN0ZXIiLHlsYWI9IndpdGggY2xzdXRlciBzdW0gb2Ygc3F1YXJlcyIpDQoNCmBgYA0KDQpBIGZ1bmRhbWVudGFsIHN0ZXAgZm9yIGFueSB1bnN1cGVydmlzZWQgYWxnb3JpdGhtIGlzIHRvIGRldGVybWluZSB0aGUgb3B0aW1hbCBudW1iZXIgb2YgY2x1c3RlcnMgaW50byB3aGljaCB0aGUgZGF0YSBtYXkgYmUgY2x1c3RlcmVkLiBUaGUgRWxib3cgTWV0aG9kIGlzIG9uZSBvZiB0aGUgbW9zdCBwb3B1bGFyIG1ldGhvZHMgdG8gZGV0ZXJtaW5lIHRoaXMgb3B0aW1hbCB2YWx1ZSBvZiBrLg0KDQpXZSBub3cgZGVtb25zdHJhdGUgdGhlIGdpdmVuIG1ldGhvZCB1c2luZyB0aGUgSy1NZWFucyBjbHVzdGVyaW5nIHRlY2huaXF1ZSANCg0KDQpJbmZlcmVuY2UgOiBCYXNlZCBvbiB0aGUgZWxib3cgY3VydmUgd2UgY2FuIHNlZSA0IGNsdXN0ZXJzIGZvcm1lZA0KDQpgYGB7cn0NCmZpdCA8LSBrbWVhbnMobXlfZGF0YSw0KQ0KbGlicmFyeShjbHVzdGVyKQ0KbGlicmFyeShmcGMpDQpwbG90Y2x1c3RlcihteV9kYXRhLGZpdCRjbHVzdGVyKQ0KcG9pbnRzKGZpdCRjZW50ZXJzLGNvbD0xOjgscGNoPTE2KQ0KYGBgDQoNCiMjIyBnZXR0aW5nIHRoZSBjbHVzdGVyIG1lYW5zDQoNCmBgYHtyfQ0KDQpteWRhdGEgPC0gZGF0YS5mcmFtZShteV9kYXRhLGZpdCRjbHVzdGVyKQ0KY2x1c3Rlcl9tZWFuIDwtIGFnZ3JlZ2F0ZShteWRhdGEsYnkgPSBsaXN0KGZpdCRjbHVzdGVyKSxGVU4gPSBtZWFuKQ0KY2x1c3Rlcl9tZWFuDQpgYGANCg0KDQoiSXQgaXMgaW1wb3J0YW50IHRvIHJlbWVtYmVyIHRoYXQgRGF0YSBBbmFseXRpY3MgUHJvamVjdHMgcmVxdWlyZSBhIGRlbGljYXRlIGJhbGFuY2UgYmV0d2VlbiBleHBlcmltZW50YXRpb24sIGludHVpdGlvbiwgYnV0IGFsc28gZm9sbG93aW5nIChvbmNlIGEgd2hpbGUpIGEgcHJvY2VzcyB0byBhdm9pZCBnZXR0aW5nIGZvb2xlZCBieSByYW5kb21uZXNzIGFuZCDigJxmaW5kaW5nIHJlc3VsdHMgYW5kIHBhdHRlcm5z4oCdIHRoYXQgYXJlIG1haW5seSBkcml2ZW4gYnkgb3VyIG93biBiaWFzZXMgYW5kIG5vdCBieSB0aGUgZmFjdHMvZGF0YSB0aGVtc2VsdmVzIi1odHRwczovL2luc2VhZGRhdGFhbmFseXRpY3MuZ2l0aHViLmlvL0lOU0VBREFuYWx5dGljcy9Db3Vyc2VTZXNzaW9ucy9TZXNzaW9uczQ1L0NsdXN0ZXJBbmFseXNpc1JlYWRpbmcuaHRtbA0KDQojIyMgQXMgS21lYW5zIGlzIHByb25lIHRvIG91dGxpZXJzIGxldHMgcmVjbHVzdGVyIHRoZW0gYWZ0ZXIgb3V0bGllciByZW1vdmFsDQoNCmBgYHtyfQ0KbXlfZGF0YTI8LW15X2RhdGENCg0Kb3V0bGllcnMzIDwtIGJveHBsb3QobXlfZGF0YTIkSW5jb21lX01vbnRobHksIHBsb3Q9RkFMU0UpDQpvdXRsaWVyczM8LW91dGxpZXJzMyRvdXQNCg0KbXlfZGF0YTIgPC0gbXlfZGF0YTJbLXdoaWNoKG15X2RhdGEyJEluY29tZV9Nb250aGx5ICVpbiUgb3V0bGllcnMzKSxdDQoNCm91dGxpZXJzNCA8LSBib3hwbG90KG15X2RhdGEyJENDQXZnLCBwbG90PUZBTFNFKQ0Kb3V0bGllcnM0PC1vdXRsaWVyczQkb3V0DQoNCm15X2RhdGEyIDwtIG15X2RhdGEyWy13aGljaChteV9kYXRhMiRDQ0F2ZyAlaW4lIG91dGxpZXJzNCksXQ0KDQpvdXRsaWVyczUgPC0gYm94cGxvdChteV9kYXRhMiRNb3J0Z2FnZSwgcGxvdD1GQUxTRSkNCm91dGxpZXJzNTwtb3V0bGllcnM1JG91dA0KDQpteV9kYXRhMiA8LSBteV9kYXRhMlstd2hpY2gobXlfZGF0YTIkTW9ydGdhZ2UgJWluJSBvdXRsaWVyczUpLF0NCg0KbnJvdyhteV9kYXRhMikNCg0KYGBgDQpJbmZlcmVuY2UgOiBPdXRsaWVycyBoYXZlIGJlZW4gc3VjY2Vzc2Z1bGx5IHJlbW92ZWQNCg0KDQojIHBsb3R0aW5nIGVsYm93IGN1cnZlIGZvciB0aGUgb3V0bGllciByZW1vdmVkIGRhdGENCg0KYGBge3J9DQp3c3MgPC0gKG5yb3cobXlfZGF0YTIpLTEpKnN1bShhcHBseShteV9kYXRhMiwyLHZhcikpDQoNCmZvcihpIGluIDI6MTUpd3NzW2ldPC0gc3VtKGZpdDI9a21lYW5zKG15X2RhdGEyLGNlbnRlcnM9aSwxNSkkd2l0aGluc3MpDQoNCnBsb3QoMToxNSx3c3MsdHlwZT0iYiIsbWFpbj0iMTUgY2x1c3RlcnMiLHhsYWI9Im5vLiBvZiBjbHVzdGVyIix5bGFiPSJ3aXRoIGNsc3V0ZXIgc3VtIG9mIHNxdWFyZXMiKQ0KYGBgDQpJbmZlcmVuY2UgOiA1IGNsdXN0ZXJzIG1ha2Ugc2Vuc2UgaGVyZQ0KDQpgYGB7cn0NCmZpdDI8LWttZWFucyhteV9kYXRhMiw1KQ0KbXlfZGF0YTMgPC0gZGF0YS5mcmFtZShteV9kYXRhMikNCmNsdXN0ZXJfbWVhbl8yIDwtIGFnZ3JlZ2F0ZShteV9kYXRhMyxieSA9IGxpc3QoZml0MiRjbHVzdGVyKSxGVU4gPSBtZWFuKQ0KY2x1c3Rlcl9tZWFuXzINCmBgYA0KDQpJbmZlcmVuY2UgOiBUaGUgNSBjbHVzdGVycyBtYWtlIG11Y2ggbW9yZSBzZW5zZSBhZnRlciBvdXRsaWVyIHJlbW92YWwNCg0KYGBge3J9DQoNCm15X2RhdGEyJGNsdXN0ZXI8LWZpdDIkY2x1c3Rlcg0KDQpsaWJyYXJ5KGRwbHlyKQ0KDQpoZWFkKG15X2RhdGEyKQ0KDQppbmRleDwtYXMuaW50ZWdlcihyb3cubmFtZXMuZGF0YS5mcmFtZShteV9kYXRhMikpDQoNClBlcnNvbmFsX2xvYW48LWJhc2VfZGF0YVtpbmRleCwxMF0NCg0KbXlfZGF0YTIkUGVyc29uYWxfbG9hbjwtUGVyc29uYWxfbG9hbg0KaGVhZChteV9kYXRhMikNCg0KYGBgDQoNCkluZmVyZW5jZSA6IFdlIGhhdmUgZ290IHRoZSB0d28gY2x1c3Rlcg0KDQojIyMgVG8gY2hlY2sgdGhlIFBlcnNvbmFsX2xvYW4gdnMgQ2x1c3RlciBiYXJjaGFydA0KDQpgYGB7cn0NCiMgR3JvdXBlZCBCYXIgUGxvdA0KY291bnRzIDwtIHRhYmxlKCBteV9kYXRhMiRQZXJzb25hbF9sb2FuLG15X2RhdGEyJGNsdXN0ZXIpDQpiYXJwbG90KGNvdW50cywgbWFpbj0iRmFtaWx5IG1lbWJlcnMgdnMgUGVyc29uYWwgTG9hbiIsDQogIHhsYWI9IlBlcnNvbmFsIExvYW4gTm8gdnMgWWVzIiwgY29sPWMoInJlZCIsImdyZWVuIiksDQogIGxlZ2VuZCA9IGMoIlBlcnNvbmFsX0xvYW5fTm8iLCJQZXJzb25hbF9Mb2FuX1llcyIpLCBiZXNpZGU9VFJVRSkNCg0KYGBgDQoNCkluZmVyZW5jZSA6IFRhcmdldGluZyB0aGUgY2x1c3RlciA0IHNlZ21lbnQgd291bGQgYmUgdGhlIGJlc3Qgb3B0aW9uIGZvciBjb252ZXJzaW9uIHJhdGUgdG8gYmUgaGlnaGVyLCBhbHNvIHRoZSBwb3B1bGF0aW9uIGlzIGNsb3NlIHRvIDUwMA0KDQpUaGlzIHdvdWxkIGhlbHAgdGhlIGNvbXBhbnkgdG8gc3BlbmQgdGhlIG1hcmtldGluZyBtb25leSBvbiB0aGUgY29ycmVjdCBjdXN0b21lcnMgcmF0aGVyIHRoYW4gd2FzdGUgaXQgb24gYWxsIGN1c3RvbWVycw0KDQoNCiMgQ2FydCBhbmQgUmFuZG9tRm9yZXN0IGFsZ29yaXRobQ0KDQoNCkNyZWF0aW5nIFRyYWluaW5nIGFuZCBUZXN0aW5nIERhdGFzZXQNClRoZSBnaXZlbiBkYXRhIHNldCBpcyBkaXZpZGVkIGludG8gVHJhaW5pbmcgYW5kIFRlc3RpbmcgZGF0YSBzZXQsIHdpdGggNzA6MzAgcHJvcG9ydGlvbi4NClRoZSBkaXN0cmlidXRpb24gb2YgUmVzcG9uZGVyIGFuZCBOb24gUmVzcG9uZGVyIENsYXNzIGlzIHZlcmlmaWVkIGluIGJvdGggdGhlIGRhdGEgc2V0cywgYW5kIGVuc3VyZWQgaXTigJlzIGNsb3NlIHRvIGVxdWFsLg0KDQpgYGB7cn0NCnNldC5zZWVkKDExMSkNCnRyYWluSW5kZXggPC0gY3JlYXRlRGF0YVBhcnRpdGlvbihQZXJzb25hbF9sb2FuLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHA9MC43LA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpc3QgPSBGQUxTRSwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0aW1lcyA9IDEpDQp0cmFpbi5kYXRhIDwtIGJhc2VfZGF0YVt0cmFpbkluZGV4LDI6bGVuZ3RoKGJhc2VfZGF0YSkgXQ0KdGVzdC5kYXRhICA8LSBiYXNlX2RhdGFbLXRyYWluSW5kZXgsMjpsZW5ndGgoYmFzZV9kYXRhKSBdDQpgYGANCg0KDQpNb2RlbCBCdWlsZGluZyAtIENBUlQgKFVuYmFsYW5jZWQgRGF0YXNldCkNClNldHRpbmcgdGhlIGNvbnRyb2wgcGFyYW1ldGVyIGlucHV0cyBmb3IgcnBhcnQNCg0KYGBge3J9DQpyLmN0cmwgPC0gcnBhcnQuY29udHJvbChtaW5zcGxpdCA9IDEwMCwNCiAgICAgICAgICAgICAgICAgICAgICAgIG1pbmJ1Y2tldCA9IDEwLA0KICAgICAgICAgICAgICAgICAgICAgICAgY3AgPSAwLA0KICAgICAgICAgICAgICAgICAgICAgICAgeHZhbCA9IDEwDQogICAgICAgICAgICAgICAgICAgICAgICApDQojRXhjbHVkZSBjb2x1bW5zIC0gIkN1c3RvbWVyIElEIiBhbmQgIkFjY3QgT3BlbmluZyBEYXRlIg0KY2FydC50cmFpbiA8LSB0cmFpbi5kYXRhDQptMSA8LSBycGFydChmb3JtdWxhID0gUGVyc29uYWxfbG9hbn4uLA0KICAgICAgICAgICAgZGF0YSA9IGNhcnQudHJhaW4sDQogICAgICAgICAgICBtZXRob2QgPSAiY2xhc3MiLA0KICAgICAgICAgICAgY29udHJvbCA9IHIuY3RybA0KICAgICAgICAgICAgKQ0KI2luc3RhbGwucGFja2FnZXMoInJhdHRsZSIpIA0KI2luc3RhbGwucGFja2FnZXMoIlJDb2xvckJyZXdlciIpIA0KbGlicmFyeShyYXR0bGUpIA0KbGlicmFyeShSQ29sb3JCcmV3ZXIpIA0KDQpmYW5jeVJwYXJ0UGxvdChtMSkgDQoNCmBgYA0KVmFyaWFibGVzIHVzZWQgaW4gdGhlIHRyZWUgY29uc3RydWN0aW9uDQoNCmBgYHtyfQ0KcHJpbnRjcChtMSkgDQpgYGANCg0KDQojIyMgUHJ1bmluZyB0aGUgY2FydCB0cmVlIHRvIGVuc3VyZSB0aGF0IHRoZSBtb2RlbCBpcyBub3Qgb3ZlcmZpdHRpbmcNCg0KIk92ZXJmaXR0aW5nIGhhcHBlbnMgd2hlbiBhIG1vZGVsIGxlYXJucyB0aGUgZGV0YWlsIGFuZCBub2lzZSBpbiB0aGUgdHJhaW5pbmcgZGF0YSB0byB0aGUgZXh0ZW50IHRoYXQgaXQgbmVnYXRpdmVseSBpbXBhY3RzIHRoZSBwZXJmb3JtYW5jZSBvZiB0aGUgbW9kZWwgb24gbmV3IGRhdGEuIFRoaXMgbWVhbnMgdGhhdCB0aGUgbm9pc2Ugb3IgcmFuZG9tIGZsdWN0dWF0aW9ucyBpbiB0aGUgdHJhaW5pbmcgZGF0YSBpcyBwaWNrZWQgdXAgYW5kIGxlYXJuZWQgYXMgY29uY2VwdHMgYnkgdGhlIG1vZGVsLiBUaGUgcHJvYmxlbSBpcyB0aGF0IHRoZXNlIGNvbmNlcHRzIGRvIG5vdCBhcHBseSB0byBuZXcgZGF0YSBhbmQgbmVnYXRpdmVseSBpbXBhY3QgdGhlIG1vZGVscyBhYmlsaXR5IHRvIGdlbmVyYWxpemUiIC1odHRwczovL21hY2hpbmVsZWFybmluZ21hc3RlcnkuY29tL292ZXJmaXR0aW5nLWFuZC11bmRlcmZpdHRpbmctd2l0aC1tYWNoaW5lLWxlYXJuaW5nLWFsZ29yaXRobXMvDQoNCg0KYGBge3J9DQpwbG90Y3AobTEpIA0KYGBgDQpQcnVuaW5nIHRoZSB0cmVlIGhhcyBzdGFydGVkDQoNCldlIGFyZSBjb25zaWRlcmluZyAwLjA0NSBhcyB0aGUgcHJ1bmVkIHBhcmFtZXRlciBhbmQgcmVidWlsZCB0aGUgdHJlZQ0KDQpgYGB7cn0NCnB0cmVlPC0gcHJ1bmUobTEsIGNwPSAwLjA0NSAsIkNQIikgDQpwcmludGNwKHB0cmVlKQ0KYGBgDQoNCg0KIyMjIFBsb3R0aW5nIG9mIHBydW5lZCB0cmVlDQoNCmBgYHtyfQ0KZmFuY3lScGFydFBsb3QocHRyZWUsIA0KICAgICAgICAgICAgICAgdW5pZm9ybSA9IFRSVUUsIA0KICAgICAgICAgICAgICAgbWFpbiA9ICJGaW5hbCBUcmVlIiwgDQogICAgICAgICAgICAgICBwYWxldHRlcyA9IGMoIkJsdWVzIiwgIk9yYW5nZXMiKQ0KICAgICAgICAgICAgICAgKQ0KYGBgDQojIyMgcHJlZGljdGluZyBvbiB0aGUgdGVzdCBzZXQNCg0KYGBge3J9DQoNCiMjIFNjb3JpbmcgSG9sZG91dCBzYW1wbGUgDQpjYXJ0LnRlc3QgPC0gdGVzdC5kYXRhDQpjYXJ0LnRlc3QkcHJlZGljdC5jbGFzcyA9IHByZWRpY3QocHRyZWUsIGNhcnQudGVzdCx0eXBlID0gImNsYXNzIikNCg0KDQoNCng8LWNhcnQudGVzdCRQZXJzb25hbF9sb2FuDQoNCmNhcnQudGVzdCRwcmVkaWN0LnNjb3JlID0gcHJlZGljdChwdHJlZSwgY2FydC50ZXN0LCB0eXBlID0gInByb2IiKQ0KbGlicmFyeShjYXJldCkNCmNvbmZ1c2lvbk1hdHJpeCh0YWJsZShhcy5mYWN0b3IoeCksY2FydC50ZXN0JHByZWRpY3QuY2xhc3MgKSkNCg0KYGBgDQoNCiMjIyBBVUMvUk9DIHBlcmZvcm1hbmNlIG1ldHJpY3MNCg0KIyMjIFJPQyBjdXJlIGZvciBwcnVuZWQgdHJlZQ0KDQpgYGB7cn0NCmxpYnJhcnkoIlJPQ1IiKQ0KUHJlZC5jYXJ0ID0gcHJlZGljdChwdHJlZSwgbmV3ZGF0YSA9IGNhcnQudGVzdCwgdHlwZSA9ICJwcm9iIilbLDJdIA0KUHJlZDIgPSBwcmVkaWN0aW9uKFByZWQuY2FydCwgY2FydC50ZXN0JFBlcnNvbmFsX2xvYW4pIA0KcGxvdChwZXJmb3JtYW5jZShQcmVkMiwgInRwciIsICJmcHIiKSkNCmFibGluZSgwLCAxLCBsdHkgPSAyKQ0KIA0KIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjDQojIw0KYGBgDQoNCiMjIyBwbG90dGluZyBhdWMNCg0KYGBge3J9DQphdWMudG1wIDwtIHBlcmZvcm1hbmNlKFByZWQyLCJhdWMiKQ0KYXVjIDwtIGFzLm51bWVyaWMoYXVjLnRtcEB5LnZhbHVlcykNCnByaW50KGF1YykNCmBgYA0KDQpJbmZlcmVuY2UgOiBUaGUgYXJlYSB1bmRlciB0aGUgY3VydmUgaXMgY2xvc2UgdG8gMC45Nw0KDQoNClJlc3VsdCA6IFRoZSBDYXJ0IG1vZGVsIGhhcyBnaXZlbiBjbG9zZSB0byA5Ny41ICUgYWNjdXJjeSBpbiBwcmVkaWN0aW5nIHRoZSBwZW9wbGUgd2hvIHdpbGwgdGFrZSBwZXJzb25hbCBsb2FuIG9uIHRoZSB0ZXN0IGRhdGENCg0KIyMjIFJhbmRvbSBGb3Jlc3QgbW9kZWwNCg0KDQpgYGB7cn0NCmxpYnJhcnkocmFuZG9tRm9yZXN0KQ0KDQpsaWJyYXJ5KGNhcmV0KQ0KbGlicmFyeShlMTA3MSkNCnRyYWluSW5kZXggPC0gY3JlYXRlRGF0YVBhcnRpdGlvbihQZXJzb25hbF9sb2FuLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHA9MC43LA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxpc3QgPSBGQUxTRSwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0aW1lcyA9IDEpDQpiYXNlX2RhdGFfMjwtYmFzZV9kYXRhWywtNV0NCnRyYWluLmRhdGEgPC0gYmFzZV9kYXRhXzJbdHJhaW5JbmRleCwyOmxlbmd0aChiYXNlX2RhdGFfMikgXQ0KY29sbmFtZXModHJhaW4uZGF0YSk8LWMoJ0FnZV9pbl95ZWFycycsJ0V4cGVyaWVuY2VfeWVhcnMnLCdJbmNvbWVfTW9udGhseScsJ0ZhbWlseV9tZW1iZXJzJywnQ0NBdmcnLCdFZHVjYXRpb24nLCdNb3J0Z2FnZScsDQogICAgICAgICAgICAgICAgICAgICAgICAnUGVyc29uYWxfbG9hbicsJ1NlY3VyaXRpZXNfQWNjb3VudCcsJ0NEX0FjY291bnQnLCdPbmxpbmUnLCdDcmVkaXRDYXJkJykNCnRyYWluLmRhdGEkUGVyc29uYWxfbG9hbjwtYXMuZmFjdG9yKHRyYWluLmRhdGEkUGVyc29uYWxfbG9hbikNCg0KdHJhaW4uZGF0YTwtbmEub21pdCh0cmFpbi5kYXRhKQ0KdGVzdC5kYXRhICA8LSBiYXNlX2RhdGFfMlstdHJhaW5JbmRleCwyOmxlbmd0aChiYXNlX2RhdGFfMikgXQ0KY29sbmFtZXModGVzdC5kYXRhKTwtYygnQWdlX2luX3llYXJzJywnRXhwZXJpZW5jZV95ZWFycycsJ0luY29tZV9Nb250aGx5JywnRmFtaWx5X21lbWJlcnMnLCdDQ0F2ZycsJ0VkdWNhdGlvbicsJ01vcnRnYWdlJywNCiAgICAgICAgICAgICAgICAgICAgICAgICdQZXJzb25hbF9sb2FuJywnU2VjdXJpdGllc19BY2NvdW50JywnQ0RfQWNjb3VudCcsJ09ubGluZScsJ0NyZWRpdENhcmQnKQ0KDQp0ZXN0LmRhdGE8LW5hLm9taXQodGVzdC5kYXRhKQ0KdGVzdC5kYXRhJFBlcnNvbmFsX2xvYW48LWFzLmZhY3Rvcih0ZXN0LmRhdGEkUGVyc29uYWxfbG9hbikNCg0KDQptb2RlbDEgPC0gcmFuZG9tRm9yZXN0KFBlcnNvbmFsX2xvYW4gfiAuLCBudHJlZSA9IDEwMCxkYXRhID0gdHJhaW4uZGF0YSwgaW1wb3J0YW5jZSA9IFRSVUUpDQptb2RlbDENClByZWRfcmYgPC0gcHJlZGljdChtb2RlbDEsIHRlc3QuZGF0YSwgdHlwZSA9ICdjbGFzcycpDQpjb25mdXNpb25NYXRyaXgodGVzdC5kYXRhJFBlcnNvbmFsX2xvYW4sIFByZWRfcmYpDQoNCg0KDQpgYGANClJlc3VsdCA6IFJhbmRvbSBmb3Jlc3QgaGFzIHBlcmZvbWVkIHZlcnkgd2VsbCB3aXRoIDk4LjklIGFjY3VyYWN5IG9uIHRoZSB0ZXN0IGRhdGENCg0KDQojIyMgUk9DIGN1cnZlIGZvciByYW5kb20gZm9yZXN0DQoNCmBgYHtyfQ0KbGlicmFyeSgiUk9DUiIpDQoNClByZWRfcmYgPC0gcHJlZGljdChtb2RlbDEsIHRlc3QuZGF0YSwgdHlwZSA9ICdwcm9iJylbLDJdDQoNCnJlcXVpcmUocFJPQykNCnJmLnJvYzwtcm9jKHRlc3QuZGF0YSRQZXJzb25hbF9sb2FuLFByZWRfcmYpDQpwbG90KHJmLnJvYykNCg0KIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjDQojIw0KYGBgDQoNCg0KSW5mZXJlbmNlIDogVGhlIFJPQyBpcyB2ZXJ5IGNsb3NlIHRvIGlkZWFsDQoNCmBgYHtyfQ0KYXVjKHJmLnJvYykNCmBgYA0KDQoNCmBgYHtyfQ0KdmFySW1wUGxvdChtb2RlbDEsICANCiAgICAgICAgICAgc29ydCA9IFQsDQogICAgICAgICAgIG4udmFyPTEwLA0KICAgICAgICAgICBtYWluPSJUb3AgMTAgLSBWYXJpYWJsZSBJbXBvcnRhbmNlIikNCmBgYA0KDQpJbmZlcmVuY2UNCg0KMSkgTW9udGhseSBJbmNvbWUgYW5kIEVkdWNhdGlvbiBpcyB0aGUgbW9zdCBzaWduaWZpY2FudCBmYWN0b3IgdGhhdCBkZWNpZGVzIHBlcnNvbmFsIGxvYW4=

Thera Bank Personal Loan Modelling

Objective

Reading the input data

1)Exploratory Data Analysis and Descriptive Statistics

Find out Class of each Feature, along with internal structure

the customers who took personal loan vs no personal loan was 90.4% and 9.6% respectively

Converting Zipcodes/Familymembers/Education/Mortgage/PersonalLoan/SecuritiesAccount/CD Account/Online/Creditcard to factors

Barplot of multiple dimensions

Boxplot for numerical data

Correlation between the numeric features

2) Cluster Analysis

getting the cluster means

As Kmeans is prone to outliers lets recluster them after outlier removal

plotting elbow curve for the outlier removed data

To check the Personal_loan vs Cluster barchart

Cart and RandomForest algorithm

Pruning the cart tree to ensure that the model is not overfitting

Plotting of pruned tree

predicting on the test set

AUC/ROC performance metrics

ROC cure for pruned tree

plotting auc

Random Forest model

ROC curve for random forest