Refer to http://archive.ics.uci.edu/ml/datasets/Statlog+(German+Credit+Data)) for variable description. The response variable is Class and all others are predictors.
Only run the following code once to install the package caret. The German credit scoring data in provided in that package.
install.packages('caret')
library(caret) #this package contains the german data with its numeric format
## Warning: package 'caret' was built under R version 4.1.3
## Loading required package: ggplot2
## Loading required package: lattice
data(GermanCredit)
GermanCredit$Class <- GermanCredit$Class == "Good" # use this code to convert `Class` into True or False (equivalent to 1 or 0)
str(GermanCredit)
## 'data.frame': 1000 obs. of 62 variables:
## $ Duration : int 6 48 12 42 24 36 24 36 12 30 ...
## $ Amount : int 1169 5951 2096 7882 4870 9055 2835 6948 3059 5234 ...
## $ InstallmentRatePercentage : int 4 2 2 2 3 2 3 2 2 4 ...
## $ ResidenceDuration : int 4 2 3 4 4 4 4 2 4 2 ...
## $ Age : int 67 22 49 45 53 35 53 35 61 28 ...
## $ NumberExistingCredits : int 2 1 1 1 2 1 1 1 1 2 ...
## $ NumberPeopleMaintenance : int 1 1 2 2 2 2 1 1 1 1 ...
## $ Telephone : num 0 1 1 1 1 0 1 0 1 1 ...
## $ ForeignWorker : num 1 1 1 1 1 1 1 1 1 1 ...
## $ Class : logi TRUE FALSE TRUE TRUE FALSE TRUE ...
## $ CheckingAccountStatus.lt.0 : num 1 0 0 1 1 0 0 0 0 0 ...
## $ CheckingAccountStatus.0.to.200 : num 0 1 0 0 0 0 0 1 0 1 ...
## $ CheckingAccountStatus.gt.200 : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CheckingAccountStatus.none : num 0 0 1 0 0 1 1 0 1 0 ...
## $ CreditHistory.NoCredit.AllPaid : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CreditHistory.ThisBank.AllPaid : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CreditHistory.PaidDuly : num 0 1 0 1 0 1 1 1 1 0 ...
## $ CreditHistory.Delay : num 0 0 0 0 1 0 0 0 0 0 ...
## $ CreditHistory.Critical : num 1 0 1 0 0 0 0 0 0 1 ...
## $ Purpose.NewCar : num 0 0 0 0 1 0 0 0 0 1 ...
## $ Purpose.UsedCar : num 0 0 0 0 0 0 0 1 0 0 ...
## $ Purpose.Furniture.Equipment : num 0 0 0 1 0 0 1 0 0 0 ...
## $ Purpose.Radio.Television : num 1 1 0 0 0 0 0 0 1 0 ...
## $ Purpose.DomesticAppliance : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Purpose.Repairs : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Purpose.Education : num 0 0 1 0 0 1 0 0 0 0 ...
## $ Purpose.Vacation : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Purpose.Retraining : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Purpose.Business : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Purpose.Other : num 0 0 0 0 0 0 0 0 0 0 ...
## $ SavingsAccountBonds.lt.100 : num 0 1 1 1 1 0 0 1 0 1 ...
## $ SavingsAccountBonds.100.to.500 : num 0 0 0 0 0 0 0 0 0 0 ...
## $ SavingsAccountBonds.500.to.1000 : num 0 0 0 0 0 0 1 0 0 0 ...
## $ SavingsAccountBonds.gt.1000 : num 0 0 0 0 0 0 0 0 1 0 ...
## $ SavingsAccountBonds.Unknown : num 1 0 0 0 0 1 0 0 0 0 ...
## $ EmploymentDuration.lt.1 : num 0 0 0 0 0 0 0 0 0 0 ...
## $ EmploymentDuration.1.to.4 : num 0 1 0 0 1 1 0 1 0 0 ...
## $ EmploymentDuration.4.to.7 : num 0 0 1 1 0 0 0 0 1 0 ...
## $ EmploymentDuration.gt.7 : num 1 0 0 0 0 0 1 0 0 0 ...
## $ EmploymentDuration.Unemployed : num 0 0 0 0 0 0 0 0 0 1 ...
## $ Personal.Male.Divorced.Seperated : num 0 0 0 0 0 0 0 0 1 0 ...
## $ Personal.Female.NotSingle : num 0 1 0 0 0 0 0 0 0 0 ...
## $ Personal.Male.Single : num 1 0 1 1 1 1 1 1 0 0 ...
## $ Personal.Male.Married.Widowed : num 0 0 0 0 0 0 0 0 0 1 ...
## $ Personal.Female.Single : num 0 0 0 0 0 0 0 0 0 0 ...
## $ OtherDebtorsGuarantors.None : num 1 1 1 0 1 1 1 1 1 1 ...
## $ OtherDebtorsGuarantors.CoApplicant : num 0 0 0 0 0 0 0 0 0 0 ...
## $ OtherDebtorsGuarantors.Guarantor : num 0 0 0 1 0 0 0 0 0 0 ...
## $ Property.RealEstate : num 1 1 1 0 0 0 0 0 1 0 ...
## $ Property.Insurance : num 0 0 0 1 0 0 1 0 0 0 ...
## $ Property.CarOther : num 0 0 0 0 0 0 0 1 0 1 ...
## $ Property.Unknown : num 0 0 0 0 1 1 0 0 0 0 ...
## $ OtherInstallmentPlans.Bank : num 0 0 0 0 0 0 0 0 0 0 ...
## $ OtherInstallmentPlans.Stores : num 0 0 0 0 0 0 0 0 0 0 ...
## $ OtherInstallmentPlans.None : num 1 1 1 1 1 1 1 1 1 1 ...
## $ Housing.Rent : num 0 0 0 0 0 0 0 1 0 0 ...
## $ Housing.Own : num 1 1 1 0 0 0 1 0 1 1 ...
## $ Housing.ForFree : num 0 0 0 1 1 1 0 0 0 0 ...
## $ Job.UnemployedUnskilled : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Job.UnskilledResident : num 0 0 1 0 0 1 0 0 1 0 ...
## $ Job.SkilledEmployee : num 1 1 0 1 1 0 1 0 0 0 ...
## $ Job.Management.SelfEmp.HighlyQualified: num 0 0 0 0 0 0 0 1 0 1 ...
Your observation: Most of the variables appear to be boolean, the values can only be either 0 or 1 for most of them and we need to change the datavtypes to treat the variables accordingly.
#This is an optional code that drop variables that provide no information in the data
GermanCredit = GermanCredit[,-c(14,19,27,30,35,40,44,45,48,52,55,58,62)]
Summary helps us visualize the data, and the basic distribution of the values stored within the variables.
summary(GermanCredit)
## Duration Amount InstallmentRatePercentage ResidenceDuration
## Min. : 4.0 Min. : 250 Min. :1.000 Min. :1.000
## 1st Qu.:12.0 1st Qu.: 1366 1st Qu.:2.000 1st Qu.:2.000
## Median :18.0 Median : 2320 Median :3.000 Median :3.000
## Mean :20.9 Mean : 3271 Mean :2.973 Mean :2.845
## 3rd Qu.:24.0 3rd Qu.: 3972 3rd Qu.:4.000 3rd Qu.:4.000
## Max. :72.0 Max. :18424 Max. :4.000 Max. :4.000
## Age NumberExistingCredits NumberPeopleMaintenance Telephone
## Min. :19.00 Min. :1.000 Min. :1.000 Min. :0.000
## 1st Qu.:27.00 1st Qu.:1.000 1st Qu.:1.000 1st Qu.:0.000
## Median :33.00 Median :1.000 Median :1.000 Median :1.000
## Mean :35.55 Mean :1.407 Mean :1.155 Mean :0.596
## 3rd Qu.:42.00 3rd Qu.:2.000 3rd Qu.:1.000 3rd Qu.:1.000
## Max. :75.00 Max. :4.000 Max. :2.000 Max. :1.000
## ForeignWorker Class CheckingAccountStatus.lt.0
## Min. :0.000 Mode :logical Min. :0.000
## 1st Qu.:1.000 FALSE:300 1st Qu.:0.000
## Median :1.000 TRUE :700 Median :0.000
## Mean :0.963 Mean :0.274
## 3rd Qu.:1.000 3rd Qu.:1.000
## Max. :1.000 Max. :1.000
## CheckingAccountStatus.0.to.200 CheckingAccountStatus.gt.200
## Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.000
## Mean :0.269 Mean :0.063
## 3rd Qu.:1.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000
## CreditHistory.NoCredit.AllPaid CreditHistory.ThisBank.AllPaid
## Min. :0.00 Min. :0.000
## 1st Qu.:0.00 1st Qu.:0.000
## Median :0.00 Median :0.000
## Mean :0.04 Mean :0.049
## 3rd Qu.:0.00 3rd Qu.:0.000
## Max. :1.00 Max. :1.000
## CreditHistory.PaidDuly CreditHistory.Delay Purpose.NewCar Purpose.UsedCar
## Min. :0.00 Min. :0.000 Min. :0.000 Min. :0.000
## 1st Qu.:0.00 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000
## Median :1.00 Median :0.000 Median :0.000 Median :0.000
## Mean :0.53 Mean :0.088 Mean :0.234 Mean :0.103
## 3rd Qu.:1.00 3rd Qu.:0.000 3rd Qu.:0.000 3rd Qu.:0.000
## Max. :1.00 Max. :1.000 Max. :1.000 Max. :1.000
## Purpose.Furniture.Equipment Purpose.Radio.Television Purpose.DomesticAppliance
## Min. :0.000 Min. :0.00 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.00 1st Qu.:0.000
## Median :0.000 Median :0.00 Median :0.000
## Mean :0.181 Mean :0.28 Mean :0.012
## 3rd Qu.:0.000 3rd Qu.:1.00 3rd Qu.:0.000
## Max. :1.000 Max. :1.00 Max. :1.000
## Purpose.Repairs Purpose.Education Purpose.Retraining Purpose.Business
## Min. :0.000 Min. :0.00 Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.00 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.00 Median :0.000 Median :0.000
## Mean :0.022 Mean :0.05 Mean :0.009 Mean :0.097
## 3rd Qu.:0.000 3rd Qu.:0.00 3rd Qu.:0.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.00 Max. :1.000 Max. :1.000
## SavingsAccountBonds.lt.100 SavingsAccountBonds.100.to.500
## Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000
## Median :1.000 Median :0.000
## Mean :0.603 Mean :0.103
## 3rd Qu.:1.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000
## SavingsAccountBonds.500.to.1000 SavingsAccountBonds.gt.1000
## Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.000
## Mean :0.063 Mean :0.048
## 3rd Qu.:0.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000
## EmploymentDuration.lt.1 EmploymentDuration.1.to.4 EmploymentDuration.4.to.7
## Min. :0.000 Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.000 Median :0.000
## Mean :0.172 Mean :0.339 Mean :0.174
## 3rd Qu.:0.000 3rd Qu.:1.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000 Max. :1.000
## EmploymentDuration.gt.7 Personal.Male.Divorced.Seperated
## Min. :0.000 Min. :0.00
## 1st Qu.:0.000 1st Qu.:0.00
## Median :0.000 Median :0.00
## Mean :0.253 Mean :0.05
## 3rd Qu.:1.000 3rd Qu.:0.00
## Max. :1.000 Max. :1.00
## Personal.Female.NotSingle Personal.Male.Single OtherDebtorsGuarantors.None
## Min. :0.00 Min. :0.000 Min. :0.000
## 1st Qu.:0.00 1st Qu.:0.000 1st Qu.:1.000
## Median :0.00 Median :1.000 Median :1.000
## Mean :0.31 Mean :0.548 Mean :0.907
## 3rd Qu.:1.00 3rd Qu.:1.000 3rd Qu.:1.000
## Max. :1.00 Max. :1.000 Max. :1.000
## OtherDebtorsGuarantors.CoApplicant Property.RealEstate Property.Insurance
## Min. :0.000 Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.000 Median :0.000
## Mean :0.041 Mean :0.282 Mean :0.232
## 3rd Qu.:0.000 3rd Qu.:1.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000 Max. :1.000
## Property.CarOther OtherInstallmentPlans.Bank OtherInstallmentPlans.Stores
## Min. :0.000 Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.000 Median :0.000
## Mean :0.332 Mean :0.139 Mean :0.047
## 3rd Qu.:1.000 3rd Qu.:0.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000 Max. :1.000
## Housing.Rent Housing.Own Job.UnemployedUnskilled Job.UnskilledResident
## Min. :0.000 Min. :0.000 Min. :0.000 Min. :0.0
## 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.0
## Median :0.000 Median :1.000 Median :0.000 Median :0.0
## Mean :0.179 Mean :0.713 Mean :0.022 Mean :0.2
## 3rd Qu.:0.000 3rd Qu.:1.000 3rd Qu.:0.000 3rd Qu.:0.0
## Max. :1.000 Max. :1.000 Max. :1.000 Max. :1.0
## Job.SkilledEmployee
## Min. :0.00
## 1st Qu.:0.00
## Median :1.00
## Mean :0.63
## 3rd Qu.:1.00
## Max. :1.00
#Check for any missing values
colSums(is.na(GermanCredit))
## Duration Amount
## 0 0
## InstallmentRatePercentage ResidenceDuration
## 0 0
## Age NumberExistingCredits
## 0 0
## NumberPeopleMaintenance Telephone
## 0 0
## ForeignWorker Class
## 0 0
## CheckingAccountStatus.lt.0 CheckingAccountStatus.0.to.200
## 0 0
## CheckingAccountStatus.gt.200 CreditHistory.NoCredit.AllPaid
## 0 0
## CreditHistory.ThisBank.AllPaid CreditHistory.PaidDuly
## 0 0
## CreditHistory.Delay Purpose.NewCar
## 0 0
## Purpose.UsedCar Purpose.Furniture.Equipment
## 0 0
## Purpose.Radio.Television Purpose.DomesticAppliance
## 0 0
## Purpose.Repairs Purpose.Education
## 0 0
## Purpose.Retraining Purpose.Business
## 0 0
## SavingsAccountBonds.lt.100 SavingsAccountBonds.100.to.500
## 0 0
## SavingsAccountBonds.500.to.1000 SavingsAccountBonds.gt.1000
## 0 0
## EmploymentDuration.lt.1 EmploymentDuration.1.to.4
## 0 0
## EmploymentDuration.4.to.7 EmploymentDuration.gt.7
## 0 0
## Personal.Male.Divorced.Seperated Personal.Female.NotSingle
## 0 0
## Personal.Male.Single OtherDebtorsGuarantors.None
## 0 0
## OtherDebtorsGuarantors.CoApplicant Property.RealEstate
## 0 0
## Property.Insurance Property.CarOther
## 0 0
## OtherInstallmentPlans.Bank OtherInstallmentPlans.Stores
## 0 0
## Housing.Rent Housing.Own
## 0 0
## Job.UnemployedUnskilled Job.UnskilledResident
## 0 0
## Job.SkilledEmployee
## 0
Now, we check the de distribution of the variable we are analyzing.
ggplot(GermanCredit, aes(x = GermanCredit$Class)) +
geom_bar(stat = "count", fill = "blue", alpha = 0.7) +
ggtitle("Distribution class of credit (GOOD= TRUE, BAD= FALSE)") +
xlab("Month") +
ylab("Class")
## Warning: Use of `GermanCredit$Class` is discouraged.
## i Use `Class` instead.
We see that there are more Trues for the variable class than falses, but what is the proportion?
mean(GermanCredit$Class)
## [1] 0.7
We check for outlier in the variable.
boxplot(GermanCredit, las =2, cex.axis=0.7 )
Th most noticeable variable with outliers is “Amount”, we will take a closer look to the summary of its statistics, but will not make any changes at the moment.
summary(GermanCredit$Amount)
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 250 1366 2320 3271 3972 18424
Your observation: The distribution the German Credit class was about 70% for good credit and 30% for bad credit. I also noticed that most of the variables in the German Credit data set could be changed into boolean, with some exceptions. Finally, there is one variable with tremendous outliers, that I think could be important, so I decided to keep them as they are for the time being, but if needed in the next steps I will handle them appropriately.
2023 for reproducibility.set.seed(2023)
index <- sample(1:nrow(GermanCredit),nrow(GermanCredit)*0.80)
GermanCrdt.train = GermanCredit[index,]
GermanCrdt.test = GermanCredit[-index,]
Your observation: Nothing much to see, just a simple splitting of the data into training and testing data.
#After analyzing the data I realized that all of the variables with a max value of 1 are boolean, they can either be true or false and nothing more, so lets change their dta type
for (column in names(GermanCredit)[-1]) {
max_value <- max(GermanCredit[[column]])
if (max_value == 1){
GermanCredit[column] <- (GermanCredit[[column]] == max_value)
}
}
GermanCredit$InstallmentRatePercentage<- as.factor(GermanCredit$InstallmentRatePercentage)
GermanCredit$ResidenceDuration<- as.factor(GermanCredit$ResidenceDuration)
GermanCredit$NumberExistingCredits<- as.factor(GermanCredit$NumberExistingCredits)
GermanCredit$NumberPeopleMaintenance<- as.factor(GermanCredit$NumberPeopleMaintenance)
str(GermanCredit)
## 'data.frame': 1000 obs. of 49 variables:
## $ Duration : int 6 48 12 42 24 36 24 36 12 30 ...
## $ Amount : int 1169 5951 2096 7882 4870 9055 2835 6948 3059 5234 ...
## $ InstallmentRatePercentage : Factor w/ 4 levels "1","2","3","4": 4 2 2 2 3 2 3 2 2 4 ...
## $ ResidenceDuration : Factor w/ 4 levels "1","2","3","4": 4 2 3 4 4 4 4 2 4 2 ...
## $ Age : int 67 22 49 45 53 35 53 35 61 28 ...
## $ NumberExistingCredits : Factor w/ 4 levels "1","2","3","4": 2 1 1 1 2 1 1 1 1 2 ...
## $ NumberPeopleMaintenance : Factor w/ 2 levels "1","2": 1 1 2 2 2 2 1 1 1 1 ...
## $ Telephone : logi FALSE TRUE TRUE TRUE TRUE FALSE ...
## $ ForeignWorker : logi TRUE TRUE TRUE TRUE TRUE TRUE ...
## $ Class : logi TRUE FALSE TRUE TRUE FALSE TRUE ...
## $ CheckingAccountStatus.lt.0 : logi TRUE FALSE FALSE TRUE TRUE FALSE ...
## $ CheckingAccountStatus.0.to.200 : logi FALSE TRUE FALSE FALSE FALSE FALSE ...
## $ CheckingAccountStatus.gt.200 : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ CreditHistory.NoCredit.AllPaid : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ CreditHistory.ThisBank.AllPaid : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ CreditHistory.PaidDuly : logi FALSE TRUE FALSE TRUE FALSE TRUE ...
## $ CreditHistory.Delay : logi FALSE FALSE FALSE FALSE TRUE FALSE ...
## $ Purpose.NewCar : logi FALSE FALSE FALSE FALSE TRUE FALSE ...
## $ Purpose.UsedCar : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ Purpose.Furniture.Equipment : logi FALSE FALSE FALSE TRUE FALSE FALSE ...
## $ Purpose.Radio.Television : logi TRUE TRUE FALSE FALSE FALSE FALSE ...
## $ Purpose.DomesticAppliance : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ Purpose.Repairs : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ Purpose.Education : logi FALSE FALSE TRUE FALSE FALSE TRUE ...
## $ Purpose.Retraining : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ Purpose.Business : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ SavingsAccountBonds.lt.100 : logi FALSE TRUE TRUE TRUE TRUE FALSE ...
## $ SavingsAccountBonds.100.to.500 : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ SavingsAccountBonds.500.to.1000 : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ SavingsAccountBonds.gt.1000 : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ EmploymentDuration.lt.1 : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ EmploymentDuration.1.to.4 : logi FALSE TRUE FALSE FALSE TRUE TRUE ...
## $ EmploymentDuration.4.to.7 : logi FALSE FALSE TRUE TRUE FALSE FALSE ...
## $ EmploymentDuration.gt.7 : logi TRUE FALSE FALSE FALSE FALSE FALSE ...
## $ Personal.Male.Divorced.Seperated : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ Personal.Female.NotSingle : logi FALSE TRUE FALSE FALSE FALSE FALSE ...
## $ Personal.Male.Single : logi TRUE FALSE TRUE TRUE TRUE TRUE ...
## $ OtherDebtorsGuarantors.None : logi TRUE TRUE TRUE FALSE TRUE TRUE ...
## $ OtherDebtorsGuarantors.CoApplicant: logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ Property.RealEstate : logi TRUE TRUE TRUE FALSE FALSE FALSE ...
## $ Property.Insurance : logi FALSE FALSE FALSE TRUE FALSE FALSE ...
## $ Property.CarOther : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ OtherInstallmentPlans.Bank : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ OtherInstallmentPlans.Stores : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ Housing.Rent : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ Housing.Own : logi TRUE TRUE TRUE FALSE FALSE FALSE ...
## $ Job.UnemployedUnskilled : logi FALSE FALSE FALSE FALSE FALSE FALSE ...
## $ Job.UnskilledResident : logi FALSE FALSE TRUE FALSE FALSE TRUE ...
## $ Job.SkilledEmployee : logi TRUE TRUE FALSE TRUE TRUE FALSE ...
summary(GermanCredit$Personal.Female.Single)
## Length Class Mode
## 0 NULL NULL
set.seed(2023)
index <- sample(1:nrow(GermanCredit), nrow(GermanCredit) * 0.80)
GermanCrdt.train <- GermanCredit[index, ]
GermanCrdt.test <- GermanCredit[-index, ]
GermanCrdt_GLM.train <- glm(Class ~ ., family = binomial, data = GermanCrdt.train)
GermanCrdt_GLM.test <- glm(Class ~ ., family = binomial, data = GermanCrdt.test)
Your observation: I made a simple logistic regression with the, but instead of using lm(), I used glm() because of the family option that lets the program know the variable is binomial
summary(GermanCrdt_GLM.train)
##
## Call:
## glm(formula = Class ~ ., family = binomial, data = GermanCrdt.train)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -2.6857 -0.7064 0.3652 0.6644 2.2909
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 7.753e+00 1.557e+00 4.979 6.38e-07
## Duration -2.479e-02 1.045e-02 -2.371 0.01773
## Amount -1.158e-04 5.005e-05 -2.314 0.02067
## InstallmentRatePercentage2 -2.268e-01 3.441e-01 -0.659 0.50987
## InstallmentRatePercentage3 -5.577e-01 3.751e-01 -1.487 0.13702
## InstallmentRatePercentage4 -7.685e-01 3.389e-01 -2.268 0.02334
## ResidenceDuration2 -6.458e-01 3.353e-01 -1.926 0.05406
## ResidenceDuration3 -3.778e-01 3.845e-01 -0.983 0.32579
## ResidenceDuration4 -2.089e-01 3.438e-01 -0.607 0.54357
## Age 1.891e-02 1.052e-02 1.798 0.07225
## NumberExistingCredits2 -3.483e-01 2.752e-01 -1.266 0.20563
## NumberExistingCredits3 -1.694e-01 7.202e-01 -0.235 0.81408
## NumberExistingCredits4 -1.057e+00 1.236e+00 -0.855 0.39248
## NumberPeopleMaintenance2 -1.383e-01 2.930e-01 -0.472 0.63691
## TelephoneTRUE -2.454e-01 2.262e-01 -1.085 0.27794
## ForeignWorkerTRUE -1.777e+00 8.280e-01 -2.146 0.03190
## CheckingAccountStatus.lt.0TRUE -2.009e+00 2.683e-01 -7.485 7.15e-14
## CheckingAccountStatus.0.to.200TRUE -1.318e+00 2.584e-01 -5.098 3.42e-07
## CheckingAccountStatus.gt.200TRUE -5.968e-01 4.461e-01 -1.338 0.18099
## CreditHistory.NoCredit.AllPaidTRUE -1.332e+00 4.852e-01 -2.745 0.00604
## CreditHistory.ThisBank.AllPaidTRUE -1.456e+00 5.129e-01 -2.838 0.00454
## CreditHistory.PaidDulyTRUE -7.562e-01 3.038e-01 -2.489 0.01282
## CreditHistory.DelayTRUE -5.459e-01 3.790e-01 -1.440 0.14974
## Purpose.NewCarTRUE -1.909e+00 8.725e-01 -2.188 0.02865
## Purpose.UsedCarTRUE -2.676e-01 8.978e-01 -0.298 0.76567
## Purpose.Furniture.EquipmentTRUE -1.085e+00 8.811e-01 -1.232 0.21803
## Purpose.Radio.TelevisionTRUE -1.080e+00 8.885e-01 -1.215 0.22438
## Purpose.DomesticApplianceTRUE -1.065e+00 1.230e+00 -0.866 0.38654
## Purpose.RepairsTRUE -1.981e+00 1.043e+00 -1.899 0.05751
## Purpose.EducationTRUE -1.948e+00 9.573e-01 -2.035 0.04188
## Purpose.RetrainingTRUE -1.046e+00 1.509e+00 -0.693 0.48812
## Purpose.BusinessTRUE -1.260e+00 9.047e-01 -1.393 0.16369
## SavingsAccountBonds.lt.100TRUE -9.675e-01 3.008e-01 -3.216 0.00130
## SavingsAccountBonds.100.to.500TRUE -7.575e-01 3.935e-01 -1.925 0.05420
## SavingsAccountBonds.500.to.1000TRUE -3.535e-01 5.307e-01 -0.666 0.50539
## SavingsAccountBonds.gt.1000TRUE -2.019e-01 6.077e-01 -0.332 0.73967
## EmploymentDuration.lt.1TRUE 1.740e-01 5.046e-01 0.345 0.73028
## EmploymentDuration.1.to.4TRUE 4.097e-01 4.809e-01 0.852 0.39426
## EmploymentDuration.4.to.7TRUE 8.700e-01 5.197e-01 1.674 0.09415
## EmploymentDuration.gt.7TRUE 3.453e-01 4.831e-01 0.715 0.47479
## Personal.Male.Divorced.SeperatedTRUE -5.406e-01 5.016e-01 -1.078 0.28113
## Personal.Female.NotSingleTRUE -2.465e-01 3.555e-01 -0.693 0.48812
## Personal.Male.SingleTRUE 2.864e-01 3.578e-01 0.801 0.42341
## OtherDebtorsGuarantors.NoneTRUE -7.684e-01 4.714e-01 -1.630 0.10309
## OtherDebtorsGuarantors.CoApplicantTRUE -1.233e+00 6.403e-01 -1.926 0.05413
## Property.RealEstateTRUE 8.033e-01 4.678e-01 1.717 0.08596
## Property.InsuranceTRUE 6.194e-01 4.539e-01 1.365 0.17232
## Property.CarOtherTRUE 4.378e-01 4.416e-01 0.991 0.32152
## OtherInstallmentPlans.BankTRUE -5.709e-01 2.722e-01 -2.097 0.03600
## OtherInstallmentPlans.StoresTRUE -4.917e-01 4.717e-01 -1.042 0.29721
## Housing.RentTRUE -5.351e-01 5.338e-01 -1.003 0.31607
## Housing.OwnTRUE 1.076e-03 5.007e-01 0.002 0.99829
## Job.UnemployedUnskilledTRUE 8.941e-01 8.715e-01 1.026 0.30493
## Job.UnskilledResidentTRUE 6.357e-02 4.004e-01 0.159 0.87385
## Job.SkilledEmployeeTRUE -4.036e-02 3.278e-01 -0.123 0.90199
##
## (Intercept) ***
## Duration *
## Amount *
## InstallmentRatePercentage2
## InstallmentRatePercentage3
## InstallmentRatePercentage4 *
## ResidenceDuration2 .
## ResidenceDuration3
## ResidenceDuration4
## Age .
## NumberExistingCredits2
## NumberExistingCredits3
## NumberExistingCredits4
## NumberPeopleMaintenance2
## TelephoneTRUE
## ForeignWorkerTRUE *
## CheckingAccountStatus.lt.0TRUE ***
## CheckingAccountStatus.0.to.200TRUE ***
## CheckingAccountStatus.gt.200TRUE
## CreditHistory.NoCredit.AllPaidTRUE **
## CreditHistory.ThisBank.AllPaidTRUE **
## CreditHistory.PaidDulyTRUE *
## CreditHistory.DelayTRUE
## Purpose.NewCarTRUE *
## Purpose.UsedCarTRUE
## Purpose.Furniture.EquipmentTRUE
## Purpose.Radio.TelevisionTRUE
## Purpose.DomesticApplianceTRUE
## Purpose.RepairsTRUE .
## Purpose.EducationTRUE *
## Purpose.RetrainingTRUE
## Purpose.BusinessTRUE
## SavingsAccountBonds.lt.100TRUE **
## SavingsAccountBonds.100.to.500TRUE .
## SavingsAccountBonds.500.to.1000TRUE
## SavingsAccountBonds.gt.1000TRUE
## EmploymentDuration.lt.1TRUE
## EmploymentDuration.1.to.4TRUE
## EmploymentDuration.4.to.7TRUE .
## EmploymentDuration.gt.7TRUE
## Personal.Male.Divorced.SeperatedTRUE
## Personal.Female.NotSingleTRUE
## Personal.Male.SingleTRUE
## OtherDebtorsGuarantors.NoneTRUE
## OtherDebtorsGuarantors.CoApplicantTRUE .
## Property.RealEstateTRUE .
## Property.InsuranceTRUE
## Property.CarOtherTRUE
## OtherInstallmentPlans.BankTRUE *
## OtherInstallmentPlans.StoresTRUE
## Housing.RentTRUE
## Housing.OwnTRUE
## Job.UnemployedUnskilledTRUE
## Job.UnskilledResidentTRUE
## Job.SkilledEmployeeTRUE
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 980.75 on 799 degrees of freedom
## Residual deviance: 712.07 on 745 degrees of freedom
## AIC: 822.07
##
## Number of Fisher Scoring iterations: 5
Your observation: Signif. codes: 0 ‘’ 0.001 ‘’ 0.01 ‘’ 0.05 ‘.’ 0.1 ‘ ’ 1 is what caugth my attention the most. They test the level of significance of the predictors when the p values are 0.001, 0.01, 0.05, and 0.1. The model appears to be more significant the smaller the p value, because the smaller the number on the significant code is the more accurate the prediction is.
pred.glm.train.GerCred <- predict(GermanCrdt_GLM.train, type="response")
hist(pred.glm.train.GerCred)
table(predict(GermanCrdt_GLM.train, type="response") > 0.5)
##
## FALSE TRUE
## 202 598
table(predict(GermanCrdt_GLM.train, type="response") > 0.2)
##
## FALSE TRUE
## 41 759
table(predict(GermanCrdt_GLM.train, type="response") > 0.1)
##
## FALSE TRUE
## 11 789
Your observation: The values vary a lot between true and false depending on the prediction coefficient.
costfunc = function(obs, pred.p, pcut){
weight1 = 1
weight0 = 1
c1 = (obs==1)&(pred.p<pcut)
c0 = (obs==0)&(pred.p>=pcut)
cost = mean(weight1*c1 + weight0*c0)
return(cost)
}
p.seq = seq(0.01, 1, 0.01)
cost = rep(0, length(p.seq))[0]
for(i in 1:length(p.seq)){
cost[i] = costfunc(obs = GermanCrdt.train$Class, pred.p = pred.glm.train.GerCred, pcut = p.seq[i])
}
pcut = p.seq[which(cost==min(cost))]
print(pcut)
## [1] 0.43 0.44
Your observation: In the class owrk, we also used mean as a pcut value, in this instance, what I found the most interesting is that the pcut value is so different than the mean.
# get binary prediction
class.glm.train.GC<- (pred.glm.train.GerCred>pcut)*1
# get confusion matrix
table(GermanCrdt.train$Class, class.glm.train.GC, dnn = c("True", "Predicted"))
## Predicted
## True 0 1
## FALSE 119 123
## TRUE 37 521
MR<- mean(GermanCrdt.train$Class!=class.glm.train.GC)
# False positive rate
FPR <- sum((GermanCrdt.train$Class == 0 & class.glm.train.GC == 1) / sum(GermanCrdt.train$Class == 0))
# False negative rate
FNR <- sum((GermanCrdt.train$Class == 1 & class.glm.train.GC == 0) / sum(GermanCrdt.train$Class == 1))
cat("FPR:", FPR, "FNR:", FNR, "MR", MR, "\n")
## FPR: 0.5082645 FNR: 0.06630824 MR 0.2
Your observation: The False positive rate is tremendously high. More than half of the positive variables have a false positive.
library(ROCR)
## Warning: package 'ROCR' was built under R version 4.1.3
pred <- prediction(pred.glm.train.GerCred, GermanCrdt.train$Class)
perf <- performance(pred, "tpr", "fpr")
plot(perf, colorize=TRUE)
#Get the AUC
unlist(slot(performance(pred, "auc"), "y.values"))
## [1] 0.8374878
Your observation: The AUC of the ROC curve for the training data suggest that the model strongly predicts the values of the class data.
pred.glm.test.GerCred1 <- predict(GermanCrdt_GLM.test, newdata = GermanCrdt.test, type="response")
class.glm.test.GC<- (pred.glm.test.GerCred1>pcut)*1
table(GermanCrdt.test$Class, class.glm.test.GC, dnn = c("True", "Predicted"))
## Predicted
## True 0 1
## FALSE 42 16
## TRUE 6 136
MR<- mean(GermanCrdt.test$Class!=class.glm.test.GC)
# False positive rate
FPR <- sum((GermanCrdt.test$Class == 0 & class.glm.test.GC == 1) / sum(GermanCrdt.test$Class == 0))
# False negative rate
FNR <- sum((GermanCrdt.test$Class == 1 & class.glm.test.GC == 0) / sum(GermanCrdt.test$Class == 1))
cat("FPR:", FPR, "FNR:", FNR, "MR", MR, "\n")
## FPR: 0.2758621 FNR: 0.04225352 MR 0.11
Your observation: The false positive rate severely decreases for the training data.
pred <- prediction(pred.glm.test.GerCred1, GermanCrdt.test$Class)
perf <- performance(pred, "tpr", "fpr")
plot(perf, colorize=TRUE)
#Get the AUC
unlist(slot(performance(pred, "auc"), "y.values"))
## [1] 0.9434191
Your observation: Although the rate of false positives is lower compared to the values in training data, the AUC suggests that this model is less strongly correlated. This can be seen when comparing both of the correlation matrix. In the training data set, the true negatives have a better proportion in the training data 119/242 against the 26/58 for the testing data set. In a similar manner, the true positives are 521/538 for training vs 125/142 for testing.
Now, let’s assume “It is worse to class a customer as good when they are bad (weight = 5), than it is to class a customer as bad when they are good (weight = 1).” Please figure out which weight should be 5 and which weight should be ### 1. Then define your cost function accordingly!
costfunc = function(obs, pred.p, pcut){
weight1 = 1
weight0 = 5
c1 = (obs==1)&(pred.p<pcut)
c0 = (obs==0)&(pred.p>=pcut)
cost = mean(weight1*c1 + weight0*c0)
return(cost)
}
p.seq = seq(0.01, 1, 0.01)
cost = rep(0, length(p.seq))
for(i in 1:length(p.seq)){
cost[i] = costfunc(obs = GermanCrdt.train$Class, pred.p = pred.glm.train.GerCred, pcut = p.seq[i])
}
plot(p.seq, cost)
# find the optimal pcut
optimal.pcut = p.seq[which(cost==min(cost))]
print(optimal.pcut)
## [1] 0.8
Your observation: By increasin the weight of the variable, the pcut point almost doubled. The value now, is much closer to the true mean of the data set for the class variable, which is 0.7.
# get binary prediction
class.glm.train.GC2<- (pred.glm.train.GerCred>optimal.pcut)*1
# get confusion matrix
table(GermanCrdt.train$Class, class.glm.train.GC, dnn = c("True", "Predicted"))
## Predicted
## True 0 1
## FALSE 119 123
## TRUE 37 521
MR<- mean(GermanCrdt.train$Class!=class.glm.train.GC2)
MR
## [1] 0.285
Your observation: Now we have a majority of true negatives, and true positives in the confusion matrix. The values predicted are being a lot better for the true negatives and a little bit worse for the true positives, but better prediction overall.
# get binary prediction
class.glm.test.GC2<- (pred.glm.test.GerCred1>optimal.pcut)*1
# get confusion matrix
table(GermanCrdt.test$Class, class.glm.test.GC, dnn = c("True", "Predicted"))
## Predicted
## True 0 1
## FALSE 42 16
## TRUE 6 136
MR<- mean(GermanCrdt.test$Class!=class.glm.test.GC2)
MR
## [1] 0.175
Your observation: In the same way, the confusion matrix demonstrates a better fit for the model overall.
Summarize your findings, including the optimal probability cut-off, MR and AUC (if calculated) for both in-sample and out-of-sample data. Discuss what you observed and make some suggestions on how can we improve the model.
The model overall improved very much during the progression of this report. The values in the confusion matrices clearly show that the model was improving severely from every minor adjustment. The equally weighted values for false positives and false negatives worked very well with the data as showed by their AUCs. One minor adjustment I would make to produce the best approximation of the values in the class column of the data set, would be to try different weights for the variables to see which weight fits the data more. This is because we got the best model for the ratio 1-5, but there maybe is a better ratio to predict the data.