Starter code for German credit scoring

Refer to http://archive.ics.uci.edu/ml/datasets/Statlog+(German+Credit+Data)) for variable description. The response variable is Class and all others are predictors.

Only run the following code once to install the package caret. The German credit scoring data in provided in that package.

install.packages('caret')

Task1: Data Preparation

1. Load the caret package and the GermanCredit dataset.

library(caret) #this package contains the german data with its numeric format
## Warning: package 'caret' was built under R version 4.1.3
## Loading required package: ggplot2
## Loading required package: lattice
data(GermanCredit)
GermanCredit$Class <-  GermanCredit$Class == "Good" # use this code to convert `Class` into True or False (equivalent to 1 or 0)
str(GermanCredit)
## 'data.frame':    1000 obs. of  62 variables:
##  $ Duration                              : int  6 48 12 42 24 36 24 36 12 30 ...
##  $ Amount                                : int  1169 5951 2096 7882 4870 9055 2835 6948 3059 5234 ...
##  $ InstallmentRatePercentage             : int  4 2 2 2 3 2 3 2 2 4 ...
##  $ ResidenceDuration                     : int  4 2 3 4 4 4 4 2 4 2 ...
##  $ Age                                   : int  67 22 49 45 53 35 53 35 61 28 ...
##  $ NumberExistingCredits                 : int  2 1 1 1 2 1 1 1 1 2 ...
##  $ NumberPeopleMaintenance               : int  1 1 2 2 2 2 1 1 1 1 ...
##  $ Telephone                             : num  0 1 1 1 1 0 1 0 1 1 ...
##  $ ForeignWorker                         : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ Class                                 : logi  TRUE FALSE TRUE TRUE FALSE TRUE ...
##  $ CheckingAccountStatus.lt.0            : num  1 0 0 1 1 0 0 0 0 0 ...
##  $ CheckingAccountStatus.0.to.200        : num  0 1 0 0 0 0 0 1 0 1 ...
##  $ CheckingAccountStatus.gt.200          : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CheckingAccountStatus.none            : num  0 0 1 0 0 1 1 0 1 0 ...
##  $ CreditHistory.NoCredit.AllPaid        : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CreditHistory.ThisBank.AllPaid        : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CreditHistory.PaidDuly                : num  0 1 0 1 0 1 1 1 1 0 ...
##  $ CreditHistory.Delay                   : num  0 0 0 0 1 0 0 0 0 0 ...
##  $ CreditHistory.Critical                : num  1 0 1 0 0 0 0 0 0 1 ...
##  $ Purpose.NewCar                        : num  0 0 0 0 1 0 0 0 0 1 ...
##  $ Purpose.UsedCar                       : num  0 0 0 0 0 0 0 1 0 0 ...
##  $ Purpose.Furniture.Equipment           : num  0 0 0 1 0 0 1 0 0 0 ...
##  $ Purpose.Radio.Television              : num  1 1 0 0 0 0 0 0 1 0 ...
##  $ Purpose.DomesticAppliance             : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Purpose.Repairs                       : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Purpose.Education                     : num  0 0 1 0 0 1 0 0 0 0 ...
##  $ Purpose.Vacation                      : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Purpose.Retraining                    : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Purpose.Business                      : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Purpose.Other                         : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ SavingsAccountBonds.lt.100            : num  0 1 1 1 1 0 0 1 0 1 ...
##  $ SavingsAccountBonds.100.to.500        : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ SavingsAccountBonds.500.to.1000       : num  0 0 0 0 0 0 1 0 0 0 ...
##  $ SavingsAccountBonds.gt.1000           : num  0 0 0 0 0 0 0 0 1 0 ...
##  $ SavingsAccountBonds.Unknown           : num  1 0 0 0 0 1 0 0 0 0 ...
##  $ EmploymentDuration.lt.1               : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ EmploymentDuration.1.to.4             : num  0 1 0 0 1 1 0 1 0 0 ...
##  $ EmploymentDuration.4.to.7             : num  0 0 1 1 0 0 0 0 1 0 ...
##  $ EmploymentDuration.gt.7               : num  1 0 0 0 0 0 1 0 0 0 ...
##  $ EmploymentDuration.Unemployed         : num  0 0 0 0 0 0 0 0 0 1 ...
##  $ Personal.Male.Divorced.Seperated      : num  0 0 0 0 0 0 0 0 1 0 ...
##  $ Personal.Female.NotSingle             : num  0 1 0 0 0 0 0 0 0 0 ...
##  $ Personal.Male.Single                  : num  1 0 1 1 1 1 1 1 0 0 ...
##  $ Personal.Male.Married.Widowed         : num  0 0 0 0 0 0 0 0 0 1 ...
##  $ Personal.Female.Single                : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ OtherDebtorsGuarantors.None           : num  1 1 1 0 1 1 1 1 1 1 ...
##  $ OtherDebtorsGuarantors.CoApplicant    : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ OtherDebtorsGuarantors.Guarantor      : num  0 0 0 1 0 0 0 0 0 0 ...
##  $ Property.RealEstate                   : num  1 1 1 0 0 0 0 0 1 0 ...
##  $ Property.Insurance                    : num  0 0 0 1 0 0 1 0 0 0 ...
##  $ Property.CarOther                     : num  0 0 0 0 0 0 0 1 0 1 ...
##  $ Property.Unknown                      : num  0 0 0 0 1 1 0 0 0 0 ...
##  $ OtherInstallmentPlans.Bank            : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ OtherInstallmentPlans.Stores          : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ OtherInstallmentPlans.None            : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ Housing.Rent                          : num  0 0 0 0 0 0 0 1 0 0 ...
##  $ Housing.Own                           : num  1 1 1 0 0 0 1 0 1 1 ...
##  $ Housing.ForFree                       : num  0 0 0 1 1 1 0 0 0 0 ...
##  $ Job.UnemployedUnskilled               : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Job.UnskilledResident                 : num  0 0 1 0 0 1 0 0 1 0 ...
##  $ Job.SkilledEmployee                   : num  1 1 0 1 1 0 1 0 0 0 ...
##  $ Job.Management.SelfEmp.HighlyQualified: num  0 0 0 0 0 0 0 1 0 1 ...

Your observation: Most of the variables appear to be boolean, the values can only be either 0 or 1 for most of them and we need to change the datavtypes to treat the variables accordingly.

#This is an optional code that drop variables that provide no information in the data
GermanCredit = GermanCredit[,-c(14,19,27,30,35,40,44,45,48,52,55,58,62)]

2. Explore the dataset to understand its structure.

Summary helps us visualize the data, and the basic distribution of the values stored within the variables.

summary(GermanCredit)
##     Duration        Amount      InstallmentRatePercentage ResidenceDuration
##  Min.   : 4.0   Min.   :  250   Min.   :1.000             Min.   :1.000    
##  1st Qu.:12.0   1st Qu.: 1366   1st Qu.:2.000             1st Qu.:2.000    
##  Median :18.0   Median : 2320   Median :3.000             Median :3.000    
##  Mean   :20.9   Mean   : 3271   Mean   :2.973             Mean   :2.845    
##  3rd Qu.:24.0   3rd Qu.: 3972   3rd Qu.:4.000             3rd Qu.:4.000    
##  Max.   :72.0   Max.   :18424   Max.   :4.000             Max.   :4.000    
##       Age        NumberExistingCredits NumberPeopleMaintenance   Telephone    
##  Min.   :19.00   Min.   :1.000         Min.   :1.000           Min.   :0.000  
##  1st Qu.:27.00   1st Qu.:1.000         1st Qu.:1.000           1st Qu.:0.000  
##  Median :33.00   Median :1.000         Median :1.000           Median :1.000  
##  Mean   :35.55   Mean   :1.407         Mean   :1.155           Mean   :0.596  
##  3rd Qu.:42.00   3rd Qu.:2.000         3rd Qu.:1.000           3rd Qu.:1.000  
##  Max.   :75.00   Max.   :4.000         Max.   :2.000           Max.   :1.000  
##  ForeignWorker     Class         CheckingAccountStatus.lt.0
##  Min.   :0.000   Mode :logical   Min.   :0.000             
##  1st Qu.:1.000   FALSE:300       1st Qu.:0.000             
##  Median :1.000   TRUE :700       Median :0.000             
##  Mean   :0.963                   Mean   :0.274             
##  3rd Qu.:1.000                   3rd Qu.:1.000             
##  Max.   :1.000                   Max.   :1.000             
##  CheckingAccountStatus.0.to.200 CheckingAccountStatus.gt.200
##  Min.   :0.000                  Min.   :0.000               
##  1st Qu.:0.000                  1st Qu.:0.000               
##  Median :0.000                  Median :0.000               
##  Mean   :0.269                  Mean   :0.063               
##  3rd Qu.:1.000                  3rd Qu.:0.000               
##  Max.   :1.000                  Max.   :1.000               
##  CreditHistory.NoCredit.AllPaid CreditHistory.ThisBank.AllPaid
##  Min.   :0.00                   Min.   :0.000                 
##  1st Qu.:0.00                   1st Qu.:0.000                 
##  Median :0.00                   Median :0.000                 
##  Mean   :0.04                   Mean   :0.049                 
##  3rd Qu.:0.00                   3rd Qu.:0.000                 
##  Max.   :1.00                   Max.   :1.000                 
##  CreditHistory.PaidDuly CreditHistory.Delay Purpose.NewCar  Purpose.UsedCar
##  Min.   :0.00           Min.   :0.000       Min.   :0.000   Min.   :0.000  
##  1st Qu.:0.00           1st Qu.:0.000       1st Qu.:0.000   1st Qu.:0.000  
##  Median :1.00           Median :0.000       Median :0.000   Median :0.000  
##  Mean   :0.53           Mean   :0.088       Mean   :0.234   Mean   :0.103  
##  3rd Qu.:1.00           3rd Qu.:0.000       3rd Qu.:0.000   3rd Qu.:0.000  
##  Max.   :1.00           Max.   :1.000       Max.   :1.000   Max.   :1.000  
##  Purpose.Furniture.Equipment Purpose.Radio.Television Purpose.DomesticAppliance
##  Min.   :0.000               Min.   :0.00             Min.   :0.000            
##  1st Qu.:0.000               1st Qu.:0.00             1st Qu.:0.000            
##  Median :0.000               Median :0.00             Median :0.000            
##  Mean   :0.181               Mean   :0.28             Mean   :0.012            
##  3rd Qu.:0.000               3rd Qu.:1.00             3rd Qu.:0.000            
##  Max.   :1.000               Max.   :1.00             Max.   :1.000            
##  Purpose.Repairs Purpose.Education Purpose.Retraining Purpose.Business
##  Min.   :0.000   Min.   :0.00      Min.   :0.000      Min.   :0.000   
##  1st Qu.:0.000   1st Qu.:0.00      1st Qu.:0.000      1st Qu.:0.000   
##  Median :0.000   Median :0.00      Median :0.000      Median :0.000   
##  Mean   :0.022   Mean   :0.05      Mean   :0.009      Mean   :0.097   
##  3rd Qu.:0.000   3rd Qu.:0.00      3rd Qu.:0.000      3rd Qu.:0.000   
##  Max.   :1.000   Max.   :1.00      Max.   :1.000      Max.   :1.000   
##  SavingsAccountBonds.lt.100 SavingsAccountBonds.100.to.500
##  Min.   :0.000              Min.   :0.000                 
##  1st Qu.:0.000              1st Qu.:0.000                 
##  Median :1.000              Median :0.000                 
##  Mean   :0.603              Mean   :0.103                 
##  3rd Qu.:1.000              3rd Qu.:0.000                 
##  Max.   :1.000              Max.   :1.000                 
##  SavingsAccountBonds.500.to.1000 SavingsAccountBonds.gt.1000
##  Min.   :0.000                   Min.   :0.000              
##  1st Qu.:0.000                   1st Qu.:0.000              
##  Median :0.000                   Median :0.000              
##  Mean   :0.063                   Mean   :0.048              
##  3rd Qu.:0.000                   3rd Qu.:0.000              
##  Max.   :1.000                   Max.   :1.000              
##  EmploymentDuration.lt.1 EmploymentDuration.1.to.4 EmploymentDuration.4.to.7
##  Min.   :0.000           Min.   :0.000             Min.   :0.000            
##  1st Qu.:0.000           1st Qu.:0.000             1st Qu.:0.000            
##  Median :0.000           Median :0.000             Median :0.000            
##  Mean   :0.172           Mean   :0.339             Mean   :0.174            
##  3rd Qu.:0.000           3rd Qu.:1.000             3rd Qu.:0.000            
##  Max.   :1.000           Max.   :1.000             Max.   :1.000            
##  EmploymentDuration.gt.7 Personal.Male.Divorced.Seperated
##  Min.   :0.000           Min.   :0.00                    
##  1st Qu.:0.000           1st Qu.:0.00                    
##  Median :0.000           Median :0.00                    
##  Mean   :0.253           Mean   :0.05                    
##  3rd Qu.:1.000           3rd Qu.:0.00                    
##  Max.   :1.000           Max.   :1.00                    
##  Personal.Female.NotSingle Personal.Male.Single OtherDebtorsGuarantors.None
##  Min.   :0.00              Min.   :0.000        Min.   :0.000              
##  1st Qu.:0.00              1st Qu.:0.000        1st Qu.:1.000              
##  Median :0.00              Median :1.000        Median :1.000              
##  Mean   :0.31              Mean   :0.548        Mean   :0.907              
##  3rd Qu.:1.00              3rd Qu.:1.000        3rd Qu.:1.000              
##  Max.   :1.00              Max.   :1.000        Max.   :1.000              
##  OtherDebtorsGuarantors.CoApplicant Property.RealEstate Property.Insurance
##  Min.   :0.000                      Min.   :0.000       Min.   :0.000     
##  1st Qu.:0.000                      1st Qu.:0.000       1st Qu.:0.000     
##  Median :0.000                      Median :0.000       Median :0.000     
##  Mean   :0.041                      Mean   :0.282       Mean   :0.232     
##  3rd Qu.:0.000                      3rd Qu.:1.000       3rd Qu.:0.000     
##  Max.   :1.000                      Max.   :1.000       Max.   :1.000     
##  Property.CarOther OtherInstallmentPlans.Bank OtherInstallmentPlans.Stores
##  Min.   :0.000     Min.   :0.000              Min.   :0.000               
##  1st Qu.:0.000     1st Qu.:0.000              1st Qu.:0.000               
##  Median :0.000     Median :0.000              Median :0.000               
##  Mean   :0.332     Mean   :0.139              Mean   :0.047               
##  3rd Qu.:1.000     3rd Qu.:0.000              3rd Qu.:0.000               
##  Max.   :1.000     Max.   :1.000              Max.   :1.000               
##   Housing.Rent    Housing.Own    Job.UnemployedUnskilled Job.UnskilledResident
##  Min.   :0.000   Min.   :0.000   Min.   :0.000           Min.   :0.0          
##  1st Qu.:0.000   1st Qu.:0.000   1st Qu.:0.000           1st Qu.:0.0          
##  Median :0.000   Median :1.000   Median :0.000           Median :0.0          
##  Mean   :0.179   Mean   :0.713   Mean   :0.022           Mean   :0.2          
##  3rd Qu.:0.000   3rd Qu.:1.000   3rd Qu.:0.000           3rd Qu.:0.0          
##  Max.   :1.000   Max.   :1.000   Max.   :1.000           Max.   :1.0          
##  Job.SkilledEmployee
##  Min.   :0.00       
##  1st Qu.:0.00       
##  Median :1.00       
##  Mean   :0.63       
##  3rd Qu.:1.00       
##  Max.   :1.00
#Check for any missing values
colSums(is.na(GermanCredit))
##                           Duration                             Amount 
##                                  0                                  0 
##          InstallmentRatePercentage                  ResidenceDuration 
##                                  0                                  0 
##                                Age              NumberExistingCredits 
##                                  0                                  0 
##            NumberPeopleMaintenance                          Telephone 
##                                  0                                  0 
##                      ForeignWorker                              Class 
##                                  0                                  0 
##         CheckingAccountStatus.lt.0     CheckingAccountStatus.0.to.200 
##                                  0                                  0 
##       CheckingAccountStatus.gt.200     CreditHistory.NoCredit.AllPaid 
##                                  0                                  0 
##     CreditHistory.ThisBank.AllPaid             CreditHistory.PaidDuly 
##                                  0                                  0 
##                CreditHistory.Delay                     Purpose.NewCar 
##                                  0                                  0 
##                    Purpose.UsedCar        Purpose.Furniture.Equipment 
##                                  0                                  0 
##           Purpose.Radio.Television          Purpose.DomesticAppliance 
##                                  0                                  0 
##                    Purpose.Repairs                  Purpose.Education 
##                                  0                                  0 
##                 Purpose.Retraining                   Purpose.Business 
##                                  0                                  0 
##         SavingsAccountBonds.lt.100     SavingsAccountBonds.100.to.500 
##                                  0                                  0 
##    SavingsAccountBonds.500.to.1000        SavingsAccountBonds.gt.1000 
##                                  0                                  0 
##            EmploymentDuration.lt.1          EmploymentDuration.1.to.4 
##                                  0                                  0 
##          EmploymentDuration.4.to.7            EmploymentDuration.gt.7 
##                                  0                                  0 
##   Personal.Male.Divorced.Seperated          Personal.Female.NotSingle 
##                                  0                                  0 
##               Personal.Male.Single        OtherDebtorsGuarantors.None 
##                                  0                                  0 
## OtherDebtorsGuarantors.CoApplicant                Property.RealEstate 
##                                  0                                  0 
##                 Property.Insurance                  Property.CarOther 
##                                  0                                  0 
##         OtherInstallmentPlans.Bank       OtherInstallmentPlans.Stores 
##                                  0                                  0 
##                       Housing.Rent                        Housing.Own 
##                                  0                                  0 
##            Job.UnemployedUnskilled              Job.UnskilledResident 
##                                  0                                  0 
##                Job.SkilledEmployee 
##                                  0

Now, we check the de distribution of the variable we are analyzing.

ggplot(GermanCredit, aes(x = GermanCredit$Class)) +
  geom_bar(stat = "count", fill = "blue", alpha = 0.7) +
  ggtitle("Distribution class of credit (GOOD= TRUE, BAD= FALSE)") +
  xlab("Month") +
  ylab("Class")
## Warning: Use of `GermanCredit$Class` is discouraged.
## i Use `Class` instead.

We see that there are more Trues for the variable class than falses, but what is the proportion?

mean(GermanCredit$Class)
## [1] 0.7

We check for outlier in the variable.

boxplot(GermanCredit, las =2, cex.axis=0.7 )

Th most noticeable variable with outliers is “Amount”, we will take a closer look to the summary of its statistics, but will not make any changes at the moment.

summary(GermanCredit$Amount)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##     250    1366    2320    3271    3972   18424

Your observation: The distribution the German Credit class was about 70% for good credit and 30% for bad credit. I also noticed that most of the variables in the German Credit data set could be changed into boolean, with some exceptions. Finally, there is one variable with tremendous outliers, that I think could be important, so I decided to keep them as they are for the time being, but if needed in the next steps I will handle them appropriately.

3. Split the dataset into training and test set. Please use the random seed as 2023 for reproducibility.

set.seed(2023)
index <- sample(1:nrow(GermanCredit),nrow(GermanCredit)*0.80)
GermanCrdt.train = GermanCredit[index,]
GermanCrdt.test = GermanCredit[-index,]

Your observation: Nothing much to see, just a simple splitting of the data into training and testing data.

Task 2: Model Fitting

#After analyzing the data I realized that all of the variables with a max value of 1 are boolean, they can either be true or false and nothing more, so lets change their dta type
for (column in names(GermanCredit)[-1]) {
  max_value <- max(GermanCredit[[column]])
  if (max_value == 1){
    GermanCredit[column] <- (GermanCredit[[column]] == max_value)
  }
}
GermanCredit$InstallmentRatePercentage<- as.factor(GermanCredit$InstallmentRatePercentage)
GermanCredit$ResidenceDuration<- as.factor(GermanCredit$ResidenceDuration)
GermanCredit$NumberExistingCredits<- as.factor(GermanCredit$NumberExistingCredits)
GermanCredit$NumberPeopleMaintenance<- as.factor(GermanCredit$NumberPeopleMaintenance)
str(GermanCredit)
## 'data.frame':    1000 obs. of  49 variables:
##  $ Duration                          : int  6 48 12 42 24 36 24 36 12 30 ...
##  $ Amount                            : int  1169 5951 2096 7882 4870 9055 2835 6948 3059 5234 ...
##  $ InstallmentRatePercentage         : Factor w/ 4 levels "1","2","3","4": 4 2 2 2 3 2 3 2 2 4 ...
##  $ ResidenceDuration                 : Factor w/ 4 levels "1","2","3","4": 4 2 3 4 4 4 4 2 4 2 ...
##  $ Age                               : int  67 22 49 45 53 35 53 35 61 28 ...
##  $ NumberExistingCredits             : Factor w/ 4 levels "1","2","3","4": 2 1 1 1 2 1 1 1 1 2 ...
##  $ NumberPeopleMaintenance           : Factor w/ 2 levels "1","2": 1 1 2 2 2 2 1 1 1 1 ...
##  $ Telephone                         : logi  FALSE TRUE TRUE TRUE TRUE FALSE ...
##  $ ForeignWorker                     : logi  TRUE TRUE TRUE TRUE TRUE TRUE ...
##  $ Class                             : logi  TRUE FALSE TRUE TRUE FALSE TRUE ...
##  $ CheckingAccountStatus.lt.0        : logi  TRUE FALSE FALSE TRUE TRUE FALSE ...
##  $ CheckingAccountStatus.0.to.200    : logi  FALSE TRUE FALSE FALSE FALSE FALSE ...
##  $ CheckingAccountStatus.gt.200      : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ CreditHistory.NoCredit.AllPaid    : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ CreditHistory.ThisBank.AllPaid    : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ CreditHistory.PaidDuly            : logi  FALSE TRUE FALSE TRUE FALSE TRUE ...
##  $ CreditHistory.Delay               : logi  FALSE FALSE FALSE FALSE TRUE FALSE ...
##  $ Purpose.NewCar                    : logi  FALSE FALSE FALSE FALSE TRUE FALSE ...
##  $ Purpose.UsedCar                   : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ Purpose.Furniture.Equipment       : logi  FALSE FALSE FALSE TRUE FALSE FALSE ...
##  $ Purpose.Radio.Television          : logi  TRUE TRUE FALSE FALSE FALSE FALSE ...
##  $ Purpose.DomesticAppliance         : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ Purpose.Repairs                   : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ Purpose.Education                 : logi  FALSE FALSE TRUE FALSE FALSE TRUE ...
##  $ Purpose.Retraining                : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ Purpose.Business                  : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ SavingsAccountBonds.lt.100        : logi  FALSE TRUE TRUE TRUE TRUE FALSE ...
##  $ SavingsAccountBonds.100.to.500    : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ SavingsAccountBonds.500.to.1000   : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ SavingsAccountBonds.gt.1000       : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ EmploymentDuration.lt.1           : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ EmploymentDuration.1.to.4         : logi  FALSE TRUE FALSE FALSE TRUE TRUE ...
##  $ EmploymentDuration.4.to.7         : logi  FALSE FALSE TRUE TRUE FALSE FALSE ...
##  $ EmploymentDuration.gt.7           : logi  TRUE FALSE FALSE FALSE FALSE FALSE ...
##  $ Personal.Male.Divorced.Seperated  : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ Personal.Female.NotSingle         : logi  FALSE TRUE FALSE FALSE FALSE FALSE ...
##  $ Personal.Male.Single              : logi  TRUE FALSE TRUE TRUE TRUE TRUE ...
##  $ OtherDebtorsGuarantors.None       : logi  TRUE TRUE TRUE FALSE TRUE TRUE ...
##  $ OtherDebtorsGuarantors.CoApplicant: logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ Property.RealEstate               : logi  TRUE TRUE TRUE FALSE FALSE FALSE ...
##  $ Property.Insurance                : logi  FALSE FALSE FALSE TRUE FALSE FALSE ...
##  $ Property.CarOther                 : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ OtherInstallmentPlans.Bank        : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ OtherInstallmentPlans.Stores      : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ Housing.Rent                      : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ Housing.Own                       : logi  TRUE TRUE TRUE FALSE FALSE FALSE ...
##  $ Job.UnemployedUnskilled           : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
##  $ Job.UnskilledResident             : logi  FALSE FALSE TRUE FALSE FALSE TRUE ...
##  $ Job.SkilledEmployee               : logi  TRUE TRUE FALSE TRUE TRUE FALSE ...
summary(GermanCredit$Personal.Female.Single)
## Length  Class   Mode 
##      0   NULL   NULL

1. Fit a logistic regression model using the training set. Please use all variables, but make sure the variable types are right.

set.seed(2023)
index <- sample(1:nrow(GermanCredit), nrow(GermanCredit) * 0.80)
GermanCrdt.train <- GermanCredit[index, ]
GermanCrdt.test <- GermanCredit[-index, ]

GermanCrdt_GLM.train <- glm(Class ~ ., family = binomial, data = GermanCrdt.train)
GermanCrdt_GLM.test <- glm(Class ~ ., family = binomial, data = GermanCrdt.test)

Your observation: I made a simple logistic regression with the, but instead of using lm(), I used glm() because of the family option that lets the program know the variable is binomial

2. Summarize the model and interpret the coefficients (pick at least one coefficient you think important to discuss in detail).

summary(GermanCrdt_GLM.train)
## 
## Call:
## glm(formula = Class ~ ., family = binomial, data = GermanCrdt.train)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -2.6857  -0.7064   0.3652   0.6644   2.2909  
## 
## Coefficients:
##                                          Estimate Std. Error z value Pr(>|z|)
## (Intercept)                             7.753e+00  1.557e+00   4.979 6.38e-07
## Duration                               -2.479e-02  1.045e-02  -2.371  0.01773
## Amount                                 -1.158e-04  5.005e-05  -2.314  0.02067
## InstallmentRatePercentage2             -2.268e-01  3.441e-01  -0.659  0.50987
## InstallmentRatePercentage3             -5.577e-01  3.751e-01  -1.487  0.13702
## InstallmentRatePercentage4             -7.685e-01  3.389e-01  -2.268  0.02334
## ResidenceDuration2                     -6.458e-01  3.353e-01  -1.926  0.05406
## ResidenceDuration3                     -3.778e-01  3.845e-01  -0.983  0.32579
## ResidenceDuration4                     -2.089e-01  3.438e-01  -0.607  0.54357
## Age                                     1.891e-02  1.052e-02   1.798  0.07225
## NumberExistingCredits2                 -3.483e-01  2.752e-01  -1.266  0.20563
## NumberExistingCredits3                 -1.694e-01  7.202e-01  -0.235  0.81408
## NumberExistingCredits4                 -1.057e+00  1.236e+00  -0.855  0.39248
## NumberPeopleMaintenance2               -1.383e-01  2.930e-01  -0.472  0.63691
## TelephoneTRUE                          -2.454e-01  2.262e-01  -1.085  0.27794
## ForeignWorkerTRUE                      -1.777e+00  8.280e-01  -2.146  0.03190
## CheckingAccountStatus.lt.0TRUE         -2.009e+00  2.683e-01  -7.485 7.15e-14
## CheckingAccountStatus.0.to.200TRUE     -1.318e+00  2.584e-01  -5.098 3.42e-07
## CheckingAccountStatus.gt.200TRUE       -5.968e-01  4.461e-01  -1.338  0.18099
## CreditHistory.NoCredit.AllPaidTRUE     -1.332e+00  4.852e-01  -2.745  0.00604
## CreditHistory.ThisBank.AllPaidTRUE     -1.456e+00  5.129e-01  -2.838  0.00454
## CreditHistory.PaidDulyTRUE             -7.562e-01  3.038e-01  -2.489  0.01282
## CreditHistory.DelayTRUE                -5.459e-01  3.790e-01  -1.440  0.14974
## Purpose.NewCarTRUE                     -1.909e+00  8.725e-01  -2.188  0.02865
## Purpose.UsedCarTRUE                    -2.676e-01  8.978e-01  -0.298  0.76567
## Purpose.Furniture.EquipmentTRUE        -1.085e+00  8.811e-01  -1.232  0.21803
## Purpose.Radio.TelevisionTRUE           -1.080e+00  8.885e-01  -1.215  0.22438
## Purpose.DomesticApplianceTRUE          -1.065e+00  1.230e+00  -0.866  0.38654
## Purpose.RepairsTRUE                    -1.981e+00  1.043e+00  -1.899  0.05751
## Purpose.EducationTRUE                  -1.948e+00  9.573e-01  -2.035  0.04188
## Purpose.RetrainingTRUE                 -1.046e+00  1.509e+00  -0.693  0.48812
## Purpose.BusinessTRUE                   -1.260e+00  9.047e-01  -1.393  0.16369
## SavingsAccountBonds.lt.100TRUE         -9.675e-01  3.008e-01  -3.216  0.00130
## SavingsAccountBonds.100.to.500TRUE     -7.575e-01  3.935e-01  -1.925  0.05420
## SavingsAccountBonds.500.to.1000TRUE    -3.535e-01  5.307e-01  -0.666  0.50539
## SavingsAccountBonds.gt.1000TRUE        -2.019e-01  6.077e-01  -0.332  0.73967
## EmploymentDuration.lt.1TRUE             1.740e-01  5.046e-01   0.345  0.73028
## EmploymentDuration.1.to.4TRUE           4.097e-01  4.809e-01   0.852  0.39426
## EmploymentDuration.4.to.7TRUE           8.700e-01  5.197e-01   1.674  0.09415
## EmploymentDuration.gt.7TRUE             3.453e-01  4.831e-01   0.715  0.47479
## Personal.Male.Divorced.SeperatedTRUE   -5.406e-01  5.016e-01  -1.078  0.28113
## Personal.Female.NotSingleTRUE          -2.465e-01  3.555e-01  -0.693  0.48812
## Personal.Male.SingleTRUE                2.864e-01  3.578e-01   0.801  0.42341
## OtherDebtorsGuarantors.NoneTRUE        -7.684e-01  4.714e-01  -1.630  0.10309
## OtherDebtorsGuarantors.CoApplicantTRUE -1.233e+00  6.403e-01  -1.926  0.05413
## Property.RealEstateTRUE                 8.033e-01  4.678e-01   1.717  0.08596
## Property.InsuranceTRUE                  6.194e-01  4.539e-01   1.365  0.17232
## Property.CarOtherTRUE                   4.378e-01  4.416e-01   0.991  0.32152
## OtherInstallmentPlans.BankTRUE         -5.709e-01  2.722e-01  -2.097  0.03600
## OtherInstallmentPlans.StoresTRUE       -4.917e-01  4.717e-01  -1.042  0.29721
## Housing.RentTRUE                       -5.351e-01  5.338e-01  -1.003  0.31607
## Housing.OwnTRUE                         1.076e-03  5.007e-01   0.002  0.99829
## Job.UnemployedUnskilledTRUE             8.941e-01  8.715e-01   1.026  0.30493
## Job.UnskilledResidentTRUE               6.357e-02  4.004e-01   0.159  0.87385
## Job.SkilledEmployeeTRUE                -4.036e-02  3.278e-01  -0.123  0.90199
##                                           
## (Intercept)                            ***
## Duration                               *  
## Amount                                 *  
## InstallmentRatePercentage2                
## InstallmentRatePercentage3                
## InstallmentRatePercentage4             *  
## ResidenceDuration2                     .  
## ResidenceDuration3                        
## ResidenceDuration4                        
## Age                                    .  
## NumberExistingCredits2                    
## NumberExistingCredits3                    
## NumberExistingCredits4                    
## NumberPeopleMaintenance2                  
## TelephoneTRUE                             
## ForeignWorkerTRUE                      *  
## CheckingAccountStatus.lt.0TRUE         ***
## CheckingAccountStatus.0.to.200TRUE     ***
## CheckingAccountStatus.gt.200TRUE          
## CreditHistory.NoCredit.AllPaidTRUE     ** 
## CreditHistory.ThisBank.AllPaidTRUE     ** 
## CreditHistory.PaidDulyTRUE             *  
## CreditHistory.DelayTRUE                   
## Purpose.NewCarTRUE                     *  
## Purpose.UsedCarTRUE                       
## Purpose.Furniture.EquipmentTRUE           
## Purpose.Radio.TelevisionTRUE              
## Purpose.DomesticApplianceTRUE             
## Purpose.RepairsTRUE                    .  
## Purpose.EducationTRUE                  *  
## Purpose.RetrainingTRUE                    
## Purpose.BusinessTRUE                      
## SavingsAccountBonds.lt.100TRUE         ** 
## SavingsAccountBonds.100.to.500TRUE     .  
## SavingsAccountBonds.500.to.1000TRUE       
## SavingsAccountBonds.gt.1000TRUE           
## EmploymentDuration.lt.1TRUE               
## EmploymentDuration.1.to.4TRUE             
## EmploymentDuration.4.to.7TRUE          .  
## EmploymentDuration.gt.7TRUE               
## Personal.Male.Divorced.SeperatedTRUE      
## Personal.Female.NotSingleTRUE             
## Personal.Male.SingleTRUE                  
## OtherDebtorsGuarantors.NoneTRUE           
## OtherDebtorsGuarantors.CoApplicantTRUE .  
## Property.RealEstateTRUE                .  
## Property.InsuranceTRUE                    
## Property.CarOtherTRUE                     
## OtherInstallmentPlans.BankTRUE         *  
## OtherInstallmentPlans.StoresTRUE          
## Housing.RentTRUE                          
## Housing.OwnTRUE                           
## Job.UnemployedUnskilledTRUE               
## Job.UnskilledResidentTRUE                 
## Job.SkilledEmployeeTRUE                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 980.75  on 799  degrees of freedom
## Residual deviance: 712.07  on 745  degrees of freedom
## AIC: 822.07
## 
## Number of Fisher Scoring iterations: 5

Your observation: Signif. codes: 0 ‘’ 0.001 ‘’ 0.01 ‘’ 0.05 ‘.’ 0.1 ‘ ’ 1 is what caugth my attention the most. They test the level of significance of the predictors when the p values are 0.001, 0.01, 0.05, and 0.1. The model appears to be more significant the smaller the p value, because the smaller the number on the significant code is the more accurate the prediction is.

Task 3: Optimal Probability Cut-off, with weight0 = 1 and weight1 = ### 1.

1. Use the training set to predict probabilities.

pred.glm.train.GerCred <- predict(GermanCrdt_GLM.train, type="response")
hist(pred.glm.train.GerCred)

table(predict(GermanCrdt_GLM.train, type="response") > 0.5)
## 
## FALSE  TRUE 
##   202   598
table(predict(GermanCrdt_GLM.train, type="response") > 0.2)
## 
## FALSE  TRUE 
##    41   759
table(predict(GermanCrdt_GLM.train, type="response") > 0.1)
## 
## FALSE  TRUE 
##    11   789

Your observation: The values vary a lot between true and false depending on the prediction coefficient.

2. Find the optimal probability cut-off point using the MR (misclassification rate) or equivalently the equal-weight cost.

costfunc = function(obs, pred.p, pcut){
    weight1 = 1  
    weight0 = 1  
    c1 = (obs==1)&(pred.p<pcut)    
    c0 = (obs==0)&(pred.p>=pcut)   
    cost = mean(weight1*c1 + weight0*c0)  
    return(cost) 
} 
p.seq = seq(0.01, 1, 0.01) 
cost = rep(0, length(p.seq))[0]  
for(i in 1:length(p.seq)){ 
    cost[i] = costfunc(obs = GermanCrdt.train$Class, pred.p = pred.glm.train.GerCred, pcut = p.seq[i])  
}

pcut = p.seq[which(cost==min(cost))]
print(pcut) 
## [1] 0.43 0.44

Your observation: In the class owrk, we also used mean as a pcut value, in this instance, what I found the most interesting is that the pcut value is so different than the mean.

Task 4: Model Evaluation

1. Using the optimal probability cut-off point obtained in ### 3.2, generate confusion matrix and obtain MR for the the training set.

# get binary prediction
class.glm.train.GC<- (pred.glm.train.GerCred>pcut)*1
# get confusion matrix
table(GermanCrdt.train$Class, class.glm.train.GC, dnn = c("True", "Predicted"))
##        Predicted
## True      0   1
##   FALSE 119 123
##   TRUE   37 521
MR<- mean(GermanCrdt.train$Class!=class.glm.train.GC) 
# False positive rate
FPR <- sum((GermanCrdt.train$Class == 0 & class.glm.train.GC == 1) / sum(GermanCrdt.train$Class == 0)) 

# False negative rate
FNR <- sum((GermanCrdt.train$Class == 1 & class.glm.train.GC == 0) / sum(GermanCrdt.train$Class == 1)) 

cat("FPR:", FPR, "FNR:", FNR, "MR", MR, "\n")
## FPR: 0.5082645 FNR: 0.06630824 MR 0.2

Your observation: The False positive rate is tremendously high. More than half of the positive variables have a false positive.

2. Using the optimal probability cut-off point obtained in ### 3.2, generate the ROC curve and calculate the AUC for the training set.

library(ROCR)
## Warning: package 'ROCR' was built under R version 4.1.3
pred <- prediction(pred.glm.train.GerCred, GermanCrdt.train$Class)
perf <- performance(pred, "tpr", "fpr")
plot(perf, colorize=TRUE)

#Get the AUC
unlist(slot(performance(pred, "auc"), "y.values"))
## [1] 0.8374878

Your observation: The AUC of the ROC curve for the training data suggest that the model strongly predicts the values of the class data.

3. Using the same cut-off point, generate confusion matrix and obtain MR for the test set.

pred.glm.test.GerCred1 <- predict(GermanCrdt_GLM.test, newdata = GermanCrdt.test, type="response")
class.glm.test.GC<- (pred.glm.test.GerCred1>pcut)*1
table(GermanCrdt.test$Class, class.glm.test.GC, dnn = c("True", "Predicted"))
##        Predicted
## True      0   1
##   FALSE  42  16
##   TRUE    6 136
MR<- mean(GermanCrdt.test$Class!=class.glm.test.GC) 
# False positive rate
FPR <- sum((GermanCrdt.test$Class == 0 & class.glm.test.GC == 1) / sum(GermanCrdt.test$Class == 0)) 

# False negative rate
FNR <- sum((GermanCrdt.test$Class == 1 & class.glm.test.GC == 0) / sum(GermanCrdt.test$Class == 1)) 

cat("FPR:", FPR, "FNR:", FNR, "MR", MR, "\n")
## FPR: 0.2758621 FNR: 0.04225352 MR 0.11

Your observation: The false positive rate severely decreases for the training data.

4. Using the same cut-off point, generate the ROC curve and calculate the AUC for the test set.

pred <- prediction(pred.glm.test.GerCred1, GermanCrdt.test$Class)
perf <- performance(pred, "tpr", "fpr")
plot(perf, colorize=TRUE)

#Get the AUC
unlist(slot(performance(pred, "auc"), "y.values"))
## [1] 0.9434191

Your observation: Although the rate of false positives is lower compared to the values in training data, the AUC suggests that this model is less strongly correlated. This can be seen when comparing both of the correlation matrix. In the training data set, the true negatives have a better proportion in the training data 119/242 against the 26/58 for the testing data set. In a similar manner, the true positives are 521/538 for training vs 125/142 for testing.

Task 5: Using different weights

Now, let’s assume “It is worse to class a customer as good when they are bad (weight = 5), than it is to class a customer as bad when they are good (weight = 1).” Please figure out which weight should be 5 and which weight should be ### 1. Then define your cost function accordingly!

1. Obtain optimal probability cut-off point again, with the new weights.

costfunc = function(obs, pred.p, pcut){
    weight1 = 1   
    weight0 = 5    
    c1 = (obs==1)&(pred.p<pcut)    
    c0 = (obs==0)&(pred.p>=pcut)   
    cost = mean(weight1*c1 + weight0*c0)  
    return(cost) 
} 
p.seq = seq(0.01, 1, 0.01) 
cost = rep(0, length(p.seq))  
for(i in 1:length(p.seq)){ 
    cost[i] = costfunc(obs = GermanCrdt.train$Class, pred.p = pred.glm.train.GerCred, pcut = p.seq[i])  
}
plot(p.seq, cost)

# find the optimal pcut
optimal.pcut = p.seq[which(cost==min(cost))]
print(optimal.pcut)
## [1] 0.8

Your observation: By increasin the weight of the variable, the pcut point almost doubled. The value now, is much closer to the true mean of the data set for the class variable, which is 0.7.

2. Obtain the confusion matrix and MR for the training set.

# get binary prediction
class.glm.train.GC2<- (pred.glm.train.GerCred>optimal.pcut)*1
# get confusion matrix
table(GermanCrdt.train$Class, class.glm.train.GC, dnn = c("True", "Predicted"))
##        Predicted
## True      0   1
##   FALSE 119 123
##   TRUE   37 521
MR<- mean(GermanCrdt.train$Class!=class.glm.train.GC2) 
MR
## [1] 0.285

Your observation: Now we have a majority of true negatives, and true positives in the confusion matrix. The values predicted are being a lot better for the true negatives and a little bit worse for the true positives, but better prediction overall.

3. Obtain the confusion matrix and MR for the test set.

# get binary prediction
class.glm.test.GC2<- (pred.glm.test.GerCred1>optimal.pcut)*1
# get confusion matrix
table(GermanCrdt.test$Class, class.glm.test.GC, dnn = c("True", "Predicted"))
##        Predicted
## True      0   1
##   FALSE  42  16
##   TRUE    6 136
MR<- mean(GermanCrdt.test$Class!=class.glm.test.GC2) 
MR
## [1] 0.175

Your observation: In the same way, the confusion matrix demonstrates a better fit for the model overall.

Task 6: Report

Summarize your findings, including the optimal probability cut-off, MR and AUC (if calculated) for both in-sample and out-of-sample data. Discuss what you observed and make some suggestions on how can we improve the model.

The model overall improved very much during the progression of this report. The values in the confusion matrices clearly show that the model was improving severely from every minor adjustment. The equally weighted values for false positives and false negatives worked very well with the data as showed by their AUCs. One minor adjustment I would make to produce the best approximation of the values in the class column of the data set, would be to try different weights for the variables to see which weight fits the data more. This is because we got the best model for the ratio 1-5, but there maybe is a better ratio to predict the data.