Starter code for German credit scoring

Refer to http://archive.ics.uci.edu/ml/datasets/Statlog+(German+Credit+Data)) for variable description. The response variable is Class and all others are predictors.

Only run the following code once to install the package caret. The German credit scoring data in provided in that package.

install.packages('caret')

Task1: Data Preparation

1. Loading the caret package and the GermanCredit dataset.

library(caret) #this package contains the german data with its numeric format
## Warning: package 'caret' was built under R version 4.3.2
## Loading required package: ggplot2
## Warning: package 'ggplot2' was built under R version 4.3.2
## Loading required package: lattice
data(GermanCredit)
GermanCredit$Class <-  as.numeric(GermanCredit$Class == "Good") # use this code to convert `Class` into True or False (equivalent to 1 or 0)
GermanCredit$Class <- as.factor(GermanCredit$Class) #make sure `Class` is a factor as SVM require a factor response
str(GermanCredit)
## 'data.frame':    1000 obs. of  62 variables:
##  $ Duration                              : int  6 48 12 42 24 36 24 36 12 30 ...
##  $ Amount                                : int  1169 5951 2096 7882 4870 9055 2835 6948 3059 5234 ...
##  $ InstallmentRatePercentage             : int  4 2 2 2 3 2 3 2 2 4 ...
##  $ ResidenceDuration                     : int  4 2 3 4 4 4 4 2 4 2 ...
##  $ Age                                   : int  67 22 49 45 53 35 53 35 61 28 ...
##  $ NumberExistingCredits                 : int  2 1 1 1 2 1 1 1 1 2 ...
##  $ NumberPeopleMaintenance               : int  1 1 2 2 2 2 1 1 1 1 ...
##  $ Telephone                             : num  0 1 1 1 1 0 1 0 1 1 ...
##  $ ForeignWorker                         : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ Class                                 : Factor w/ 2 levels "0","1": 2 1 2 2 1 2 2 2 2 1 ...
##  $ CheckingAccountStatus.lt.0            : num  1 0 0 1 1 0 0 0 0 0 ...
##  $ CheckingAccountStatus.0.to.200        : num  0 1 0 0 0 0 0 1 0 1 ...
##  $ CheckingAccountStatus.gt.200          : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CheckingAccountStatus.none            : num  0 0 1 0 0 1 1 0 1 0 ...
##  $ CreditHistory.NoCredit.AllPaid        : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CreditHistory.ThisBank.AllPaid        : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ CreditHistory.PaidDuly                : num  0 1 0 1 0 1 1 1 1 0 ...
##  $ CreditHistory.Delay                   : num  0 0 0 0 1 0 0 0 0 0 ...
##  $ CreditHistory.Critical                : num  1 0 1 0 0 0 0 0 0 1 ...
##  $ Purpose.NewCar                        : num  0 0 0 0 1 0 0 0 0 1 ...
##  $ Purpose.UsedCar                       : num  0 0 0 0 0 0 0 1 0 0 ...
##  $ Purpose.Furniture.Equipment           : num  0 0 0 1 0 0 1 0 0 0 ...
##  $ Purpose.Radio.Television              : num  1 1 0 0 0 0 0 0 1 0 ...
##  $ Purpose.DomesticAppliance             : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Purpose.Repairs                       : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Purpose.Education                     : num  0 0 1 0 0 1 0 0 0 0 ...
##  $ Purpose.Vacation                      : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Purpose.Retraining                    : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Purpose.Business                      : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Purpose.Other                         : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ SavingsAccountBonds.lt.100            : num  0 1 1 1 1 0 0 1 0 1 ...
##  $ SavingsAccountBonds.100.to.500        : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ SavingsAccountBonds.500.to.1000       : num  0 0 0 0 0 0 1 0 0 0 ...
##  $ SavingsAccountBonds.gt.1000           : num  0 0 0 0 0 0 0 0 1 0 ...
##  $ SavingsAccountBonds.Unknown           : num  1 0 0 0 0 1 0 0 0 0 ...
##  $ EmploymentDuration.lt.1               : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ EmploymentDuration.1.to.4             : num  0 1 0 0 1 1 0 1 0 0 ...
##  $ EmploymentDuration.4.to.7             : num  0 0 1 1 0 0 0 0 1 0 ...
##  $ EmploymentDuration.gt.7               : num  1 0 0 0 0 0 1 0 0 0 ...
##  $ EmploymentDuration.Unemployed         : num  0 0 0 0 0 0 0 0 0 1 ...
##  $ Personal.Male.Divorced.Seperated      : num  0 0 0 0 0 0 0 0 1 0 ...
##  $ Personal.Female.NotSingle             : num  0 1 0 0 0 0 0 0 0 0 ...
##  $ Personal.Male.Single                  : num  1 0 1 1 1 1 1 1 0 0 ...
##  $ Personal.Male.Married.Widowed         : num  0 0 0 0 0 0 0 0 0 1 ...
##  $ Personal.Female.Single                : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ OtherDebtorsGuarantors.None           : num  1 1 1 0 1 1 1 1 1 1 ...
##  $ OtherDebtorsGuarantors.CoApplicant    : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ OtherDebtorsGuarantors.Guarantor      : num  0 0 0 1 0 0 0 0 0 0 ...
##  $ Property.RealEstate                   : num  1 1 1 0 0 0 0 0 1 0 ...
##  $ Property.Insurance                    : num  0 0 0 1 0 0 1 0 0 0 ...
##  $ Property.CarOther                     : num  0 0 0 0 0 0 0 1 0 1 ...
##  $ Property.Unknown                      : num  0 0 0 0 1 1 0 0 0 0 ...
##  $ OtherInstallmentPlans.Bank            : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ OtherInstallmentPlans.Stores          : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ OtherInstallmentPlans.None            : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ Housing.Rent                          : num  0 0 0 0 0 0 0 1 0 0 ...
##  $ Housing.Own                           : num  1 1 1 0 0 0 1 0 1 1 ...
##  $ Housing.ForFree                       : num  0 0 0 1 1 1 0 0 0 0 ...
##  $ Job.UnemployedUnskilled               : num  0 0 0 0 0 0 0 0 0 0 ...
##  $ Job.UnskilledResident                 : num  0 0 1 0 0 1 0 0 1 0 ...
##  $ Job.SkilledEmployee                   : num  1 1 0 1 1 0 1 0 0 0 ...
##  $ Job.Management.SelfEmp.HighlyQualified: num  0 0 0 0 0 0 0 1 0 1 ...

There are a lots of values that appear to be binomial variables

GermanCredit = GermanCredit[,-c(14,19,27,30,35,40,44,45,48,52,55,58,62)]
head(GermanCredit)
##   Duration Amount InstallmentRatePercentage ResidenceDuration Age
## 1        6   1169                         4                 4  67
## 2       48   5951                         2                 2  22
## 3       12   2096                         2                 3  49
## 4       42   7882                         2                 4  45
## 5       24   4870                         3                 4  53
## 6       36   9055                         2                 4  35
##   NumberExistingCredits NumberPeopleMaintenance Telephone ForeignWorker Class
## 1                     2                       1         0             1     1
## 2                     1                       1         1             1     0
## 3                     1                       2         1             1     1
## 4                     1                       2         1             1     1
## 5                     2                       2         1             1     0
## 6                     1                       2         0             1     1
##   CheckingAccountStatus.lt.0 CheckingAccountStatus.0.to.200
## 1                          1                              0
## 2                          0                              1
## 3                          0                              0
## 4                          1                              0
## 5                          1                              0
## 6                          0                              0
##   CheckingAccountStatus.gt.200 CreditHistory.NoCredit.AllPaid
## 1                            0                              0
## 2                            0                              0
## 3                            0                              0
## 4                            0                              0
## 5                            0                              0
## 6                            0                              0
##   CreditHistory.ThisBank.AllPaid CreditHistory.PaidDuly CreditHistory.Delay
## 1                              0                      0                   0
## 2                              0                      1                   0
## 3                              0                      0                   0
## 4                              0                      1                   0
## 5                              0                      0                   1
## 6                              0                      1                   0
##   Purpose.NewCar Purpose.UsedCar Purpose.Furniture.Equipment
## 1              0               0                           0
## 2              0               0                           0
## 3              0               0                           0
## 4              0               0                           1
## 5              1               0                           0
## 6              0               0                           0
##   Purpose.Radio.Television Purpose.DomesticAppliance Purpose.Repairs
## 1                        1                         0               0
## 2                        1                         0               0
## 3                        0                         0               0
## 4                        0                         0               0
## 5                        0                         0               0
## 6                        0                         0               0
##   Purpose.Education Purpose.Retraining Purpose.Business
## 1                 0                  0                0
## 2                 0                  0                0
## 3                 1                  0                0
## 4                 0                  0                0
## 5                 0                  0                0
## 6                 1                  0                0
##   SavingsAccountBonds.lt.100 SavingsAccountBonds.100.to.500
## 1                          0                              0
## 2                          1                              0
## 3                          1                              0
## 4                          1                              0
## 5                          1                              0
## 6                          0                              0
##   SavingsAccountBonds.500.to.1000 SavingsAccountBonds.gt.1000
## 1                               0                           0
## 2                               0                           0
## 3                               0                           0
## 4                               0                           0
## 5                               0                           0
## 6                               0                           0
##   EmploymentDuration.lt.1 EmploymentDuration.1.to.4 EmploymentDuration.4.to.7
## 1                       0                         0                         0
## 2                       0                         1                         0
## 3                       0                         0                         1
## 4                       0                         0                         1
## 5                       0                         1                         0
## 6                       0                         1                         0
##   EmploymentDuration.gt.7 Personal.Male.Divorced.Seperated
## 1                       1                                0
## 2                       0                                0
## 3                       0                                0
## 4                       0                                0
## 5                       0                                0
## 6                       0                                0
##   Personal.Female.NotSingle Personal.Male.Single OtherDebtorsGuarantors.None
## 1                         0                    1                           1
## 2                         1                    0                           1
## 3                         0                    1                           1
## 4                         0                    1                           0
## 5                         0                    1                           1
## 6                         0                    1                           1
##   OtherDebtorsGuarantors.CoApplicant Property.RealEstate Property.Insurance
## 1                                  0                   1                  0
## 2                                  0                   1                  0
## 3                                  0                   1                  0
## 4                                  0                   0                  1
## 5                                  0                   0                  0
## 6                                  0                   0                  0
##   Property.CarOther OtherInstallmentPlans.Bank OtherInstallmentPlans.Stores
## 1                 0                          0                            0
## 2                 0                          0                            0
## 3                 0                          0                            0
## 4                 0                          0                            0
## 5                 0                          0                            0
## 6                 0                          0                            0
##   Housing.Rent Housing.Own Job.UnemployedUnskilled Job.UnskilledResident
## 1            0           1                       0                     0
## 2            0           1                       0                     0
## 3            0           1                       0                     1
## 4            0           0                       0                     0
## 5            0           0                       0                     0
## 6            0           0                       0                     1
##   Job.SkilledEmployee
## 1                   1
## 2                   1
## 3                   0
## 4                   1
## 5                   1
## 6                   0

2. Exploring the dataset to understand its structure.

Summary helps us visualize the data, and the basic distribution of the values stored within the variables.

summary(GermanCredit)
##     Duration        Amount      InstallmentRatePercentage ResidenceDuration
##  Min.   : 4.0   Min.   :  250   Min.   :1.000             Min.   :1.000    
##  1st Qu.:12.0   1st Qu.: 1366   1st Qu.:2.000             1st Qu.:2.000    
##  Median :18.0   Median : 2320   Median :3.000             Median :3.000    
##  Mean   :20.9   Mean   : 3271   Mean   :2.973             Mean   :2.845    
##  3rd Qu.:24.0   3rd Qu.: 3972   3rd Qu.:4.000             3rd Qu.:4.000    
##  Max.   :72.0   Max.   :18424   Max.   :4.000             Max.   :4.000    
##       Age        NumberExistingCredits NumberPeopleMaintenance   Telephone    
##  Min.   :19.00   Min.   :1.000         Min.   :1.000           Min.   :0.000  
##  1st Qu.:27.00   1st Qu.:1.000         1st Qu.:1.000           1st Qu.:0.000  
##  Median :33.00   Median :1.000         Median :1.000           Median :1.000  
##  Mean   :35.55   Mean   :1.407         Mean   :1.155           Mean   :0.596  
##  3rd Qu.:42.00   3rd Qu.:2.000         3rd Qu.:1.000           3rd Qu.:1.000  
##  Max.   :75.00   Max.   :4.000         Max.   :2.000           Max.   :1.000  
##  ForeignWorker   Class   CheckingAccountStatus.lt.0
##  Min.   :0.000   0:300   Min.   :0.000             
##  1st Qu.:1.000   1:700   1st Qu.:0.000             
##  Median :1.000           Median :0.000             
##  Mean   :0.963           Mean   :0.274             
##  3rd Qu.:1.000           3rd Qu.:1.000             
##  Max.   :1.000           Max.   :1.000             
##  CheckingAccountStatus.0.to.200 CheckingAccountStatus.gt.200
##  Min.   :0.000                  Min.   :0.000               
##  1st Qu.:0.000                  1st Qu.:0.000               
##  Median :0.000                  Median :0.000               
##  Mean   :0.269                  Mean   :0.063               
##  3rd Qu.:1.000                  3rd Qu.:0.000               
##  Max.   :1.000                  Max.   :1.000               
##  CreditHistory.NoCredit.AllPaid CreditHistory.ThisBank.AllPaid
##  Min.   :0.00                   Min.   :0.000                 
##  1st Qu.:0.00                   1st Qu.:0.000                 
##  Median :0.00                   Median :0.000                 
##  Mean   :0.04                   Mean   :0.049                 
##  3rd Qu.:0.00                   3rd Qu.:0.000                 
##  Max.   :1.00                   Max.   :1.000                 
##  CreditHistory.PaidDuly CreditHistory.Delay Purpose.NewCar  Purpose.UsedCar
##  Min.   :0.00           Min.   :0.000       Min.   :0.000   Min.   :0.000  
##  1st Qu.:0.00           1st Qu.:0.000       1st Qu.:0.000   1st Qu.:0.000  
##  Median :1.00           Median :0.000       Median :0.000   Median :0.000  
##  Mean   :0.53           Mean   :0.088       Mean   :0.234   Mean   :0.103  
##  3rd Qu.:1.00           3rd Qu.:0.000       3rd Qu.:0.000   3rd Qu.:0.000  
##  Max.   :1.00           Max.   :1.000       Max.   :1.000   Max.   :1.000  
##  Purpose.Furniture.Equipment Purpose.Radio.Television Purpose.DomesticAppliance
##  Min.   :0.000               Min.   :0.00             Min.   :0.000            
##  1st Qu.:0.000               1st Qu.:0.00             1st Qu.:0.000            
##  Median :0.000               Median :0.00             Median :0.000            
##  Mean   :0.181               Mean   :0.28             Mean   :0.012            
##  3rd Qu.:0.000               3rd Qu.:1.00             3rd Qu.:0.000            
##  Max.   :1.000               Max.   :1.00             Max.   :1.000            
##  Purpose.Repairs Purpose.Education Purpose.Retraining Purpose.Business
##  Min.   :0.000   Min.   :0.00      Min.   :0.000      Min.   :0.000   
##  1st Qu.:0.000   1st Qu.:0.00      1st Qu.:0.000      1st Qu.:0.000   
##  Median :0.000   Median :0.00      Median :0.000      Median :0.000   
##  Mean   :0.022   Mean   :0.05      Mean   :0.009      Mean   :0.097   
##  3rd Qu.:0.000   3rd Qu.:0.00      3rd Qu.:0.000      3rd Qu.:0.000   
##  Max.   :1.000   Max.   :1.00      Max.   :1.000      Max.   :1.000   
##  SavingsAccountBonds.lt.100 SavingsAccountBonds.100.to.500
##  Min.   :0.000              Min.   :0.000                 
##  1st Qu.:0.000              1st Qu.:0.000                 
##  Median :1.000              Median :0.000                 
##  Mean   :0.603              Mean   :0.103                 
##  3rd Qu.:1.000              3rd Qu.:0.000                 
##  Max.   :1.000              Max.   :1.000                 
##  SavingsAccountBonds.500.to.1000 SavingsAccountBonds.gt.1000
##  Min.   :0.000                   Min.   :0.000              
##  1st Qu.:0.000                   1st Qu.:0.000              
##  Median :0.000                   Median :0.000              
##  Mean   :0.063                   Mean   :0.048              
##  3rd Qu.:0.000                   3rd Qu.:0.000              
##  Max.   :1.000                   Max.   :1.000              
##  EmploymentDuration.lt.1 EmploymentDuration.1.to.4 EmploymentDuration.4.to.7
##  Min.   :0.000           Min.   :0.000             Min.   :0.000            
##  1st Qu.:0.000           1st Qu.:0.000             1st Qu.:0.000            
##  Median :0.000           Median :0.000             Median :0.000            
##  Mean   :0.172           Mean   :0.339             Mean   :0.174            
##  3rd Qu.:0.000           3rd Qu.:1.000             3rd Qu.:0.000            
##  Max.   :1.000           Max.   :1.000             Max.   :1.000            
##  EmploymentDuration.gt.7 Personal.Male.Divorced.Seperated
##  Min.   :0.000           Min.   :0.00                    
##  1st Qu.:0.000           1st Qu.:0.00                    
##  Median :0.000           Median :0.00                    
##  Mean   :0.253           Mean   :0.05                    
##  3rd Qu.:1.000           3rd Qu.:0.00                    
##  Max.   :1.000           Max.   :1.00                    
##  Personal.Female.NotSingle Personal.Male.Single OtherDebtorsGuarantors.None
##  Min.   :0.00              Min.   :0.000        Min.   :0.000              
##  1st Qu.:0.00              1st Qu.:0.000        1st Qu.:1.000              
##  Median :0.00              Median :1.000        Median :1.000              
##  Mean   :0.31              Mean   :0.548        Mean   :0.907              
##  3rd Qu.:1.00              3rd Qu.:1.000        3rd Qu.:1.000              
##  Max.   :1.00              Max.   :1.000        Max.   :1.000              
##  OtherDebtorsGuarantors.CoApplicant Property.RealEstate Property.Insurance
##  Min.   :0.000                      Min.   :0.000       Min.   :0.000     
##  1st Qu.:0.000                      1st Qu.:0.000       1st Qu.:0.000     
##  Median :0.000                      Median :0.000       Median :0.000     
##  Mean   :0.041                      Mean   :0.282       Mean   :0.232     
##  3rd Qu.:0.000                      3rd Qu.:1.000       3rd Qu.:0.000     
##  Max.   :1.000                      Max.   :1.000       Max.   :1.000     
##  Property.CarOther OtherInstallmentPlans.Bank OtherInstallmentPlans.Stores
##  Min.   :0.000     Min.   :0.000              Min.   :0.000               
##  1st Qu.:0.000     1st Qu.:0.000              1st Qu.:0.000               
##  Median :0.000     Median :0.000              Median :0.000               
##  Mean   :0.332     Mean   :0.139              Mean   :0.047               
##  3rd Qu.:1.000     3rd Qu.:0.000              3rd Qu.:0.000               
##  Max.   :1.000     Max.   :1.000              Max.   :1.000               
##   Housing.Rent    Housing.Own    Job.UnemployedUnskilled Job.UnskilledResident
##  Min.   :0.000   Min.   :0.000   Min.   :0.000           Min.   :0.0          
##  1st Qu.:0.000   1st Qu.:0.000   1st Qu.:0.000           1st Qu.:0.0          
##  Median :0.000   Median :1.000   Median :0.000           Median :0.0          
##  Mean   :0.179   Mean   :0.713   Mean   :0.022           Mean   :0.2          
##  3rd Qu.:0.000   3rd Qu.:1.000   3rd Qu.:0.000           3rd Qu.:0.0          
##  Max.   :1.000   Max.   :1.000   Max.   :1.000           Max.   :1.0          
##  Job.SkilledEmployee
##  Min.   :0.00       
##  1st Qu.:0.00       
##  Median :1.00       
##  Mean   :0.63       
##  3rd Qu.:1.00       
##  Max.   :1.00
#Check for any missing values
colSums(is.na(GermanCredit))
##                           Duration                             Amount 
##                                  0                                  0 
##          InstallmentRatePercentage                  ResidenceDuration 
##                                  0                                  0 
##                                Age              NumberExistingCredits 
##                                  0                                  0 
##            NumberPeopleMaintenance                          Telephone 
##                                  0                                  0 
##                      ForeignWorker                              Class 
##                                  0                                  0 
##         CheckingAccountStatus.lt.0     CheckingAccountStatus.0.to.200 
##                                  0                                  0 
##       CheckingAccountStatus.gt.200     CreditHistory.NoCredit.AllPaid 
##                                  0                                  0 
##     CreditHistory.ThisBank.AllPaid             CreditHistory.PaidDuly 
##                                  0                                  0 
##                CreditHistory.Delay                     Purpose.NewCar 
##                                  0                                  0 
##                    Purpose.UsedCar        Purpose.Furniture.Equipment 
##                                  0                                  0 
##           Purpose.Radio.Television          Purpose.DomesticAppliance 
##                                  0                                  0 
##                    Purpose.Repairs                  Purpose.Education 
##                                  0                                  0 
##                 Purpose.Retraining                   Purpose.Business 
##                                  0                                  0 
##         SavingsAccountBonds.lt.100     SavingsAccountBonds.100.to.500 
##                                  0                                  0 
##    SavingsAccountBonds.500.to.1000        SavingsAccountBonds.gt.1000 
##                                  0                                  0 
##            EmploymentDuration.lt.1          EmploymentDuration.1.to.4 
##                                  0                                  0 
##          EmploymentDuration.4.to.7            EmploymentDuration.gt.7 
##                                  0                                  0 
##   Personal.Male.Divorced.Seperated          Personal.Female.NotSingle 
##                                  0                                  0 
##               Personal.Male.Single        OtherDebtorsGuarantors.None 
##                                  0                                  0 
## OtherDebtorsGuarantors.CoApplicant                Property.RealEstate 
##                                  0                                  0 
##                 Property.Insurance                  Property.CarOther 
##                                  0                                  0 
##         OtherInstallmentPlans.Bank       OtherInstallmentPlans.Stores 
##                                  0                                  0 
##                       Housing.Rent                        Housing.Own 
##                                  0                                  0 
##            Job.UnemployedUnskilled              Job.UnskilledResident 
##                                  0                                  0 
##                Job.SkilledEmployee 
##                                  0

Now, we check the de distribution of the variable we are analyzing.

ggplot(GermanCredit, aes(x = GermanCredit$Class)) +
  geom_bar(stat = "count", fill = "blue", alpha = 0.7) +
  ggtitle("Distribution class of credit (GOOD= TRUE, BAD= FALSE)") +
  xlab("Month") +
  ylab("Class")
## Warning: Use of `GermanCredit$Class` is discouraged.
## ℹ Use `Class` instead.

We check for outliers in the variable.

boxplot(GermanCredit, las =2, cex.axis=0.7 )

Th most noticeable variable with outliers is “Amount”, we will take a closer look to the summary of its statistics, but will not make any changes at the moment.

summary(GermanCredit$Amount)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##     250    1366    2320    3271    3972   18424

I noticed that most of the variables in the German Credit data set could be changed into boolean, with some exceptions. Finally, there is one variable with tremendous outliers, that I think could be important, so I decided to keep them as they are for the time being, but if needed in the next steps I will handle them appropriately.

3. Splitting the dataset into training and test set. Using random seed as 2023 for reproducibility.

library(e1071)
## Warning: package 'e1071' was built under R version 4.3.2
#Separating the traning and the testing sets into an 80/20 split
set.seed(2023)
index <- sample(1:nrow(GermanCredit),nrow(GermanCredit)*0.80)
training.data <- GermanCredit[index,]
test.data <- GermanCredit[-index,]

Task 2: SVM without weighted class cost

1. SVM model using the training set.

training.data$Class <- factor(training.data$Class)
Ger.credit.svm = svm(Class ~ ., data = training.data, kernel = 'linear')
summary(Ger.credit.svm)
## 
## Call:
## svm(formula = Class ~ ., data = training.data, kernel = "linear")
## 
## 
## Parameters:
##    SVM-Type:  C-classification 
##  SVM-Kernel:  linear 
##        cost:  1 
## 
## Number of Support Vectors:  418
## 
##  ( 201 217 )
## 
## 
## Number of Classes:  2 
## 
## Levels: 
##  0 1

There are 418 support vectors, 201 are 0 and 217 are 1. Everything else is an input, no added cost and linear classification.

2. training set to get prediected classes.

pred_class_train <- predict(Ger.credit.svm, training.data)

Takes the svm of the training data and applies it to itself.

3. Confusion matrix and MR on training set.

Cnfsn_mtrix_trn = table(true = training.data$Class,
                      pred = pred_class_train)
Cnfsn_mtrix_trn
##     pred
## true   0   1
##    0 142 100
##    1  68 490

Missclassification rate of the training data:

MR<- 1 - sum(diag(Cnfsn_mtrix_trn))/sum(Cnfsn_mtrix_trn)
MR
## [1] 0.21

There is a good number of true positives 490/558 and true negatives 142/242. This is clearly reflected in the low number in the MR.

4. testing set to get prediected classes.

pred_class_test <- predict(Ger.credit.svm, test.data)

5. Confusion matrix and MR on testing set.

Cnfsn_mtrix_tst = table(true = test.data$Class, pred = pred_class_test)
Cnfsn_mtrix_tst
##     pred
## true   0   1
##    0  32  26
##    1  24 118

Missclassification rate of the test data:

MR<- 1 - sum(diag(Cnfsn_mtrix_tst))/sum(Cnfsn_mtrix_tst)
MR
## [1] 0.25

Not as good of a confusion matrix with only 118/142 true positives and 32/58 true negatives, which caught my attention. Overall, a good fit of the model since the MR got just a little bit lower, the model is still effective predicting the class variable.

Task 3: SVM with weighted class cost, and probabilities enabled

1. Fit a SVM model using the training set with weight of 2 on “1” and weight of 1 on “0”. Please use all variables, but make sure the variable types are right.

Ger.credit.svm_asymmetric = svm(Class ~ .,
                            data = training.data, 
                            kernel = 'linear',
                            class.weights = c("0" = 1, "1" = 2),
                            probability = TRUE)

The new SVM model contains a weight of 2 for the value of 1 and a 1 for the value of 0.

2. Use the training set to get prediected probabilities and classes.

pred_class_train_a <- predict(Ger.credit.svm_asymmetric,
                          newdata = training.data,
                          probability = TRUE)

Prediction for the asymmetric smv with the training dataset.

3. Obtain confusion matrix and MR on training set

table( true = training.data$Class, pred = pred_class_train_a)
##     pred
## true   0   1
##    0  85 157
##    1  21 537
MR<-1 - sum(diag(Cnfsn_mtrix_tst))/sum(Cnfsn_mtrix_tst)
MR
## [1] 0.25

Despite having an ok MR, the confusion matrix suggest that it is predicting too many true values, thus the high false positive cases.

4. Obtain ROC and AUC on training set

str(pred_class_train_a)
##  Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 1 ...
##  - attr(*, "names")= chr [1:800] "885" "464" "431" "361" ...
##  - attr(*, "probabilities")= num [1:800, 1:2] 0.3264 0.2188 0.0568 0.3154 0.325 ...
##   ..- attr(*, "dimnames")=List of 2
##   .. ..$ : chr [1:800] "885" "464" "431" "361" ...
##   .. ..$ : chr [1:2] "0" "1"
pred_class_train_a = attr(pred_class_train_a, "probabilities")[, 2] 
library(ROCR)
pred <- prediction(pred_class_train_a, training.data$Class)
perf <- performance(pred, "tpr", "fpr")
plot(perf, colorize=TRUE)

# Obtain AUC for training data

#Get the AUC
unlist(slot(performance(pred, "auc"), "y.values"))
## [1] 0.8111911

The AUC indicates a good fit of the model using the predicted probabilities for the training dataset.

5. Use the testing set to get prediected probabilities and classes.

pred_class_test_a = predict(Ger.credit.svm_asymmetric,
                         newdata = test.data,
                         probability = TRUE)

6. Obtain confusion matrix and MR on testing set.

table( true = test.data$Class, pred = pred_class_test_a)
##     pred
## true   0   1
##    0  21  37
##    1   9 133
1 - sum(diag(Cnfsn_mtrix_tst))/sum(Cnfsn_mtrix_tst)
## [1] 0.25

The false negatives are still higher than the true negatives. Yet, it is at a lower rate.

7. Obtain ROC and AUC on testing set.

str(pred_class_test_a)
##  Factor w/ 2 levels "0","1": 2 2 1 2 2 2 2 2 2 2 ...
##  - attr(*, "names")= chr [1:200] "1" "14" "15" "17" ...
##  - attr(*, "probabilities")= num [1:200, 1:2] 0.112 0.395 0.555 0.098 0.228 ...
##   ..- attr(*, "dimnames")=List of 2
##   .. ..$ : chr [1:200] "1" "14" "15" "17" ...
##   .. ..$ : chr [1:2] "0" "1"
pred_class_test_a = attr(pred_class_test_a, "probabilities")[, 2]
#ROC
pred <- prediction(pred_class_test_a, test.data$Class)
perf <- performance(pred, "tpr", "fpr")
plot(perf, colorize=TRUE)

#Get the AUC
unlist(slot(performance(pred, "auc"), "y.values"))
## [1] 0.7793832

The AUC shows that the svm with the weighted values is statistically significant to the data set. The value is a little bit on the lower side of good, but a good fit overall.

Task 4: Report

Since the MR of both testing data sets is the same, my conclusion is that the un-weighted model fits the data better due to the values on the confusion matrices. The values on the weighted data had alarming false positive rates compared to the un-weighted. Yet, both models showed to be good predictors for the Class variable in the GermanCredit data.