Refer to http://archive.ics.uci.edu/ml/datasets/Statlog+(German+Credit+Data)) for variable description. The response variable is Class and all others are predictors.
Only run the following code once to install the package caret. The German credit scoring data in provided in that package.
install.packages('caret')
library(caret) #this package contains the german data with its numeric format
## Loading required package: ggplot2
## Loading required package: lattice
data(GermanCredit)
GermanCredit$Class <- as.numeric(GermanCredit$Class == "Good") # use this code to convert `Class` into True or False (equivalent to 1 or 0)
GermanCredit$Class <- as.factor(GermanCredit$Class) #make sure `Class` is a factor as SVM require a factor response
str(GermanCredit)
## 'data.frame': 1000 obs. of 62 variables:
## $ Duration : int 6 48 12 42 24 36 24 36 12 30 ...
## $ Amount : int 1169 5951 2096 7882 4870 9055 2835 6948 3059 5234 ...
## $ InstallmentRatePercentage : int 4 2 2 2 3 2 3 2 2 4 ...
## $ ResidenceDuration : int 4 2 3 4 4 4 4 2 4 2 ...
## $ Age : int 67 22 49 45 53 35 53 35 61 28 ...
## $ NumberExistingCredits : int 2 1 1 1 2 1 1 1 1 2 ...
## $ NumberPeopleMaintenance : int 1 1 2 2 2 2 1 1 1 1 ...
## $ Telephone : num 0 1 1 1 1 0 1 0 1 1 ...
## $ ForeignWorker : num 1 1 1 1 1 1 1 1 1 1 ...
## $ Class : Factor w/ 2 levels "0","1": 2 1 2 2 1 2 2 2 2 1 ...
## $ CheckingAccountStatus.lt.0 : num 1 0 0 1 1 0 0 0 0 0 ...
## $ CheckingAccountStatus.0.to.200 : num 0 1 0 0 0 0 0 1 0 1 ...
## $ CheckingAccountStatus.gt.200 : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CheckingAccountStatus.none : num 0 0 1 0 0 1 1 0 1 0 ...
## $ CreditHistory.NoCredit.AllPaid : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CreditHistory.ThisBank.AllPaid : num 0 0 0 0 0 0 0 0 0 0 ...
## $ CreditHistory.PaidDuly : num 0 1 0 1 0 1 1 1 1 0 ...
## $ CreditHistory.Delay : num 0 0 0 0 1 0 0 0 0 0 ...
## $ CreditHistory.Critical : num 1 0 1 0 0 0 0 0 0 1 ...
## $ Purpose.NewCar : num 0 0 0 0 1 0 0 0 0 1 ...
## $ Purpose.UsedCar : num 0 0 0 0 0 0 0 1 0 0 ...
## $ Purpose.Furniture.Equipment : num 0 0 0 1 0 0 1 0 0 0 ...
## $ Purpose.Radio.Television : num 1 1 0 0 0 0 0 0 1 0 ...
## $ Purpose.DomesticAppliance : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Purpose.Repairs : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Purpose.Education : num 0 0 1 0 0 1 0 0 0 0 ...
## $ Purpose.Vacation : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Purpose.Retraining : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Purpose.Business : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Purpose.Other : num 0 0 0 0 0 0 0 0 0 0 ...
## $ SavingsAccountBonds.lt.100 : num 0 1 1 1 1 0 0 1 0 1 ...
## $ SavingsAccountBonds.100.to.500 : num 0 0 0 0 0 0 0 0 0 0 ...
## $ SavingsAccountBonds.500.to.1000 : num 0 0 0 0 0 0 1 0 0 0 ...
## $ SavingsAccountBonds.gt.1000 : num 0 0 0 0 0 0 0 0 1 0 ...
## $ SavingsAccountBonds.Unknown : num 1 0 0 0 0 1 0 0 0 0 ...
## $ EmploymentDuration.lt.1 : num 0 0 0 0 0 0 0 0 0 0 ...
## $ EmploymentDuration.1.to.4 : num 0 1 0 0 1 1 0 1 0 0 ...
## $ EmploymentDuration.4.to.7 : num 0 0 1 1 0 0 0 0 1 0 ...
## $ EmploymentDuration.gt.7 : num 1 0 0 0 0 0 1 0 0 0 ...
## $ EmploymentDuration.Unemployed : num 0 0 0 0 0 0 0 0 0 1 ...
## $ Personal.Male.Divorced.Seperated : num 0 0 0 0 0 0 0 0 1 0 ...
## $ Personal.Female.NotSingle : num 0 1 0 0 0 0 0 0 0 0 ...
## $ Personal.Male.Single : num 1 0 1 1 1 1 1 1 0 0 ...
## $ Personal.Male.Married.Widowed : num 0 0 0 0 0 0 0 0 0 1 ...
## $ Personal.Female.Single : num 0 0 0 0 0 0 0 0 0 0 ...
## $ OtherDebtorsGuarantors.None : num 1 1 1 0 1 1 1 1 1 1 ...
## $ OtherDebtorsGuarantors.CoApplicant : num 0 0 0 0 0 0 0 0 0 0 ...
## $ OtherDebtorsGuarantors.Guarantor : num 0 0 0 1 0 0 0 0 0 0 ...
## $ Property.RealEstate : num 1 1 1 0 0 0 0 0 1 0 ...
## $ Property.Insurance : num 0 0 0 1 0 0 1 0 0 0 ...
## $ Property.CarOther : num 0 0 0 0 0 0 0 1 0 1 ...
## $ Property.Unknown : num 0 0 0 0 1 1 0 0 0 0 ...
## $ OtherInstallmentPlans.Bank : num 0 0 0 0 0 0 0 0 0 0 ...
## $ OtherInstallmentPlans.Stores : num 0 0 0 0 0 0 0 0 0 0 ...
## $ OtherInstallmentPlans.None : num 1 1 1 1 1 1 1 1 1 1 ...
## $ Housing.Rent : num 0 0 0 0 0 0 0 1 0 0 ...
## $ Housing.Own : num 1 1 1 0 0 0 1 0 1 1 ...
## $ Housing.ForFree : num 0 0 0 1 1 1 0 0 0 0 ...
## $ Job.UnemployedUnskilled : num 0 0 0 0 0 0 0 0 0 0 ...
## $ Job.UnskilledResident : num 0 0 1 0 0 1 0 0 1 0 ...
## $ Job.SkilledEmployee : num 1 1 0 1 1 0 1 0 0 0 ...
## $ Job.Management.SelfEmp.HighlyQualified: num 0 0 0 0 0 0 0 1 0 1 ...
Your observation: There are a lots of values that appear to be binomial variables
#This is an optional code that drop variables that provide no information in the data
GermanCredit = GermanCredit[,-c(14,19,27,30,35,40,44,45,48,52,55,58,62)]
head(GermanCredit)
## Duration Amount InstallmentRatePercentage ResidenceDuration Age
## 1 6 1169 4 4 67
## 2 48 5951 2 2 22
## 3 12 2096 2 3 49
## 4 42 7882 2 4 45
## 5 24 4870 3 4 53
## 6 36 9055 2 4 35
## NumberExistingCredits NumberPeopleMaintenance Telephone ForeignWorker Class
## 1 2 1 0 1 1
## 2 1 1 1 1 0
## 3 1 2 1 1 1
## 4 1 2 1 1 1
## 5 2 2 1 1 0
## 6 1 2 0 1 1
## CheckingAccountStatus.lt.0 CheckingAccountStatus.0.to.200
## 1 1 0
## 2 0 1
## 3 0 0
## 4 1 0
## 5 1 0
## 6 0 0
## CheckingAccountStatus.gt.200 CreditHistory.NoCredit.AllPaid
## 1 0 0
## 2 0 0
## 3 0 0
## 4 0 0
## 5 0 0
## 6 0 0
## CreditHistory.ThisBank.AllPaid CreditHistory.PaidDuly CreditHistory.Delay
## 1 0 0 0
## 2 0 1 0
## 3 0 0 0
## 4 0 1 0
## 5 0 0 1
## 6 0 1 0
## Purpose.NewCar Purpose.UsedCar Purpose.Furniture.Equipment
## 1 0 0 0
## 2 0 0 0
## 3 0 0 0
## 4 0 0 1
## 5 1 0 0
## 6 0 0 0
## Purpose.Radio.Television Purpose.DomesticAppliance Purpose.Repairs
## 1 1 0 0
## 2 1 0 0
## 3 0 0 0
## 4 0 0 0
## 5 0 0 0
## 6 0 0 0
## Purpose.Education Purpose.Retraining Purpose.Business
## 1 0 0 0
## 2 0 0 0
## 3 1 0 0
## 4 0 0 0
## 5 0 0 0
## 6 1 0 0
## SavingsAccountBonds.lt.100 SavingsAccountBonds.100.to.500
## 1 0 0
## 2 1 0
## 3 1 0
## 4 1 0
## 5 1 0
## 6 0 0
## SavingsAccountBonds.500.to.1000 SavingsAccountBonds.gt.1000
## 1 0 0
## 2 0 0
## 3 0 0
## 4 0 0
## 5 0 0
## 6 0 0
## EmploymentDuration.lt.1 EmploymentDuration.1.to.4 EmploymentDuration.4.to.7
## 1 0 0 0
## 2 0 1 0
## 3 0 0 1
## 4 0 0 1
## 5 0 1 0
## 6 0 1 0
## EmploymentDuration.gt.7 Personal.Male.Divorced.Seperated
## 1 1 0
## 2 0 0
## 3 0 0
## 4 0 0
## 5 0 0
## 6 0 0
## Personal.Female.NotSingle Personal.Male.Single OtherDebtorsGuarantors.None
## 1 0 1 1
## 2 1 0 1
## 3 0 1 1
## 4 0 1 0
## 5 0 1 1
## 6 0 1 1
## OtherDebtorsGuarantors.CoApplicant Property.RealEstate Property.Insurance
## 1 0 1 0
## 2 0 1 0
## 3 0 1 0
## 4 0 0 1
## 5 0 0 0
## 6 0 0 0
## Property.CarOther OtherInstallmentPlans.Bank OtherInstallmentPlans.Stores
## 1 0 0 0
## 2 0 0 0
## 3 0 0 0
## 4 0 0 0
## 5 0 0 0
## 6 0 0 0
## Housing.Rent Housing.Own Job.UnemployedUnskilled Job.UnskilledResident
## 1 0 1 0 0
## 2 0 1 0 0
## 3 0 1 0 1
## 4 0 0 0 0
## 5 0 0 0 0
## 6 0 0 0 1
## Job.SkilledEmployee
## 1 1
## 2 1
## 3 0
## 4 1
## 5 1
## 6 0
Summary helps us visualize the data, and the basic distribution of the values stored within the variables.
summary(GermanCredit)
## Duration Amount InstallmentRatePercentage ResidenceDuration
## Min. : 4.0 Min. : 250 Min. :1.000 Min. :1.000
## 1st Qu.:12.0 1st Qu.: 1366 1st Qu.:2.000 1st Qu.:2.000
## Median :18.0 Median : 2320 Median :3.000 Median :3.000
## Mean :20.9 Mean : 3271 Mean :2.973 Mean :2.845
## 3rd Qu.:24.0 3rd Qu.: 3972 3rd Qu.:4.000 3rd Qu.:4.000
## Max. :72.0 Max. :18424 Max. :4.000 Max. :4.000
## Age NumberExistingCredits NumberPeopleMaintenance Telephone
## Min. :19.00 Min. :1.000 Min. :1.000 Min. :0.000
## 1st Qu.:27.00 1st Qu.:1.000 1st Qu.:1.000 1st Qu.:0.000
## Median :33.00 Median :1.000 Median :1.000 Median :1.000
## Mean :35.55 Mean :1.407 Mean :1.155 Mean :0.596
## 3rd Qu.:42.00 3rd Qu.:2.000 3rd Qu.:1.000 3rd Qu.:1.000
## Max. :75.00 Max. :4.000 Max. :2.000 Max. :1.000
## ForeignWorker Class CheckingAccountStatus.lt.0
## Min. :0.000 0:300 Min. :0.000
## 1st Qu.:1.000 1:700 1st Qu.:0.000
## Median :1.000 Median :0.000
## Mean :0.963 Mean :0.274
## 3rd Qu.:1.000 3rd Qu.:1.000
## Max. :1.000 Max. :1.000
## CheckingAccountStatus.0.to.200 CheckingAccountStatus.gt.200
## Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.000
## Mean :0.269 Mean :0.063
## 3rd Qu.:1.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000
## CreditHistory.NoCredit.AllPaid CreditHistory.ThisBank.AllPaid
## Min. :0.00 Min. :0.000
## 1st Qu.:0.00 1st Qu.:0.000
## Median :0.00 Median :0.000
## Mean :0.04 Mean :0.049
## 3rd Qu.:0.00 3rd Qu.:0.000
## Max. :1.00 Max. :1.000
## CreditHistory.PaidDuly CreditHistory.Delay Purpose.NewCar Purpose.UsedCar
## Min. :0.00 Min. :0.000 Min. :0.000 Min. :0.000
## 1st Qu.:0.00 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000
## Median :1.00 Median :0.000 Median :0.000 Median :0.000
## Mean :0.53 Mean :0.088 Mean :0.234 Mean :0.103
## 3rd Qu.:1.00 3rd Qu.:0.000 3rd Qu.:0.000 3rd Qu.:0.000
## Max. :1.00 Max. :1.000 Max. :1.000 Max. :1.000
## Purpose.Furniture.Equipment Purpose.Radio.Television Purpose.DomesticAppliance
## Min. :0.000 Min. :0.00 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.00 1st Qu.:0.000
## Median :0.000 Median :0.00 Median :0.000
## Mean :0.181 Mean :0.28 Mean :0.012
## 3rd Qu.:0.000 3rd Qu.:1.00 3rd Qu.:0.000
## Max. :1.000 Max. :1.00 Max. :1.000
## Purpose.Repairs Purpose.Education Purpose.Retraining Purpose.Business
## Min. :0.000 Min. :0.00 Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.00 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.00 Median :0.000 Median :0.000
## Mean :0.022 Mean :0.05 Mean :0.009 Mean :0.097
## 3rd Qu.:0.000 3rd Qu.:0.00 3rd Qu.:0.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.00 Max. :1.000 Max. :1.000
## SavingsAccountBonds.lt.100 SavingsAccountBonds.100.to.500
## Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000
## Median :1.000 Median :0.000
## Mean :0.603 Mean :0.103
## 3rd Qu.:1.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000
## SavingsAccountBonds.500.to.1000 SavingsAccountBonds.gt.1000
## Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.000
## Mean :0.063 Mean :0.048
## 3rd Qu.:0.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000
## EmploymentDuration.lt.1 EmploymentDuration.1.to.4 EmploymentDuration.4.to.7
## Min. :0.000 Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.000 Median :0.000
## Mean :0.172 Mean :0.339 Mean :0.174
## 3rd Qu.:0.000 3rd Qu.:1.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000 Max. :1.000
## EmploymentDuration.gt.7 Personal.Male.Divorced.Seperated
## Min. :0.000 Min. :0.00
## 1st Qu.:0.000 1st Qu.:0.00
## Median :0.000 Median :0.00
## Mean :0.253 Mean :0.05
## 3rd Qu.:1.000 3rd Qu.:0.00
## Max. :1.000 Max. :1.00
## Personal.Female.NotSingle Personal.Male.Single OtherDebtorsGuarantors.None
## Min. :0.00 Min. :0.000 Min. :0.000
## 1st Qu.:0.00 1st Qu.:0.000 1st Qu.:1.000
## Median :0.00 Median :1.000 Median :1.000
## Mean :0.31 Mean :0.548 Mean :0.907
## 3rd Qu.:1.00 3rd Qu.:1.000 3rd Qu.:1.000
## Max. :1.00 Max. :1.000 Max. :1.000
## OtherDebtorsGuarantors.CoApplicant Property.RealEstate Property.Insurance
## Min. :0.000 Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.000 Median :0.000
## Mean :0.041 Mean :0.282 Mean :0.232
## 3rd Qu.:0.000 3rd Qu.:1.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000 Max. :1.000
## Property.CarOther OtherInstallmentPlans.Bank OtherInstallmentPlans.Stores
## Min. :0.000 Min. :0.000 Min. :0.000
## 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000
## Median :0.000 Median :0.000 Median :0.000
## Mean :0.332 Mean :0.139 Mean :0.047
## 3rd Qu.:1.000 3rd Qu.:0.000 3rd Qu.:0.000
## Max. :1.000 Max. :1.000 Max. :1.000
## Housing.Rent Housing.Own Job.UnemployedUnskilled Job.UnskilledResident
## Min. :0.000 Min. :0.000 Min. :0.000 Min. :0.0
## 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.0
## Median :0.000 Median :1.000 Median :0.000 Median :0.0
## Mean :0.179 Mean :0.713 Mean :0.022 Mean :0.2
## 3rd Qu.:0.000 3rd Qu.:1.000 3rd Qu.:0.000 3rd Qu.:0.0
## Max. :1.000 Max. :1.000 Max. :1.000 Max. :1.0
## Job.SkilledEmployee
## Min. :0.00
## 1st Qu.:0.00
## Median :1.00
## Mean :0.63
## 3rd Qu.:1.00
## Max. :1.00
#Check for any missing values
colSums(is.na(GermanCredit))
## Duration Amount
## 0 0
## InstallmentRatePercentage ResidenceDuration
## 0 0
## Age NumberExistingCredits
## 0 0
## NumberPeopleMaintenance Telephone
## 0 0
## ForeignWorker Class
## 0 0
## CheckingAccountStatus.lt.0 CheckingAccountStatus.0.to.200
## 0 0
## CheckingAccountStatus.gt.200 CreditHistory.NoCredit.AllPaid
## 0 0
## CreditHistory.ThisBank.AllPaid CreditHistory.PaidDuly
## 0 0
## CreditHistory.Delay Purpose.NewCar
## 0 0
## Purpose.UsedCar Purpose.Furniture.Equipment
## 0 0
## Purpose.Radio.Television Purpose.DomesticAppliance
## 0 0
## Purpose.Repairs Purpose.Education
## 0 0
## Purpose.Retraining Purpose.Business
## 0 0
## SavingsAccountBonds.lt.100 SavingsAccountBonds.100.to.500
## 0 0
## SavingsAccountBonds.500.to.1000 SavingsAccountBonds.gt.1000
## 0 0
## EmploymentDuration.lt.1 EmploymentDuration.1.to.4
## 0 0
## EmploymentDuration.4.to.7 EmploymentDuration.gt.7
## 0 0
## Personal.Male.Divorced.Seperated Personal.Female.NotSingle
## 0 0
## Personal.Male.Single OtherDebtorsGuarantors.None
## 0 0
## OtherDebtorsGuarantors.CoApplicant Property.RealEstate
## 0 0
## Property.Insurance Property.CarOther
## 0 0
## OtherInstallmentPlans.Bank OtherInstallmentPlans.Stores
## 0 0
## Housing.Rent Housing.Own
## 0 0
## Job.UnemployedUnskilled Job.UnskilledResident
## 0 0
## Job.SkilledEmployee
## 0
Now, we check the de distribution of the variable we are analyzing.
ggplot(GermanCredit, aes(x = GermanCredit$Class)) +
geom_bar(stat = "count", fill = "blue", alpha = 0.7) +
ggtitle("Distribution class of credit (GOOD= TRUE, BAD= FALSE)") +
xlab("Month") +
ylab("Class")
## Warning: Use of `GermanCredit$Class` is discouraged.
## ℹ Use `Class` instead.
We check for outliers in the variable.
boxplot(GermanCredit, las =2, cex.axis=0.7 )
Th most noticeable variable with outliers is “Amount”, we will take a closer look to the summary of its statistics, but will not make any changes at the moment.
summary(GermanCredit$Amount)
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 250 1366 2320 3271 3972 18424
Your observation: I noticed that most of the variables in the German Credit data set could be changed into boolean, with some exceptions. Finally, there is one variable with tremendous outliers, that I think could be important, so I decided to keep them as they are for the time being, but if needed in the next steps I will handle them appropriately.
2023 for reproducibility.library(e1071)
#Separating the traning and the testing sets into an 80/20 split
set.seed(2023)
index <- sample(1:nrow(GermanCredit),nrow(GermanCredit)*0.80)
training.data <- GermanCredit[index,]
test.data <- GermanCredit[-index,]
Your observation: Just a regular 80/20 separation for training and testting data for the svm model
training.data$Class <- factor(training.data$Class)
Ger.credit.svm = svm(Class ~ ., data = training.data, kernel = 'linear')
summary(Ger.credit.svm)
##
## Call:
## svm(formula = Class ~ ., data = training.data, kernel = "linear")
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: linear
## cost: 1
##
## Number of Support Vectors: 418
##
## ( 201 217 )
##
##
## Number of Classes: 2
##
## Levels:
## 0 1
Your observation: There are 418 support vectors, 201 are 0 and 217 are 1. Everything else is an input, no added cost and linear classification.
pred_class_train <- predict(Ger.credit.svm, training.data)
Your observation: Takes the svm of the training data and applies it to itself.
Cnfsn_mtrix_trn = table(true = training.data$Class,
pred = pred_class_train)
Cnfsn_mtrix_trn
## pred
## true 0 1
## 0 142 100
## 1 68 490
Missclassification rate of the training data:
MR<- 1 - sum(diag(Cnfsn_mtrix_trn))/sum(Cnfsn_mtrix_trn)
MR
## [1] 0.21
Your observation: There is a good number of true positives 490/558 and true negatives 142/242. This is clearly reflected in the low number in the MR.
pred_class_test <- predict(Ger.credit.svm, test.data)
Your observation: Now we are getting the predictions of the svm using the testing data.
Cnfsn_mtrix_tst = table(true = test.data$Class, pred = pred_class_test)
Cnfsn_mtrix_tst
## pred
## true 0 1
## 0 32 26
## 1 24 118
Missclassification rate of the test data:
MR<- 1 - sum(diag(Cnfsn_mtrix_tst))/sum(Cnfsn_mtrix_tst)
MR
## [1] 0.25
Your observation: Not as good of a confusion matrix with only 118/142 true positives and 32/58 true negatives, which caught my attention. Overall, a good fit of the model since the MR got just a little bit lower, the model is still effective predicting the class variable.
Ger.credit.svm_asymmetric = svm(Class ~ .,
data = training.data,
kernel = 'linear',
class.weights = c("0" = 1, "1" = 2),
probability = TRUE)
Your observation: The new SVM model contains a weight of 2 for the value of 1 and a 1 for the value of 0.
pred_class_train_a <- predict(Ger.credit.svm_asymmetric,
newdata = training.data,
probability = TRUE)
Your observation: We are getting a prediction for the asymmetric smv with the training dataset.
table( true = training.data$Class, pred = pred_class_train_a)
## pred
## true 0 1
## 0 85 157
## 1 21 537
MR<-1 - sum(diag(Cnfsn_mtrix_tst))/sum(Cnfsn_mtrix_tst)
MR
## [1] 0.25
Your observation: Despite having an ok MR, the confusion matrix suggest that it is predicting too many true values, thus the high false positive cases.
str(pred_class_train_a)
## Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 1 ...
## - attr(*, "names")= chr [1:800] "885" "464" "431" "361" ...
## - attr(*, "probabilities")= num [1:800, 1:2] 0.3264 0.2188 0.0568 0.3154 0.325 ...
## ..- attr(*, "dimnames")=List of 2
## .. ..$ : chr [1:800] "885" "464" "431" "361" ...
## .. ..$ : chr [1:2] "0" "1"
pred_class_train_a = attr(pred_class_train_a, "probabilities")[, 2]
library(ROCR)
pred <- prediction(pred_class_train_a, training.data$Class)
perf <- performance(pred, "tpr", "fpr")
plot(perf, colorize=TRUE)
# Obtain AUC for training data
#Get the AUC
unlist(slot(performance(pred, "auc"), "y.values"))
## [1] 0.8111911
Your observation: The AUC indicates a good fit of the model using the predicted probabilities for the training dataset.
pred_class_test_a = predict(Ger.credit.svm_asymmetric,
newdata = test.data,
probability = TRUE)
Your observation: We are feeding the asymmetric svm with the test data.
table( true = test.data$Class, pred = pred_class_test_a)
## pred
## true 0 1
## 0 21 37
## 1 9 133
1 - sum(diag(Cnfsn_mtrix_tst))/sum(Cnfsn_mtrix_tst)
## [1] 0.25
Your observation: The false negatives are still higher than the true negatives. Yet, it is at a lower rate.
str(pred_class_test_a)
## Factor w/ 2 levels "0","1": 2 2 1 2 2 2 2 2 2 2 ...
## - attr(*, "names")= chr [1:200] "1" "14" "15" "17" ...
## - attr(*, "probabilities")= num [1:200, 1:2] 0.112 0.395 0.555 0.098 0.228 ...
## ..- attr(*, "dimnames")=List of 2
## .. ..$ : chr [1:200] "1" "14" "15" "17" ...
## .. ..$ : chr [1:2] "0" "1"
pred_class_test_a = attr(pred_class_test_a, "probabilities")[, 2]
#ROC
pred <- prediction(pred_class_test_a, test.data$Class)
perf <- performance(pred, "tpr", "fpr")
plot(perf, colorize=TRUE)
#Get the AUC
unlist(slot(performance(pred, "auc"), "y.values"))
## [1] 0.7793832
Your observation: The AUC shows that the svm with the weighted values is statistically significant to the data set. The value is a little bit on the lower side of good, but a good fit overall.
Since the MR of both testing data sets is the same, my conclusion is that the un-weighted model fits the data better due to the values on the confusion matrices. The values on the weighted data had alarming false positive rates compared to the un-weighted. Yet, both models showed to be good predictors for the Class variable in the GermanCredit data.
I think svm is easier to code with very similar results. Specially when we want to see if we should adjust the weights of the values in the data. This model was easier, because we were just adjusting the values inside the svm call instead of finding the p-cut values besides the logistic regression and then use them for calculations.