Nicole-Brown-Algorithms-II-HW-8
Problem 5. We have seen that we can fit an SVM with a non-linear kernel in order to perform classification using a non-linear decision boundary.We will now see that we can also obtain a non-linear decision boundary by performing logistic regression using non-linear transformations of the features.
(a) Generate a data set with n = 500 and p = 2, such that the observations belong to two classes with a quadratic decision boundary between them.
set.seed(12345)
x1=runif (500) -0.5
x2=runif (500) -0.5
y=1*( x1^2-x2^2 > 0)
df<-as.data.frame(cbind(x1,x2,y))
head(y)## [1] 0 1 0 0 0 0(b) Plot the observations, colored according to their class labels. Your plot should display X1 on the x-axis, and X2 on the yaxis.
plot(df$x1,df$x2, col=factor(y))
(c) Fit a logistic regression model to the data, using X1 and X2 as predictors.
library(caret)
control=trainControl(method = "repeatedcv",number = 10, repeats=3)
log.fit<-train(y~.,data=df, method='glm', family="binomial", trControl=control)
summary(log.fit)##
## Call:
## NULL
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -1.385 -1.207 1.011 1.130 1.328
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 0.07563 0.09032 0.837 0.402
## x1 0.46494 0.31237 1.488 0.137
## x2 0.41377 0.32373 1.278 0.201
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 692.18 on 499 degrees of freedom
## Residual deviance: 688.04 on 497 degrees of freedom
## AIC: 694.04
##
## Number of Fisher Scoring iterations: 4(d) Apply this model to the training data in order to obtain a predicted class label for each training observation. Plot the observations, colored according to the predicted class labels. The decision boundary should be linear.
df.probs<-predict(log.fit,df)as.data.frame(df.predict<-(ifelse(df.probs>.5, 1,0)))
newdf<-cbind(df,df.predict)#df.predict
plot(x1,x2, col=factor(newdf$df.predict))
(e) Now fit a logistic regression model to the data using non-linear functions of X1 and X2 as predictors (e.g. X21 , X1×X2, log(X2),and so forth).
set.seed(12345)
log.fit=glm(y~poly(x1,2)+poly(x2,2) ,family='binomial', data=df)
summary(log.fit)##
## Call:
## glm(formula = y ~ poly(x1, 2) + poly(x2, 2), family = "binomial",
## data = df)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -0.004146 0.000000 0.000000 0.000000 0.003513
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 462.4 5743.0 0.081 0.936
## poly(x1, 2)1 5879.3 74242.3 0.079 0.937
## poly(x1, 2)2 110432.7 1471937.1 0.075 0.940
## poly(x2, 2)1 5928.8 84203.8 0.070 0.944
## poly(x2, 2)2 -106934.1 1452101.5 -0.074 0.941
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 6.9218e+02 on 499 degrees of freedom
## Residual deviance: 3.4329e-05 on 495 degrees of freedom
## AIC: 10
##
## Number of Fisher Scoring iterations: 25(f) Apply this model to the training data in order to obtain a predicted class label for each training observation. Plot the observations, colored according to the predicted class labels. The decision boundary should be obviously non-linear. If it is not, then repeat (a)-(e) until you come up with an example in which the predicted class labels are obviously non-linear.
df.probs<-predict(log.fit,df)as.data.frame(df.predict<-(ifelse(df.probs>.5, 1,0)))
newdf<-cbind(df,df.predict)#df.predict
plot(x1,x2, col=factor(newdf$df.predict))
(g) Fit a support vector classifier to the data with X1 and X2 as predictors. Obtain a class prediction for each training observation. Plot the observations, colored according to the predicted class labels.
set.seed(12345)
svm.fit<-train(y~x1+x2, data=df, method = 'svmLinear', trControl=control, preProcess = c("center","scale"))
svm.fit## Support Vector Machines with Linear Kernel
##
## 500 samples
## 2 predictor
##
## Pre-processing: centered (2), scaled (2)
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 450, 450, 450, 450, 450, 450, ...
## Resampling results:
##
## RMSE Rsquared MAE
## 0.6028379 0.01581338 0.5123253
##
## Tuning parameter 'C' was held constant at a value of 1svm.probs<-predict(svm.fit,df)
head(svm.probs)## 1 2 3 4 5 6
## 0.4976567 0.9371132 0.9324137 1.0524188 0.6794172 0.6626965as.data.frame(df.predict<-(ifelse(svm.probs>.5, 1,0)))
newsvmdf<-cbind(df,df.predict)plot(x1,x2, col=factor(newsvmdf$df.predict))
(h) Fit a SVM using a non-linear kernel to the data. Obtain a class prediction for each training observation. Plot the observations, colored according to the predicted class labels.
set.seed(12345)
svm.fit2<-train(y~x1+x2, dat=df, method = 'svmRadial', trControl=control, preProcess = c("center","scale"))
svm.fit2## Support Vector Machines with Radial Basis Function Kernel
##
## 500 samples
## 2 predictor
##
## Pre-processing: centered (2), scaled (2)
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 450, 450, 450, 450, 450, 450, ...
## Resampling results across tuning parameters:
##
## C RMSE Rsquared MAE
## 0.25 0.2483821 0.7592557 0.1819114
## 0.50 0.2411481 0.7696975 0.1732061
## 1.00 0.2355450 0.7787067 0.1664917
##
## Tuning parameter 'sigma' was held constant at a value of 1.209616
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 1.209616 and C = 1.svm.probs<-predict(svm.fit2,df)as.data.frame(df.predict<-(ifelse(svm.probs>.5, 1,0)))
newsvmdf<-cbind(df,df.predict)#df.predict
plot(x1,x2, col=factor(newsvmdf$df.predict))
(i) Comment on your results.
It can be seen that the Radial basis SVM model produces the best cross validated r square.
Problem 7. In this problem, you will use support vector approaches in order to predict whether a given car gets high or low gas mileage based on the [Auto]: https://rdrr.io/cran/ISLR/man/Auto.html data set.
(a) Create a binary variable that takes on a 1 for cars with gas mileage above the median, and a 0 for cars with gas mileage below the median.
library(ISLR)
attach(Auto)
library(dplyr)
dim(Auto)## [1] 392 9Auto$mpglevel<-ifelse(mpg>median(mpg),1,0)
table(Auto$mpglevel)##
## 0 1
## 196 196Auto$mpglevel<-as.factor(Auto$mpglevel)(b) Fit a support vector classifier to the data with various values of cost, in order to predict whether a car gets high or low gas mileage. Report the cross-validation errors associated with different values of this parameter. Comment on your results.
library(e1071)
set.seed(1)
tune.out=tune(svm,mpglevel~.,data=Auto,kernel ="linear",ranges=list(cost=c(0.001, 0.01, 0.1, 1,5,10,100)))
summary(tune.out)##
## Parameter tuning of 'svm':
##
## - sampling method: 10-fold cross validation
##
## - best parameters:
## cost
## 1
##
## - best performance: 0.01025641
##
## - Detailed performance results:
## cost error dispersion
## 1 1e-03 0.09442308 0.04519425
## 2 1e-02 0.07653846 0.03617137
## 3 1e-01 0.04596154 0.03378238
## 4 1e+00 0.01025641 0.01792836
## 5 5e+00 0.02051282 0.02648194
## 6 1e+01 0.02051282 0.02648194
## 7 1e+02 0.03076923 0.03151981tune.out$best.model##
## Call:
## best.tune(method = svm, train.x = mpglevel ~ ., data = Auto, ranges = list(cost = c(0.001,
## 0.01, 0.1, 1, 5, 10, 100)), kernel = "linear")
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: linear
## cost: 1
##
## Number of Support Vectors: 56tune.out$best.performance## [1] 0.01025641The best linear SVM kernal is associated with a cost of 1 and a gamma of 0.003205128. It produced an cross validated error rate of 0.07424404. This leveraged a 10-fold cross validation method.
(c) Now repeat (b), this time using SVMs with radial and polynomial basis kernels, with different values of gamma and degree and cost. Comment on your results.
set.seed(1)
tune.out2=tune(svm,mpglevel~.,data=Auto,kernel ="polynomial",ranges=list(cost=c(0.001, 0.01, 0.1, 1,5,10,100), degree=c(2,3,4)))
summary(tune.out2)##
## Parameter tuning of 'svm':
##
## - sampling method: 10-fold cross validation
##
## - best parameters:
## cost degree
## 100 2
##
## - best performance: 0.3013462
##
## - Detailed performance results:
## cost degree error dispersion
## 1 1e-03 2 0.5511538 0.04366593
## 2 1e-02 2 0.5511538 0.04366593
## 3 1e-01 2 0.5511538 0.04366593
## 4 1e+00 2 0.5511538 0.04366593
## 5 5e+00 2 0.5511538 0.04366593
## 6 1e+01 2 0.5130128 0.08963366
## 7 1e+02 2 0.3013462 0.09961961
## 8 1e-03 3 0.5511538 0.04366593
## 9 1e-02 3 0.5511538 0.04366593
## 10 1e-01 3 0.5511538 0.04366593
## 11 1e+00 3 0.5511538 0.04366593
## 12 5e+00 3 0.5511538 0.04366593
## 13 1e+01 3 0.5511538 0.04366593
## 14 1e+02 3 0.3446154 0.09821588
## 15 1e-03 4 0.5511538 0.04366593
## 16 1e-02 4 0.5511538 0.04366593
## 17 1e-01 4 0.5511538 0.04366593
## 18 1e+00 4 0.5511538 0.04366593
## 19 5e+00 4 0.5511538 0.04366593
## 20 1e+01 4 0.5511538 0.04366593
## 21 1e+02 4 0.5511538 0.04366593The best polynomial model contains the following tuning parameters:
##
## Call:
## best.tune(method = svm, train.x = mpglevel ~ ., data = Auto, ranges = list(cost = c(0.001,
## 0.01, 0.1, 1, 5, 10, 100), degree = c(2, 3, 4)), kernel = "polynomial")
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: polynomial
## cost: 100
## degree: 2
## coef.0: 0
##
## Number of Support Vectors: 355## [1] 0.3013462The best polynomial SVM kernal is associated with a cost of 100, gamma of 0.003205128 and a degree of 3. It produced an error rate of 0.20262.
set.seed(1)
tune.out3=tune(svm,mpglevel~.,data=Auto,kernel ="radial",ranges=list(cost=c(0.001, 0.01, 0.1, 1,5,10,100), gamma=c(0.01, 0.1, 1, 5, 10, 100)))The best radial model contains the following tuning parameters:
##
## Call:
## best.tune(method = svm, train.x = mpglevel ~ ., data = Auto, ranges = list(cost = c(0.001,
## 0.01, 0.1, 1, 5, 10, 100), gamma = c(0.01, 0.1, 1, 5, 10, 100)),
## kernel = "radial")
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: radial
## cost: 100
##
## Number of Support Vectors: 57
##
## ( 27 30 )
##
##
## Number of Classes: 2
##
## Levels:
## 0 1## [1] 0.01282051The best radial SVM kernal is associated with a cost of 10 and gamma of 0.01. It produced an error rate of 0.03829745.
(d) Make some plots to back up your assertions in (b) and (c). Hint: In the lab, we used the plot() function for svm objects only in cases with p = 2. When p > 2, you can use the plot() function to create plots displaying pairs of variables at a time. Essentially, instead of typing > plot(svmfit , dat) where svmfit contains your fitted model and dat is a data frame containing your data, you can type > plot(svmfit , dat , x1∼x4) in order to plot just the first and fourth variables. However, you must replace x1 and x4 with the correct variable names. To find out more, type ?plot.svm.
linear.svm <- svm(mpglevel ~ ., data = Auto, kernel = "linear", cost = 1)
poly.svm <- svm(mpglevel ~ ., data = Auto, kernel = "polynomial", cost = 100, degree = 3)
radial.svm <- svm(mpglevel ~ ., data = Auto, kernel = "radial", cost = 5, gamma = 0.1)
plotpairs = function(fit) {
for (name in names(Auto)[!(names(Auto) %in% c("mpg", "mpglevel", "name"))]) {
plot(fit, Auto, as.formula(paste("mpg~", name, sep = "")))
}
}
plotpairs(linear.svm)
plotpairs(poly.svm)
plotpairs(radial.svm)
Problem 8. This problem involves the [OJ]: https://rdrr.io/cran/ISLR/man/OJ.html data set which is part of the ISLR package.
(a) Create a training set containing a random sample of 800 observations, and a test set containing the remaining observations.
set.seed(12345)
library(ISLR)
attach(OJ)
oj.intrain <- createDataPartition(OJ$Purchase, p = 0.746, list = FALSE)
oj.train <- OJ[oj.intrain,]
oj.test <- OJ[-oj.intrain,]
dim(oj.train)## [1] 800 18(b) Fit a support vector classifier to the training data using cost=0.01, with Purchase as the response and the other variables as predictors. Use the summary() function to produce summary statistics, and describe the results obtained.
library(e1071)
set.seed(12345)
svm.oj<-svm(Purchase~., data = oj.train, kernel = "linear", cost = 0.01)
summary(svm.oj)##
## Call:
## svm(formula = Purchase ~ ., data = oj.train, kernel = "linear", cost = 0.01)
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: linear
## cost: 0.01
##
## Number of Support Vectors: 449
##
## ( 225 224 )
##
##
## Number of Classes: 2
##
## Levels:
## CH MMThe SVM linear classifier with a Cost of 0.01 produces a total of 449 support vectors with 225 being classified as CH and 224 classified as MM.
(c) What are the training and test error rates?
#train error rate
train.pred <- predict(svm.oj, oj.train)
table(oj.train$Purchase, train.pred)## train.pred
## CH MM
## CH 432 56
## MM 85 227(85+56)/800## [1] 0.17625Train Error Rate is 17.63%
test.pred <- predict(svm.oj, oj.test)
table(oj.test$Purchase, test.pred)## test.pred
## CH MM
## CH 147 18
## MM 24 81(24+18)/270## [1] 0.1555556Test error rate is 15.56%
(d) Use the tune() function to select an optimal cost. Consider values in the range 0.01 to 10.
set.seed(12345)
tune.out = tune(svm, Purchase ~., data = oj.train, kernel = "linear", ranges = list(cost=c(0.001, 0.01, 0.1, 1, 5, 10)))
summary(tune.out)##
## Parameter tuning of 'svm':
##
## - sampling method: 10-fold cross validation
##
## - best parameters:
## cost
## 5
##
## - best performance: 0.18
##
## - Detailed performance results:
## cost error dispersion
## 1 1e-03 0.34250 0.08563488
## 2 1e-02 0.18125 0.04535738
## 3 1e-01 0.18500 0.04322101
## 4 1e+00 0.18500 0.04556741
## 5 5e+00 0.18000 0.04417453
## 6 1e+01 0.18000 0.04533824tune.out$best.model##
## Call:
## best.tune(method = svm, train.x = Purchase ~ ., data = oj.train,
## ranges = list(cost = c(0.001, 0.01, 0.1, 1, 5, 10)), kernel = "linear")
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: linear
## cost: 5
##
## Number of Support Vectors: 344tune.out$best.performance## [1] 0.18The best linear model has a cost of 5 and an error rate of 18%.
(e) Compute the training and test error rates using this new value for cost.
set.seed(12345)
svm.oj.best<-svm(Purchase~., data = oj.train, cost=5, kernal="linear")
svm.train.pred<-predict(svm.oj.best, oj.train)
table(oj.train$Purchase, svm.train.pred)## svm.train.pred
## CH MM
## CH 448 40
## MM 82 230(82+40)/800## [1] 0.1525Train Error is 15.25%
svm.test.pred<-predict(svm.oj.best, oj.test)
table(oj.test$Purchase, svm.test.pred)## svm.test.pred
## CH MM
## CH 147 18
## MM 30 75(30+18)/270## [1] 0.1777778Test error is 17.78%
(f) Repeat parts (b) through (e) using a support vector machine with a radial kernel. Use the default value for gamma.
set.seed(12345)
svm.rad = svm(Purchase~., data = oj.train, kernel = "radial", cost = 0.01)
summary(svm.rad)##
## Call:
## svm(formula = Purchase ~ ., data = oj.train, kernel = "radial", cost = 0.01)
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: radial
## cost: 0.01
##
## Number of Support Vectors: 627
##
## ( 315 312 )
##
##
## Number of Classes: 2
##
## Levels:
## CH MMThe SVM radial classifier with a Cost of 0.01 produces a total of 627 support vectors with 315 being classified as CH and 312 classified as MM.
rad.train.pred<-predict(svm.rad, oj.train)
table(oj.train$Purchase, rad.train.pred)## rad.train.pred
## CH MM
## CH 488 0
## MM 312 0(312+0)/800## [1] 0.39Train error rate is 39%
rad.test.pred<-predict(svm.rad, oj.test)
table(oj.test$Purchase, rad.test.pred)## rad.test.pred
## CH MM
## CH 165 0
## MM 105 0(105+0)/270## [1] 0.3888889Test error rate is 39%
set.seed(12345)
rad.best.tune<-tune(svm, Purchase~., data = oj.train, kernal = "radial", ranges = list(cost=c(0.001, 0.01, 0.1, 1, 5, 10)))
summary(rad.best.tune)##
## Parameter tuning of 'svm':
##
## - sampling method: 10-fold cross validation
##
## - best parameters:
## cost
## 1
##
## - best performance: 0.175
##
## - Detailed performance results:
## cost error dispersion
## 1 1e-03 0.39000 0.08032054
## 2 1e-02 0.39000 0.08032054
## 3 1e-01 0.19250 0.03129164
## 4 1e+00 0.17500 0.03679900
## 5 5e+00 0.18375 0.03910900
## 6 1e+01 0.19125 0.03064696rad.best.tune$best.model##
## Call:
## best.tune(method = svm, train.x = Purchase ~ ., data = oj.train,
## ranges = list(cost = c(0.001, 0.01, 0.1, 1, 5, 10)), kernal = "radial")
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: radial
## cost: 1
##
## Number of Support Vectors: 387The best tuned radial has a cost of 1 and produces an error of 17.5%.
set.seed(12345)
svm.rad.best = svm(Purchase~., data = oj.train, kernel = "radial", cost = 1)
summary(svm.rad.best)##
## Call:
## svm(formula = Purchase ~ ., data = oj.train, kernel = "radial", cost = 1)
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: radial
## cost: 1
##
## Number of Support Vectors: 387
##
## ( 197 190 )
##
##
## Number of Classes: 2
##
## Levels:
## CH MMThe best tuned model radial results in 387 support vectors with 197 being classified as CH and 190 as MM
rad.train.predbest<-predict(svm.rad.best, oj.train)
table(oj.train$Purchase, rad.train.predbest)## rad.train.predbest
## CH MM
## CH 446 42
## MM 78 234(78+42)/800## [1] 0.15Best tuned radial train error is 15%
rad.test.predbest<-predict(svm.rad.best, oj.test)
table(oj.test$Purchase, rad.test.predbest)## rad.test.predbest
## CH MM
## CH 148 17
## MM 30 75(30+17)/270## [1] 0.1740741Best tuned radial test error is 17.4%
(g) Repeat parts (b) through (e) using a support vector machine with a polynomial kernel. Set degree=2.
set.seed(12345)
svm.poly = svm(Purchase~., data = oj.train, kernel = "poly", degree=2)
summary(svm.poly)##
## Call:
## svm(formula = Purchase ~ ., data = oj.train, kernel = "poly", degree = 2)
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: polynomial
## cost: 1
## degree: 2
## coef.0: 0
##
## Number of Support Vectors: 463
##
## ( 234 229 )
##
##
## Number of Classes: 2
##
## Levels:
## CH MMThe SVM polynomial classifier with a degree of 2 produces a total of 463 support vectors with 234 being classified as CH and 229 classified as MM.
poly.train.pred<-predict(svm.poly, oj.train)
table(oj.train$Purchase, poly.train.pred)## poly.train.pred
## CH MM
## CH 455 33
## MM 114 198(114+33)/800## [1] 0.18375Train error rate is 18.38%
poly.test.pred<-predict(svm.poly, oj.test)
table(oj.test$Purchase, poly.test.pred)## poly.test.pred
## CH MM
## CH 155 10
## MM 38 67(38+10)/270## [1] 0.1777778Test error rate is 17.78%
set.seed(12345)
poly.best.tune<-tune(svm, Purchase~., data = oj.train, kernel = "poly", ranges = list(cost=c(0.001, 0.01, 0.1, 1, 5, 10), degree=c(2,3,4)))
summary(poly.best.tune)##
## Parameter tuning of 'svm':
##
## - sampling method: 10-fold cross validation
##
## - best parameters:
## cost degree
## 5 2
##
## - best performance: 0.18125
##
## - Detailed performance results:
## cost degree error dispersion
## 1 1e-03 2 0.39000 0.08032054
## 2 1e-02 2 0.39125 0.07796233
## 3 1e-01 2 0.32000 0.05533986
## 4 1e+00 2 0.20500 0.02713137
## 5 5e+00 2 0.18125 0.02960973
## 6 1e+01 2 0.18125 0.03691676
## 7 1e-03 3 0.39000 0.08032054
## 8 1e-02 3 0.37375 0.06958458
## 9 1e-01 3 0.29750 0.05737305
## 10 1e+00 3 0.18750 0.05496211
## 11 5e+00 3 0.19000 0.05583955
## 12 1e+01 3 0.20250 0.03622844
## 13 1e-03 4 0.39000 0.08032054
## 14 1e-02 4 0.37375 0.06958458
## 15 1e-01 4 0.32625 0.06022239
## 16 1e+00 4 0.23500 0.04594683
## 17 5e+00 4 0.20750 0.04456581
## 18 1e+01 4 0.21250 0.04409586poly.best.tune$best.model##
## Call:
## best.tune(method = svm, train.x = Purchase ~ ., data = oj.train,
## ranges = list(cost = c(0.001, 0.01, 0.1, 1, 5, 10), degree = c(2,
## 3, 4)), kernel = "poly")
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: polynomial
## cost: 5
## degree: 2
## coef.0: 0
##
## Number of Support Vectors: 382The best tuned polynomial has a cost of 5, a degree of 2 and produces an error of 18.1%.
set.seed(12345)
svm.poly.best = svm(Purchase~., data = oj.train, kernel = "poly", cost = 5, degree= 2)
summary(svm.poly.best)##
## Call:
## svm(formula = Purchase ~ ., data = oj.train, kernel = "poly", cost = 5,
## degree = 2)
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: polynomial
## cost: 5
## degree: 2
## coef.0: 0
##
## Number of Support Vectors: 382
##
## ( 194 188 )
##
##
## Number of Classes: 2
##
## Levels:
## CH MMThe best tuned poly model radial results in 382 support vectors with 194 being classified as CH and 188 as MM
poly.train.predbest<-predict(svm.poly.best, oj.train)
table(oj.train$Purchase, poly.train.predbest)## poly.train.predbest
## CH MM
## CH 450 38
## MM 92 220(92+38)/800## [1] 0.1625Best tuned poly train error is 16.25%
poly.test.predbest<-predict(svm.poly.best, oj.test)
table(oj.test$Purchase, poly.test.predbest)## poly.test.predbest
## CH MM
## CH 152 13
## MM 28 77(27+21)/270## [1] 0.1777778Best tuned poly test error is 17.78%
(h) Overall, which approach seems to give the best results on this data?
The best classifier is the Radial SVM. It produces a test error rate of 17.74% and has a tuning parameter of Cost=1.