#Note: If you ever forget the tuning parameters for a particular model, the
#modelLookup ( ) function Can be used to find them. Simply supply the method’s name.
#Part 1:
#1. What is data mining? Explain. Process of discovering useful information in large data repositories. DM techniques scour large dataset to find novel and useful pattern that might be unknown. Also, used to predict future observations.
#2. What are support vectors? Explain. Observations that lie on the edge of an area in space which presents a boundary between two observed classes.
#3. State what support vectors are used for. It identifies the margin between classes.
#4. What do we mean when we say that the data is linearly separable? Separated data with a straight line
#5. List five model builders.
#6. Support vector machines (SVM) are unsupervised learning algorithms. TRUE or FALSE? Falsi
#7. SVMs are sensitive to the choice of ____Tuning__________ ________Options________.
#8. The SVM model only deals with _Support Vectors___.
#9. What is a binary classification model? Model that has more than two input variables
#10. SVMs often perform a non-linear mapping of the original data into a very high dimensional space where the classes can be separated linearly by a hyperplane. What method do we use to find solutions for this problem at a lower dimension? Lagrangian relaxation.
#11. SVMs solve the problem of linearly separable binary classification by doing what? Finding the maximum margin hyperplane
#12. What will the SVMs do when the cases are not linearly separable? Move the data into higher dimension where feasible.
#13. The relation,
#K(x, z) = ∅(𝑥) ∙ ∅(𝑧)
#is associated with what method that is used to find solutions. Why is it important? The Kernel Trick. It transforms two vectors into a much higher dimension, r.
#Part 2: This part of the assignment is based on the six algorithms sent to you via email
#1, In Listing 12.16: Example of Naïve Bayes algorithm for classification in caret,
#a. Explain the trainControl( ) function - Provides a set of control parameters that dictate how the train() function evaluates and selects the optimal model.
#b. Explain the argument “metric = “Accuracy””? Accuracy is the metric accross resamples
#2. In Listing 12.17: Example of SVM algorithm for classification,
#a. Explain the argument “kernel = “rbfdot””, The RBF kernel calculates the similarity between two data points based on their Euclidean distance.
#b. Explain what is happening in the argument, “PimaIndiansDiabetes [ ,1:8], and
#explain the indexing, It uses the all rows of PIMA data but only used the columns(Variables) 1 to 8.
#c. Explain and draw conclusions about the output? It fit the dataset to the model then uses columns-Variables 1 : 8 to predict the outputs then verified the accuracy of the model's predicted output to the actual outputs
#3. In Listing 12.18: Explain the SVM algorithm for regression. Explain and draw
#conclusions about the output as well.
#load packages
library(kernlab)
## Warning: package 'kernlab' was built under R version 4.5.2
library(mlbench)
## Warning: package 'mlbench' was built under R version 4.5.2
#load data
data("BostonHousing")
#fit model
fit <- ksvm(medv~.,BostonHousing, kernel="rbfdot")
#summarize the fit
print(fit)
## Support Vector Machine object of class "ksvm"
##
## SV type: eps-svr (regression)
## parameter : epsilon = 0.1 cost C = 1
##
## Gaussian Radial Basis kernel function.
## Hyperparameter : sigma = 0.105148416696812
##
## Number of Support Vectors : 335
##
## Objective Function Value : -75.2978
## Training error : 0.091037
#make predictions
predictions <- predict(fit, BostonHousing)
#summize accuracy
mse <- mean((BostonHousing$medv - predictions)^2)
print(mse)
## [1] 7.700518
#4. In Listing 12.19: Example of SVM algorithm for classification in caret,
#a. Explain the arguments ”method =”cv”” and “number=5”
#b. Explain and draw conclusions about the output
#load packages
library(caret)
## Warning: package 'caret' was built under R version 4.5.2
## Loading required package: ggplot2
##
## Attaching package: 'ggplot2'
## The following object is masked from 'package:kernlab':
##
## alpha
## Loading required package: lattice
library(mlbench)
#load dataset
data("PimaIndiansDiabetes")
#train
set.seed(7)
trainControl<- trainControl(method = "cv", number = 5)
fit.svmRadial<-train(diabetes~., data=PimaIndiansDiabetes, method="svmRadial",
metric="Accuracy", trControl=trainControl)
#summize fit
print(fit.svmRadial)
## Support Vector Machines with Radial Basis Function Kernel
##
## 768 samples
## 8 predictor
## 2 classes: 'neg', 'pos'
##
## No pre-processing
## Resampling: Cross-Validated (5 fold)
## Summary of sample sizes: 614, 614, 615, 614, 615
## Resampling results across tuning parameters:
##
## C Accuracy Kappa
## 0.25 0.7604278 0.4360310
## 0.50 0.7656056 0.4552142
## 1.00 0.7590952 0.4409422
##
## Tuning parameter 'sigma' was held constant at a value of 0.124824
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were sigma = 0.124824 and C = 0.5.
#5. In Listing 12.20: Example of SVM algorithm for regression in caret,
#a. Explain the arguments “method = “svmRadial”” and “metric = “RMSE””
#b. Explain and draw conclusions about the output
#load packages
#library(caret)
#library(mlbench)
#load dataset
data("BostonHousing")
#train
set.seed(7)
trainControl<-trainControl(method="cv",number=5)
fit.svmRadial<- train(medv~., data=BostonHousing,method="svmRadial", metric="RMSE",
trControl=trainControl)
#summarize fit
print(fit.svmRadial)
## Support Vector Machines with Radial Basis Function Kernel
##
## 506 samples
## 13 predictor
##
## No pre-processing
## Resampling: Cross-Validated (5 fold)
## Summary of sample sizes: 407, 404, 405, 404, 404
## Resampling results across tuning parameters:
##
## C RMSE Rsquared MAE
## 0.25 4.842935 0.7527188 2.823026
## 0.50 4.303449 0.7969666 2.540125
## 1.00 3.830139 0.8333874 2.327301
##
## Tuning parameter 'sigma' was held constant at a value of 0.1057462
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.1057462 and C = 1.
#6. In Listing 12.4.4: Classification and Regression Trees,
#a. Explain the following argument “diabetes~., “
#b. Explain the argument, “type = “class””
#c. Explain and draw conclusions about the output
#load packages
library(rpart)
#library(mlbench)
#load dataset
data("PimaIndiansDiabetes")
#fit model
fit <- rpart(diabetes~., data=PimaIndiansDiabetes)
#summarize the fit
print(fit)
## n= 768
##
## node), split, n, loss, yval, (yprob)
## * denotes terminal node
##
## 1) root 768 268 neg (0.65104167 0.34895833)
## 2) glucose< 127.5 485 94 neg (0.80618557 0.19381443)
## 4) age< 28.5 271 23 neg (0.91512915 0.08487085) *
## 5) age>=28.5 214 71 neg (0.66822430 0.33177570)
## 10) mass< 26.35 41 2 neg (0.95121951 0.04878049) *
## 11) mass>=26.35 173 69 neg (0.60115607 0.39884393)
## 22) glucose< 99.5 55 10 neg (0.81818182 0.18181818) *
## 23) glucose>=99.5 118 59 neg (0.50000000 0.50000000)
## 46) pedigree< 0.561 84 34 neg (0.59523810 0.40476190)
## 92) pedigree< 0.2 21 4 neg (0.80952381 0.19047619) *
## 93) pedigree>=0.2 63 30 neg (0.52380952 0.47619048)
## 186) pregnant>=1.5 52 21 neg (0.59615385 0.40384615)
## 372) pressure>=67 40 12 neg (0.70000000 0.30000000) *
## 373) pressure< 67 12 3 pos (0.25000000 0.75000000) *
## 187) pregnant< 1.5 11 2 pos (0.18181818 0.81818182) *
## 47) pedigree>=0.561 34 9 pos (0.26470588 0.73529412) *
## 3) glucose>=127.5 283 109 pos (0.38515901 0.61484099)
## 6) mass< 29.95 76 24 neg (0.68421053 0.31578947)
## 12) glucose< 145.5 41 6 neg (0.85365854 0.14634146) *
## 13) glucose>=145.5 35 17 pos (0.48571429 0.51428571)
## 26) insulin< 14.5 21 8 neg (0.61904762 0.38095238) *
## 27) insulin>=14.5 14 4 pos (0.28571429 0.71428571) *
## 7) mass>=29.95 207 57 pos (0.27536232 0.72463768)
## 14) glucose< 157.5 115 45 pos (0.39130435 0.60869565)
## 28) age< 30.5 50 23 neg (0.54000000 0.46000000)
## 56) pressure>=61 40 13 neg (0.67500000 0.32500000)
## 112) mass< 41.8 31 7 neg (0.77419355 0.22580645) *
## 113) mass>=41.8 9 3 pos (0.33333333 0.66666667) *
## 57) pressure< 61 10 0 pos (0.00000000 1.00000000) *
## 29) age>=30.5 65 18 pos (0.27692308 0.72307692) *
## 15) glucose>=157.5 92 12 pos (0.13043478 0.86956522) *
#make predictions
predictions<- predict(fit, PimaIndiansDiabetes[ ,1:8],type="class")
#summarize accuracy
table(predictions, PimaIndiansDiabetes$diabetes)
##
## predictions neg pos
## neg 449 72
## pos 51 196