DATA624
Jackie Yee
2026-04-26
Question 7.2
7.2 Friedman (1991) introduced several benchmark data sets create by sim-ulation. One of these simulations used the following nonlinear equation to create data: y =10 sin(πx1x2) + 20(x3 − 0.5)2 +10x4 +5x5 +N(0,σ2) where the x values are random variables uniformly distributed between [0, 1] (there are also 5 other non-informative variables also created in the simula-tion). The package mlbench contains a function called mlbench.friedman1 that simulates these data:
library(caret)## Warning: package 'caret' was built under R version 4.5.1## Loading required package: lattice##
## Attaching package: 'caret'## The following objects are masked from 'package:fabletools':
##
## MAE, RMSElibrary(mlbench)## Warning: package 'mlbench' was built under R version 4.5.2set.seed(200)
trainingData <- mlbench.friedman1(200, sd = 1)
## We convert the 'x' data from a matrix to a data frame
## One reason is that this will give the columns names.
trainingData$x <- data.frame(trainingData$x)
## Look at the data using
featurePlot(trainingData$x, trainingData$y)
## or other methods.
## This creates a list with a vector 'y' and a matrix
## of predictors 'x'. Also simulate a large test set to
## estimate the true error rate with good precision:
testData <- mlbench.friedman1(5000, sd = 1)
testData$x <- data.frame(testData$x)Tune several models on these data. For example:
knnModel <- train(x = trainingData$x, y = trainingData$y, method = "knn",
preProcess = c("center", "scale"), tuneLength = 10)
knnModel## k-Nearest Neighbors
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Bootstrapped (25 reps)
## Summary of sample sizes: 200, 200, 200, 200, 200, 200, ...
## Resampling results across tuning parameters:
##
## k RMSE Rsquared MAE
## 5 3.466085 0.5121775 2.816838
## 7 3.349428 0.5452823 2.727410
## 9 3.264276 0.5785990 2.660026
## 11 3.214216 0.6024244 2.603767
## 13 3.196510 0.6176570 2.591935
## 15 3.184173 0.6305506 2.577482
## 17 3.183130 0.6425367 2.567787
## 19 3.198752 0.6483184 2.592683
## 21 3.188993 0.6611428 2.588787
## 23 3.200458 0.6638353 2.604529
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 17.knnPred <- predict(knnModel, newdata = testData$x)
## The function 'postResample' can be used to get the test set
## perforamnce values
knnPR <- postResample(pred = knnPred, obs = testData$y)
knnPR ## RMSE Rsquared MAE
## 3.2040595 0.6819919 2.5683461NM Model
We will begin by building other models such as neural network, SVM and MARS.
findCorrelation(cor(trainingData$x), cutoff = .75)## integer(0)Building out the Averaged Neural Network
nnetGrid <- expand.grid(
.decay = c(0, 0.01, 0.1),
.size = c(1:10),
.bag = c(FALSE, TRUE)
)
ctrl <- trainControl(method = "cv")
nnmod <- train(trainingData$x, trainingData$y,
method = "avNNet",
tuneGrid = nnetGrid,
trControl = ctrl,
preProcess = c("center", "scale"),
linout = TRUE,
trace = FALSE,
maxit = 500,
MaxNWts = 10 * (ncol(trainingData$x) + 1) + 10 + 1)## Warning: executing %dopar% sequentially: no parallel backend registerednnmod## Model Averaged Neural Network
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
## Resampling results across tuning parameters:
##
## decay size bag RMSE Rsquared MAE
## 0.00 1 FALSE 2.478794 0.7638408 1.966188
## 0.00 1 TRUE 2.596747 0.7444602 2.094435
## 0.00 2 FALSE 2.491706 0.7593193 2.002956
## 0.00 2 TRUE 2.522951 0.7586125 2.024724
## 0.00 3 FALSE 2.168391 0.8188071 1.719245
## 0.00 3 TRUE 2.437305 0.7717807 1.947359
## 0.00 4 FALSE 1.902601 0.8611192 1.481721
## 0.00 4 TRUE 2.557905 0.7551978 1.928723
## 0.00 5 FALSE 2.352714 0.7931905 1.751301
## 0.00 5 TRUE 3.411130 0.6605821 2.450705
## 0.00 6 FALSE 2.578437 0.7506957 2.026519
## 0.00 6 TRUE 3.235597 0.6712055 2.468968
## 0.00 7 FALSE 5.778286 0.4736580 3.112061
## 0.00 7 TRUE 5.475215 0.4182845 3.875270
## 0.00 8 FALSE 3.806285 0.6047105 2.623582
## 0.00 8 TRUE 4.273412 0.5798483 3.081382
## 0.00 9 FALSE 5.095874 0.4899494 3.204571
## 0.00 9 TRUE 5.114142 0.5019823 3.339065
## 0.00 10 FALSE 3.849069 0.5504444 2.633995
## 0.00 10 TRUE 3.253265 0.6678400 2.547286
## 0.01 1 FALSE 2.437343 0.7689713 1.935159
## 0.01 1 TRUE 2.486361 0.7613587 1.968174
## 0.01 2 FALSE 2.516177 0.7533749 1.997962
## 0.01 2 TRUE 2.558313 0.7520374 2.003602
## 0.01 3 FALSE 2.047737 0.8352246 1.621316
## 0.01 3 TRUE 2.451372 0.7646222 1.970692
## 0.01 4 FALSE 2.048678 0.8358409 1.601835
## 0.01 4 TRUE 2.402370 0.7836622 1.885380
## 0.01 5 FALSE 2.140483 0.8303723 1.687525
## 0.01 5 TRUE 2.612549 0.7516072 2.115569
## 0.01 6 FALSE 2.180342 0.8212561 1.757742
## 0.01 6 TRUE 2.718382 0.7291119 2.181814
## 0.01 7 FALSE 2.372565 0.7838706 1.896419
## 0.01 7 TRUE 2.905560 0.7181912 2.321369
## 0.01 8 FALSE 2.515781 0.7727538 1.998660
## 0.01 8 TRUE 2.976130 0.7056945 2.362309
## 0.01 9 FALSE 2.442924 0.7691628 1.992318
## 0.01 9 TRUE 3.102105 0.6727280 2.480653
## 0.01 10 FALSE 2.635540 0.7272163 2.094795
## 0.01 10 TRUE 2.653328 0.7179808 2.160876
## 0.10 1 FALSE 2.450881 0.7652326 1.942933
## 0.10 1 TRUE 2.462778 0.7622961 1.965710
## 0.10 2 FALSE 2.540402 0.7534280 2.042851
## 0.10 2 TRUE 2.449406 0.7692308 1.987570
## 0.10 3 FALSE 2.208993 0.8151676 1.769552
## 0.10 3 TRUE 2.339193 0.7856447 1.855318
## 0.10 4 FALSE 2.085835 0.8303957 1.668884
## 0.10 4 TRUE 2.227976 0.8057848 1.729116
## 0.10 5 FALSE 2.074115 0.8386194 1.689254
## 0.10 5 TRUE 2.355224 0.7920519 1.952832
## 0.10 6 FALSE 2.227288 0.8085555 1.751421
## 0.10 6 TRUE 2.449053 0.7675714 1.966840
## 0.10 7 FALSE 2.295272 0.7981585 1.827043
## 0.10 7 TRUE 2.741629 0.7320245 2.225974
## 0.10 8 FALSE 2.382892 0.7895902 1.876622
## 0.10 8 TRUE 2.489989 0.7739297 1.952249
## 0.10 9 FALSE 2.319274 0.7928340 1.881329
## 0.10 9 TRUE 2.571649 0.7557121 2.020368
## 0.10 10 FALSE 2.380105 0.7799089 1.893941
## 0.10 10 TRUE 2.367558 0.7834815 1.874068
##
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 4, decay = 0 and bag = FALSE.nnPred <- predict(nnmod, newdata = testData$x)
postResample(nnPred, obs = testData$y)## RMSE Rsquared MAE
## 3.2548735 0.6722128 2.0841466SVM Model
Building out the Support Vector Machine Model
set.seed(593)
# tune
svmMod <- train(trainingData$x, trainingData$y,
method = "svmRadial",
preProc = c("center", "scale"),
tuneLength = 16)
svmMod## Support Vector Machines with Radial Basis Function Kernel
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Bootstrapped (25 reps)
## Summary of sample sizes: 200, 200, 200, 200, 200, 200, ...
## Resampling results across tuning parameters:
##
## C RMSE Rsquared MAE
## 0.25 2.549253 0.7776535 2.016965
## 0.50 2.337303 0.7914547 1.831224
## 1.00 2.209272 0.8084283 1.726631
## 2.00 2.121453 0.8206207 1.653912
## 4.00 2.081593 0.8257583 1.622902
## 8.00 2.064126 0.8280981 1.613390
## 16.00 2.063698 0.8280749 1.613449
## 32.00 2.063698 0.8280749 1.613449
## 64.00 2.063698 0.8280749 1.613449
## 128.00 2.063698 0.8280749 1.613449
## 256.00 2.063698 0.8280749 1.613449
## 512.00 2.063698 0.8280749 1.613449
## 1024.00 2.063698 0.8280749 1.613449
## 2048.00 2.063698 0.8280749 1.613449
## 4096.00 2.063698 0.8280749 1.613449
## 8192.00 2.063698 0.8280749 1.613449
##
## Tuning parameter 'sigma' was held constant at a value of 0.06533273
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.06533273 and C = 16.svmRPred <- predict(svmMod, testData$x)
postResample(svmRPred, testData$y)## RMSE Rsquared MAE
## 2.0794165 0.8247607 1.5797228MARS Model
Building out the Multivariate Adaptive Regression Splines Model
marsGrid <- expand.grid(.degree = 1:2, .nprune = 2:38)set.seed(7921)
marsMod <- train(trainingData$x, trainingData$y,
method = "earth",
tuneGrid = marsGrid,
trControl = ctrl)
marsMod## Multivariate Adaptive Regression Spline
##
## 200 samples
## 10 predictor
##
## No pre-processing
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
## Resampling results across tuning parameters:
##
## degree nprune RMSE Rsquared MAE
## 1 2 4.315676 0.2671928 3.5746995
## 1 3 3.623770 0.4766764 2.8577341
## 1 4 2.555717 0.7451824 2.0580220
## 1 5 2.257718 0.7950282 1.8110849
## 1 6 2.198788 0.8096925 1.6952491
## 1 7 1.831017 0.8724203 1.4489371
## 1 8 1.720465 0.8871890 1.3516418
## 1 9 1.691144 0.8903847 1.3323384
## 1 10 1.652973 0.8972317 1.3296797
## 1 11 1.657552 0.8985504 1.3284029
## 1 12 1.638089 0.8991183 1.3000436
## 1 13 1.639284 0.8993956 1.2950567
## 1 14 1.646789 0.8979711 1.2982198
## 1 15 1.646789 0.8979711 1.2982198
## 1 16 1.646789 0.8979711 1.2982198
## 1 17 1.646789 0.8979711 1.2982198
## 1 18 1.646789 0.8979711 1.2982198
## 1 19 1.646789 0.8979711 1.2982198
## 1 20 1.646789 0.8979711 1.2982198
## 1 21 1.646789 0.8979711 1.2982198
## 1 22 1.646789 0.8979711 1.2982198
## 1 23 1.646789 0.8979711 1.2982198
## 1 24 1.646789 0.8979711 1.2982198
## 1 25 1.646789 0.8979711 1.2982198
## 1 26 1.646789 0.8979711 1.2982198
## 1 27 1.646789 0.8979711 1.2982198
## 1 28 1.646789 0.8979711 1.2982198
## 1 29 1.646789 0.8979711 1.2982198
## 1 30 1.646789 0.8979711 1.2982198
## 1 31 1.646789 0.8979711 1.2982198
## 1 32 1.646789 0.8979711 1.2982198
## 1 33 1.646789 0.8979711 1.2982198
## 1 34 1.646789 0.8979711 1.2982198
## 1 35 1.646789 0.8979711 1.2982198
## 1 36 1.646789 0.8979711 1.2982198
## 1 37 1.646789 0.8979711 1.2982198
## 1 38 1.646789 0.8979711 1.2982198
## 2 2 4.315676 0.2671928 3.5746995
## 2 3 3.623770 0.4766764 2.8577341
## 2 4 2.545673 0.7454550 2.0388668
## 2 5 2.242259 0.7973687 1.7903417
## 2 6 2.210834 0.8053749 1.7073359
## 2 7 1.753167 0.8827754 1.3817020
## 2 8 1.730365 0.8829303 1.3178946
## 2 9 1.442424 0.9182270 1.1276864
## 2 10 1.375973 0.9271518 1.0641284
## 2 11 1.355236 0.9294170 1.0490884
## 2 12 1.291320 0.9360552 1.0074066
## 2 13 1.329782 0.9320000 1.0201254
## 2 14 1.265062 0.9392004 0.9834551
## 2 15 1.233152 0.9419609 0.9608637
## 2 16 1.275747 0.9382412 0.9886187
## 2 17 1.269969 0.9387661 0.9903473
## 2 18 1.269969 0.9387661 0.9903473
## 2 19 1.269969 0.9387661 0.9903473
## 2 20 1.269969 0.9387661 0.9903473
## 2 21 1.269969 0.9387661 0.9903473
## 2 22 1.269969 0.9387661 0.9903473
## 2 23 1.269969 0.9387661 0.9903473
## 2 24 1.269969 0.9387661 0.9903473
## 2 25 1.269969 0.9387661 0.9903473
## 2 26 1.269969 0.9387661 0.9903473
## 2 27 1.269969 0.9387661 0.9903473
## 2 28 1.269969 0.9387661 0.9903473
## 2 29 1.269969 0.9387661 0.9903473
## 2 30 1.269969 0.9387661 0.9903473
## 2 31 1.269969 0.9387661 0.9903473
## 2 32 1.269969 0.9387661 0.9903473
## 2 33 1.269969 0.9387661 0.9903473
## 2 34 1.269969 0.9387661 0.9903473
## 2 35 1.269969 0.9387661 0.9903473
## 2 36 1.269969 0.9387661 0.9903473
## 2 37 1.269969 0.9387661 0.9903473
## 2 38 1.269969 0.9387661 0.9903473
##
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nprune = 15 and degree = 2.marsPred <- predict(marsMod, testData$x)
postResample(marsPred, testData$y)## RMSE Rsquared MAE
## 1.1589948 0.9460418 0.9250230Which models appear to give the best performance?**
From the models, we can see that MARS appears to give the best performance through the RSME, MAE and R^2 value.
Does MARS select the informative predictors (those named X1–X5)?**
varImp(marsMod)## earth variable importance
##
## Overall
## X1 100.00
## X4 75.24
## X2 48.73
## X5 15.52
## X3 0.00The MARS model did select the appropriate 5 predictors, although it should be noted that X3 has an importance of 0.
Question 7.5
7.5. Exercise 6.3 describes data for a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several nonlinear regression models.
library(AppliedPredictiveModeling)## Warning: package 'AppliedPredictiveModeling' was built under R version 4.5.3data(ChemicalManufacturingProcess)
ChemicalManufacturingProcesssum(is.na(ChemicalManufacturingProcess))## [1] 106miss <- preProcess(ChemicalManufacturingProcess, method = "bagImpute")
Chemical <- predict(miss, ChemicalManufacturingProcess)
Chemical <- Chemical[, -nearZeroVar(Chemical)]
sum(is.na(Chemical))## [1] 0index <- createDataPartition(Chemical$Yield, p = .8, list = FALSE)
trainX <- Chemical[index, -1]
trainY <- Chemical[index, 1]
testX <- Chemical[-index, -1]
testY <- Chemical[-index, 1](a) Which nonlinear regression model gives the optimal resampling and test set performance?
Building KNN Model
knnModel <- train(trainX, trainY,
method = "knn",
preProc = c("center", "scale"),
tuneLength = 10)
knnModel## k-Nearest Neighbors
##
## 144 samples
## 56 predictor
##
## Pre-processing: centered (56), scaled (56)
## Resampling: Bootstrapped (25 reps)
## Summary of sample sizes: 144, 144, 144, 144, 144, 144, ...
## Resampling results across tuning parameters:
##
## k RMSE Rsquared MAE
## 5 1.379703 0.4020736 1.102902
## 7 1.366067 0.4103742 1.100953
## 9 1.354120 0.4207913 1.099029
## 11 1.352314 0.4258107 1.105125
## 13 1.355193 0.4250063 1.104362
## 15 1.368565 0.4158785 1.113761
## 17 1.381911 0.4083701 1.124273
## 19 1.386402 0.4098804 1.130656
## 21 1.395035 0.4078110 1.139015
## 23 1.404177 0.4007548 1.148577
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 11.knnPred <- predict(knnModel, testX)
postResample(pred = knnPred, testY)## RMSE Rsquared MAE
## 1.8175127 0.3844161 1.4332955Building the Neural Network Model
findCorrelation(cor(trainX), cutoff = .75)## [1] 2 7 6 1 4 11 43 26 40 20 25 53 37 56 38 42 29 51nnetGrid <- expand.grid(.decay = c(0, 0.01, .1),
.size = c(1:10),
.bag = FALSE)
ctrl <- trainControl(method = "cv")
nnMod <- train(trainX, trainY,
method = "avNNet",
tuneGrid = nnetGrid,
trControl = ctrl,
preProcess = c("center", "scale"),
linout = TRUE,
trace = FALSE,
maxit = 500,
MaxNWts = 10 * (ncol(trainX) + 1) + 10 + 1)
nnMod## Model Averaged Neural Network
##
## 144 samples
## 56 predictor
##
## Pre-processing: centered (56), scaled (56)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 129, 129, 130, 130, 131, 130, ...
## Resampling results across tuning parameters:
##
## decay size RMSE Rsquared MAE
## 0.00 1 1.421088 0.4353822 1.1736401
## 0.00 2 1.406119 0.4224303 1.1427541
## 0.00 3 1.750686 0.3167411 1.4178585
## 0.00 4 1.679128 0.3703060 1.3400561
## 0.00 5 1.680078 0.3038884 1.3135751
## 0.00 6 1.917258 0.3311338 1.5557395
## 0.00 7 2.775803 0.2518893 2.0752212
## 0.00 8 3.670803 0.2566044 2.7217279
## 0.00 9 5.785061 0.1544394 3.9008151
## 0.00 10 7.371345 0.1387379 5.0634375
## 0.01 1 1.276252 0.5352462 1.0088114
## 0.01 2 1.247151 0.5543998 0.9848569
## 0.01 3 1.614255 0.4219773 1.3107561
## 0.01 4 1.905854 0.3173106 1.4282043
## 0.01 5 1.610231 0.4249579 1.2681481
## 0.01 6 1.667155 0.3686994 1.3431863
## 0.01 7 1.429050 0.4655228 1.1334984
## 0.01 8 1.419816 0.5111092 1.1365799
## 0.01 9 1.843094 0.5140191 1.4006433
## 0.01 10 2.204821 0.3252498 1.6833401
## 0.10 1 1.270346 0.5474731 0.9916518
## 0.10 2 1.566743 0.4195979 1.2397987
## 0.10 3 1.941322 0.3623579 1.4345481
## 0.10 4 1.769434 0.4172465 1.2788047
## 0.10 5 1.755986 0.4226148 1.3699622
## 0.10 6 2.161755 0.3584110 1.4842873
## 0.10 7 1.899403 0.3377718 1.3979362
## 0.10 8 1.942708 0.3145920 1.4149571
## 0.10 9 1.502637 0.4008087 1.1749525
## 0.10 10 1.756645 0.3167959 1.3845105
##
## Tuning parameter 'bag' was held constant at a value of FALSE
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 2, decay = 0.01 and bag = FALSE.Building the MARS Model
marsGrid <- expand.grid(.degree = 1:2, .nprune = 2:38)
set.seed(54231)marsMod <- train(trainX, trainY,
method = "earth",
tuneGrid = marsGrid,
trControl = trainControl(method = "cv"))
marsMod## Multivariate Adaptive Regression Spline
##
## 144 samples
## 56 predictor
##
## No pre-processing
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 129, 131, 129, 130, 131, 130, ...
## Resampling results across tuning parameters:
##
## degree nprune RMSE Rsquared MAE
## 1 2 1.340799 0.4649719 1.1004585
## 1 3 1.209146 0.5651755 0.9787654
## 1 4 1.173317 0.5832292 0.9377987
## 1 5 1.165887 0.5810809 0.9334496
## 1 6 1.180940 0.5651958 0.9547805
## 1 7 1.202844 0.5511704 0.9682488
## 1 8 1.235556 0.5365197 0.9981821
## 1 9 1.221780 0.5479321 0.9868238
## 1 10 1.241276 0.5372488 1.0002643
## 1 11 1.249812 0.5328813 0.9988527
## 1 12 1.255850 0.5178299 0.9845619
## 1 13 1.245558 0.5240588 0.9803102
## 1 14 1.245048 0.5216954 0.9737635
## 1 15 1.270364 0.5070310 0.9986802
## 1 16 1.270364 0.5070310 0.9986802
## 1 17 1.270364 0.5070310 0.9986802
## 1 18 1.270364 0.5070310 0.9986802
## 1 19 1.270364 0.5070310 0.9986802
## 1 20 1.270364 0.5070310 0.9986802
## 1 21 1.270364 0.5070310 0.9986802
## 1 22 1.270364 0.5070310 0.9986802
## 1 23 1.270364 0.5070310 0.9986802
## 1 24 1.270364 0.5070310 0.9986802
## 1 25 1.270364 0.5070310 0.9986802
## 1 26 1.270364 0.5070310 0.9986802
## 1 27 1.270364 0.5070310 0.9986802
## 1 28 1.270364 0.5070310 0.9986802
## 1 29 1.270364 0.5070310 0.9986802
## 1 30 1.270364 0.5070310 0.9986802
## 1 31 1.270364 0.5070310 0.9986802
## 1 32 1.270364 0.5070310 0.9986802
## 1 33 1.270364 0.5070310 0.9986802
## 1 34 1.270364 0.5070310 0.9986802
## 1 35 1.270364 0.5070310 0.9986802
## 1 36 1.270364 0.5070310 0.9986802
## 1 37 1.270364 0.5070310 0.9986802
## 1 38 1.270364 0.5070310 0.9986802
## 2 2 1.340799 0.4649719 1.1004585
## 2 3 1.244779 0.5382957 1.0145906
## 2 4 1.300142 0.4939130 1.0223280
## 2 5 1.384043 0.4429660 1.1070918
## 2 6 1.378476 0.4604166 1.0768455
## 2 7 1.360565 0.4718864 1.0467721
## 2 8 1.351435 0.4806186 1.0442373
## 2 9 1.281255 0.5315188 1.0027102
## 2 10 1.311374 0.5268555 1.0173540
## 2 11 1.286310 0.5317447 0.9976560
## 2 12 1.285330 0.5234754 0.9998362
## 2 13 1.249041 0.5401741 0.9644277
## 2 14 1.276860 0.5412646 0.9915412
## 2 15 1.251177 0.5619222 0.9619268
## 2 16 1.230806 0.5744942 0.9459742
## 2 17 1.229839 0.5738698 0.9419521
## 2 18 1.251608 0.5655265 0.9694469
## 2 19 1.255777 0.5637853 0.9787747
## 2 20 1.255396 0.5627015 0.9810978
## 2 21 1.256398 0.5635398 0.9813296
## 2 22 1.257578 0.5637732 0.9846323
## 2 23 1.279136 0.5569298 1.0057649
## 2 24 1.289256 0.5568107 1.0167075
## 2 25 1.290622 0.5561302 1.0137420
## 2 26 1.290622 0.5561302 1.0137420
## 2 27 1.297179 0.5546504 1.0192554
## 2 28 1.301385 0.5517945 1.0212303
## 2 29 1.302481 0.5514189 1.0196536
## 2 30 1.292147 0.5550387 1.0085670
## 2 31 1.292147 0.5550387 1.0085670
## 2 32 1.292147 0.5550387 1.0085670
## 2 33 1.292147 0.5550387 1.0085670
## 2 34 1.292147 0.5550387 1.0085670
## 2 35 1.292147 0.5550387 1.0085670
## 2 36 1.292147 0.5550387 1.0085670
## 2 37 1.292147 0.5550387 1.0085670
## 2 38 1.292147 0.5550387 1.0085670
##
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nprune = 5 and degree = 1.marsPred <- predict(marsMod, testX)
postResample(marsPred, testY)## RMSE Rsquared MAE
## 1.1084315 0.7929977 0.8890066Building the Support Vector Machine Model
set.seed(7631)
svmMod <- train(trainX, trainY,
method = "svmRadial",
preProc = c("center", "scale"),
tuneLength = 16,
trControl = trainControl(method = "cv"))
svmMod## Support Vector Machines with Radial Basis Function Kernel
##
## 144 samples
## 56 predictor
##
## Pre-processing: centered (56), scaled (56)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 130, 130, 130, 130, 128, 130, ...
## Resampling results across tuning parameters:
##
## C RMSE Rsquared MAE
## 0.25 1.325010 0.4807991 1.0876172
## 0.50 1.263589 0.4965131 1.0281657
## 1.00 1.209587 0.5341332 0.9783153
## 2.00 1.174721 0.5607496 0.9503839
## 4.00 1.150579 0.5744105 0.9281115
## 8.00 1.104243 0.6167904 0.8890677
## 16.00 1.088928 0.6360630 0.8776651
## 32.00 1.088928 0.6360630 0.8776651
## 64.00 1.088928 0.6360630 0.8776651
## 128.00 1.088928 0.6360630 0.8776651
## 256.00 1.088928 0.6360630 0.8776651
## 512.00 1.088928 0.6360630 0.8776651
## 1024.00 1.088928 0.6360630 0.8776651
## 2048.00 1.088928 0.6360630 0.8776651
## 4096.00 1.088928 0.6360630 0.8776651
## 8192.00 1.088928 0.6360630 0.8776651
##
## Tuning parameter 'sigma' was held constant at a value of 0.01256242
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.01256242 and C = 16.svmPred <- predict(svmMod, testX)
postResample(svmPred, testY)## RMSE Rsquared MAE
## 1.3903990 0.6769041 1.0302360rbind(knn = postResample(knnPred, testY),
nn = postResample(nnPred, testY),
mars = postResample(marsPred, testY),
svmR = postResample(svmRPred, testY))## RMSE Rsquared MAE
## knn 1.817513 0.384416111 1.4332955
## nn NA 0.004567578 NA
## mars 1.108432 0.792997690 0.8890066
## svmR NA 0.007140254 NAFrom the performance test results, we can see that the mars model has the best performance when comparing the 3 metrics.
(b) Which predictors are most important in the optimal nonlinear regression model? Do either the biological or process variables dominate the list? How do the top ten important predictors compare to the top ten predictors from the optimal linear model?
varImp(marsMod)## earth variable importance
##
## Overall
## ManufacturingProcess32 100.00
## ManufacturingProcess09 50.37
## ManufacturingProcess13 22.66
## ManufacturingProcess39 0.00Interestingly, although the MARS model is shown to be the optimal nonlinear regression model, it seems like there aren’t enough predictors in the list. So I will be answering the question with the second most optimal model: SVM.
varImp(svmMod)## loess r-squared variable importance
##
## only 20 most important variables shown (out of 56)
##
## Overall
## ManufacturingProcess32 100.00
## BiologicalMaterial06 89.77
## ManufacturingProcess36 70.54
## BiologicalMaterial02 67.07
## BiologicalMaterial03 65.76
## BiologicalMaterial12 63.75
## ManufacturingProcess13 63.64
## ManufacturingProcess29 59.14
## ManufacturingProcess31 58.91
## BiologicalMaterial04 56.00
## ManufacturingProcess09 51.88
## BiologicalMaterial11 48.19
## ManufacturingProcess33 44.22
## ManufacturingProcess06 44.20
## BiologicalMaterial01 42.67
## BiologicalMaterial08 42.32
## ManufacturingProcess17 42.30
## ManufacturingProcess04 38.29
## ManufacturingProcess02 32.06
## ManufacturingProcess27 27.95The process predictors that still play an important role are the manufacturing process, as they remain dominating the list of important predictors. Compared to my top 10 predictors from my optimal linear model, ManufacturingProcess09 and ManufacturingProcess36 are still important predictors. But the non-linear model contains biological processes while my optimal linear model did not.
Top 10 Predictors: - ManufacturingProcess32 -
ManufacturingProcess13
- BiologicalMaterial06 -
ManufacturingProcess17 - BiologicalMaterial03
- ManufacturingProcess09 -
ManufacturingProcess36
- BiologicalMaterial12 -
BiologicalMaterial02
- ManufacturingProcess06
(c) Explore the relationships between the top predictors and the response for the predictors that are unique to the optimal nonlinear regression model. Do these plots reveal intuition about the biological or process predictors and their relationship with yield?
In the plot from the assignment, the plot reveals intuition about the biological or process predictors and their relationship with yield as a linear relationship. For all biological/process predictors on the graph, we can see that they are positively correlated with the exception of ManufacturingProcess13 and ManufacturingProcess17.
We can also create a correlation plot to see the top predictors from our nonlinear regression model and compare them with the graph in the assignment.
Through creating a correlation plot of the top predictors and the response for the predictors that are unique to the optimal nonlinear regression model, we can see that ManufacturingProcess32 and ManufacturingProcess09 persists with mid to high correlation to the yield. But 3 of the top ten biological/processes variables are negatively correlative to yield.
topPreds <- varImp(svmMod)$importance %>%
arrange(-Overall) %>%
head(10)
Chemical %>%
select(c("Yield", row.names(topPreds))) %>%
cor() %>%
corrplot()