Friedman (1991) introduced several benchmark data sets create by simulation. One of these simulations used a nonlinear equation to create data where the x values are random variables uniformly distributed between [0,1] (there are also 5 other non-informative variables also created in the simulation). The package mlbench contains a function called mlbench.friedman1 that simulates data.
Tune several models on these data.
## k-Nearest Neighbors
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Bootstrapped (25 reps)
## Summary of sample sizes: 200, 200, 200, 200, 200, 200, ...
## Resampling results across tuning parameters:
##
## k RMSE Rsquared MAE
## 5 3.466085 0.5121775 2.816838
## 7 3.349428 0.5452823 2.727410
## 9 3.264276 0.5785990 2.660026
## 11 3.214216 0.6024244 2.603767
## 13 3.196510 0.6176570 2.591935
## 15 3.184173 0.6305506 2.577482
## 17 3.183130 0.6425367 2.567787
## 19 3.198752 0.6483184 2.592683
## 21 3.188993 0.6611428 2.588787
## 23 3.200458 0.6638353 2.604529
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 17.
## RMSE Rsquared MAE
## 3.2040595 0.6819919 2.5683461
Which models appear to give the best performance? Does MARS select the informative predictors?
## Warning: executing %dopar% sequentially: no parallel backend registered
## Warning: model fit failed for Fold01: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold01: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold01: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold01: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold01: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold01: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold01: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold01: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold01: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold01: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold01: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold01: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold01: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold01: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold01: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold02: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold02: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold02: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold02: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold02: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold02: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold02: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold02: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold02: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold02: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold02: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold02: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold02: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold02: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold02: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold03: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold03: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold03: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold03: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold03: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold03: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold03: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold03: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold03: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold03: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold03: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold03: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold03: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold03: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold03: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold04: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold04: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold04: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold04: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold04: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold04: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold04: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold04: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold04: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold04: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold04: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold04: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold04: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold04: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold04: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold05: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold05: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold05: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold05: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold05: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold05: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold05: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold05: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold05: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold05: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold05: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold05: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold05: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold05: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold05: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold06: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold06: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold06: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold06: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold06: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold06: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold06: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold06: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold06: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold06: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold06: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold06: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold06: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold06: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold06: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold07: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold07: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold07: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold07: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold07: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold07: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold07: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold07: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold07: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold07: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold07: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold07: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold07: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold07: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold07: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold08: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold08: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold08: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold08: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold08: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold08: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold08: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold08: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold08: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold08: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold08: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold08: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold08: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold08: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold08: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold09: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold09: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold09: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold09: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold09: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold09: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold09: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold09: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold09: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold09: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold09: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold09: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold09: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold09: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold09: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold10: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold10: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold10: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (73) weights"
## Warning: model fit failed for Fold10: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold10: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold10: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (85) weights"
## Warning: model fit failed for Fold10: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold10: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold10: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (97) weights"
## Warning: model fit failed for Fold10: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold10: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold10: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (109) weights"
## Warning: model fit failed for Fold10: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold10: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning: model fit failed for Fold10: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (121) weights"
## Warning in nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,
## : There were missing values in resampled performance measures.
## Warning in train.default(x = trainingData$x, y = trainingData$y, method =
## "avNNet", : missing values found in aggregated results
## Model Averaged Neural Network
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
## Resampling results across tuning parameters:
##
## decay size RMSE Rsquared MAE
## 0.00 1 2.434845 0.7683498 1.921367
## 0.00 2 2.497822 0.7558233 1.993325
## 0.00 3 2.037885 0.8419795 1.609413
## 0.00 4 1.900063 0.8584928 1.529545
## 0.00 5 2.176661 0.8092998 1.628603
## 0.00 6 NaN NaN NaN
## 0.00 7 NaN NaN NaN
## 0.00 8 NaN NaN NaN
## 0.00 9 NaN NaN NaN
## 0.00 10 NaN NaN NaN
## 0.01 1 2.437231 0.7689665 1.934978
## 0.01 2 2.510986 0.7596191 1.988260
## 0.01 3 1.999944 0.8419567 1.555751
## 0.01 4 2.003357 0.8445288 1.549723
## 0.01 5 2.104801 0.8296459 1.664982
## 0.01 6 NaN NaN NaN
## 0.01 7 NaN NaN NaN
## 0.01 8 NaN NaN NaN
## 0.01 9 NaN NaN NaN
## 0.01 10 NaN NaN NaN
## 0.10 1 2.450897 0.7652309 1.942945
## 0.10 2 2.489399 0.7606443 1.997060
## 0.10 3 2.200693 0.8155496 1.786599
## 0.10 4 2.059322 0.8432340 1.651716
## 0.10 5 2.189025 0.8133603 1.729453
## 0.10 6 NaN NaN NaN
## 0.10 7 NaN NaN NaN
## 0.10 8 NaN NaN NaN
## 0.10 9 NaN NaN NaN
## 0.10 10 NaN NaN NaN
##
## Tuning parameter 'bag' was held constant at a value of FALSE
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 4, decay = 0 and bag = FALSE.
## RMSE Rsquared MAE
## 2.496722 0.784618 1.685182
## Multivariate Adaptive Regression Spline
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
## Resampling results across tuning parameters:
##
## degree nprune RMSE Rsquared MAE
## 1 2 4.334325 0.2599883 3.607719
## 1 3 3.599334 0.4805557 2.888987
## 1 4 2.637145 0.7290848 2.087677
## 1 5 2.283872 0.7939684 1.817343
## 1 6 2.125875 0.8183677 1.647491
## 1 7 1.766013 0.8733619 1.410328
## 1 8 1.671282 0.8842102 1.324258
## 1 9 1.645406 0.8867947 1.322041
## 1 10 1.597968 0.8926582 1.297518
## 1 11 1.540109 0.8996361 1.237949
## 1 12 1.545349 0.8992979 1.243771
## 1 13 1.535169 0.9010122 1.233571
## 1 14 1.529405 0.9018457 1.223874
## 1 15 1.529405 0.9018457 1.223874
## 2 2 4.334325 0.2599883 3.607719
## 2 3 3.707393 0.4465094 2.974346
## 2 4 2.741265 0.6998652 2.158615
## 2 5 2.374112 0.7658466 1.896371
## 2 6 2.208958 0.7957704 1.785614
## 2 7 1.919044 0.8456712 1.543549
## 2 8 1.724766 0.8791033 1.346494
## 2 9 1.540450 0.9027946 1.232856
## 2 10 1.464049 0.9152778 1.169922
## 2 11 1.411899 0.9200138 1.130318
## 2 12 1.376559 0.9230165 1.100356
## 2 13 1.370289 0.9235617 1.083929
## 2 14 1.331399 0.9299364 1.037307
## 2 15 1.332740 0.9284297 1.037727
##
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nprune = 14 and degree = 2.
## RMSE Rsquared MAE
## 1.2779993 0.9338365 1.0147070
## Support Vector Machines with Radial Basis Function Kernel
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
## Resampling results across tuning parameters:
##
## C RMSE Rsquared MAE
## 0.25 2.504105 0.7940789 1.987142
## 0.50 2.219946 0.8148914 1.750249
## 1.00 2.028115 0.8388693 1.590383
## 2.00 1.899331 0.8561464 1.486326
## 4.00 1.815632 0.8669708 1.424246
## 8.00 1.798299 0.8702910 1.427678
## 16.00 1.797165 0.8702715 1.431259
## 32.00 1.795246 0.8705225 1.429235
## 64.00 1.795246 0.8705225 1.429235
## 128.00 1.795246 0.8705225 1.429235
##
## Tuning parameter 'sigma' was held constant at a value of 0.06104815
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.06104815 and C = 32.
## RMSE Rsquared MAE
## 2.0693488 0.8263553 1.5718972
The MARS model performed the best out of all other models. Additionally, the model selected the informative predictors correctly.
## earth variable importance
##
## Overall
## X1 100.00
## X4 75.40
## X2 49.00
## X5 15.72
## X3 0.00
Exercise 6.3 describes data for a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several nonlinear regression models.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## k-Nearest Neighbors
##
## 140 samples
## 57 predictor
##
## Pre-processing: centered (57), scaled (57)
## Resampling: Bootstrapped (25 reps)
## Summary of sample sizes: 140, 140, 140, 140, 140, 140, ...
## Resampling results across tuning parameters:
##
## k RMSE Rsquared MAE
## 5 1.419482 0.3991458 1.121494
## 7 1.391003 0.4171854 1.100294
## 9 1.388923 0.4189765 1.106286
## 11 1.381783 0.4232204 1.097156
## 13 1.378699 0.4309197 1.094673
## 15 1.384882 0.4299953 1.106791
## 17 1.387211 0.4313584 1.111412
## 19 1.396096 0.4285464 1.116807
## 21 1.401936 0.4275811 1.120130
## 23 1.413424 0.4199238 1.131370
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 13.
## Warning: model fit failed for Fold01: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold01: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold01: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold01: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold01: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold01: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold01: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold01: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold01: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold01: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold01: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold01: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold01: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold01: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold01: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold02: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold02: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold02: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold02: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold02: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold02: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold02: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold02: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold02: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold02: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold02: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold02: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold02: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold02: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold02: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold03: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold03: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold03: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold03: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold03: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold03: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold03: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold03: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold03: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold03: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold03: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold03: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold03: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold03: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold03: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold04: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold04: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold04: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold04: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold04: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold04: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold04: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold04: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold04: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold04: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold04: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold04: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold04: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold04: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold04: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold05: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold05: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold05: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold05: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold05: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold05: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold05: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold05: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold05: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold05: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold05: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold05: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold05: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold05: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold05: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning: model fit failed for Fold06: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold07: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold07: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold07: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold07: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold07: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold07: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold07: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold07: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold07: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold07: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold07: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold07: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold07: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold07: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold07: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold08: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold08: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold08: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold08: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold08: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold08: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold08: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold08: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold08: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold08: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold08: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold08: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold08: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold08: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold08: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold09: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold09: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold09: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold09: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold09: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold09: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold09: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold09: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold09: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold09: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold09: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold09: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold09: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold09: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold09: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold10: decay=0.00, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold10: decay=0.01, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold10: decay=0.10, size= 6, bag=FALSE Error in { : task 1 failed - "too many (355) weights"
## Warning: model fit failed for Fold10: decay=0.00, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold10: decay=0.01, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold10: decay=0.10, size= 7, bag=FALSE Error in { : task 1 failed - "too many (414) weights"
## Warning: model fit failed for Fold10: decay=0.00, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold10: decay=0.01, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold10: decay=0.10, size= 8, bag=FALSE Error in { : task 1 failed - "too many (473) weights"
## Warning: model fit failed for Fold10: decay=0.00, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold10: decay=0.01, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold10: decay=0.10, size= 9, bag=FALSE Error in { : task 1 failed - "too many (532) weights"
## Warning: model fit failed for Fold10: decay=0.00, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold10: decay=0.01, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning: model fit failed for Fold10: decay=0.10, size=10, bag=FALSE Error in { : task 1 failed - "too many (591) weights"
## Warning in nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,
## : There were missing values in resampled performance measures.
## Warning in train.default(x, y, weights = w, ...): missing values found in
## aggregated results
## Model Averaged Neural Network
##
## 140 samples
## 57 predictor
##
## Pre-processing: centered (57), scaled (57)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 125, 127, 126, 126, 125, 127, ...
## Resampling results across tuning parameters:
##
## decay size RMSE Rsquared MAE
## 0.00 1 1.541538 0.3618972 1.254688
## 0.00 2 1.405357 0.4508104 1.154767
## 0.00 3 1.568578 0.3983051 1.257665
## 0.00 4 2.108765 0.2202162 1.650086
## 0.00 5 1.796619 0.3674858 1.431505
## 0.00 6 NaN NaN NaN
## 0.00 7 NaN NaN NaN
## 0.00 8 NaN NaN NaN
## 0.00 9 NaN NaN NaN
## 0.00 10 NaN NaN NaN
## 0.01 1 1.468310 0.4669686 1.215813
## 0.01 2 1.364636 0.5257126 1.114417
## 0.01 3 1.539984 0.4627881 1.195273
## 0.01 4 1.988214 0.2956067 1.580750
## 0.01 5 1.702632 0.3711392 1.358215
## 0.01 6 NaN NaN NaN
## 0.01 7 NaN NaN NaN
## 0.01 8 NaN NaN NaN
## 0.01 9 NaN NaN NaN
## 0.01 10 NaN NaN NaN
## 0.10 1 1.737852 0.3714519 1.305442
## 0.10 2 1.496306 0.4912668 1.187494
## 0.10 3 1.802945 0.4325921 1.371090
## 0.10 4 2.072661 0.3884972 1.374417
## 0.10 5 2.168559 0.3824470 1.488328
## 0.10 6 NaN NaN NaN
## 0.10 7 NaN NaN NaN
## 0.10 8 NaN NaN NaN
## 0.10 9 NaN NaN NaN
## 0.10 10 NaN NaN NaN
##
## Tuning parameter 'bag' was held constant at a value of FALSE
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 2, decay = 0.01 and bag = FALSE.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Multivariate Adaptive Regression Spline
##
## 140 samples
## 57 predictor
##
## Pre-processing: centered (57), scaled (57)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 125, 126, 127, 125, 127, 127, ...
## Resampling results across tuning parameters:
##
## degree nprune RMSE Rsquared MAE
## 1 2 1.316981 0.4973498 1.0742105
## 1 3 1.218946 0.5537952 0.9929862
## 1 4 1.081111 0.6524961 0.9180206
## 1 5 1.065269 0.6655141 0.9068569
## 1 6 1.193724 0.5783257 1.0038057
## 1 7 1.202812 0.5798186 1.0036378
## 1 8 1.198245 0.5844033 0.9880349
## 1 9 1.232941 0.5565316 1.0141454
## 1 10 1.242385 0.5486842 1.0074270
## 1 11 1.252755 0.5473025 1.0174107
## 1 12 1.240987 0.5510414 1.0160233
## 1 13 1.244577 0.5470990 1.0216679
## 1 14 1.255504 0.5388626 1.0304026
## 1 15 1.245918 0.5457323 1.0225746
## 2 2 1.316981 0.4973498 1.0742105
## 2 3 1.196283 0.5836105 0.9890718
## 2 4 1.154878 0.5986732 0.9542092
## 2 5 1.182427 0.5663294 0.9837644
## 2 6 1.157402 0.5914123 0.9476130
## 2 7 1.201650 0.5612003 0.9876394
## 2 8 1.207122 0.5569707 0.9778457
## 2 9 1.268709 0.5399029 1.0109698
## 2 10 1.278447 0.5431554 1.0186445
## 2 11 1.292836 0.5322338 1.0248477
## 2 12 1.292600 0.5353797 1.0246384
## 2 13 1.308686 0.5284899 1.0422811
## 2 14 1.284571 0.5585503 1.0011376
## 2 15 1.299311 0.5505901 1.0281504
##
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nprune = 5 and degree = 1.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
## Warning in preProcess.default(thresh = 0.95, k = 5, freqCut = 19, uniqueCut =
## 10, : These variables have zero variances: BiologicalMaterial07
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
## Support Vector Machines with Radial Basis Function Kernel
##
## 140 samples
## 57 predictor
##
## Pre-processing: centered (57), scaled (57)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 126, 126, 125, 127, 125, 127, ...
## Resampling results across tuning parameters:
##
## C RMSE Rsquared MAE
## 0.25 1.361050 0.5015436 1.0969294
## 0.50 1.265866 0.5329777 1.0233138
## 1.00 1.195120 0.5761482 0.9631696
## 2.00 1.156202 0.5952330 0.9364056
## 4.00 1.161206 0.5923048 0.9450628
## 8.00 1.185117 0.5719019 0.9619071
## 16.00 1.174586 0.5790990 0.9532898
## 32.00 1.174257 0.5793419 0.9531748
## 64.00 1.174257 0.5793419 0.9531748
## 128.00 1.174257 0.5793419 0.9531748
##
## Tuning parameter 'sigma' was held constant at a value of 0.01314925
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.01314925 and C = 2.
Which nonlinear regression model gives the optimal resampling and test set performance?
The SVM model seems to have the best RMSE and Rsquare values when compared to the other models.
## RMSE Rsquared MAE
## knn 1.1997154 0.6076811 0.9470024
## nnet 0.2245741 0.9854546 0.1493347
## mars 1.0031973 0.6928301 0.8115820
## svm 0.5778743 0.9135486 0.3697327
Which predictors are most important in the optimal nonlinear regression model? Do either the biological or process variables dominate the list? How do the top ten important predictors compare to the top ten predictors from the optimal linear model?
The importance within the linear model consisted of 7 ManufacturingProcess and 3 BiologicalMaterials. The nonlinear model consists of 6 ManufacturingProcess and 4 BiologicalMaterials.
## loess r-squared variable importance
##
## only 20 most important variables shown (out of 57)
##
## Overall
## ManufacturingProcess32 100.00
## ManufacturingProcess13 86.50
## BiologicalMaterial06 85.94
## BiologicalMaterial02 75.73
## ManufacturingProcess36 71.59
## ManufacturingProcess31 64.79
## ManufacturingProcess17 62.23
## BiologicalMaterial12 62.22
## BiologicalMaterial03 61.97
## ManufacturingProcess09 59.77
## BiologicalMaterial04 55.83
## ManufacturingProcess33 53.90
## ManufacturingProcess02 51.45
## ManufacturingProcess06 49.49
## ManufacturingProcess29 49.17
## BiologicalMaterial01 48.19
## ManufacturingProcess11 44.91
## BiologicalMaterial08 44.44
## BiologicalMaterial11 36.66
## ManufacturingProcess04 35.48
Explore the relationships between the top predictors and the response for the predictors that are unique to the optimal nonlinear regression model. Do these plots reveal intuition about the biological or process predictors and their relationship with yield?
The important variables in respect to the response variable Yield have a linear relationship, whether it be positively or negatively correlated. ManufacturingProcess31 seems to have a very low correlation to the response variable.