hw8
7.2. Friedman (1991) introduced several benchmark data sets create by simulation. One of these simulations used the following nonlinear equation to create data: y = 10 sin(πx1x2) + 20(x3 − 0.5)2 + 10x4 + 5x5 + N(0, σ2) where the x values are random variables uniformly distributed between [0, 1] (there are also 5 other non-informative variables also created in the simulation). The package mlbench contains a function called mlbench.friedman1 that simulates these data:
## k-Nearest Neighbors
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Bootstrapped (25 reps)
## Summary of sample sizes: 200, 200, 200, 200, 200, 200, ...
## Resampling results across tuning parameters:
##
## k RMSE Rsquared MAE
## 5 3.466085 0.5121775 2.816838
## 7 3.349428 0.5452823 2.727410
## 9 3.264276 0.5785990 2.660026
## 11 3.214216 0.6024244 2.603767
## 13 3.196510 0.6176570 2.591935
## 15 3.184173 0.6305506 2.577482
## 17 3.183130 0.6425367 2.567787
## 19 3.198752 0.6483184 2.592683
## 21 3.188993 0.6611428 2.588787
## 23 3.200458 0.6638353 2.604529
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 17.## RMSE Rsquared MAE
## 3.2040595 0.6819919 2.5683461SVM Model
## Support Vector Machines with Radial Basis Function Kernel
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
## Resampling results across tuning parameters:
##
## C RMSE Rsquared MAE
## 0.25 2.505383 0.8031869 1.999381
## 0.50 2.290725 0.8103140 1.829703
## 1.00 2.105086 0.8302040 1.677851
## 2.00 2.014620 0.8418576 1.598814
## 4.00 1.965196 0.8491165 1.567327
## 8.00 1.927668 0.8538927 1.542287
## 16.00 1.924269 0.8545304 1.539258
## 32.00 1.924269 0.8545304 1.539258
## 64.00 1.924269 0.8545304 1.539258
## 128.00 1.924269 0.8545304 1.539258
##
## Tuning parameter 'sigma' was held constant at a value of 0.06802164
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.06802164 and C = 16.## RMSE Rsquared MAE
## 2.0864652 0.8236735 1.5854649MARS Model
## Call: earth(x=trainingData$x, y=trainingData$y)
##
## coefficients
## (Intercept) 18.451984
## h(0.621722-X1) -11.074396
## h(0.601063-X2) -10.744225
## h(X3-0.281766) 20.607853
## h(0.447442-X3) 17.880232
## h(X3-0.447442) -23.282007
## h(X3-0.636458) 15.150350
## h(0.734892-X4) -10.027487
## h(X4-0.734892) 9.092045
## h(0.850094-X5) -4.723407
## h(X5-0.850094) 10.832932
## h(X6-0.361791) -1.956821
##
## Selected 12 of 18 terms, and 6 of 10 predictors
## Termination condition: Reached nk 21
## Importance: X1, X4, X2, X5, X3, X6, X7-unused, X8-unused, X9-unused, ...
## Number of terms at each degree of interaction: 1 11 (additive model)
## GCV 2.540556 RSS 397.9654 GRSq 0.8968524 RSq 0.9183982## RMSE Rsquared MAE
## 1.8136467 0.8677298 1.3911836Neural Networks Model
## Warning: executing %dopar% sequentially: no parallel backend registered## Length Class Mode
## model 5 -none- list
## repeats 1 -none- numeric
## bag 1 -none- logical
## seeds 5 -none- numeric
## names 10 -none- character## RMSE Rsquared MAE
## 1.9712485 0.8434083 1.4644855Compare Models - Conclusion: Neural Networks was the best model because of best performance results with R squared and RMSE. KNN was the worst-performing model.
7.5 Exercise 6.3 describes data for a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several nonlinear regression models.
## Warning in preProcess.default(Test, method = c("center", "scale")): These
## variables have zero variances: BiologicalMaterial07SVM
## RMSE Rsquared MAE
## 0.6642845 0.5546619 0.5608173KNN
## RMSE Rsquared MAE
## 0.7585275 0.4284176 0.6270432MARS
## RMSE Rsquared MAE
## 1.76572491 0.05438788 1.28748814Neural Networks
## RMSE Rsquared MAE
## 0.6550674 0.5645153 0.5079642Which nonlinear regression model gives the optimal resampling and test set performance?
It is the SVM model because of best performance and with best RMSE.Which predictors are most important in the optimal nonlinear regression model? Do either the biological or process variables dominate the list? How do the top ten important predictors compare to the top ten predictors from the optimal linear model?
ManufacturingProcess32 and BiologicalMaterial06 are most important predictors.
## loess r-squared variable importance
##
## only 20 most important variables shown (out of 57)
##
## Overall
## ManufacturingProcess32 100.00
## BiologicalMaterial06 94.06
## BiologicalMaterial03 81.27
## ManufacturingProcess13 80.63
## ManufacturingProcess36 79.17
## ManufacturingProcess31 76.84
## BiologicalMaterial02 76.04
## ManufacturingProcess17 75.92
## ManufacturingProcess09 73.04
## BiologicalMaterial12 69.48
## ManufacturingProcess06 66.28
## BiologicalMaterial11 59.72
## ManufacturingProcess33 58.60
## ManufacturingProcess29 54.77
## BiologicalMaterial04 53.93
## ManufacturingProcess11 49.55
## BiologicalMaterial01 45.62
## BiologicalMaterial08 44.93
## BiologicalMaterial09 40.88
## ManufacturingProcess30 40.31- Explore the relationships between the top predictors and the
response for the predictors that are unique to the optimal nonlinear
regression model. Do these plots reveal intuition about the biological
or process predictors and their relationship with yield?
There is a positive relationship with yield for the biological variables. There are several process variables that have either negative relationships or do not have clear relationships with yield. There are also several process variables with positive relationships with yield. Yes, these plots reveal intuition about the biological or process predictors and their relationship with yield.
