Exercises 7.2 & 7.5 from the K&J book. The rpubs version of this work can be found here, and source/data can be found on github here.

#clear the workspace
rm(list = ls())

#load req's packages
library(mlbench)
library(caret)
library(knitr)
library(AppliedPredictiveModeling)
library(corrplot)

7.2

Friedman (1991) introduced several benchmark data sets create by simulation. One of these simulations used the following nonlinear equation to create data:

\(y=10sin(\pi x_1x_2)+20(x_3-0.5)^2 + 10x_4+5x_5+N(0,\sigma^2)\)

where the x values are random variables uniformly distributed between [0, 1] (there are also 5 other non-informative variables also created in the simulation). The package mlbench contains a function called mlbench.friedman1 that simulates these data:

set.seed(200)
trainingData <- mlbench.friedman1(200, sd = 1)
## We convert the 'x' data from a matrix to a data frame
## One reason is that this will give the columns names.
trainingData$x <- data.frame(trainingData$x)
## Look at the data using
featurePlot(trainingData$x, trainingData$y)

## or other methods.

## This creates a list with a vector 'y' and a matrix
## of predictors 'x'. Also simulate a large test set to
## estimate the true error rate with good precision:
testData <- mlbench.friedman1(5000, sd = 1)
testData$x <- data.frame(testData$x)

Tune several models on these data. For example:

knnModel <- train(x = trainingData$x, 
                  y = trainingData$y,
                  method = "knn",
                  preProcess = c("center", "scale"),
                  tuneLength = 10)
knnModel
## k-Nearest Neighbors 
## 
## 200 samples
##  10 predictor
## 
## Pre-processing: centered (10), scaled (10) 
## Resampling: Bootstrapped (25 reps) 
## Summary of sample sizes: 200, 200, 200, 200, 200, 200, ... 
## Resampling results across tuning parameters:
## 
##   k   RMSE      Rsquared   MAE     
##    5  3.466085  0.5121775  2.816838
##    7  3.349428  0.5452823  2.727410
##    9  3.264276  0.5785990  2.660026
##   11  3.214216  0.6024244  2.603767
##   13  3.196510  0.6176570  2.591935
##   15  3.184173  0.6305506  2.577482
##   17  3.183130  0.6425367  2.567787
##   19  3.198752  0.6483184  2.592683
##   21  3.188993  0.6611428  2.588787
##   23  3.200458  0.6638353  2.604529
## 
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 17.
knnPred <- predict(knnModel, newdata = testData$x)
postResample(pred = knnPred, obs = testData$y)
##      RMSE  Rsquared       MAE 
## 3.2040595 0.6819919 2.5683461

Which models appear to give the best performance? Does MARS select the informative predictors (those named \(X1-X5\))?

MARS:

set.seed(19)  #set the see for the 7.2
grid.MARS <- expand.grid(degree =1:2, nprune=seq(2,14,by=2))
model.MARS <- train(x = trainingData$x, 
                    y = trainingData$y, 
                    method='bagEarth', 
                    tuneGrid = grid.MARS, 
                    trControl = trainControl(method = "cv"))
## Loading required package: earth
## Loading required package: Formula
## Loading required package: plotmo
## Loading required package: plotrix
## Loading required package: TeachingDemos
model.MARS
## Bagged MARS 
## 
## 200 samples
##  10 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ... 
## Resampling results across tuning parameters:
## 
##   degree  nprune  RMSE      Rsquared   MAE      
##   1        2      3.872886  0.5574803  3.1941640
##   1        4      2.633475  0.7410836  2.1128858
##   1        6      1.972961  0.8489500  1.5654086
##   1        8      1.626131  0.8939049  1.2611676
##   1       10      1.597489  0.8984315  1.2257064
##   1       12      1.591183  0.8997029  1.2309777
##   1       14      1.615752  0.8970973  1.2422983
##   2        2      3.839686  0.6053134  3.1622985
##   2        4      2.609962  0.7450580  2.0968921
##   2        6      1.973944  0.8510582  1.5795255
##   2        8      1.521918  0.9083429  1.1879655
##   2       10      1.270410  0.9375227  1.0158870
##   2       12      1.191376  0.9441512  0.9574594
##   2       14      1.202423  0.9436500  0.9654653
## 
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nprune = 12 and degree = 2.
pred.MARS <- predict(model.MARS, newdata = testData$x)
postResample(pred = pred.MARS, obs = testData$y)
##      RMSE  Rsquared       MAE 
## 1.1284736 0.9490171 0.8964610

Neural Network:

grid.NN <- expand.grid(size = seq(from = 1, to = 5, by = 1),
                        decay = seq(from = 0.01, to = 0.1, by = 0.01), 
                        bag = F)

maxW.NN <- 5 * (ncol(trainingData$x) + 1) + 5 + 1
model.NN <- train(x = trainingData$x,
                    y = trainingData$y,
                    method = "avNNet",
                    preProcess = c("center", "scale"),
                    tuneGrid = grid.NN,
                    trControl = trainControl(method = "cv"),
                    linout = TRUE,
                    trace = FALSE,
                    MaxNWts = maxW.NN,
                    maxit = 500)
## Warning: executing %dopar% sequentially: no parallel backend registered
model.NN
## Model Averaged Neural Network 
## 
## 200 samples
##  10 predictor
## 
## Pre-processing: centered (10), scaled (10) 
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ... 
## Resampling results across tuning parameters:
## 
##   size  decay  RMSE      Rsquared   MAE     
##   1     0.01   2.400622  0.7714386  1.897629
##   1     0.02   2.399843  0.7714852  1.895721
##   1     0.03   2.399920  0.7714152  1.895046
##   1     0.04   2.400168  0.7713292  1.894515
##   1     0.05   2.400890  0.7711732  1.894608
##   1     0.06   2.401542  0.7710361  1.895117
##   1     0.07   2.402297  0.7708871  1.895686
##   1     0.08   2.403115  0.7707332  1.896336
##   1     0.09   2.403789  0.7705797  1.896881
##   1     0.10   2.404708  0.7704335  1.897459
##   2     0.01   2.424842  0.7684080  1.927782
##   2     0.02   2.453302  0.7632762  1.940758
##   2     0.03   2.410481  0.7753408  1.898656
##   2     0.04   2.489221  0.7551772  1.969690
##   2     0.05   2.400486  0.7740147  1.875456
##   2     0.06   2.496520  0.7575745  1.991294
##   2     0.07   2.482649  0.7594988  1.988898
##   2     0.08   2.493447  0.7578581  1.955523
##   2     0.09   2.479188  0.7576474  1.968391
##   2     0.10   2.461433  0.7664001  1.918090
##   3     0.01   2.185140  0.8164435  1.763261
##   3     0.02   2.076960  0.8312257  1.649566
##   3     0.03   2.129401  0.8240269  1.707017
##   3     0.04   2.174907  0.8148989  1.719764
##   3     0.05   2.247129  0.8010927  1.815262
##   3     0.06   2.109751  0.8244457  1.683526
##   3     0.07   2.124562  0.8231077  1.699436
##   3     0.08   2.136488  0.8226152  1.699475
##   3     0.09   2.124756  0.8200458  1.709112
##   3     0.10   2.027132  0.8369401  1.588769
##   4     0.01   1.980588  0.8472038  1.605826
##   4     0.02   2.056921  0.8365647  1.633403
##   4     0.03   1.968777  0.8458685  1.569599
##   4     0.04   2.001161  0.8459572  1.584765
##   4     0.05   2.044591  0.8355813  1.620653
##   4     0.06   2.097238  0.8292814  1.701473
##   4     0.07   1.947761  0.8546748  1.563598
##   4     0.08   1.966863  0.8481986  1.531310
##   4     0.09   2.086539  0.8280100  1.628341
##   4     0.10   2.061633  0.8338011  1.639661
##   5     0.01   2.055161  0.8333678  1.629186
##   5     0.02   2.032069  0.8359462  1.630763
##   5     0.03   2.137228  0.8205219  1.692230
##   5     0.04   2.005580  0.8446996  1.599752
##   5     0.05   2.067853  0.8316243  1.670586
##   5     0.06   1.991971  0.8453077  1.598048
##   5     0.07   1.969556  0.8502192  1.589085
##   5     0.08   1.985552  0.8437606  1.584983
##   5     0.09   2.058434  0.8308109  1.689876
##   5     0.10   2.080779  0.8243130  1.660202
## 
## Tuning parameter 'bag' was held constant at a value of FALSE
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 4, decay = 0.07 and bag = FALSE.
pred.NN <- predict(model.NN, newdata = testData$x)
postResample(pred = pred.NN, obs = testData$y)
##     RMSE Rsquared      MAE 
## 2.120147 0.821799 1.591204

SVM:

model.SVM <- train(x = trainingData$x,
                   y = trainingData$y,
                   method='svmRadial',
                   tuneLength = 9,
                   trControl = trainControl(method = "cv"))


model.SVM
## Support Vector Machines with Radial Basis Function Kernel 
## 
## 200 samples
##  10 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ... 
## Resampling results across tuning parameters:
## 
##   C      RMSE      Rsquared   MAE     
##    0.25  2.505555  0.7966271  2.022634
##    0.50  2.279753  0.8109619  1.832437
##    1.00  2.122294  0.8322147  1.706683
##    2.00  1.998040  0.8511927  1.598674
##    4.00  1.874576  0.8670713  1.494258
##    8.00  1.836062  0.8705888  1.456335
##   16.00  1.833865  0.8705680  1.451881
##   32.00  1.833884  0.8705678  1.451888
##   64.00  1.833884  0.8705678  1.451888
## 
## Tuning parameter 'sigma' was held constant at a value of 0.06178128
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.06178128 and C = 16.
pred.SVM <- predict(model.SVM, newdata = testData$x)
postResample(pred = pred.SVM , obs = testData$y)
##      RMSE  Rsquared       MAE 
## 2.0710406 0.8260815 1.5731468

Summary

First we will look at model performance:

model.perf <- cbind(data.frame(postResample(pred = knnPred, obs = testData$y)),
                    data.frame(postResample(pred = pred.MARS, obs = testData$y)),
                    data.frame(postResample(pred = pred.NN, obs = testData$y)),
                    data.frame(postResample(pred = pred.SVM, obs = testData$y)))

colnames(model.perf) <- c("KNN","MARS","NN","SVM")

kable(round(model.perf,2))
KNN MARS NN SVM
RMSE 3.20 1.13 2.12 2.07
Rsquared 0.68 0.95 0.82 0.83
MAE 2.57 0.90 1.59 1.57

For this particular problem, we can see that the MARS model the superior performer. MARS shows the smallest error stats and the higest Rsquared.

var.importance <- cbind(varImp(knnModel)[[1]],
                        varImp(model.MARS)[[1]],
                        varImp(model.NN)[[1]],
                        varImp(model.SVM)[[1]])

colnames(var.importance) <- c("KNN","MARS","NN","SVM")

kable(round(var.importance))
KNN MARS NN SVM
X1 96 100 96 96
X2 90 84 90 90
X3 30 69 30 30
X4 100 50 100 100
X5 45 40 45 45
X6 1 2 1 1
X7 0 0 0 0
X8 3 0 3 3
X9 6 0 6 6
X10 6 0 6 6

All of the models correctly identify \(X1-X5\) as the important variables while almost identifying \(X6-X10\) as not important.

Interestingly, MARS performs the best and also shows 2 distinct differences: For the MARS model, \(X3\) is significantly over-weight as compared to other models and \(X4\) is signifcantly underweight vs. the other models.

7.5

Exercise 6.3 describes data for a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several nonlinear regression models.

set.seed(19) #reset the seed for Q 7.5 
data(ChemicalManufacturingProcess)
chem <- ChemicalManufacturingProcess


#impute using knn
chem.imp <- preProcess(chem[,2:ncol(chem)], method=c('knnImpute'))
chem <- cbind(chem$Yield,predict(chem.imp, chem[,2:ncol(chem)]))
colnames(chem)[1] <- "Yield"

#split 70/30
n <-  floor(0.70 * nrow(chem))
idx <- sample(seq_len(nrow(chem)), size = n)
train <- chem[idx, ]
test <- chem[-idx, ]
knnModel <- train(x=train[,-1],
                  y=train$Yield,
                  method = "knn",
                  preProcess = c("center", "scale"),
                  tuneLength = 10)
knnModel
## k-Nearest Neighbors 
## 
## 123 samples
##  57 predictor
## 
## Pre-processing: centered (57), scaled (57) 
## Resampling: Bootstrapped (25 reps) 
## Summary of sample sizes: 123, 123, 123, 123, 123, 123, ... 
## Resampling results across tuning parameters:
## 
##   k   RMSE      Rsquared   MAE     
##    5  1.433790  0.3687691  1.136364
##    7  1.410122  0.3732909  1.120532
##    9  1.414762  0.3668017  1.121848
##   11  1.421107  0.3607650  1.138411
##   13  1.437570  0.3437580  1.151992
##   15  1.443993  0.3408658  1.164805
##   17  1.453462  0.3358697  1.172275
##   19  1.466765  0.3268167  1.184108
##   21  1.471999  0.3259854  1.191366
##   23  1.484340  0.3177842  1.205743
## 
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 7.
knnPred <- predict(knnModel, newdata = test[,-1])
postResample(pred = knnPred, obs = test$Yield)
##      RMSE  Rsquared       MAE 
## 1.3705821 0.5888388 1.0641004
grid.MARS <- expand.grid(degree =1:2, nprune=seq(2,14,by=2))
model.MARS <- train(x=train[,-1],
                    y=train$Yield, 
                    method='bagEarth', 
                    tuneGrid = grid.MARS, 
                    trControl = trainControl(method = "cv"))
model.MARS
## Bagged MARS 
## 
## 123 samples
##  57 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 111, 111, 111, 110, 111, 111, ... 
## Resampling results across tuning parameters:
## 
##   degree  nprune  RMSE      Rsquared   MAE     
##   1        2      1.350900  0.4941127  1.108734
##   1        4      1.221729  0.5606339  1.005637
##   1        6      1.317405  0.5269844  1.036788
##   1        8      1.450711  0.4857636  1.091201
##   1       10      1.513744  0.5216115  1.071108
##   1       12      1.819527  0.4806020  1.166368
##   1       14      1.851979  0.4982395  1.189922
##   2        2      1.369968  0.5002618  1.132796
##   2        4      1.263521  0.5200987  1.033068
##   2        6      1.436696  0.5185892  1.062468
##   2        8      1.593507  0.5342204  1.097654
##   2       10      1.419092  0.5333126  1.025104
##   2       12      1.527123  0.5480292  1.070069
##   2       14      1.467996  0.5114144  1.044072
## 
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nprune = 4 and degree = 1.
pred.MARS <- predict(model.MARS, newdata = test[,-1])
postResample(pred = pred.MARS, obs = test$Yield)
##      RMSE  Rsquared       MAE 
## 1.1358439 0.6864584 0.9029211
grid.NN <- expand.grid(size = seq(from = 1, to = 5, by = 1),
                        decay = seq(from = 0.01, to = 0.1, by = 0.01), 
                        bag = F)

maxW.NN <- 5 * (ncol(train[,-1]) + 1) + 5 + 1
model.NN <- train(x=train[,-1],
                 y=train$Yield,
                    method = "avNNet",
                    preProcess = c("center", "scale"),
                    tuneGrid = grid.NN,
                    trControl = trainControl(method = "cv"),
                    linout = TRUE,
                    trace = FALSE,
                    MaxNWts = maxW.NN ,
                    maxit = 500)
model.NN
## Model Averaged Neural Network 
## 
## 123 samples
##  57 predictor
## 
## Pre-processing: centered (57), scaled (57) 
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 111, 110, 111, 110, 111, 111, ... 
## Resampling results across tuning parameters:
## 
##   size  decay  RMSE      Rsquared   MAE     
##   1     0.01   1.414621  0.4820550  1.134261
##   1     0.02   1.498310  0.4790568  1.244785
##   1     0.03   1.475457  0.4649938  1.215205
##   1     0.04   1.446891  0.4828503  1.171212
##   1     0.05   1.404087  0.5105161  1.125299
##   1     0.06   1.404469  0.5268742  1.133860
##   1     0.07   1.453192  0.5021947  1.174893
##   1     0.08   1.382445  0.5245528  1.110207
##   1     0.09   1.389990  0.5427522  1.116941
##   1     0.10   1.452343  0.5204224  1.154873
##   2     0.01   1.624556  0.4733965  1.275621
##   2     0.02   1.272054  0.5803635  1.056169
##   2     0.03   1.439756  0.4765153  1.174549
##   2     0.04   1.424209  0.5220034  1.148160
##   2     0.05   1.606384  0.4473907  1.260930
##   2     0.06   1.799784  0.4045282  1.403076
##   2     0.07   1.618185  0.4500350  1.243843
##   2     0.08   1.802231  0.4156477  1.443997
##   2     0.09   1.693245  0.4569352  1.274126
##   2     0.10   1.683087  0.4193042  1.297389
##   3     0.01   1.385125  0.5622414  1.124513
##   3     0.02   1.452840  0.5227157  1.147075
##   3     0.03   1.814029  0.4160417  1.402600
##   3     0.04   1.593735  0.5085344  1.275350
##   3     0.05   1.822826  0.4470904  1.420016
##   3     0.06   2.045729  0.4356639  1.504792
##   3     0.07   1.928627  0.4526792  1.407563
##   3     0.08   2.184081  0.4198829  1.488014
##   3     0.09   2.158882  0.4189337  1.538232
##   3     0.10   2.340185  0.4382209  1.739256
##   4     0.01   1.779042  0.4643278  1.406868
##   4     0.02   2.103758  0.4217445  1.495449
##   4     0.03   1.736494  0.4568198  1.342442
##   4     0.04   2.112594  0.4025472  1.557984
##   4     0.05   2.124631  0.4132023  1.524389
##   4     0.06   1.857849  0.4891411  1.356698
##   4     0.07   2.246249  0.4132615  1.666025
##   4     0.08   2.060857  0.4313946  1.539671
##   4     0.09   2.026729  0.4273797  1.494332
##   4     0.10   2.564989  0.3605550  1.667459
##   5     0.01   2.052423  0.3918023  1.495925
##   5     0.02   2.243536  0.4221535  1.567315
##   5     0.03   2.106678  0.4411413  1.559716
##   5     0.04   2.111195  0.3960049  1.463374
##   5     0.05   2.127825  0.4056766  1.514876
##   5     0.06   2.412157  0.3861384  1.566869
##   5     0.07   2.464693  0.3756247  1.601317
##   5     0.08   2.219673  0.4103620  1.527444
##   5     0.09   2.331367  0.4205786  1.557505
##   5     0.10   2.127349  0.4275442  1.490513
## 
## Tuning parameter 'bag' was held constant at a value of FALSE
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 2, decay = 0.02 and bag = FALSE.
pred.NN <- predict(model.NN, newdata = test[,-1])
postResample(pred = pred.NN, obs = test$Yield)
##      RMSE  Rsquared       MAE 
## 1.2243400 0.6495989 1.0223198
model.SVM <- train(x=train[,-1],
                   y=train$Yield,
                   method='svmRadial',
                   tuneLength = 9,
                   trControl = trainControl(method = "cv"))


model.SVM
## Support Vector Machines with Radial Basis Function Kernel 
## 
## 123 samples
##  57 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 110, 111, 111, 111, 111, 111, ... 
## Resampling results across tuning parameters:
## 
##   C      RMSE      Rsquared   MAE      
##    0.25  1.410928  0.4654637  1.1588947
##    0.50  1.302344  0.5212314  1.0655698
##    1.00  1.212190  0.5812348  0.9768983
##    2.00  1.130613  0.6400578  0.9071572
##    4.00  1.101416  0.6501162  0.8926903
##    8.00  1.094220  0.6471883  0.8948193
##   16.00  1.094220  0.6471883  0.8948193
##   32.00  1.094220  0.6471883  0.8948193
##   64.00  1.094220  0.6471883  0.8948193
## 
## Tuning parameter 'sigma' was held constant at a value of 0.01630921
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.01630921 and C = 8.
pred.SVM <- predict(model.SVM, newdata = test[,-1])
postResample(pred = pred.SVM , obs = test$Yield)
##      RMSE  Rsquared       MAE 
## 1.3164498 0.6197054 1.0114683

A

Which nonlinear regression model gives the optimal resampling and test set performance?

model.perf <- cbind(data.frame(postResample(pred = knnPred, obs =test$Yield)),
                    data.frame(postResample(pred = pred.MARS, obs = test$Yield)),
                    data.frame(postResample(pred = pred.NN, obs = test$Yield)),
                    data.frame(postResample(pred = pred.SVM, obs = test$Yield)))

colnames(model.perf) <- c("KNN","MARS","NN","SVM")

kable(round(model.perf,2))
KNN MARS NN SVM
RMSE 1.37 1.14 1.22 1.32
Rsquared 0.59 0.69 0.65 0.62
MAE 1.06 0.90 1.02 1.01

B

Which predictors are most important in the optimal nonlinear regression model? Do either the biological or process variables dominate the list? How do the top ten important predictors compare to the top ten predictors from the optimal linear model?

var.importance <- cbind(varImp(knnModel)[[1]],
                        varImp(model.MARS)[[1]],
                        varImp(model.NN)[[1]],
                        varImp(model.SVM)[[1]])

colnames(var.importance) <- c("KNN","MARS","NN","SVM")

kable(round(var.importance))
KNN MARS NN SVM
BiologicalMaterial01 46 100 46 46
BiologicalMaterial02 64 58 64 64
BiologicalMaterial03 91 23 91 91
BiologicalMaterial04 47 3 47 47
BiologicalMaterial05 8 0 8 8
BiologicalMaterial06 95 0 95 95
BiologicalMaterial07 4 0 4 4
BiologicalMaterial08 48 0 48 48
BiologicalMaterial09 57 0 57 57
BiologicalMaterial10 22 0 22 22
BiologicalMaterial11 52 0 52 52
BiologicalMaterial12 75 0 75 75
ManufacturingProcess01 28 0 28 28
ManufacturingProcess02 32 0 32 32
ManufacturingProcess03 1 0 1 1
ManufacturingProcess04 29 0 29 29
ManufacturingProcess05 7 0 7 7
ManufacturingProcess06 64 0 64 64
ManufacturingProcess07 0 0 0 0
ManufacturingProcess08 0 0 0 0
ManufacturingProcess09 72 0 72 72
ManufacturingProcess10 29 0 29 29
ManufacturingProcess11 53 0 53 53
ManufacturingProcess12 29 0 29 29
ManufacturingProcess13 88 0 88 88
ManufacturingProcess14 19 0 19 19
ManufacturingProcess15 22 0 22 22
ManufacturingProcess16 26 0 26 26
ManufacturingProcess17 66 0 66 66
ManufacturingProcess18 32 0 32 32
ManufacturingProcess19 9 0 9 9
ManufacturingProcess20 23 0 23 23
ManufacturingProcess21 13 0 13 13
ManufacturingProcess22 0 0 0 0
ManufacturingProcess23 4 0 4 4
ManufacturingProcess24 15 0 15 15
ManufacturingProcess25 28 0 28 28
ManufacturingProcess26 22 0 22 22
ManufacturingProcess27 17 0 17 17
ManufacturingProcess28 13 0 13 13
ManufacturingProcess29 57 0 57 57
ManufacturingProcess30 42 0 42 42
ManufacturingProcess31 62 0 62 62
ManufacturingProcess32 100 0 100 100
ManufacturingProcess33 48 0 48 48
ManufacturingProcess34 6 0 6 6
ManufacturingProcess35 14 0 14 14
ManufacturingProcess36 57 0 57 57
ManufacturingProcess37 15 0 15 15
ManufacturingProcess38 0 0 0 0
ManufacturingProcess39 1 0 1 1
ManufacturingProcess40 0 0 0 0
ManufacturingProcess41 0 0 0 0
ManufacturingProcess42 0 0 0 0
ManufacturingProcess43 2 0 2 2
ManufacturingProcess44 14 0 14 14
ManufacturingProcess45 1 0 1 1

C

Explore the relationships between the top predictors and the response for the predictors that are unique to the optimal nonlinear regression model. Do these plots reveal intuition about the biological or process predictors and their relationship with yield?

#we'll look @ the 10 top predictors
top.predictors <- data.frame(sort(rowMeans(var.importance),decreasing=T)[1:15])

top.predictors <- cbind(test$Yield,test[,row.names(top.predictors)])




r.mat <- data.frame(cor(top.predictors)[1,])


r.target <- tail(data.frame(cor(top.predictors)[1,]),-1)
colnames(r.target) <- c("Corr")


ggplot(r.target,aes(x=row.names(r.target),y=Corr))+
  geom_bar(stat="identity")+
  ggtitle("Correlation to Yield")+
  theme(axis.text.x=element_text(angle = -90, hjust = 0))

We arbitrarily chose the top 15 predictors (~25% of the variables) and note that biological processes seem to have a positive correlation to target, whereas Manufacturing processes appear to show negative bias but are also mixed.