DATA624 Spring 2021 HW8 - Non Linear Regression
John K. Hancock
4/25/2021
7.2.
Friedman (1991) introduced several benchmark data sets create by simulation. One of these simulations used the following nonlinear equation to create data:
\[y = 10sin(\pi x_1x_2) + 20(x_3-0.5)^2 + 10x_4+5x_5+N(0,\sigma^2)\]
where the x values are random variables uniformly distributed between [0, 1] (there are also 5 other non-informative variables also created in the simulation). The package mlbench contains a function called mlbench.friedman1 that simulates these data:
## Warning: package 'mlbench' was built under R version 3.6.3set.seed(200)
trainingData <- mlbench.friedman1(200, sd = 1)
## We convert the 'x' data from a matrix to a data frame
## One reason is that this will give the columns names.
trainingData$x <- data.frame(trainingData$x)
## Look at the data using
featurePlot(trainingData$x, trainingData$y)
## This creates a list with a vector 'y' and a matrix
## of predictors 'x'. Also simulate a large test set to
## estimate the true error rate with good precision:
testData <- mlbench.friedman1(5000, sd = 1)
testData$x <- data.frame(testData$x)Tune several models on these data.
Which models appear to give the best performance?
The models with the best performance are
Multivariate Adaptive Regression Splines (MARS) model:
RMSE Rsquared MAE 1.1589948 0.9460418 0.9250230
Followed by:
Support Vector Machine(svmR) RMSE Rsquared MAE 1.1485881 0.6008914 0.9419527
Neural Networks (NN) RMSE Rsquared MAE 2.1113956 0.8277556 1.5739011
Does MARS select the informative predictors (those named X1-X5)?
Yes.
Overall
X1 100.00000
X4 75.23592
X2 48.72974
X5 15.51884
X3 0.00000
Book Example - KNN
knnModel <- train(x = trainingData$x,
y = trainingData$y,
method = "knn",
preProc = c("center", "scale"),
tuneLength = 10)
knnModel## k-Nearest Neighbors
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Bootstrapped (25 reps)
## Summary of sample sizes: 200, 200, 200, 200, 200, 200, ...
## Resampling results across tuning parameters:
##
## k RMSE Rsquared MAE
## 5 3.466085 0.5121775 2.816838
## 7 3.349428 0.5452823 2.727410
## 9 3.264276 0.5785990 2.660026
## 11 3.214216 0.6024244 2.603767
## 13 3.196510 0.6176570 2.591935
## 15 3.184173 0.6305506 2.577482
## 17 3.183130 0.6425367 2.567787
## 19 3.198752 0.6483184 2.592683
## 21 3.188993 0.6611428 2.588787
## 23 3.200458 0.6638353 2.604529
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 17.knnPred <- predict(knnModel, newdata = testData$x)
knn <- postResample(pred = knnPred, obs = testData$y)
knn## RMSE Rsquared MAE
## 3.2040595 0.6819919 2.5683461## loess r-squared variable importance
##
## Overall
## X4 100.0000
## X1 95.5047
## X2 89.6186
## X5 45.2170
## X3 29.9330
## X9 6.3299
## X10 5.5182
## X8 3.2527
## X6 0.8884
## X7 0.0000Neural Networks
Check for correlations:
## integer(0)Create a specific candidate set of models to evaluate:
nnetGrid <- expand.grid(decay = c(0, 0.01, .1),
size = c(1:10),
bag = FALSE)
ctrl <- trainControl(method = "cv", number=10)set.seed(100)
nnetTune <- train(x = trainingData$x,
y = trainingData$y,
method = "avNNet",
tuneGrid = nnetGrid,
trControl = ctrl,
preProc = c("center", "scale"),
linout = TRUE, trace = FALSE,
MaxNWts = 10 * (ncol(trainingData$x) + 1) + 10 + 1,
maxit = 500)## Warning: executing %dopar% sequentially: no parallel backend registered## Model Averaged Neural Network
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
## Resampling results across tuning parameters:
##
## decay size RMSE Rsquared MAE
## 0.00 1 2.392711 0.7610354 1.897330
## 0.00 2 2.410532 0.7567109 1.907478
## 0.00 3 2.043957 0.8224281 1.630751
## 0.00 4 2.289347 0.8130639 1.749187
## 0.00 5 2.445600 0.7709399 1.824446
## 0.00 6 2.898295 0.7388800 2.052725
## 0.00 7 3.351563 0.6644147 2.460366
## 0.00 8 6.513566 0.4418645 3.563297
## 0.00 9 4.484215 0.5644107 2.877950
## 0.00 10 3.422545 0.6247430 2.439739
## 0.01 1 2.385381 0.7602926 1.887906
## 0.01 2 2.425125 0.7510903 1.935991
## 0.01 3 2.151209 0.8016018 1.701951
## 0.01 4 2.091925 0.8154383 1.676653
## 0.01 5 2.169742 0.7999255 1.738715
## 0.01 6 2.262032 0.8056619 1.817195
## 0.01 7 2.318301 0.7861811 1.856908
## 0.01 8 2.413847 0.7772629 1.938009
## 0.01 9 2.317190 0.7847500 1.857641
## 0.01 10 2.480407 0.7408505 1.995656
## 0.10 1 2.393965 0.7596431 1.894191
## 0.10 2 2.423612 0.7525959 1.935872
## 0.10 3 2.169914 0.7982380 1.726854
## 0.10 4 2.059080 0.8224160 1.648610
## 0.10 5 1.975656 0.8394000 1.578979
## 0.10 6 2.152198 0.8098015 1.693056
## 0.10 7 2.161512 0.8163011 1.693526
## 0.10 8 2.273716 0.7922525 1.822713
## 0.10 9 2.315333 0.7811273 1.785409
## 0.10 10 2.334803 0.7692182 1.872733
##
## Tuning parameter 'bag' was held constant at a value of FALSE
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 5, decay = 0.1 and bag = FALSE.## Length Class Mode
## model 5 -none- list
## repeats 1 -none- numeric
## bag 1 -none- logical
## seeds 5 -none- numeric
## names 10 -none- character
## terms 3 terms call
## coefnames 10 -none- character
## xlevels 0 -none- list
## xNames 10 -none- character
## problemType 1 -none- character
## tuneValue 3 data.frame list
## obsLevels 1 -none- logical
## param 4 -none- list## size decay bag
## 25 5 0.1 FALSEnnetPred <- predict(nnetTune, newdata=testData$x)
NNET <- postResample(pred = nnetPred, obs = testData$y)
NNET## RMSE Rsquared MAE
## 2.1113956 0.8277556 1.5739011## plotmo grid: X1 X2 X3 X4 X5 X6 X7
## 0.5139349 0.5106664 0.537307 0.4445841 0.5343299 0.4975981 0.4688035
## X8 X9 X10
## 0.497961 0.5288716 0.5359218
## loess r-squared variable importance
##
## Overall
## X4 100.0000
## X1 95.5047
## X2 89.6186
## X5 45.2170
## X3 29.9330
## X9 6.3299
## X10 5.5182
## X8 3.2527
## X6 0.8884
## X7 0.0000Multivariate Adaptive Regression Splines (MARS)
Define the candidate models to test
set.seed(100)
marsGrid <- expand.grid(.degree = 1:2, .nprune = 2:38)
marsTuned <- train(x = trainingData$x, y = trainingData$y,
method = "earth",
tuneGrid = marsGrid,
trControl = ctrl)
marsTuned## Multivariate Adaptive Regression Spline
##
## 200 samples
## 10 predictor
##
## No pre-processing
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
## Resampling results across tuning parameters:
##
## degree nprune RMSE Rsquared MAE
## 1 2 4.327937 0.2544880 3.600474
## 1 3 3.572450 0.4912720 2.895811
## 1 4 2.596841 0.7183600 2.106341
## 1 5 2.370161 0.7659777 1.918669
## 1 6 2.276141 0.7881481 1.810001
## 1 7 1.766728 0.8751831 1.390215
## 1 8 1.780946 0.8723243 1.401345
## 1 9 1.665091 0.8819775 1.325515
## 1 10 1.663804 0.8821283 1.327657
## 1 11 1.657738 0.8822967 1.331730
## 1 12 1.653784 0.8827903 1.331504
## 1 13 1.648496 0.8823663 1.316407
## 1 14 1.639073 0.8841742 1.312833
## 1 15 1.639073 0.8841742 1.312833
## 1 16 1.639073 0.8841742 1.312833
## 1 17 1.639073 0.8841742 1.312833
## 1 18 1.639073 0.8841742 1.312833
## 1 19 1.639073 0.8841742 1.312833
## 1 20 1.639073 0.8841742 1.312833
## 1 21 1.639073 0.8841742 1.312833
## 1 22 1.639073 0.8841742 1.312833
## 1 23 1.639073 0.8841742 1.312833
## 1 24 1.639073 0.8841742 1.312833
## 1 25 1.639073 0.8841742 1.312833
## 1 26 1.639073 0.8841742 1.312833
## 1 27 1.639073 0.8841742 1.312833
## 1 28 1.639073 0.8841742 1.312833
## 1 29 1.639073 0.8841742 1.312833
## 1 30 1.639073 0.8841742 1.312833
## 1 31 1.639073 0.8841742 1.312833
## 1 32 1.639073 0.8841742 1.312833
## 1 33 1.639073 0.8841742 1.312833
## 1 34 1.639073 0.8841742 1.312833
## 1 35 1.639073 0.8841742 1.312833
## 1 36 1.639073 0.8841742 1.312833
## 1 37 1.639073 0.8841742 1.312833
## 1 38 1.639073 0.8841742 1.312833
## 2 2 4.327937 0.2544880 3.600474
## 2 3 3.572450 0.4912720 2.895811
## 2 4 2.661826 0.7070510 2.173471
## 2 5 2.404015 0.7578971 1.975387
## 2 6 2.243927 0.7914805 1.783072
## 2 7 1.856336 0.8605482 1.435682
## 2 8 1.754607 0.8763186 1.396841
## 2 9 1.603578 0.8938666 1.261361
## 2 10 1.492421 0.9084998 1.168700
## 2 11 1.317350 0.9292504 1.033926
## 2 12 1.304327 0.9320133 1.019108
## 2 13 1.277510 0.9323681 1.002927
## 2 14 1.269626 0.9350024 1.003346
## 2 15 1.266217 0.9359400 1.013893
## 2 16 1.268470 0.9354868 1.011414
## 2 17 1.268470 0.9354868 1.011414
## 2 18 1.268470 0.9354868 1.011414
## 2 19 1.268470 0.9354868 1.011414
## 2 20 1.268470 0.9354868 1.011414
## 2 21 1.268470 0.9354868 1.011414
## 2 22 1.268470 0.9354868 1.011414
## 2 23 1.268470 0.9354868 1.011414
## 2 24 1.268470 0.9354868 1.011414
## 2 25 1.268470 0.9354868 1.011414
## 2 26 1.268470 0.9354868 1.011414
## 2 27 1.268470 0.9354868 1.011414
## 2 28 1.268470 0.9354868 1.011414
## 2 29 1.268470 0.9354868 1.011414
## 2 30 1.268470 0.9354868 1.011414
## 2 31 1.268470 0.9354868 1.011414
## 2 32 1.268470 0.9354868 1.011414
## 2 33 1.268470 0.9354868 1.011414
## 2 34 1.268470 0.9354868 1.011414
## 2 35 1.268470 0.9354868 1.011414
## 2 36 1.268470 0.9354868 1.011414
## 2 37 1.268470 0.9354868 1.011414
## 2 38 1.268470 0.9354868 1.011414
##
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nprune = 15 and degree = 2.marsPred <- predict(marsTuned, newdata=testData$x)
MARS <- postResample(pred = marsPred, obs = testData$y)
MARS## RMSE Rsquared MAE
## 1.1589948 0.9460418 0.9250230## plotmo grid: X1 X2 X3 X4 X5 X6 X7
## 0.5139349 0.5106664 0.537307 0.4445841 0.5343299 0.4975981 0.4688035
## X8 X9 X10
## 0.497961 0.5288716 0.5359218
## earth variable importance
##
## Overall
## X1 100.00
## X4 75.24
## X2 48.73
## X5 15.52
## X3 0.00Support Vector Machines (SVM)
set.seed(100)
svmLTuned <- train(trainingData$x, trainingData$y,
method = "svmLinear",
preProc = c("center", "scale"),
tuneLength = 14,
trControl = trainControl(method = "cv"))
svmLTuned## Support Vector Machines with Linear Kernel
##
## 200 samples
## 10 predictor
##
## Pre-processing: centered (10), scaled (10)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
## Resampling results:
##
## RMSE Rsquared MAE
## 2.414092 0.7548203 1.965221
##
## Tuning parameter 'C' was held constant at a value of 1svmLPred <- predict(svmLTuned, newdata=testData$x)
svmL<- postResample(pred = svmLPred, obs = testData$y)
svmL## RMSE Rsquared MAE
## 2.7633860 0.6973384 2.0970616## plotmo grid: X1 X2 X3 X4 X5 X6 X7
## 0.5139349 0.5106664 0.537307 0.4445841 0.5343299 0.4975981 0.4688035
## X8 X9 X10
## 0.497961 0.5288716 0.5359218
## loess r-squared variable importance
##
## Overall
## X4 100.0000
## X1 95.5047
## X2 89.6186
## X5 45.2170
## X3 29.9330
## X9 6.3299
## X10 5.5182
## X8 3.2527
## X6 0.8884
## X7 0.00007.5.
Exercise 6.3 describes data for a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several nonlinear regression models.
Start R and use these commands to load the data:
## Warning: package 'AppliedPredictiveModeling' was built under R version 3.6.3The matrix processPredictors contains the 57 predictors (12 describing the input biological material and 45 describing the process predictors) for the 176 manufacturing runs. yield contains the percent yield for each run.
processPredictors <- ChemicalManufacturingProcess[2:58]
print(paste0("The number of columns is ", ncol(processPredictors), " and the number of rows is ", nrow(processPredictors)))## [1] "The number of columns is 57 and the number of rows is 176"missingData <- as.data.frame(colSums(is.na(processPredictors)))
colnames(missingData) <- c("NAs")
missingData <- cbind(Predictors = rownames(missingData), missingData)
rownames(missingData) <- 1:nrow(missingData)
missingData <- missingData[missingData$NAs != 0,] %>%
arrange(desc(NAs))
head(missingData)## Predictors NAs
## 1 ManufacturingProcess03 15
## 2 ManufacturingProcess11 10
## 3 ManufacturingProcess10 9
## 4 ManufacturingProcess25 5
## 5 ManufacturingProcess26 5
## 6 ManufacturingProcess27 5missingData %>%
ggplot() +
geom_bar(aes(x=Predictors, y=NAs,fill=factor(NAs)), stat = 'identity', ) +
labs(x='Predictor', y="NAs", title='Number of missing values') +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) + coord_flip() 
I used a kNN imputation strategy to fill in for the missing predictors I also used the default value of k=5.
set.seed(24)
knnImputedValues = preProcess(processPredictors, "knnImpute")
processPredictors <- try(predict(knnImputedValues, processPredictors), silent = TRUE)## Warning in kNN(processPredictors, variable = c(missingData$Predictors), :
## Nothing to impute, because no NA are present (also after using makeNA)missingData <- as.data.frame(colSums(is.na(processPredictors_imputed)))
colnames(missingData) <- c("NAs")
missingData <- cbind(Predictors = rownames(missingData), missingData)
rownames(missingData) <- 1:nrow(missingData)
missingData <- missingData[missingData$NAs != 0,]
head(missingData)## [1] Predictors NAs
## <0 rows> (or 0-length row.names)- Pre-Process the Data - Remove nearZeroVar predictors:
## [1] "BiologicalMaterial07"## [1] -0.1313107 -0.1313107 -0.1313107 -0.1313107 -0.1313107 -0.1313107
## [7] -0.1313107 -0.1313107 -0.1313107 -0.1313107 -0.1313107 -0.1313107
## [13] -0.1313107 -0.1313107 -0.1313107 -0.1313107 -0.1313107 -0.1313107
## [19] -0.1313107 -0.1313107- Pre-Process the Data - Check and remove Multi-Colinearity. We can remove these variables for multi-colinearity:
## [1] "BiologicalMaterial02" "BiologicalMaterial06" "BiologicalMaterial08"
## [4] "BiologicalMaterial01" "BiologicalMaterial04" "BiologicalMaterial12"
## [7] "ManufacturingProcess32" "ManufacturingProcess29" "ManufacturingProcess15"
## [10] "ManufacturingProcess09" "ManufacturingProcess13" "ManufacturingProcess14"
## [13] "ManufacturingProcess42" "ManufacturingProcess45" "ManufacturingProcess39"
## [16] "ManufacturingProcess26" "ManufacturingProcess25" "ManufacturingProcess31"
## [19] "ManufacturingProcess18" "ManufacturingProcess40"# Look at correlation between variables
corr <- round(cor(processPredictors_imputed), 1)
ggcorrplot(corr,
type="lower",
lab=TRUE,
lab_size=3,
method="circle",
colors=c("tomato2", "white", "springgreen3"),
title="Correlation of variables in Training Data Set",
ggtheme=theme_bw)
removePredictors <- findCorrelation(cor(processPredictors_imputed), 0.90, names = TRUE)
removePredictors## [1] "BiologicalMaterial02" "BiologicalMaterial04" "BiologicalMaterial12"
## [4] "ManufacturingProcess29" "ManufacturingProcess42" "ManufacturingProcess27"
## [7] "ManufacturingProcess25" "ManufacturingProcess31" "ManufacturingProcess18"
## [10] "ManufacturingProcess40"## [1] "BiologicalMaterial01" "BiologicalMaterial03" "BiologicalMaterial05"
## [4] "BiologicalMaterial06" "BiologicalMaterial08" "BiologicalMaterial09"
## [7] "BiologicalMaterial10" "BiologicalMaterial11" "ManufacturingProcess01"
## [10] "ManufacturingProcess02" "ManufacturingProcess03" "ManufacturingProcess04"
## [13] "ManufacturingProcess05" "ManufacturingProcess06" "ManufacturingProcess07"
## [16] "ManufacturingProcess08" "ManufacturingProcess09" "ManufacturingProcess10"
## [19] "ManufacturingProcess11" "ManufacturingProcess12" "ManufacturingProcess13"
## [22] "ManufacturingProcess14" "ManufacturingProcess15" "ManufacturingProcess16"
## [25] "ManufacturingProcess17" "ManufacturingProcess19" "ManufacturingProcess20"
## [28] "ManufacturingProcess21" "ManufacturingProcess22" "ManufacturingProcess23"
## [31] "ManufacturingProcess24" "ManufacturingProcess26" "ManufacturingProcess28"
## [34] "ManufacturingProcess30" "ManufacturingProcess32" "ManufacturingProcess33"
## [37] "ManufacturingProcess34" "ManufacturingProcess35" "ManufacturingProcess36"
## [40] "ManufacturingProcess37" "ManufacturingProcess38" "ManufacturingProcess39"
## [43] "ManufacturingProcess41" "ManufacturingProcess43" "ManufacturingProcess44"
## [46] "ManufacturingProcess45"chemmfgproc <- cbind(ChemicalManufacturingProcess$Yield, processPredictors_imputed)
names(chemmfgproc)[names(chemmfgproc) == "ChemicalManufacturingProcess$Yield"] <- "Yield"
head(chemmfgproc )## Yield BiologicalMaterial01 BiologicalMaterial03 BiologicalMaterial05
## 1 38.00 -0.2261036 -2.68303622 0.4941942
## 2 42.44 2.2391498 -0.05623504 0.4128555
## 3 42.03 2.2391498 -0.05623504 0.4128555
## 4 41.42 2.2391498 -0.05623504 0.4128555
## 5 42.49 1.4827653 1.13594780 -0.3734185
## 6 43.57 -0.4081962 -0.59859075 1.7305423
## BiologicalMaterial06 BiologicalMaterial08 BiologicalMaterial09
## 1 -1.3828880 -1.233131 -3.3962895
## 2 1.1290767 2.282619 -0.7227225
## 3 1.1290767 2.282619 -0.7227225
## 4 1.1290767 2.282619 -0.7227225
## 5 1.5348350 1.071310 -0.1205678
## 6 0.6192092 1.189487 -1.7343424
## BiologicalMaterial10 BiologicalMaterial11 ManufacturingProcess01
## 1 1.1005296 -1.838655 0.2154105
## 2 1.1005296 1.393395 -6.1497028
## 3 1.1005296 1.393395 -6.1497028
## 4 1.1005296 1.393395 -6.1497028
## 5 0.4162193 0.136256 -0.2784345
## 6 1.6346255 1.022062 0.4348971
## ManufacturingProcess02 ManufacturingProcess03 ManufacturingProcess04
## 1 0.5662872 0.3765810 0.5655598
## 2 -1.9692525 0.1979962 -2.3669726
## 3 -1.9692525 0.1087038 -3.1638563
## 4 -1.9692525 0.4658734 -3.3232331
## 5 -1.9692525 0.1087038 -2.2075958
## 6 -1.9692525 0.5551658 -1.2513352
## ManufacturingProcess05 ManufacturingProcess06 ManufacturingProcess07
## 1 -0.44593467 -0.5414997 -0.1596700
## 2 0.99933318 0.9625383 -0.9580199
## 3 0.06246417 -0.1117745 1.0378549
## 4 0.42279841 2.1850322 -0.9580199
## 5 0.84537219 -0.6304083 1.0378549
## 6 0.49486525 0.5550403 1.0378549
## ManufacturingProcess08 ManufacturingProcess09 ManufacturingProcess10
## 1 -0.3095182 -1.7201524 -0.07700901
## 2 0.8941637 0.5883746 0.52297397
## 3 0.8941637 -0.3815947 0.31428424
## 4 -1.1119728 -0.4785917 -0.02483658
## 5 0.8941637 -0.4527258 -0.39004361
## 6 0.8941637 -0.2199332 0.28819802
## ManufacturingProcess11 ManufacturingProcess12 ManufacturingProcess13
## 1 -0.09157342 -0.4806937 0.97711512
## 2 1.08204765 -0.4806937 -0.50030980
## 3 0.55112383 -0.4806937 0.28765016
## 4 0.80261406 -0.4806937 0.28765016
## 5 0.10403009 -0.4806937 0.09066017
## 6 1.41736795 -0.4806937 -0.50030980
## ManufacturingProcess14 ManufacturingProcess15 ManufacturingProcess16
## 1 0.8093999 1.1846438 0.3303945
## 2 0.2775205 0.9617071 0.1455765
## 3 0.4425865 0.8245152 0.1455765
## 4 0.7910592 1.0817499 0.1967569
## 5 2.5334227 3.3282665 0.4754056
## 6 2.4050380 3.1396277 0.6261033
## ManufacturingProcess17 ManufacturingProcess19 ManufacturingProcess20
## 1 0.9263296 0.4563798 0.3109942
## 2 -0.2753953 1.5095063 0.1849230
## 3 0.3655246 1.0926437 0.1849230
## 4 0.3655246 0.9829430 0.1562704
## 5 -0.3555103 1.6192070 0.2938027
## 6 -0.7560852 1.9044287 0.3998171
## ManufacturingProcess21 ManufacturingProcess22 ManufacturingProcess23
## 1 0.2109804 0.05833309 0.8317688
## 2 0.2109804 -0.72230090 -1.8147683
## 3 0.2109804 -0.42205706 -1.2132826
## 4 0.2109804 -0.12181322 -0.6117969
## 5 -0.6884239 0.77891831 0.5911745
## 6 -0.5599376 1.07916216 -1.2132826
## ManufacturingProcess24 ManufacturingProcess26 ManufacturingProcess28
## 1 0.8907291 0.1256347 0.7826636
## 2 -1.0060115 0.1966227 0.8779201
## 3 -0.8335805 0.2159831 0.8588688
## 4 -0.6611496 0.2052273 0.8588688
## 5 1.5804530 0.2912733 0.8969714
## 6 -1.3508734 0.2417969 0.9160227
## ManufacturingProcess30 ManufacturingProcess32 ManufacturingProcess33
## 1 0.7566948 -0.4568829 0.9890307
## 2 0.7566948 1.9517531 0.9890307
## 3 0.2444430 2.6928719 0.9890307
## 4 0.2444430 2.3223125 1.7943843
## 5 -0.1653585 2.3223125 2.5997378
## 6 0.9615956 2.6928719 2.5997378
## ManufacturingProcess34 ManufacturingProcess35 ManufacturingProcess36
## 1 -1.7202722 -0.88694718 -0.6557774
## 2 1.9568096 1.14638329 -0.6557774
## 3 1.9568096 1.23880740 -1.8000420
## 4 0.1182687 0.03729394 -1.8000420
## 5 0.1182687 -2.55058120 -2.9443066
## 6 0.1182687 -0.51725073 -1.8000420
## ManufacturingProcess37 ManufacturingProcess38 ManufacturingProcess39
## 1 -1.1540243 0.7174727 0.2317270
## 2 2.2161351 -0.8224687 0.2317270
## 3 -0.7046697 -0.8224687 0.2317270
## 4 0.4187168 -0.8224687 0.2317270
## 5 -1.8280562 -0.8224687 0.2981503
## 6 -1.3787016 -0.8224687 0.2317270
## ManufacturingProcess41 ManufacturingProcess43 ManufacturingProcess44
## 1 -0.06900773 2.40564734 -0.01588055
## 2 2.34626280 -0.01374656 0.29467248
## 3 -0.44058781 0.10146268 -0.01588055
## 4 -0.44058781 0.21667191 -0.01588055
## 5 -0.44058781 0.21667191 -0.32643359
## 6 -0.44058781 1.48397347 -0.01588055
## ManufacturingProcess45
## 1 0.64371849
## 2 0.15220242
## 3 0.39796046
## 4 -0.09355562
## 5 -0.09355562
## 6 -0.339313655. Split the data into Training and Test Set
chemmfgproc_train <- initial_split(chemmfgproc, prop = 0.8, strata = "Yield")
train_chemmfgproc <- training(chemmfgproc_train)
test_chemmfgproc <- testing(chemmfgproc_train)
print (paste0("The number of observations in the training set is ", nrow(train_chemmfgproc)))## [1] "The number of observations in the training set is 144"## [1] "The number of observations in the test set is 32"(a)
Which nonlinear regression model gives the optimal resampling and test set performance?
Define x and y from the training data:
trainingData_x <- train_chemmfgproc[2:47]
trainingData_y <- train_chemmfgproc$Yield
testData_x <- test_chemmfgproc[2:47]
testData_y <- test_chemmfgproc$Yieldknn
knnModel <- train(x = trainingData_x, y = trainingData_y,
method = "knn",
preProc = c("center", "scale"),
tuneLength = 10)
knnModel## k-Nearest Neighbors
##
## 144 samples
## 46 predictor
##
## Pre-processing: centered (46), scaled (46)
## Resampling: Bootstrapped (25 reps)
## Summary of sample sizes: 144, 144, 144, 144, 144, 144, ...
## Resampling results across tuning parameters:
##
## k RMSE Rsquared MAE
## 5 1.460664 0.4028876 1.126581
## 7 1.453214 0.4050023 1.130873
## 9 1.470496 0.3944659 1.158138
## 11 1.476206 0.3915436 1.170962
## 13 1.474888 0.3975906 1.170766
## 15 1.481775 0.3934704 1.184795
## 17 1.481231 0.3978893 1.187896
## 19 1.486936 0.3941312 1.197999
## 21 1.485838 0.4007873 1.196170
## 23 1.494588 0.3986187 1.204249
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 7.knnPred <- predict(knnModel, newdata = testData_x)
knn <- postResample(pred = knnPred, obs = testData_y)
knn## RMSE Rsquared MAE
## 1.3914277 0.4055338 1.1167857## loess r-squared variable importance
##
## only 20 most important variables shown (out of 46)
##
## Overall
## ManufacturingProcess13 100.00
## ManufacturingProcess32 97.55
## BiologicalMaterial06 90.98
## ManufacturingProcess17 90.24
## ManufacturingProcess09 83.51
## BiologicalMaterial03 80.05
## ManufacturingProcess06 69.35
## ManufacturingProcess36 67.70
## BiologicalMaterial11 62.52
## ManufacturingProcess11 56.51
## BiologicalMaterial08 51.74
## ManufacturingProcess30 50.42
## ManufacturingProcess33 48.03
## BiologicalMaterial01 42.98
## ManufacturingProcess02 41.50
## ManufacturingProcess12 34.65
## BiologicalMaterial09 33.77
## ManufacturingProcess04 28.67
## ManufacturingProcess01 24.02
## ManufacturingProcess24 23.98NN
Create a specific candidate set of models to evaluate:
nnetGrid <- expand.grid(decay = c(0, 0.01, .1),
size = c(1,5,10),
bag = FALSE)
ctrl <- trainControl(method = "cv", number=10)set.seed(600)
nnetTune <- train(x = trainingData_x,
y = trainingData_y,
method = "avNNet",
tuneGrid = nnetGrid,
trControl = ctrl,
preProc = c("center", "scale"),
linout = TRUE,
trace = FALSE,
maxit = 10)
nnetTune## Model Averaged Neural Network
##
## 144 samples
## 46 predictor
##
## Pre-processing: centered (46), scaled (46)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 129, 128, 130, 132, 130, 131, ...
## Resampling results across tuning parameters:
##
## decay size RMSE Rsquared MAE
## 0.00 1 2.079239 0.2509487 1.654390
## 0.00 5 1.855700 0.3441985 1.520017
## 0.00 10 4.777061 0.2217198 3.480252
## 0.01 1 1.839715 0.2228044 1.472172
## 0.01 5 1.646069 0.4079185 1.343367
## 0.01 10 4.601379 0.1698877 3.452087
## 0.10 1 2.044865 0.1434052 1.661691
## 0.10 5 1.945397 0.3373070 1.538292
## 0.10 10 5.343721 0.1277047 3.604875
##
## Tuning parameter 'bag' was held constant at a value of FALSE
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 5, decay = 0.01 and bag = FALSE.## size decay bag
## 5 5 0.01 FALSEnnetPred <- predict(nnetTune, newdata=testData_x)
NNET <- postResample(pred = nnetPred, obs = testData_y)
NNET## RMSE Rsquared MAE
## 1.9577545 0.3452134 1.3517244## plotmo grid: BiologicalMaterial01 BiologicalMaterial03 BiologicalMaterial05
## -0.07902882 -0.1137197 0.0386975
## BiologicalMaterial06 BiologicalMaterial08 BiologicalMaterial09
## -0.120232 0.02249382 -0.04830923
## BiologicalMaterial10 BiologicalMaterial11 ManufacturingProcess01
## -0.08449564 -0.1935873 0.1056672
## ManufacturingProcess02 ManufacturingProcess03 ManufacturingProcess04
## 0.5096271 0.4212272 0.3424324
## ManufacturingProcess05 ManufacturingProcess06 ManufacturingProcess07
## -0.08658317 -0.2970009 -0.9580199
## ManufacturingProcess08 ManufacturingProcess09 ManufacturingProcess10
## 0.8941637 0.00962621 -0.1030952
## ManufacturingProcess11 ManufacturingProcess12 ManufacturingProcess13
## 0.02020002 -0.4806937 0.09066017
## ManufacturingProcess14 ManufacturingProcess15 ManufacturingProcess16
## 0.07577316 -0.07580629 0.07449264
## ManufacturingProcess17 ManufacturingProcess19 ManufacturingProcess20
## 0.04506468 -0.05921344 0.07604324
## ManufacturingProcess21 ManufacturingProcess22 ManufacturingProcess23
## -0.1744786 -0.1218132 -0.01031118
## ManufacturingProcess24 ManufacturingProcess26 ManufacturingProcess28
## -0.1438567 0.07400713 0.7255096
## ManufacturingProcess30 ManufacturingProcess32 ManufacturingProcess33
## 0.03954225 -0.08632349 0.1434095
## ManufacturingProcess34 ManufacturingProcess35 ManufacturingProcess36
## 0.1182687 -0.05513017 -0.4269245
## ManufacturingProcess37 ManufacturingProcess38 ManufacturingProcess39
## -0.03063781 0.7174727 0.231727
## ManufacturingProcess41 ManufacturingProcess43 ManufacturingProcess44
## -0.4405878 -0.1289558 0.2946725
## ManufacturingProcess45
## 0.1522024
## loess r-squared variable importance
##
## only 20 most important variables shown (out of 46)
##
## Overall
## ManufacturingProcess13 100.00
## ManufacturingProcess32 97.55
## BiologicalMaterial06 90.98
## ManufacturingProcess17 90.24
## ManufacturingProcess09 83.51
## BiologicalMaterial03 80.05
## ManufacturingProcess06 69.35
## ManufacturingProcess36 67.70
## BiologicalMaterial11 62.52
## ManufacturingProcess11 56.51
## BiologicalMaterial08 51.74
## ManufacturingProcess30 50.42
## ManufacturingProcess33 48.03
## BiologicalMaterial01 42.98
## ManufacturingProcess02 41.50
## ManufacturingProcess12 34.65
## BiologicalMaterial09 33.77
## ManufacturingProcess04 28.67
## ManufacturingProcess01 24.02
## ManufacturingProcess24 23.98MARS
set.seed(100)
marsGrid <- expand.grid(.degree = 1:2, .nprune = 2:38)
marsTuned <- train(x = trainingData_x, y = trainingData_y,
method = "earth",
tuneGrid = marsGrid,
trControl = ctrl)
marsTuned## Multivariate Adaptive Regression Spline
##
## 144 samples
## 46 predictor
##
## No pre-processing
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 129, 130, 130, 130, 130, 130, ...
## Resampling results across tuning parameters:
##
## degree nprune RMSE Rsquared MAE
## 1 2 1.448515 0.4440339 1.1333246
## 1 3 1.224236 0.5721310 1.0058963
## 1 4 1.244391 0.5720038 0.9927964
## 1 5 1.253067 0.5550116 1.0024962
## 1 6 1.259736 0.5559211 1.0247774
## 1 7 1.294419 0.5368010 1.0514481
## 1 8 1.278354 0.5500121 1.0341437
## 1 9 1.268940 0.5686326 1.0407235
## 1 10 1.274156 0.5697414 1.0258097
## 1 11 1.278179 0.5725027 1.0248152
## 1 12 1.305037 0.5580925 1.0550739
## 1 13 1.304524 0.5578616 1.0650023
## 1 14 1.317963 0.5556105 1.0855094
## 1 15 1.333923 0.5473088 1.0998437
## 1 16 1.329814 0.5495838 1.0955290
## 1 17 1.346683 0.5450200 1.1115220
## 1 18 1.346683 0.5450200 1.1115220
## 1 19 1.346683 0.5450200 1.1115220
## 1 20 1.346683 0.5450200 1.1115220
## 1 21 1.346683 0.5450200 1.1115220
## 1 22 1.346683 0.5450200 1.1115220
## 1 23 1.346683 0.5450200 1.1115220
## 1 24 1.346683 0.5450200 1.1115220
## 1 25 1.346683 0.5450200 1.1115220
## 1 26 1.346683 0.5450200 1.1115220
## 1 27 1.346683 0.5450200 1.1115220
## 1 28 1.346683 0.5450200 1.1115220
## 1 29 1.346683 0.5450200 1.1115220
## 1 30 1.346683 0.5450200 1.1115220
## 1 31 1.346683 0.5450200 1.1115220
## 1 32 1.346683 0.5450200 1.1115220
## 1 33 1.346683 0.5450200 1.1115220
## 1 34 1.346683 0.5450200 1.1115220
## 1 35 1.346683 0.5450200 1.1115220
## 1 36 1.346683 0.5450200 1.1115220
## 1 37 1.346683 0.5450200 1.1115220
## 1 38 1.346683 0.5450200 1.1115220
## 2 2 1.448515 0.4440339 1.1333246
## 2 3 1.257926 0.5584563 1.0325833
## 2 4 1.310491 0.5473734 1.0297392
## 2 5 1.316233 0.5570648 1.0479154
## 2 6 1.348028 0.5367897 1.1023681
## 2 7 1.277303 0.5783217 1.0318610
## 2 8 1.232429 0.6100417 0.9845343
## 2 9 1.289296 0.5869412 1.0295541
## 2 10 1.249687 0.6138497 0.9920105
## 2 11 1.180129 0.6325412 0.9296795
## 2 12 1.141832 0.6425260 0.8979901
## 2 13 1.204524 0.6261197 0.9305343
## 2 14 1.202326 0.6338662 0.9308999
## 2 15 1.182341 0.6471334 0.9003371
## 2 16 1.187881 0.6503646 0.8961610
## 2 17 1.197772 0.6464490 0.9031182
## 2 18 1.197649 0.6456106 0.9055504
## 2 19 1.272723 0.6267058 0.9542015
## 2 20 1.259142 0.6314385 0.9437099
## 2 21 1.336659 0.6167057 0.9786114
## 2 22 1.362935 0.5987279 1.0013902
## 2 23 1.358780 0.6028033 0.9939780
## 2 24 1.355097 0.6053408 0.9902009
## 2 25 1.355097 0.6053408 0.9902009
## 2 26 1.355097 0.6053408 0.9902009
## 2 27 1.355097 0.6053408 0.9902009
## 2 28 1.355097 0.6053408 0.9902009
## 2 29 1.355097 0.6053408 0.9902009
## 2 30 1.355097 0.6053408 0.9902009
## 2 31 1.355097 0.6053408 0.9902009
## 2 32 1.355097 0.6053408 0.9902009
## 2 33 1.355097 0.6053408 0.9902009
## 2 34 1.355097 0.6053408 0.9902009
## 2 35 1.355097 0.6053408 0.9902009
## 2 36 1.355097 0.6053408 0.9902009
## 2 37 1.355097 0.6053408 0.9902009
## 2 38 1.355097 0.6053408 0.9902009
##
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nprune = 12 and degree = 2.marsPred <- predict(marsTuned, newdata=testData_x)
MARS <- postResample(pred = marsPred, obs = testData_y)
MARS## RMSE Rsquared MAE
## 1.847295 0.232907 1.128391## plotmo grid: BiologicalMaterial01 BiologicalMaterial03 BiologicalMaterial05
## -0.07902882 -0.1137197 0.0386975
## BiologicalMaterial06 BiologicalMaterial08 BiologicalMaterial09
## -0.120232 0.02249382 -0.04830923
## BiologicalMaterial10 BiologicalMaterial11 ManufacturingProcess01
## -0.08449564 -0.1935873 0.1056672
## ManufacturingProcess02 ManufacturingProcess03 ManufacturingProcess04
## 0.5096271 0.4212272 0.3424324
## ManufacturingProcess05 ManufacturingProcess06 ManufacturingProcess07
## -0.08658317 -0.2970009 -0.9580199
## ManufacturingProcess08 ManufacturingProcess09 ManufacturingProcess10
## 0.8941637 0.00962621 -0.1030952
## ManufacturingProcess11 ManufacturingProcess12 ManufacturingProcess13
## 0.02020002 -0.4806937 0.09066017
## ManufacturingProcess14 ManufacturingProcess15 ManufacturingProcess16
## 0.07577316 -0.07580629 0.07449264
## ManufacturingProcess17 ManufacturingProcess19 ManufacturingProcess20
## 0.04506468 -0.05921344 0.07604324
## ManufacturingProcess21 ManufacturingProcess22 ManufacturingProcess23
## -0.1744786 -0.1218132 -0.01031118
## ManufacturingProcess24 ManufacturingProcess26 ManufacturingProcess28
## -0.1438567 0.07400713 0.7255096
## ManufacturingProcess30 ManufacturingProcess32 ManufacturingProcess33
## 0.03954225 -0.08632349 0.1434095
## ManufacturingProcess34 ManufacturingProcess35 ManufacturingProcess36
## 0.1182687 -0.05513017 -0.4269245
## ManufacturingProcess37 ManufacturingProcess38 ManufacturingProcess39
## -0.03063781 0.7174727 0.231727
## ManufacturingProcess41 ManufacturingProcess43 ManufacturingProcess44
## -0.4405878 -0.1289558 0.2946725
## ManufacturingProcess45
## 0.1522024
## earth variable importance
##
## Overall
## ManufacturingProcess32 100.000
## ManufacturingProcess09 63.492
## ManufacturingProcess24 31.634
## ManufacturingProcess39 28.781
## ManufacturingProcess17 20.797
## BiologicalMaterial09 12.950
## ManufacturingProcess33 12.950
## ManufacturingProcess44 5.424
## ManufacturingProcess28 0.000svmR
set.seed(100)
svmRTuned <- train(trainingData_x, trainingData_y,
method = "svmRadial",
preProc = c("center", "scale"),
tuneLength = 14,
trControl = trainControl(method = "cv"))
svmRTuned ## Support Vector Machines with Radial Basis Function Kernel
##
## 144 samples
## 46 predictor
##
## Pre-processing: centered (46), scaled (46)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 129, 130, 130, 130, 130, 130, ...
## Resampling results across tuning parameters:
##
## C RMSE Rsquared MAE
## 0.25 1.402333 0.5152740 1.1540993
## 0.50 1.281362 0.5640935 1.0580769
## 1.00 1.153024 0.6367293 0.9351356
## 2.00 1.119224 0.6530985 0.8848283
## 4.00 1.130527 0.6382115 0.8860747
## 8.00 1.165579 0.6142273 0.9275068
## 16.00 1.166279 0.6130560 0.9331774
## 32.00 1.166279 0.6130560 0.9331774
## 64.00 1.166279 0.6130560 0.9331774
## 128.00 1.166279 0.6130560 0.9331774
## 256.00 1.166279 0.6130560 0.9331774
## 512.00 1.166279 0.6130560 0.9331774
## 1024.00 1.166279 0.6130560 0.9331774
## 2048.00 1.166279 0.6130560 0.9331774
##
## Tuning parameter 'sigma' was held constant at a value of 0.01251632
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.01251632 and C = 2.svmRPred <- predict(svmRTuned, newdata=testData_x)
svmR<- postResample(pred = svmRPred, obs = testData_y)
svmR## RMSE Rsquared MAE
## 1.1485881 0.6008914 0.9419527## plotmo grid: BiologicalMaterial01 BiologicalMaterial03 BiologicalMaterial05
## -0.07902882 -0.1137197 0.0386975
## BiologicalMaterial06 BiologicalMaterial08 BiologicalMaterial09
## -0.120232 0.02249382 -0.04830923
## BiologicalMaterial10 BiologicalMaterial11 ManufacturingProcess01
## -0.08449564 -0.1935873 0.1056672
## ManufacturingProcess02 ManufacturingProcess03 ManufacturingProcess04
## 0.5096271 0.4212272 0.3424324
## ManufacturingProcess05 ManufacturingProcess06 ManufacturingProcess07
## -0.08658317 -0.2970009 -0.9580199
## ManufacturingProcess08 ManufacturingProcess09 ManufacturingProcess10
## 0.8941637 0.00962621 -0.1030952
## ManufacturingProcess11 ManufacturingProcess12 ManufacturingProcess13
## 0.02020002 -0.4806937 0.09066017
## ManufacturingProcess14 ManufacturingProcess15 ManufacturingProcess16
## 0.07577316 -0.07580629 0.07449264
## ManufacturingProcess17 ManufacturingProcess19 ManufacturingProcess20
## 0.04506468 -0.05921344 0.07604324
## ManufacturingProcess21 ManufacturingProcess22 ManufacturingProcess23
## -0.1744786 -0.1218132 -0.01031118
## ManufacturingProcess24 ManufacturingProcess26 ManufacturingProcess28
## -0.1438567 0.07400713 0.7255096
## ManufacturingProcess30 ManufacturingProcess32 ManufacturingProcess33
## 0.03954225 -0.08632349 0.1434095
## ManufacturingProcess34 ManufacturingProcess35 ManufacturingProcess36
## 0.1182687 -0.05513017 -0.4269245
## ManufacturingProcess37 ManufacturingProcess38 ManufacturingProcess39
## -0.03063781 0.7174727 0.231727
## ManufacturingProcess41 ManufacturingProcess43 ManufacturingProcess44
## -0.4405878 -0.1289558 0.2946725
## ManufacturingProcess45
## 0.1522024
## loess r-squared variable importance
##
## only 20 most important variables shown (out of 46)
##
## Overall
## ManufacturingProcess13 100.00
## ManufacturingProcess32 97.55
## BiologicalMaterial06 90.98
## ManufacturingProcess17 90.24
## ManufacturingProcess09 83.51
## BiologicalMaterial03 80.05
## ManufacturingProcess06 69.35
## ManufacturingProcess36 67.70
## BiologicalMaterial11 62.52
## ManufacturingProcess11 56.51
## BiologicalMaterial08 51.74
## ManufacturingProcess30 50.42
## ManufacturingProcess33 48.03
## BiologicalMaterial01 42.98
## ManufacturingProcess02 41.50
## ManufacturingProcess12 34.65
## BiologicalMaterial09 33.77
## ManufacturingProcess04 28.67
## ManufacturingProcess01 24.02
## ManufacturingProcess24 23.98Comparison of RMSE
RMSE_Comparison <- as.data.frame(rbind(c("KNN",knn),c("NNET",NNET),c("SVMR",svmR), c("MARS", MARS)))
colnames(RMSE_Comparison) <- c("Non Linear Model", "RMSE", "Rsquared", "MAE")
RMSE_Comparison %>%
arrange(RMSE)## Non Linear Model RMSE Rsquared MAE
## 1 SVMR 1.14858805281709 0.600891407446318 0.941952746423919
## 2 KNN 1.39142774640632 0.405533849581907 1.11678571428571
## 3 MARS 1.847294521408 0.232906996262002 1.12839058974759
## 4 NNET 1.95775447531253 0.345213412346812 1.35172435279523(b)
Which predictors are most important in the optimal nonlinear regression model?
The top ten most important predictors from the optimal nonlinear regression model, svmRadial, are:
ManufacturingProcess13 100.00000
ManufacturingProcess32 97.55308
BiologicalMaterial06 90.97712
ManufacturingProcess17 90.23700
ManufacturingProcess09 83.51391
BiologicalMaterial03 80.05192
ManufacturingProcess06 69.35247
ManufacturingProcess36 67.70372
BiologicalMaterial11 62.51836
ManufacturingProcess11 56.50844
Do either the biological or process variables dominate the list?
The manufacturing processes account for 7 of the top ten variables while the biological ones only account for three.
How do the top ten important predictors compare to the top ten predictors from the optimal linear model?
The top ten important predictors from the optimal non-linear model were somewhat identical to the optimal linear model, partial least squares with some major differences. The most important predictor was different. Biological material 11 and Manufacturing Process 11 were on the list for the optimal non-linear model, but it wasn’t on the list for the optimal linear model which had Biological material 08 and Manufacturing Process 33.
ManufacturingProcess32 100.00000
ManufacturingProcess09 92.24799
ManufacturingProcess13 89.29483
ManufacturingProcess17 79.75857
ManufacturingProcess36 76.58516
BiologicalMaterial06 71.80696
BiologicalMaterial08 68.31074
ManufacturingProcess06 65.80944
ManufacturingProcess33 62.39126
BiologicalMaterial03 60.56164
SVM
## loess r-squared variable importance
##
## only 20 most important variables shown (out of 46)
##
## Overall
## ManufacturingProcess13 100.00
## ManufacturingProcess32 97.55
## BiologicalMaterial06 90.98
## ManufacturingProcess17 90.24
## ManufacturingProcess09 83.51
## BiologicalMaterial03 80.05
## ManufacturingProcess06 69.35
## ManufacturingProcess36 67.70
## BiologicalMaterial11 62.52
## ManufacturingProcess11 56.51
## BiologicalMaterial08 51.74
## ManufacturingProcess30 50.42
## ManufacturingProcess33 48.03
## BiologicalMaterial01 42.98
## ManufacturingProcess02 41.50
## ManufacturingProcess12 34.65
## BiologicalMaterial09 33.77
## ManufacturingProcess04 28.67
## ManufacturingProcess01 24.02
## ManufacturingProcess24 23.98KNN
## loess r-squared variable importance
##
## only 20 most important variables shown (out of 46)
##
## Overall
## ManufacturingProcess13 100.00
## ManufacturingProcess32 97.55
## BiologicalMaterial06 90.98
## ManufacturingProcess17 90.24
## ManufacturingProcess09 83.51
## BiologicalMaterial03 80.05
## ManufacturingProcess06 69.35
## ManufacturingProcess36 67.70
## BiologicalMaterial11 62.52
## ManufacturingProcess11 56.51
## BiologicalMaterial08 51.74
## ManufacturingProcess30 50.42
## ManufacturingProcess33 48.03
## BiologicalMaterial01 42.98
## ManufacturingProcess02 41.50
## ManufacturingProcess12 34.65
## BiologicalMaterial09 33.77
## ManufacturingProcess04 28.67
## ManufacturingProcess01 24.02
## ManufacturingProcess24 23.98NNET
## loess r-squared variable importance
##
## only 20 most important variables shown (out of 46)
##
## Overall
## ManufacturingProcess13 100.00
## ManufacturingProcess32 97.55
## BiologicalMaterial06 90.98
## ManufacturingProcess17 90.24
## ManufacturingProcess09 83.51
## BiologicalMaterial03 80.05
## ManufacturingProcess06 69.35
## ManufacturingProcess36 67.70
## BiologicalMaterial11 62.52
## ManufacturingProcess11 56.51
## BiologicalMaterial08 51.74
## ManufacturingProcess30 50.42
## ManufacturingProcess33 48.03
## BiologicalMaterial01 42.98
## ManufacturingProcess02 41.50
## ManufacturingProcess12 34.65
## BiologicalMaterial09 33.77
## ManufacturingProcess04 28.67
## ManufacturingProcess01 24.02
## ManufacturingProcess24 23.98MARS
## earth variable importance
##
## Overall
## ManufacturingProcess32 100.000
## ManufacturingProcess09 63.492
## ManufacturingProcess24 31.634
## ManufacturingProcess39 28.781
## ManufacturingProcess17 20.797
## BiologicalMaterial09 12.950
## ManufacturingProcess33 12.950
## ManufacturingProcess44 5.424
## ManufacturingProcess28 0.000(c)
Explore the relationships between the top predictors and the response for the predictors that are unique to the optimal nonlinear regression model. Do these plots reveal intuition about the biological or process predictors and their relationship with yield?
As stated earlier, there were two important predictors that were unique to the optimal nonlinear regression model Manufacturing Process 11 and Biological Material 11. When each is plotted against the response variable, Yield, there is a very weak correlation between the two. There does not seem to be any intuition revealed.
## [1] 0.3525799## [1] 0.3549143
## Warning: Removed 10 rows containing missing values (geom_point).
LS0tDQp0aXRsZTogIkRBVEE2MjQgU3ByaW5nIDIwMjEgSFc4IC0gTm9uIExpbmVhciBSZWdyZXNzaW9uIg0KYXV0aG9yOiAiSm9obiBLLiBIYW5jb2NrIg0KZGF0ZTogIjQvMjUvMjAyMSINCm91dHB1dDoNCiAgaHRtbF9kb2N1bWVudDoNCiAgICBjb2RlX2Rvd25sb2FkOiB5ZXMNCiAgICBjb2RlX2ZvbGRpbmc6IHNob3cNCiAgICBoaWdobGlnaHQ6IHB5Z21lbnRzDQogICAgbnVtYmVyX3NlY3Rpb25zOiBubw0KICAgIHRoZW1lOiBjZXJ1bGVhbg0KICAgIHRvYzogeWVzDQogICAgdG9jX2Zsb2F0OiB5ZXMNCiAgcGRmX2RvY3VtZW50Og0KICAgIHRvYzogeWVzDQotLS0NCg0KYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9DQprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IFRSVUUpDQpgYGANCg0KDQpgYGB7ciwgaW5jbHVkZT1GQUxTRSwgd2FybmluZz1GQUxTRX0NCmxpYnJhcnkodGlkeXZlcnNlKQ0KbGlicmFyeShnZ3Bsb3QyKQ0KbGlicmFyeShnZ2NvcnJwbG90KQ0KbGlicmFyeShjYXJldCkNCmxpYnJhcnkoZWxhc3RpY25ldCkNCmxpYnJhcnkobGFycykNCmxpYnJhcnkocGxzKQ0KbGlicmFyeShuYW5pYXIpDQpsaWJyYXJ5KGhlYXRtYXBseSkNCmxpYnJhcnkoVklNKQ0KbGlicmFyeShJQ1NOUCkNCmxpYnJhcnkocnNhbXBsZSkNCmxpYnJhcnkoZ2xtbmV0KQ0KbGlicmFyeSgibWljZSIpDQpsaWJyYXJ5KCJlMTA3MSIpDQpsaWJyYXJ5KFJBTk4pDQpsaWJyYXJ5KGVhcnRoKQ0KbGlicmFyeShrZXJubGFiKQ0KbGlicmFyeShubmV0KQ0KDQoNCmBgYA0KDQoNCiMjIDcuMi4gDQoNCjxpPkZyaWVkbWFuICgxOTkxKSBpbnRyb2R1Y2VkIHNldmVyYWwgYmVuY2htYXJrIGRhdGEgc2V0cyBjcmVhdGUgYnkgc2ltdWxhdGlvbi4gT25lIG9mIHRoZXNlIHNpbXVsYXRpb25zIHVzZWQgdGhlIGZvbGxvd2luZyBub25saW5lYXIgZXF1YXRpb24gdG8NCmNyZWF0ZSBkYXRhOjwvaT4NCg0KJCR5ID0gMTBzaW4oXHBpIHhfMXhfMikgKyAyMCh4XzMtMC41KV4yICsgMTB4XzQrNXhfNStOKDAsXHNpZ21hXjIpJCQNCg0KPGk+d2hlcmUgdGhlIHggdmFsdWVzIGFyZSByYW5kb20gdmFyaWFibGVzIHVuaWZvcm1seSBkaXN0cmlidXRlZCBiZXR3ZWVuIFswLCAxXSAodGhlcmUgYXJlIGFsc28gNSBvdGhlciBub24taW5mb3JtYXRpdmUgdmFyaWFibGVzIGFsc28gY3JlYXRlZCBpbiB0aGUgc2ltdWxhdGlvbikuIFRoZSBwYWNrYWdlIG1sYmVuY2ggY29udGFpbnMgYSBmdW5jdGlvbiBjYWxsZWQgbWxiZW5jaC5mcmllZG1hbjEgdGhhdCBzaW11bGF0ZXMgdGhlc2UgZGF0YTo8L2k+DQoNCmBgYHtyfQ0KbGlicmFyeShtbGJlbmNoKQ0Kc2V0LnNlZWQoMjAwKQ0KdHJhaW5pbmdEYXRhIDwtIG1sYmVuY2guZnJpZWRtYW4xKDIwMCwgc2QgPSAxKQ0KIyMgV2UgY29udmVydCB0aGUgJ3gnIGRhdGEgZnJvbSBhIG1hdHJpeCB0byBhIGRhdGEgZnJhbWUNCiMjIE9uZSByZWFzb24gaXMgdGhhdCB0aGlzIHdpbGwgZ2l2ZSB0aGUgY29sdW1ucyBuYW1lcy4NCnRyYWluaW5nRGF0YSR4IDwtIGRhdGEuZnJhbWUodHJhaW5pbmdEYXRhJHgpDQojIyBMb29rIGF0IHRoZSBkYXRhIHVzaW5nDQpmZWF0dXJlUGxvdCh0cmFpbmluZ0RhdGEkeCwgdHJhaW5pbmdEYXRhJHkpDQojIyBvciBvdGhlciBtZXRob2RzLg0KDQoNCmBgYA0KDQoNCg0KYGBge3J9DQojIyBUaGlzIGNyZWF0ZXMgYSBsaXN0IHdpdGggYSB2ZWN0b3IgJ3knIGFuZCBhIG1hdHJpeA0KIyMgb2YgcHJlZGljdG9ycyAneCcuIEFsc28gc2ltdWxhdGUgYSBsYXJnZSB0ZXN0IHNldCB0bw0KIyMgZXN0aW1hdGUgdGhlIHRydWUgZXJyb3IgcmF0ZSB3aXRoIGdvb2QgcHJlY2lzaW9uOg0KdGVzdERhdGEgPC0gbWxiZW5jaC5mcmllZG1hbjEoNTAwMCwgc2QgPSAxKQ0KdGVzdERhdGEkeCA8LSBkYXRhLmZyYW1lKHRlc3REYXRhJHgpDQpgYGANCjxpPlR1bmUgc2V2ZXJhbCBtb2RlbHMgb24gdGhlc2UgZGF0YS48L2k+PGJyIC8+DQo8aT5XaGljaCBtb2RlbHMgYXBwZWFyIHRvIGdpdmUgdGhlIGJlc3QgcGVyZm9ybWFuY2U/PC9pPjxiciAvPg0KDQpUaGUgbW9kZWxzIHdpdGggdGhlIGJlc3QgcGVyZm9ybWFuY2UgYXJlDQoNCk11bHRpdmFyaWF0ZSBBZGFwdGl2ZSBSZWdyZXNzaW9uIFNwbGluZXMgKE1BUlMpIG1vZGVsOg0KDQogICAgIFJNU0UgIFJzcXVhcmVkICAgICAgIE1BRSANCjEuMTU4OTk0OCAwLjk0NjA0MTggMC45MjUwMjMwIA0KDQoNCkZvbGxvd2VkIGJ5Og0KDQpTdXBwb3J0IFZlY3RvciBNYWNoaW5lKHN2bVIpDQogICAgIFJNU0UgIFJzcXVhcmVkICAgICAgIE1BRSANCjEuMTQ4NTg4MSAwLjYwMDg5MTQgMC45NDE5NTI3IA0KDQoNCk5ldXJhbCBOZXR3b3JrcyAoTk4pDQogICAgIFJNU0UgIFJzcXVhcmVkICAgICAgIE1BRSANCjIuMTExMzk1NiAwLjgyNzc1NTYgMS41NzM5MDExIA0KDQoNCg0KPGk+RG9lcyBNQVJTIHNlbGVjdCB0aGUgaW5mb3JtYXRpdmUgcHJlZGljdG9ycyAodGhvc2UgbmFtZWQgWDEtWDUpPzwvaT4NCg0KWWVzLiANCg0KCQ0KT3ZlcmFsbA0KDQpYMQkxMDAuMDAwMDAJCQkNClg0CTc1LjIzNTkyCQkJDQpYMgk0OC43Mjk3NAkJCQ0KWDUJMTUuNTE4ODQJCQkNClgzCTAuMDAwMDAJDQoNCg0KIyMjIEJvb2sgRXhhbXBsZSAtIEtOTiANCg0KYGBge3J9DQprbm5Nb2RlbCA8LSB0cmFpbih4ID0gdHJhaW5pbmdEYXRhJHgsDQogeSA9IHRyYWluaW5nRGF0YSR5LA0KIG1ldGhvZCA9ICJrbm4iLA0KIHByZVByb2MgPSBjKCJjZW50ZXIiLCAic2NhbGUiKSwNCiB0dW5lTGVuZ3RoID0gMTApDQprbm5Nb2RlbA0KYGBgDQoNCg0KYGBge3J9DQprbm5QcmVkIDwtIHByZWRpY3Qoa25uTW9kZWwsIG5ld2RhdGEgPSB0ZXN0RGF0YSR4KQ0KDQprbm4gPC0gcG9zdFJlc2FtcGxlKHByZWQgPSBrbm5QcmVkLCBvYnMgPSB0ZXN0RGF0YSR5KQ0Ka25uDQpgYGANCg0KYGBge3J9DQp2YXJJbXAoa25uTW9kZWwpDQpgYGANCg0KDQoNCiMjIyBOZXVyYWwgTmV0d29ya3MNCg0KQ2hlY2sgZm9yIGNvcnJlbGF0aW9uczoNCg0KYGBge3J9DQp0b29IaWdoIDwtIGZpbmRDb3JyZWxhdGlvbihjb3IodHJhaW5pbmdEYXRhJHgpLCBjdXRvZmYgPSAuNzUpDQp0b29IaWdoDQpgYGANCg0KQ3JlYXRlIGEgc3BlY2lmaWMgY2FuZGlkYXRlIHNldCBvZiBtb2RlbHMgdG8gZXZhbHVhdGU6DQoNCmBgYHtyfQ0Kbm5ldEdyaWQgPC0gZXhwYW5kLmdyaWQoZGVjYXkgPSBjKDAsIDAuMDEsIC4xKSwNCiAgICAgICAgICAgICAgICAgICAgICAgIHNpemUgPSBjKDE6MTApLA0KICAgICAgICAgICAgICAgICAgICAgICAgYmFnID0gRkFMU0UpDQoNCmN0cmwgPC0gdHJhaW5Db250cm9sKG1ldGhvZCA9ICJjdiIsIG51bWJlcj0xMCkNCg0KDQpgYGANCg0KDQpgYGB7cn0NCnNldC5zZWVkKDEwMCkNCg0KDQpubmV0VHVuZSA8LSB0cmFpbih4ID0gdHJhaW5pbmdEYXRhJHgsIA0KICAgICAgICAgICAgICAgICAgeSA9IHRyYWluaW5nRGF0YSR5LCAgDQogICAgICAgICAgICAgICAgICBtZXRob2QgPSAiYXZOTmV0IiwgIA0KICAgICAgICAgICAgICAgICAgdHVuZUdyaWQgPSBubmV0R3JpZCwgIA0KICAgICAgICAgICAgICAgICAgdHJDb250cm9sID0gY3RybCwNCiAgICAgICAgICAgICAgICAgIHByZVByb2MgPSBjKCJjZW50ZXIiLCAic2NhbGUiKSwgIA0KICAgICAgICAgICAgICAgICAgbGlub3V0ID0gVFJVRSwgIHRyYWNlID0gRkFMU0UsICANCiAgICAgICAgICAgICAgICAgIE1heE5XdHMgPSAxMCAqIChuY29sKHRyYWluaW5nRGF0YSR4KSArIDEpICsgMTAgKyAxLCANCiAgICAgICAgICAgICAgICAgIG1heGl0ID0gNTAwKQ0KICAgICAgICAgICAgICAgICAgDQpgYGANCg0KYGBge3J9DQpubmV0VHVuZQ0KYGBgDQoNCmBgYHtyfQ0Kc3VtbWFyeShubmV0VHVuZSkNCmBgYA0KDQoNCmBgYHtyfQ0Kbm5ldFR1bmUkYmVzdFR1bmUNCmBgYA0KDQoNCmBgYHtyfQ0Kbm5ldFByZWQgPC0gcHJlZGljdChubmV0VHVuZSwgbmV3ZGF0YT10ZXN0RGF0YSR4KQ0KTk5FVCA8LSBwb3N0UmVzYW1wbGUocHJlZCA9IG5uZXRQcmVkLCBvYnMgPSB0ZXN0RGF0YSR5KQ0KTk5FVA0KYGBgDQoNCmBgYHtyfQ0KcGxvdG1vKG5uZXRUdW5lKQ0KYGBgDQoNCmBgYHtyfQ0KdmFySW1wKG5uZXRUdW5lKQ0KYGBgDQoNCg0KIyMjIE11bHRpdmFyaWF0ZSBBZGFwdGl2ZSBSZWdyZXNzaW9uIFNwbGluZXMgKE1BUlMpDQoNCg0KRGVmaW5lIHRoZSBjYW5kaWRhdGUgbW9kZWxzIHRvIHRlc3QNCg0KYGBge3J9DQpzZXQuc2VlZCgxMDApDQptYXJzR3JpZCA8LSBleHBhbmQuZ3JpZCguZGVncmVlID0gMToyLCAubnBydW5lID0gMjozOCkNCm1hcnNUdW5lZCA8LSB0cmFpbih4ID0gdHJhaW5pbmdEYXRhJHgsIHkgPSB0cmFpbmluZ0RhdGEkeSwNCiAgICAgICAgICAgICAgICAgIG1ldGhvZCA9ICJlYXJ0aCIsIA0KICAgICAgICAgICAgICAgICAgdHVuZUdyaWQgPSBtYXJzR3JpZCwNCiAgICAgICAgICAgICAgICAgIHRyQ29udHJvbCA9IGN0cmwpDQptYXJzVHVuZWQNCmBgYA0KDQpgYGB7cn0NCm1hcnNQcmVkIDwtIHByZWRpY3QobWFyc1R1bmVkLCBuZXdkYXRhPXRlc3REYXRhJHgpDQpNQVJTIDwtIHBvc3RSZXNhbXBsZShwcmVkID0gbWFyc1ByZWQsIG9icyA9IHRlc3REYXRhJHkpDQpNQVJTDQpgYGANCg0KDQpgYGB7cn0NCnBsb3RtbyhtYXJzVHVuZWQpDQpgYGANCg0KYGBge3J9DQp2YXJJbXAobWFyc1R1bmVkKQ0KYGBgDQoNCg0KDQoNCiMjIyBTdXBwb3J0IFZlY3RvciBNYWNoaW5lcyAoU1ZNKQ0KDQpgYGB7cn0NCnNldC5zZWVkKDEwMCkNCnN2bUxUdW5lZCA8LSB0cmFpbih0cmFpbmluZ0RhdGEkeCwgdHJhaW5pbmdEYXRhJHksDQogICAgICAgICAgICAgICAgICAgbWV0aG9kID0gInN2bUxpbmVhciIsDQogICAgICAgICAgICAgICAgICAgcHJlUHJvYyA9IGMoImNlbnRlciIsICJzY2FsZSIpLA0KICAgICAgICAgICAgICAgICAgIHR1bmVMZW5ndGggPSAxNCwNCiAgICAgICAgICAgICAgICAgICB0ckNvbnRyb2wgPSB0cmFpbkNvbnRyb2wobWV0aG9kID0gImN2IikpDQpzdm1MVHVuZWQNCmBgYA0KDQoNCmBgYHtyfQ0Kc3ZtTFByZWQgPC0gcHJlZGljdChzdm1MVHVuZWQsIG5ld2RhdGE9dGVzdERhdGEkeCkNCnN2bUw8LSBwb3N0UmVzYW1wbGUocHJlZCA9IHN2bUxQcmVkLCBvYnMgPSB0ZXN0RGF0YSR5KQ0Kc3ZtTA0KYGBgDQoNCg0KYGBge3J9DQpwbG90bW8oc3ZtTFR1bmVkKQ0KYGBgDQoNCg0KDQpgYGB7cn0NCnZhckltcChzdm1MVHVuZWQpDQpgYGANCg0KDQoNCiMjIDcuNS4NCg0KRXhlcmNpc2UgNi4zIGRlc2NyaWJlcyBkYXRhIGZvciBhIGNoZW1pY2FsIG1hbnVmYWN0dXJpbmcgcHJvY2Vzcy4gVXNlIHRoZSBzYW1lIGRhdGEgaW1wdXRhdGlvbiwgZGF0YSBzcGxpdHRpbmcsIGFuZCBwcmUtcHJvY2Vzc2luZyBzdGVwcyBhcyBiZWZvcmUNCmFuZCB0cmFpbiBzZXZlcmFsIG5vbmxpbmVhciByZWdyZXNzaW9uIG1vZGVscy4NCg0KDQpTdGFydCBSIGFuZCB1c2UgdGhlc2UgY29tbWFuZHMgdG8gbG9hZCB0aGUgZGF0YToNCg0KYGBge3IgbWVzc2FnZT1GQUxTRX0NCmxpYnJhcnkoQXBwbGllZFByZWRpY3RpdmVNb2RlbGluZykNCmRhdGEoQ2hlbWljYWxNYW51ZmFjdHVyaW5nUHJvY2VzcykNCmBgYA0KDQpUaGUgbWF0cml4IHByb2Nlc3NQcmVkaWN0b3JzIGNvbnRhaW5zIHRoZSA1NyBwcmVkaWN0b3JzICgxMiBkZXNjcmliaW5nIHRoZSBpbnB1dCBiaW9sb2dpY2FsIG1hdGVyaWFsIGFuZCA0NSBkZXNjcmliaW5nIHRoZSBwcm9jZXNzIHByZWRpY3RvcnMpIGZvciB0aGUgMTc2IG1hbnVmYWN0dXJpbmcgcnVucy4geWllbGQgY29udGFpbnMgdGhlIHBlcmNlbnQgeWllbGQgZm9yIGVhY2ggcnVuLg0KDQpgYGB7cn0NCnByb2Nlc3NQcmVkaWN0b3JzIDwtICBDaGVtaWNhbE1hbnVmYWN0dXJpbmdQcm9jZXNzWzI6NThdDQpwcmludChwYXN0ZTAoIlRoZSBudW1iZXIgb2YgY29sdW1ucyBpcyAiLCBuY29sKHByb2Nlc3NQcmVkaWN0b3JzKSwgIiBhbmQgdGhlIG51bWJlciBvZiByb3dzIGlzICIsIG5yb3cocHJvY2Vzc1ByZWRpY3RvcnMpKSkNCmBgYA0KDQoNCmBgYHtyfQ0KbWlzc2luZ0RhdGEgPC0gYXMuZGF0YS5mcmFtZShjb2xTdW1zKGlzLm5hKHByb2Nlc3NQcmVkaWN0b3JzKSkpDQpjb2xuYW1lcyhtaXNzaW5nRGF0YSkgPC0gYygiTkFzIikgDQptaXNzaW5nRGF0YSA8LSBjYmluZChQcmVkaWN0b3JzID0gcm93bmFtZXMobWlzc2luZ0RhdGEpLCBtaXNzaW5nRGF0YSkNCnJvd25hbWVzKG1pc3NpbmdEYXRhKSA8LSAxOm5yb3cobWlzc2luZ0RhdGEpDQptaXNzaW5nRGF0YSA8LSBtaXNzaW5nRGF0YVttaXNzaW5nRGF0YSROQXMgIT0gMCxdICU+JSANCiAgICAgICAgICAgICAgICBhcnJhbmdlKGRlc2MoTkFzKSkNCmhlYWQobWlzc2luZ0RhdGEpDQpgYGANCg0KDQpgYGB7ciAgZmlnLmhlaWdodD03LCBmaWcuYWxpZ249J2NlbnRlcid9DQptaXNzaW5nRGF0YSAgJT4lDQogIGdncGxvdCgpICsNCiAgICBnZW9tX2JhcihhZXMoeD1QcmVkaWN0b3JzLCB5PU5BcyxmaWxsPWZhY3RvcihOQXMpKSwgc3RhdCA9ICdpZGVudGl0eScsICkgKw0KICAgIGxhYnMoeD0nUHJlZGljdG9yJywgeT0iTkFzIiwgdGl0bGU9J051bWJlciBvZiBtaXNzaW5nIHZhbHVlcycpICsNCiAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA0NSwgaGp1c3QgPSAxKSkgKyBjb29yZF9mbGlwKCkgDQogIA0KYGBgDQoNCg0KDQpJIHVzZWQgYSBrTk4gaW1wdXRhdGlvbiBzdHJhdGVneSB0byBmaWxsIGluIGZvciB0aGUgbWlzc2luZyBwcmVkaWN0b3JzICBJIGFsc28gdXNlZCB0aGUgZGVmYXVsdCB2YWx1ZSBvZiBrPTUuIDxici8+DQoNCg0KYGBge3J9DQpzZXQuc2VlZCgyNCkNCmtubkltcHV0ZWRWYWx1ZXMgPSBwcmVQcm9jZXNzKHByb2Nlc3NQcmVkaWN0b3JzLCAia25uSW1wdXRlIikNCnByb2Nlc3NQcmVkaWN0b3JzIDwtIHRyeShwcmVkaWN0KGtubkltcHV0ZWRWYWx1ZXMsIHByb2Nlc3NQcmVkaWN0b3JzKSwgc2lsZW50ID0gVFJVRSkNCg0KYGBgDQoNCg0KYGBge3J9DQpwcm9jZXNzUHJlZGljdG9yc19pbXB1dGVkIDwtIGtOTihwcm9jZXNzUHJlZGljdG9ycywgdmFyaWFibGUgPSBjKG1pc3NpbmdEYXRhJFByZWRpY3RvcnMpLCBrPTUpDQpwcm9jZXNzUHJlZGljdG9yc19pbXB1dGVkIDwtIHByb2Nlc3NQcmVkaWN0b3JzX2ltcHV0ZWRbMTo1N10NCg0KYGBgDQoNCg0KDQpgYGB7cn0NCm1pc3NpbmdEYXRhIDwtIGFzLmRhdGEuZnJhbWUoY29sU3Vtcyhpcy5uYShwcm9jZXNzUHJlZGljdG9yc19pbXB1dGVkKSkpDQpjb2xuYW1lcyhtaXNzaW5nRGF0YSkgPC0gYygiTkFzIikgDQptaXNzaW5nRGF0YSA8LSBjYmluZChQcmVkaWN0b3JzID0gcm93bmFtZXMobWlzc2luZ0RhdGEpLCBtaXNzaW5nRGF0YSkNCnJvd25hbWVzKG1pc3NpbmdEYXRhKSA8LSAxOm5yb3cobWlzc2luZ0RhdGEpDQptaXNzaW5nRGF0YSA8LSBtaXNzaW5nRGF0YVttaXNzaW5nRGF0YSROQXMgIT0gMCxdDQpoZWFkKG1pc3NpbmdEYXRhKQ0KYGBgDQoNCjEuIFByZS1Qcm9jZXNzIHRoZSBEYXRhIC0gUmVtb3ZlIG5lYXJaZXJvVmFyIHByZWRpY3RvcnM6DQoNCmBgYHtyfQ0KY2FyZXQ6Om5lYXJaZXJvVmFyKHByb2Nlc3NQcmVkaWN0b3JzX2ltcHV0ZWQsIG5hbWVzID0gVFJVRSkNCmBgYA0KDQpgYGB7cn0NCnByb2Nlc3NQcmVkaWN0b3JzX2ltcHV0ZWQkQmlvbG9naWNhbE1hdGVyaWFsMDdbMToyMF0NCmBgYA0KDQoNCmBgYHtyfQ0KIHByb2Nlc3NQcmVkaWN0b3JzX2ltcHV0ZWQgPC0gc3Vic2V0ICggcHJvY2Vzc1ByZWRpY3RvcnNfaW1wdXRlZCwgc2VsZWN0ID0gLUJpb2xvZ2ljYWxNYXRlcmlhbDA3KQ0KYGBgDQoNCg0KMi4gUHJlLVByb2Nlc3MgdGhlIERhdGEgLSBDaGVjayBhbmQgcmVtb3ZlIE11bHRpLUNvbGluZWFyaXR5LiBXZSBjYW4gcmVtb3ZlIHRoZXNlIHZhcmlhYmxlcyBmb3IgbXVsdGktY29saW5lYXJpdHk6DQoNCmBgYHtyfQ0KdG9vSGlnaCA8LSBmaW5kQ29ycmVsYXRpb24oY29yKHByb2Nlc3NQcmVkaWN0b3JzX2ltcHV0ZWQpLCBjdXRvZmYgPSAuNzUsIG5hbWVzID0gVFJVRSkNCnRvb0hpZ2gNCmBgYA0KDQoNCg0KYGBge3IgZmlnLmhlaWdodD0xMCwgZmlnLndpZHRoPSAxMCwgZmlnLmFsaWduPSdjZW50ZXInfQ0KIyBMb29rIGF0IGNvcnJlbGF0aW9uIGJldHdlZW4gdmFyaWFibGVzDQoNCmNvcnIgPC0gcm91bmQoY29yKHByb2Nlc3NQcmVkaWN0b3JzX2ltcHV0ZWQpLCAxKQ0KDQpnZ2NvcnJwbG90KGNvcnIsDQogICAgICAgICAgIHR5cGU9Imxvd2VyIiwNCiAgICAgICAgICAgbGFiPVRSVUUsDQogICAgICAgICAgIGxhYl9zaXplPTMsDQogICAgICAgICAgIG1ldGhvZD0iY2lyY2xlIiwNCiAgICAgICAgICAgY29sb3JzPWMoInRvbWF0bzIiLCAid2hpdGUiLCAic3ByaW5nZ3JlZW4zIiksDQogICAgICAgICAgIHRpdGxlPSJDb3JyZWxhdGlvbiBvZiB2YXJpYWJsZXMgaW4gVHJhaW5pbmcgRGF0YSBTZXQiLA0KICAgICAgICAgICBnZ3RoZW1lPXRoZW1lX2J3KQ0KDQpgYGANCg0KDQpgYGB7cn0NCnJlbW92ZVByZWRpY3RvcnMgPC0gZmluZENvcnJlbGF0aW9uKGNvcihwcm9jZXNzUHJlZGljdG9yc19pbXB1dGVkKSwgMC45MCwgbmFtZXMgPSBUUlVFKQ0KcmVtb3ZlUHJlZGljdG9ycw0KDQpgYGANCg0KYGBge3J9DQpwcm9jZXNzUHJlZGljdG9yc19pbXB1dGVkWyAsYyhyZW1vdmVQcmVkaWN0b3JzKV0gPC0gbGlzdChOVUxMKQ0KY29sbmFtZXMocHJvY2Vzc1ByZWRpY3RvcnNfaW1wdXRlZCkNCmBgYA0KDQoNCmBgYHtyfQ0KY2hlbW1mZ3Byb2MgPC0gY2JpbmQoQ2hlbWljYWxNYW51ZmFjdHVyaW5nUHJvY2VzcyRZaWVsZCwgcHJvY2Vzc1ByZWRpY3RvcnNfaW1wdXRlZCkNCm5hbWVzKGNoZW1tZmdwcm9jKVtuYW1lcyhjaGVtbWZncHJvYykgPT0gIkNoZW1pY2FsTWFudWZhY3R1cmluZ1Byb2Nlc3MkWWllbGQiXSA8LSAiWWllbGQiDQpoZWFkKGNoZW1tZmdwcm9jICkNCmBgYA0KPGk+NS4gU3BsaXQgdGhlIGRhdGEgaW50byBUcmFpbmluZyBhbmQgVGVzdCBTZXQ8L2k+DQoNCmBgYHtyfQ0KY2hlbW1mZ3Byb2NfdHJhaW4gPC0gaW5pdGlhbF9zcGxpdChjaGVtbWZncHJvYywgcHJvcCA9IDAuOCwgc3RyYXRhID0gIllpZWxkIikNCnRyYWluX2NoZW1tZmdwcm9jICA8LSB0cmFpbmluZyhjaGVtbWZncHJvY190cmFpbikNCnRlc3RfY2hlbW1mZ3Byb2MgIDwtIHRlc3RpbmcoY2hlbW1mZ3Byb2NfdHJhaW4pDQpwcmludCAocGFzdGUwKCJUaGUgbnVtYmVyIG9mIG9ic2VydmF0aW9ucyBpbiB0aGUgdHJhaW5pbmcgc2V0IGlzICIsIG5yb3codHJhaW5fY2hlbW1mZ3Byb2MpKSkNCnByaW50IChwYXN0ZTAoIlRoZSBudW1iZXIgb2Ygb2JzZXJ2YXRpb25zIGluIHRoZSB0ZXN0IHNldCBpcyAiLCBucm93KHRlc3RfY2hlbW1mZ3Byb2MpKSkNCmBgYA0KDQojIyMgKGEpDQoNCldoaWNoIG5vbmxpbmVhciByZWdyZXNzaW9uIG1vZGVsIGdpdmVzIHRoZSBvcHRpbWFsIHJlc2FtcGxpbmcgYW5kIHRlc3Qgc2V0IHBlcmZvcm1hbmNlPw0KDQpEZWZpbmUgeCBhbmQgeSBmcm9tIHRoZSB0cmFpbmluZyBkYXRhOg0KDQpgYGB7cn0NCnRyYWluaW5nRGF0YV94IDwtIHRyYWluX2NoZW1tZmdwcm9jWzI6NDddDQp0cmFpbmluZ0RhdGFfeSA8LSB0cmFpbl9jaGVtbWZncHJvYyRZaWVsZA0KDQp0ZXN0RGF0YV94IDwtIHRlc3RfY2hlbW1mZ3Byb2NbMjo0N10NCnRlc3REYXRhX3kgPC0gdGVzdF9jaGVtbWZncHJvYyRZaWVsZA0KDQpgYGANCg0KDQo8Yj5rbm48L2I+DQoNCmBgYHtyfQ0Ka25uTW9kZWwgPC0gdHJhaW4oeCA9IHRyYWluaW5nRGF0YV94LCB5ID0gdHJhaW5pbmdEYXRhX3ksDQogbWV0aG9kID0gImtubiIsDQogcHJlUHJvYyA9IGMoImNlbnRlciIsICJzY2FsZSIpLA0KIHR1bmVMZW5ndGggPSAxMCkNCmtubk1vZGVsDQpgYGANCg0KDQpgYGB7cn0NCmtublByZWQgPC0gcHJlZGljdChrbm5Nb2RlbCwgbmV3ZGF0YSA9IHRlc3REYXRhX3gpDQprbm4gPC0gcG9zdFJlc2FtcGxlKHByZWQgPSBrbm5QcmVkLCBvYnMgPSB0ZXN0RGF0YV95KQ0Ka25uDQpgYGANCg0KYGBge3J9DQp2YXJJbXAoa25uTW9kZWwpDQpgYGANCg0KDQoNCjxiPk5OPC9iPg0KDQpDcmVhdGUgYSBzcGVjaWZpYyBjYW5kaWRhdGUgc2V0IG9mIG1vZGVscyB0byBldmFsdWF0ZToNCg0KYGBge3J9DQpubmV0R3JpZCA8LSBleHBhbmQuZ3JpZChkZWNheSA9IGMoMCwgMC4wMSwgLjEpLA0KICAgICAgICAgICAgICAgICAgICAgICAgc2l6ZSA9IGMoMSw1LDEwKSwNCiAgICAgICAgICAgICAgICAgICAgICAgIGJhZyA9IEZBTFNFKQ0KDQpjdHJsIDwtIHRyYWluQ29udHJvbChtZXRob2QgPSAiY3YiLCBudW1iZXI9MTApDQoNCmBgYA0KDQoNCmBgYHtyLCBtZXNzYWdlPUZBTFNFfQ0Kc2V0LnNlZWQoNjAwKQ0KDQoNCm5uZXRUdW5lIDwtIHRyYWluKHggPSB0cmFpbmluZ0RhdGFfeCwgDQogICAgICAgICAgICAgICAgICB5ID0gdHJhaW5pbmdEYXRhX3ksICANCiAgICAgICAgICAgICAgICAgIG1ldGhvZCA9ICJhdk5OZXQiLCAgDQogICAgICAgICAgICAgICAgICB0dW5lR3JpZCA9IG5uZXRHcmlkLCAgDQogICAgICAgICAgICAgICAgICB0ckNvbnRyb2wgPSBjdHJsLA0KICAgICAgICAgICAgICAgICAgcHJlUHJvYyA9IGMoImNlbnRlciIsICJzY2FsZSIpLCAgDQogICAgICAgICAgICAgICAgICBsaW5vdXQgPSBUUlVFLA0KICAgICAgICAgICAgICAgICAgdHJhY2UgPSBGQUxTRSwgIA0KICAgICAgICAgICAgICAgICAgbWF4aXQgPSAxMCkNCm5uZXRUdW5lDQogICAgICAgICAgICAgICAgICANCmBgYA0KDQoNCmBgYHtyfQ0Kbm5ldFR1bmUkYmVzdFR1bmUNCmBgYA0KDQoNCmBgYHtyfQ0Kbm5ldFByZWQgPC0gcHJlZGljdChubmV0VHVuZSwgbmV3ZGF0YT10ZXN0RGF0YV94KQ0KTk5FVCA8LSBwb3N0UmVzYW1wbGUocHJlZCA9IG5uZXRQcmVkLCBvYnMgPSB0ZXN0RGF0YV95KQ0KTk5FVA0KYGBgDQoNCmBgYHtyfQ0KcGxvdG1vKG5uZXRUdW5lKQ0KYGBgDQoNCmBgYHtyfQ0KdmFySW1wKG5uZXRUdW5lKQ0KYGBgDQoNCg0KDQo8Yj5NQVJTPC9iPg0KDQoNCmBgYHtyfQ0Kc2V0LnNlZWQoMTAwKQ0KbWFyc0dyaWQgPC0gZXhwYW5kLmdyaWQoLmRlZ3JlZSA9IDE6MiwgLm5wcnVuZSA9IDI6MzgpDQptYXJzVHVuZWQgPC0gdHJhaW4oeCA9IHRyYWluaW5nRGF0YV94LCB5ID0gdHJhaW5pbmdEYXRhX3ksDQogICAgICAgICAgICAgICAgICBtZXRob2QgPSAiZWFydGgiLCANCiAgICAgICAgICAgICAgICAgIHR1bmVHcmlkID0gbWFyc0dyaWQsDQogICAgICAgICAgICAgICAgICB0ckNvbnRyb2wgPSBjdHJsKQ0KbWFyc1R1bmVkDQpgYGANCg0KYGBge3J9DQptYXJzUHJlZCA8LSBwcmVkaWN0KG1hcnNUdW5lZCwgbmV3ZGF0YT10ZXN0RGF0YV94KQ0KTUFSUyA8LSBwb3N0UmVzYW1wbGUocHJlZCA9IG1hcnNQcmVkLCBvYnMgPSB0ZXN0RGF0YV95KQ0KTUFSUw0KYGBgDQoNCg0KYGBge3J9DQpwbG90bW8obWFyc1R1bmVkKQ0KYGBgDQoNCmBgYHtyfQ0KdmFySW1wKG1hcnNUdW5lZCkNCmBgYA0KDQoNCg0KPGI+c3ZtUjwvYj4NCg0KYGBge3J9DQpzZXQuc2VlZCgxMDApDQpzdm1SVHVuZWQgPC0gdHJhaW4odHJhaW5pbmdEYXRhX3gsIHRyYWluaW5nRGF0YV95LA0KICAgICAgICAgICAgICAgICAgIG1ldGhvZCA9ICJzdm1SYWRpYWwiLA0KICAgICAgICAgICAgICAgICAgIHByZVByb2MgPSBjKCJjZW50ZXIiLCAic2NhbGUiKSwNCiAgICAgICAgICAgICAgICAgICB0dW5lTGVuZ3RoID0gMTQsDQogICAgICAgICAgICAgICAgICAgdHJDb250cm9sID0gdHJhaW5Db250cm9sKG1ldGhvZCA9ICJjdiIpKQ0Kc3ZtUlR1bmVkIA0KYGBgDQoNCg0KYGBge3J9DQpzdm1SUHJlZCA8LSBwcmVkaWN0KHN2bVJUdW5lZCwgbmV3ZGF0YT10ZXN0RGF0YV94KQ0Kc3ZtUjwtIHBvc3RSZXNhbXBsZShwcmVkID0gc3ZtUlByZWQsIG9icyA9IHRlc3REYXRhX3kpDQpzdm1SDQpgYGANCg0KDQpgYGB7cn0NCnBsb3Rtbyhzdm1SVHVuZWQpDQpgYGANCg0KDQoNCmBgYHtyfQ0KdmFySW1wKHN2bVJUdW5lZCkNCmBgYA0KDQoNCjxiPkNvbXBhcmlzb24gb2YgUk1TRTwvYj4NCg0KYGBge3J9DQpSTVNFX0NvbXBhcmlzb24gPC0gYXMuZGF0YS5mcmFtZShyYmluZChjKCJLTk4iLGtubiksYygiTk5FVCIsTk5FVCksYygiU1ZNUiIsc3ZtUiksIGMoIk1BUlMiLCBNQVJTKSkpDQpjb2xuYW1lcyhSTVNFX0NvbXBhcmlzb24pIDwtIGMoIk5vbiBMaW5lYXIgTW9kZWwiLCAiUk1TRSIsICJSc3F1YXJlZCIsICJNQUUiKQ0KUk1TRV9Db21wYXJpc29uICU+JSANCiAgYXJyYW5nZShSTVNFKQ0KYGBgDQoNCg0KDQoNCiMjIyAoYikNCg0KPGk+V2hpY2ggcHJlZGljdG9ycyBhcmUgbW9zdCBpbXBvcnRhbnQgaW4gdGhlIG9wdGltYWwgbm9ubGluZWFyIHJlZ3Jlc3Npb24gbW9kZWw/PC9pPg0KDQpUaGUgdG9wIHRlbiBtb3N0IGltcG9ydGFudCBwcmVkaWN0b3JzIGZyb20gdGhlIG9wdGltYWwgbm9ubGluZWFyIHJlZ3Jlc3Npb24gbW9kZWwsIHN2bVJhZGlhbCwgYXJlOg0KDQpNYW51ZmFjdHVyaW5nUHJvY2VzczEzCTEwMC4wMDAwMAkJCQ0KTWFudWZhY3R1cmluZ1Byb2Nlc3MzMgk5Ny41NTMwOAkJCQ0KQmlvbG9naWNhbE1hdGVyaWFsMDYJOTAuOTc3MTIJCQkNCk1hbnVmYWN0dXJpbmdQcm9jZXNzMTcJOTAuMjM3MDAJCQkNCk1hbnVmYWN0dXJpbmdQcm9jZXNzMDkJODMuNTEzOTEJCQkNCkJpb2xvZ2ljYWxNYXRlcmlhbDAzCTgwLjA1MTkyCQkJDQpNYW51ZmFjdHVyaW5nUHJvY2VzczA2CTY5LjM1MjQ3CQkJDQpNYW51ZmFjdHVyaW5nUHJvY2VzczM2CTY3LjcwMzcyCQkJDQpCaW9sb2dpY2FsTWF0ZXJpYWwxMQk2Mi41MTgzNgkJCQ0KTWFudWZhY3R1cmluZ1Byb2Nlc3MxMQk1Ni41MDg0NAkJCQ0KDQoNCg0KPGk+RG8gZWl0aGVyIHRoZSBiaW9sb2dpY2FsIG9yIHByb2Nlc3MgdmFyaWFibGVzIGRvbWluYXRlIHRoZSBsaXN0PzwvaT4NCg0KVGhlIG1hbnVmYWN0dXJpbmcgcHJvY2Vzc2VzIGFjY291bnQgZm9yIDcgb2YgdGhlIHRvcCB0ZW4gdmFyaWFibGVzIHdoaWxlIHRoZSBiaW9sb2dpY2FsIG9uZXMgb25seSBhY2NvdW50IGZvciB0aHJlZS4gDQoNCjxpPkhvdyBkbyB0aGUgdG9wIHRlbiBpbXBvcnRhbnQgcHJlZGljdG9ycyBjb21wYXJlIHRvIHRoZSB0b3AgdGVuIHByZWRpY3RvcnMgZnJvbSB0aGUgb3B0aW1hbCBsaW5lYXIgbW9kZWw/PC9pPg0KDQpUaGUgdG9wIHRlbiBpbXBvcnRhbnQgcHJlZGljdG9ycyBmcm9tIHRoZSBvcHRpbWFsIG5vbi1saW5lYXIgbW9kZWwgd2VyZSBzb21ld2hhdCBpZGVudGljYWwgdG8gdGhlIG9wdGltYWwgbGluZWFyIG1vZGVsLCBwYXJ0aWFsIGxlYXN0IHNxdWFyZXMgd2l0aCBzb21lIG1ham9yIGRpZmZlcmVuY2VzLiBUaGUgbW9zdCBpbXBvcnRhbnQgcHJlZGljdG9yIHdhcyBkaWZmZXJlbnQuICBCaW9sb2dpY2FsIG1hdGVyaWFsIDExIGFuZCAgTWFudWZhY3R1cmluZyBQcm9jZXNzIDExIHdlcmUgb24gdGhlIGxpc3QgZm9yIHRoZSAgb3B0aW1hbCBub24tbGluZWFyIG1vZGVsLCBidXQgaXQgd2Fzbid0IG9uIHRoZSBsaXN0IGZvciB0aGUgb3B0aW1hbCBsaW5lYXIgbW9kZWwgd2hpY2ggaGFkIEJpb2xvZ2ljYWwgbWF0ZXJpYWwgMDggYW5kIE1hbnVmYWN0dXJpbmcgUHJvY2VzcyAzMy4gIA0KDQpNYW51ZmFjdHVyaW5nUHJvY2VzczMyCTEwMC4wMDAwMAkJCQ0KTWFudWZhY3R1cmluZ1Byb2Nlc3MwOQk5Mi4yNDc5OQkJCQ0KTWFudWZhY3R1cmluZ1Byb2Nlc3MxMwk4OS4yOTQ4MwkJCQ0KTWFudWZhY3R1cmluZ1Byb2Nlc3MxNwk3OS43NTg1NwkJCQ0KTWFudWZhY3R1cmluZ1Byb2Nlc3MzNgk3Ni41ODUxNgkJCQ0KQmlvbG9naWNhbE1hdGVyaWFsMDYJNzEuODA2OTYJCQkNCkJpb2xvZ2ljYWxNYXRlcmlhbDA4CTY4LjMxMDc0CQkJDQpNYW51ZmFjdHVyaW5nUHJvY2VzczA2CTY1LjgwOTQ0CQkJDQpNYW51ZmFjdHVyaW5nUHJvY2VzczMzCTYyLjM5MTI2CQkJDQpCaW9sb2dpY2FsTWF0ZXJpYWwwMwk2MC41NjE2NAkNCg0KDQoNCg0KDQpTVk0NCg0KYGBge3J9DQp2YXJJbXAoc3ZtUlR1bmVkKQ0KYGBgDQoNCg0KS05ODQoNCg0KYGBge3J9DQp2YXJJbXAoa25uTW9kZWwpDQpgYGANCg0KDQpOTkVUDQoNCg0KYGBge3J9DQp2YXJJbXAobm5ldFR1bmUpDQpgYGANCg0KDQoNCg0KDQpNQVJTDQoNCg0KYGBge3J9DQp2YXJJbXAobWFyc1R1bmVkKQ0KYGBgDQoNCg0KDQojIyMgKGMpDQoNCjxpPkV4cGxvcmUgdGhlIHJlbGF0aW9uc2hpcHMgYmV0d2VlbiB0aGUgdG9wIHByZWRpY3RvcnMgYW5kIHRoZSByZXNwb25zZSBmb3IgdGhlIHByZWRpY3RvcnMgdGhhdCBhcmUgdW5pcXVlIHRvIHRoZSBvcHRpbWFsIG5vbmxpbmVhciByZWdyZXNzaW9uIG1vZGVsLg0KRG8gdGhlc2UgcGxvdHMgcmV2ZWFsIGludHVpdGlvbiBhYm91dCB0aGUgYmlvbG9naWNhbCBvciBwcm9jZXNzIHByZWRpY3RvcnMgYW5kIHRoZWlyIHJlbGF0aW9uc2hpcCB3aXRoIHlpZWxkPzwvaT4NCg0KQXMgc3RhdGVkIGVhcmxpZXIsIHRoZXJlIHdlcmUgdHdvIGltcG9ydGFudCBwcmVkaWN0b3JzIHRoYXQgd2VyZSB1bmlxdWUgdG8gdGhlIG9wdGltYWwgbm9ubGluZWFyIHJlZ3Jlc3Npb24gbW9kZWwgTWFudWZhY3R1cmluZyBQcm9jZXNzIDExIGFuZCBCaW9sb2dpY2FsIE1hdGVyaWFsIDExLiBXaGVuIGVhY2ggaXMgcGxvdHRlZCBhZ2FpbnN0IHRoZSByZXNwb25zZSB2YXJpYWJsZSwgWWllbGQsIHRoZXJlIGlzIGEgdmVyeSB3ZWFrIGNvcnJlbGF0aW9uIGJldHdlZW4gdGhlIHR3by4gVGhlcmUgZG9lcyBub3Qgc2VlbSB0byBiZSBhbnkgaW50dWl0aW9uIHJldmVhbGVkLiANCg0KDQoNCmBgYHtyfQ0KY29yKGNoZW1tZmdwcm9jJFlpZWxkLCBjaGVtbWZncHJvYyRNYW51ZmFjdHVyaW5nUHJvY2VzczExKQ0KYGBgDQoNCmBgYHtyfQ0KY29yKGNoZW1tZmdwcm9jJFlpZWxkLCBjaGVtbWZncHJvYyRCaW9sb2dpY2FsTWF0ZXJpYWwxMSkNCmBgYA0KDQoNCg0KYGBge3J9DQpnZ3Bsb3QoQ2hlbWljYWxNYW51ZmFjdHVyaW5nUHJvY2VzcywgYWVzKEJpb2xvZ2ljYWxNYXRlcmlhbDExLCBZaWVsZCkpICsNCiAgZ2VvbV9wb2ludCgpDQpgYGANCg0KYGBge3J9DQpnZ3Bsb3QoQ2hlbWljYWxNYW51ZmFjdHVyaW5nUHJvY2VzcywgYWVzKE1hbnVmYWN0dXJpbmdQcm9jZXNzMTEsIFlpZWxkKSkgKw0KICBnZW9tX3BvaW50KCkNCmBgYA0KDQoNCg0KDQoNCg0KDQoNCg==