V1 V2 V3 V4
Min. :0.002806 Min. :0.0004409 Min. :0.003296 Min. :0.00819
1st Qu.:0.217222 1st Qu.:0.2795877 1st Qu.:0.284872 1st Qu.:0.23178
Median :0.513935 Median :0.5106664 Median :0.537307 Median :0.44458
Mean :0.481461 Mean :0.5126936 Mean :0.508570 Mean :0.46939
3rd Qu.:0.680814 3rd Qu.:0.7326078 3rd Qu.:0.746155 3rd Qu.:0.68454
Max. :0.998992 Max. :0.9840194 Max. :0.999923 Max. :0.99929
V5 V6 V7 V8
Min. :0.001756 Min. :0.002901 Min. :0.0003388 Min. :0.004698
1st Qu.:0.285408 1st Qu.:0.276341 1st Qu.:0.2092393 1st Qu.:0.289413
Median :0.534330 Median :0.497598 Median :0.4688035 Median :0.497961
Mean :0.517948 Mean :0.496841 Mean :0.4636166 Mean :0.512865
3rd Qu.:0.748180 3rd Qu.:0.749616 3rd Qu.:0.6752972 3rd Qu.:0.727624
Max. :0.991577 Max. :0.998987 Max. :0.9943477 Max. :0.999561
V9 V10
Min. :0.0176 Min. :0.003172
1st Qu.:0.2304 1st Qu.:0.317826
Median :0.5289 Median :0.535922
Mean :0.5016 Mean :0.542409
3rd Qu.:0.7218 3rd Qu.:0.789736
Max. :0.9910 Max. :0.999992
summary(trainingData$y)
Min. 1st Qu. Median Mean 3rd Qu. Max.
3.556 10.756 14.556 14.416 17.970 28.382
# Examine predictors and responsefeaturePlot(x = trainingData$x,y = trainingData$y)
# tune several models on these dataset.seed(200)knnModel <-train(x = trainingData$x,y = trainingData$y,method ="knn",preProcess =c("center", "scale"),trControl = control,tuneLength =10)knnModel
k-Nearest Neighbors
200 samples
10 predictor
Pre-processing: centered (10), scaled (10)
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
Resampling results across tuning parameters:
k RMSE Rsquared MAE
5 3.288773 0.5889898 2.725944
7 3.144376 0.6434361 2.577807
9 3.124264 0.6689980 2.517219
11 3.077985 0.6915901 2.489324
13 3.103767 0.7026394 2.506703
15 3.127439 0.7062228 2.513829
17 3.149433 0.7114210 2.526318
19 3.153927 0.7249773 2.528158
21 3.121909 0.7450302 2.523428
23 3.171824 0.7389961 2.565383
RMSE was used to select the optimal model using the smallest value.
The final value used for the model was k = 11.
Support Vector Machines with Linear Kernel
200 samples
10 predictor
Pre-processing: centered (10), scaled (10)
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
Resampling results:
RMSE Rsquared MAE
2.432316 0.765497 1.96225
Tuning parameter 'C' was held constant at a value of 1
Support Vector Machines with Radial Basis Function Kernel
200 samples
10 predictor
Pre-processing: centered (10), scaled (10)
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 180, 180, 180, 180, 180, 180, ...
Resampling results across tuning parameters:
C RMSE Rsquared MAE
0.25 2.502240 0.7991647 1.996420
0.50 2.251774 0.8122993 1.775708
1.00 2.071314 0.8353324 1.624308
2.00 1.954727 0.8525581 1.515420
4.00 1.884376 0.8615784 1.470561
8.00 1.861357 0.8643203 1.475030
16.00 1.852625 0.8668207 1.476040
32.00 1.852625 0.8668207 1.476040
64.00 1.852625 0.8668207 1.476040
128.00 1.852625 0.8668207 1.476040
Tuning parameter 'sigma' was held constant at a value of 0.06299324
RMSE was used to select the optimal model using the smallest value.
The final values used for the model were sigma = 0.06299324 and C = 16.
# A tibble: 1 × 4
Model RMSE Rsquared MAE
<chr> <dbl> <dbl> <dbl>
1 MARS 1.28 0.934 1.02
Interpretation
The MARS model performed better than the other five regression models because it achieved the lowest RMSE on the test dataset. This indicates that MARS produced more accurate predictions compared to KNN, linear SVM, radial SVM, Neural Network.
Additionally, the actual versus predicted plot shows that the MARS predictions are closely aligned with the actual values, indicating a good model fit. Since the Friedman dataset contains nonlinear relationships and interactions between predictors, MARS can effectively capture these patterns using piecewise regression functions.
Based on the RMSE results and the actual versus predicted plot, MARS is the best-performing model among the six models evaluated in this analysis.
##Exercise 7.5
Exercise 6.3 describes data for a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several nonlinear regression models.
'data.frame': 176 obs. of 58 variables:
$ Yield : num 38 42.4 42 41.4 42.5 ...
$ BiologicalMaterial01 : num 6.25 8.01 8.01 8.01 7.47 6.12 7.48 6.94 6.94 6.94 ...
$ BiologicalMaterial02 : num 49.6 61 61 61 63.3 ...
$ BiologicalMaterial03 : num 57 67.5 67.5 67.5 72.2 ...
$ BiologicalMaterial04 : num 12.7 14.7 14.7 14.7 14 ...
$ BiologicalMaterial05 : num 19.5 19.4 19.4 19.4 17.9 ...
$ BiologicalMaterial06 : num 43.7 53.1 53.1 53.1 54.7 ...
$ BiologicalMaterial07 : num 100 100 100 100 100 100 100 100 100 100 ...
$ BiologicalMaterial08 : num 16.7 19 19 19 18.2 ...
$ BiologicalMaterial09 : num 11.4 12.6 12.6 12.6 12.8 ...
$ BiologicalMaterial10 : num 3.46 3.46 3.46 3.46 3.05 3.78 3.04 3.85 3.85 3.85 ...
$ BiologicalMaterial11 : num 138 154 154 154 148 ...
$ BiologicalMaterial12 : num 18.8 21.1 21.1 21.1 21.1 ...
$ ManufacturingProcess01: num NA 0 0 0 10.7 12 11.5 12 12 12 ...
$ ManufacturingProcess02: num NA 0 0 0 0 0 0 0 0 0 ...
$ ManufacturingProcess03: num NA NA NA NA NA NA 1.56 1.55 1.56 1.55 ...
$ ManufacturingProcess04: num NA 917 912 911 918 924 933 929 928 938 ...
$ ManufacturingProcess05: num NA 1032 1004 1015 1028 ...
$ ManufacturingProcess06: num NA 210 207 213 206 ...
$ ManufacturingProcess07: num NA 177 178 177 178 178 177 178 177 177 ...
$ ManufacturingProcess08: num NA 178 178 177 178 178 178 178 177 177 ...
$ ManufacturingProcess09: num 43 46.6 45.1 44.9 45 ...
$ ManufacturingProcess10: num NA NA NA NA NA NA 11.6 10.2 9.7 10.1 ...
$ ManufacturingProcess11: num NA NA NA NA NA NA 11.5 11.3 11.1 10.2 ...
$ ManufacturingProcess12: num NA 0 0 0 0 0 0 0 0 0 ...
$ ManufacturingProcess13: num 35.5 34 34.8 34.8 34.6 34 32.4 33.6 33.9 34.3 ...
$ ManufacturingProcess14: num 4898 4869 4878 4897 4992 ...
$ ManufacturingProcess15: num 6108 6095 6087 6102 6233 ...
$ ManufacturingProcess16: num 4682 4617 4617 4635 4733 ...
$ ManufacturingProcess17: num 35.5 34 34.8 34.8 33.9 33.4 33.8 33.6 33.9 35.3 ...
$ ManufacturingProcess18: num 4865 4867 4877 4872 4886 ...
$ ManufacturingProcess19: num 6049 6097 6078 6073 6102 ...
$ ManufacturingProcess20: num 4665 4621 4621 4611 4659 ...
$ ManufacturingProcess21: num 0 0 0 0 -0.7 -0.6 1.4 0 0 1 ...
$ ManufacturingProcess22: num NA 3 4 5 8 9 1 2 3 4 ...
$ ManufacturingProcess23: num NA 0 1 2 4 1 1 2 3 1 ...
$ ManufacturingProcess24: num NA 3 4 5 18 1 1 2 3 4 ...
$ ManufacturingProcess25: num 4873 4869 4897 4892 4930 ...
$ ManufacturingProcess26: num 6074 6107 6116 6111 6151 ...
$ ManufacturingProcess27: num 4685 4630 4637 4630 4684 ...
$ ManufacturingProcess28: num 10.7 11.2 11.1 11.1 11.3 11.4 11.2 11.1 11.3 11.4 ...
$ ManufacturingProcess29: num 21 21.4 21.3 21.3 21.6 21.7 21.2 21.2 21.5 21.7 ...
$ ManufacturingProcess30: num 9.9 9.9 9.4 9.4 9 10.1 11.2 10.9 10.5 9.8 ...
$ ManufacturingProcess31: num 69.1 68.7 69.3 69.3 69.4 68.2 67.6 67.9 68 68.5 ...
$ ManufacturingProcess32: num 156 169 173 171 171 173 159 161 160 164 ...
$ ManufacturingProcess33: num 66 66 66 68 70 70 65 65 65 66 ...
$ ManufacturingProcess34: num 2.4 2.6 2.6 2.5 2.5 2.5 2.5 2.5 2.5 2.5 ...
$ ManufacturingProcess35: num 486 508 509 496 468 490 475 478 491 488 ...
$ ManufacturingProcess36: num 0.019 0.019 0.018 0.018 0.017 0.018 0.019 0.019 0.019 0.019 ...
$ ManufacturingProcess37: num 0.5 2 0.7 1.2 0.2 0.4 0.8 1 1.2 1.8 ...
$ ManufacturingProcess38: num 3 2 2 2 2 2 2 2 3 3 ...
$ ManufacturingProcess39: num 7.2 7.2 7.2 7.2 7.3 7.2 7.3 7.3 7.4 7.1 ...
$ ManufacturingProcess40: num NA 0.1 0 0 0 0 0 0 0 0 ...
$ ManufacturingProcess41: num NA 0.15 0 0 0 0 0 0 0 0 ...
$ ManufacturingProcess42: num 11.6 11.1 12 10.6 11 11.5 11.7 11.4 11.4 11.3 ...
$ ManufacturingProcess43: num 3 0.9 1 1.1 1.1 2.2 0.7 0.8 0.9 0.8 ...
$ ManufacturingProcess44: num 1.8 1.9 1.8 1.8 1.7 1.8 2 2 1.9 1.9 ...
$ ManufacturingProcess45: num 2.4 2.2 2.3 2.1 2.1 2 2.2 2.2 2.1 2.4 ...
summary(ChemicalManufacturingProcess)
Yield BiologicalMaterial01 BiologicalMaterial02 BiologicalMaterial03
Min. :35.25 Min. :4.580 Min. :46.87 Min. :56.97
1st Qu.:38.75 1st Qu.:5.978 1st Qu.:52.68 1st Qu.:64.98
Median :39.97 Median :6.305 Median :55.09 Median :67.22
Mean :40.18 Mean :6.411 Mean :55.69 Mean :67.70
3rd Qu.:41.48 3rd Qu.:6.870 3rd Qu.:58.74 3rd Qu.:70.43
Max. :46.34 Max. :8.810 Max. :64.75 Max. :78.25
BiologicalMaterial04 BiologicalMaterial05 BiologicalMaterial06
Min. : 9.38 Min. :13.24 Min. :40.60
1st Qu.:11.24 1st Qu.:17.23 1st Qu.:46.05
Median :12.10 Median :18.49 Median :48.46
Mean :12.35 Mean :18.60 Mean :48.91
3rd Qu.:13.22 3rd Qu.:19.90 3rd Qu.:51.34
Max. :23.09 Max. :24.85 Max. :59.38
BiologicalMaterial07 BiologicalMaterial08 BiologicalMaterial09
Min. :100.0 Min. :15.88 Min. :11.44
1st Qu.:100.0 1st Qu.:17.06 1st Qu.:12.60
Median :100.0 Median :17.51 Median :12.84
Mean :100.0 Mean :17.49 Mean :12.85
3rd Qu.:100.0 3rd Qu.:17.88 3rd Qu.:13.13
Max. :100.8 Max. :19.14 Max. :14.08
BiologicalMaterial10 BiologicalMaterial11 BiologicalMaterial12
Min. :1.770 Min. :135.8 Min. :18.35
1st Qu.:2.460 1st Qu.:143.8 1st Qu.:19.73
Median :2.710 Median :146.1 Median :20.12
Mean :2.801 Mean :147.0 Mean :20.20
3rd Qu.:2.990 3rd Qu.:149.6 3rd Qu.:20.75
Max. :6.870 Max. :158.7 Max. :22.21
ManufacturingProcess01 ManufacturingProcess02 ManufacturingProcess03
Min. : 0.00 Min. : 0.00 Min. :1.47
1st Qu.:10.80 1st Qu.:19.30 1st Qu.:1.53
Median :11.40 Median :21.00 Median :1.54
Mean :11.21 Mean :16.68 Mean :1.54
3rd Qu.:12.15 3rd Qu.:21.50 3rd Qu.:1.55
Max. :14.10 Max. :22.50 Max. :1.60
NA's :1 NA's :3 NA's :15
ManufacturingProcess04 ManufacturingProcess05 ManufacturingProcess06
Min. :911.0 Min. : 923.0 Min. :203.0
1st Qu.:928.0 1st Qu.: 986.8 1st Qu.:205.7
Median :934.0 Median : 999.2 Median :206.8
Mean :931.9 Mean :1001.7 Mean :207.4
3rd Qu.:936.0 3rd Qu.:1008.9 3rd Qu.:208.7
Max. :946.0 Max. :1175.3 Max. :227.4
NA's :1 NA's :1 NA's :2
ManufacturingProcess07 ManufacturingProcess08 ManufacturingProcess09
Min. :177.0 Min. :177.0 Min. :38.89
1st Qu.:177.0 1st Qu.:177.0 1st Qu.:44.89
Median :177.0 Median :178.0 Median :45.73
Mean :177.5 Mean :177.6 Mean :45.66
3rd Qu.:178.0 3rd Qu.:178.0 3rd Qu.:46.52
Max. :178.0 Max. :178.0 Max. :49.36
NA's :1 NA's :1
ManufacturingProcess10 ManufacturingProcess11 ManufacturingProcess12
Min. : 7.500 Min. : 7.500 Min. : 0.0
1st Qu.: 8.700 1st Qu.: 9.000 1st Qu.: 0.0
Median : 9.100 Median : 9.400 Median : 0.0
Mean : 9.179 Mean : 9.386 Mean : 857.8
3rd Qu.: 9.550 3rd Qu.: 9.900 3rd Qu.: 0.0
Max. :11.600 Max. :11.500 Max. :4549.0
NA's :9 NA's :10 NA's :1
ManufacturingProcess13 ManufacturingProcess14 ManufacturingProcess15
Min. :32.10 Min. :4701 Min. :5904
1st Qu.:33.90 1st Qu.:4828 1st Qu.:6010
Median :34.60 Median :4856 Median :6032
Mean :34.51 Mean :4854 Mean :6039
3rd Qu.:35.20 3rd Qu.:4882 3rd Qu.:6061
Max. :38.60 Max. :5055 Max. :6233
NA's :1
ManufacturingProcess16 ManufacturingProcess17 ManufacturingProcess18
Min. : 0 Min. :31.30 Min. : 0
1st Qu.:4561 1st Qu.:33.50 1st Qu.:4813
Median :4588 Median :34.40 Median :4835
Mean :4566 Mean :34.34 Mean :4810
3rd Qu.:4619 3rd Qu.:35.10 3rd Qu.:4862
Max. :4852 Max. :40.00 Max. :4971
ManufacturingProcess19 ManufacturingProcess20 ManufacturingProcess21
Min. :5890 Min. : 0 Min. :-1.8000
1st Qu.:6001 1st Qu.:4553 1st Qu.:-0.6000
Median :6022 Median :4582 Median :-0.3000
Mean :6028 Mean :4556 Mean :-0.1642
3rd Qu.:6050 3rd Qu.:4610 3rd Qu.: 0.0000
Max. :6146 Max. :4759 Max. : 3.6000
ManufacturingProcess22 ManufacturingProcess23 ManufacturingProcess24
Min. : 0.000 Min. :0.000 Min. : 0.000
1st Qu.: 3.000 1st Qu.:2.000 1st Qu.: 4.000
Median : 5.000 Median :3.000 Median : 8.000
Mean : 5.406 Mean :3.017 Mean : 8.834
3rd Qu.: 8.000 3rd Qu.:4.000 3rd Qu.:14.000
Max. :12.000 Max. :6.000 Max. :23.000
NA's :1 NA's :1 NA's :1
ManufacturingProcess25 ManufacturingProcess26 ManufacturingProcess27
Min. : 0 Min. : 0 Min. : 0
1st Qu.:4832 1st Qu.:6020 1st Qu.:4560
Median :4855 Median :6047 Median :4587
Mean :4828 Mean :6016 Mean :4563
3rd Qu.:4877 3rd Qu.:6070 3rd Qu.:4609
Max. :4990 Max. :6161 Max. :4710
NA's :5 NA's :5 NA's :5
ManufacturingProcess28 ManufacturingProcess29 ManufacturingProcess30
Min. : 0.000 Min. : 0.00 Min. : 0.000
1st Qu.: 0.000 1st Qu.:19.70 1st Qu.: 8.800
Median :10.400 Median :19.90 Median : 9.100
Mean : 6.592 Mean :20.01 Mean : 9.161
3rd Qu.:10.750 3rd Qu.:20.40 3rd Qu.: 9.700
Max. :11.500 Max. :22.00 Max. :11.200
NA's :5 NA's :5 NA's :5
ManufacturingProcess31 ManufacturingProcess32 ManufacturingProcess33
Min. : 0.00 Min. :143.0 Min. :56.00
1st Qu.:70.10 1st Qu.:155.0 1st Qu.:62.00
Median :70.80 Median :158.0 Median :64.00
Mean :70.18 Mean :158.5 Mean :63.54
3rd Qu.:71.40 3rd Qu.:162.0 3rd Qu.:65.00
Max. :72.50 Max. :173.0 Max. :70.00
NA's :5 NA's :5
ManufacturingProcess34 ManufacturingProcess35 ManufacturingProcess36
Min. :2.300 Min. :463.0 Min. :0.01700
1st Qu.:2.500 1st Qu.:490.0 1st Qu.:0.01900
Median :2.500 Median :495.0 Median :0.02000
Mean :2.494 Mean :495.6 Mean :0.01957
3rd Qu.:2.500 3rd Qu.:501.5 3rd Qu.:0.02000
Max. :2.600 Max. :522.0 Max. :0.02200
NA's :5 NA's :5 NA's :5
ManufacturingProcess37 ManufacturingProcess38 ManufacturingProcess39
Min. :0.000 Min. :0.000 Min. :0.000
1st Qu.:0.700 1st Qu.:2.000 1st Qu.:7.100
Median :1.000 Median :3.000 Median :7.200
Mean :1.014 Mean :2.534 Mean :6.851
3rd Qu.:1.300 3rd Qu.:3.000 3rd Qu.:7.300
Max. :2.300 Max. :3.000 Max. :7.500
ManufacturingProcess40 ManufacturingProcess41 ManufacturingProcess42
Min. :0.00000 Min. :0.00000 Min. : 0.00
1st Qu.:0.00000 1st Qu.:0.00000 1st Qu.:11.40
Median :0.00000 Median :0.00000 Median :11.60
Mean :0.01771 Mean :0.02371 Mean :11.21
3rd Qu.:0.00000 3rd Qu.:0.00000 3rd Qu.:11.70
Max. :0.10000 Max. :0.20000 Max. :12.10
NA's :1 NA's :1
ManufacturingProcess43 ManufacturingProcess44 ManufacturingProcess45
Min. : 0.0000 Min. :0.000 Min. :0.000
1st Qu.: 0.6000 1st Qu.:1.800 1st Qu.:2.100
Median : 0.8000 Median :1.900 Median :2.200
Mean : 0.9119 Mean :1.805 Mean :2.138
3rd Qu.: 1.0250 3rd Qu.:1.900 3rd Qu.:2.300
Max. :11.0000 Max. :2.100 Max. :2.600
k-Nearest Neighbors
144 samples
57 predictor
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 128, 129, 130, 130, 128, 130, ...
Resampling results across tuning parameters:
k RMSE Rsquared MAE
5 1.244040 0.5408253 1.023276
7 1.312593 0.4873704 1.080397
9 1.364263 0.4542507 1.124819
11 1.359890 0.4711554 1.126593
13 1.365952 0.4563463 1.127534
15 1.372466 0.4542979 1.137705
17 1.394964 0.4353030 1.150754
19 1.411442 0.4158501 1.161529
21 1.410953 0.4260760 1.161424
23 1.432978 0.4096841 1.172731
RMSE was used to select the optimal model using the smallest value.
The final value used for the model was k = 5.
Support Vector Machines with Linear Kernel
144 samples
57 predictor
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 128, 129, 130, 130, 128, 130, ...
Resampling results:
RMSE Rsquared MAE
3.713475 0.4705089 1.810233
Tuning parameter 'C' was held constant at a value of 1
Support Vector Machines with Radial Basis Function Kernel
144 samples
57 predictor
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 128, 129, 130, 130, 128, 130, ...
Resampling results across tuning parameters:
C RMSE Rsquared MAE
0.25 1.397369 0.4977627 1.1448942
0.50 1.278128 0.5399648 1.0482201
1.00 1.176753 0.5939419 0.9543719
2.00 1.135787 0.6124316 0.9064516
4.00 1.127997 0.6215421 0.9030751
8.00 1.108598 0.6364725 0.8916759
16.00 1.107539 0.6378048 0.8910765
32.00 1.107539 0.6378048 0.8910765
64.00 1.107539 0.6378048 0.8910765
128.00 1.107539 0.6378048 0.8910765
Tuning parameter 'sigma' was held constant at a value of 0.01489089
RMSE was used to select the optimal model using the smallest value.
The final values used for the model were sigma = 0.01489089 and C = 16.
# Exclude linear SVM from nonlinear model selectionnonlinear_comparison <- comparison |>filter(Model !="SVM_Linear") |>arrange(RMSE)nonlinear_comparison
# A tibble: 4 × 4
Model RMSE Rsquared MAE
<chr> <dbl> <dbl> <dbl>
1 Neural_Network 1.23 0.645 0.955
2 SVM_Radial 1.29 0.607 0.941
3 KNN 1.37 0.549 0.972
4 MARS 1.42 0.553 1.02
# Best nonlinear modelbest_nonlinear <- nonlinear_comparison |>slice(1)best_nonlinear
# A tibble: 1 × 4
Model RMSE Rsquared MAE
<chr> <dbl> <dbl> <dbl>
1 Neural_Network 1.23 0.645 0.955
# Best cross-validation RMSE for each nonlinear modelcv_results <-bind_rows(lapply(list(KNN = knnModel,MARS = marsModel,SVM_Radial = svmRadialModel,Neural_Network = nnetModel ),function(model) { model$results |>slice_min(RMSE, n =1, with_ties =FALSE) |>select(RMSE, Rsquared, MAE) } ),.id ="Model") |>arrange(RMSE)cv_results
Model RMSE Rsquared MAE
1 SVM_Radial 1.107539 0.6378048 0.8910765
2 MARS 1.118314 0.6457536 0.8741773
3 KNN 1.244040 0.5408253 1.0232762
4 Neural_Network 1.648740 0.3599425 1.3342240
(a) Optimal Nonlinear Regression Model
Four nonlinear regression models were evaluated to predict chemical manufacturing yield: KNN, MARS, SVM Radial, and Neural Network. SVM Linear was also included for comparison.
The models were trained using 10-fold cross-validation and evaluated on an independent test dataset.
Based on cross-validation performance, SVM Radial achieved the lowest RMSE of 1.1075, followed closely by MARS with an RMSE of 1.1183. KNN and Neural Network produced higher cross-validation errors.
However, on the independent test dataset, Neural Network achieved the lowest RMSE of 1.2259, with an R-squared of 0.6448 and an MAE of 0.9553.
These results indicate that the best-performing model depends on the evaluation method. SVM Radial performed best during cross-validation, while Neural Network performed best on the test dataset.
Therefore, SVM Radial was selected as the optimal nonlinear regression model based on cross-validation RMSE, with Neural Network providing the strongest test-set performance.
Variable Importance Variable_Type
1 ManufacturingProcess32 0.6264861 Process
2 ManufacturingProcess36 0.3298613 Process
3 ManufacturingProcess09 0.2937248 Process
4 ManufacturingProcess37 0.2843384 Process
5 ManufacturingProcess13 0.2823769 Process
6 ManufacturingProcess28 0.2441497 Process
7 ManufacturingProcess11 0.2411713 Process
8 BiologicalMaterial03 0.2379343 Biological
9 ManufacturingProcess38 0.2293577 Process
10 ManufacturingProcess17 0.2237996 Process
# Count biological vs processtop10_nonlinear |>count(Variable_Type)
Permutation importance was used to identify the ten most influential predictors in the SVM Radial model.
ManufacturingProcess32 was the most important predictor, with an increase in RMSE of approximately 0.6265 when its values were randomly shuffled. ManufacturingProcess36 and ManufacturingProcess09 followed, with increases in RMSE of approximately 0.3299 and 0.2937, respectively.
Among the ten most important predictors, nine were manufacturing process variables and one was a biological material variable (BiologicalMaterial03).
These results suggest that manufacturing process variables contribute substantially to the SVM Radial model’s predictive performance. However, variable importance does not necessarily indicate a causal relationship with manufacturing yield.
c. Explore the relationships between the top predictors and the response for the predictors that are unique to the optimal nonlinear regression model. Do these plots reveal intuition about the biological or process predictors and their relationship with yield?
# Plot predictors unique to the nonlinear modelplot_data <- train_data |>select(Yield, all_of(unique_nonlinear)) |>pivot_longer(cols =all_of(unique_nonlinear),names_to ="Predictor",values_to ="Value" ) |>filter(!is.na(Value), !is.na(Yield))ggplot( plot_data,aes(x = Value, y = Yield)) +geom_point(alpha =0.5,color ="steelblue" ) +geom_smooth(method ="lm",formula = y ~ x +I(x^2),se =TRUE,color ="red" ) +facet_wrap(~ Predictor,scales ="free_x" ) +labs(title ="Unique SVM Radial Predictors vs Yield",x ="Predictor Value",y ="Manufacturing Yield (%)" ) +theme_minimal()
(c) Relationships Between Unique Nonlinear Predictors and Yield
The relationships between manufacturing yield and three predictors appearing in the SVM Radial model’s top-ten importance list, but not in the Ridge model’s top-ten list, were examined using scatterplots and quadratic regression curves.
ManufacturingProcess28 shows a potentially nonlinear relationship with yield. However, observations are concentrated in two distinct groups, making it difficult to determine the relationship between them. The wide confidence interval indicates substantial uncertainty in the fitted curve.
ManufacturingProcess37 shows a slight decrease in yield as its value increases, followed by a relatively stable relationship. However, the observations are widely scattered, suggesting a weak individual association with yield.
ManufacturingProcess38 shows a slight curved relationship, but most observations are concentrated at values of 2 and 3. Therefore, the evidence for a nonlinear relationship is limited.
Overall, these plots suggest that some manufacturing process predictors may have nonlinear associations with yield. However, the relationships are not sufficiently strong or consistent to establish clear nonlinear effects. The results also suggest that relationships between manufacturing process variables and yield may be more complex than those captured by simple individual predictor plots.