a. Fit a random forest model to all of the predictors, then estimate the variable importance scores:
library(randomForest)
randomForest 4.7-1.2
Type rfNews() to see new features/changes/bug fixes.
library(caret)
Loading required package: ggplot2
Warning: package 'ggplot2' was built under R version 4.5.2
Attaching package: 'ggplot2'
The following object is masked from 'package:randomForest':
margin
Loading required package: lattice
model1 <-randomForest(y ~ ., data = simulated, importance =TRUE, ntree=1000)model1
Call:
randomForest(formula = y ~ ., data = simulated, importance = TRUE, ntree = 1000)
Type of random forest: regression
Number of trees: 1000
No. of variables tried at each split: 3
Mean of squared residuals: 6.514982
% Var explained: 73.28
Did the random forest model significantly use the uninformative predictors (V6 – V10)?
##Answer:
The random forest model used all ten predictors (V1–V10) to predict the response variable y. However, the variable importance scores for the uninformative predictors (V6–V10) were relatively low, with some values near zero or negative. This suggests that these predictors contributed little to the model’s predictive performance compared with the informative predictors (V1–V5). Therefore, the random forest did not rely heavily on the uninformative predictors.
b. Now add an additional predictor that is highly correlated with one of the informative predictors. For example:
# Fit a new random forestset.seed(200)model2 <-randomForest( y ~ .,data = simulated2,importance =TRUE,ntree =1000)model2
Call:
randomForest(formula = y ~ ., data = simulated2, importance = TRUE, ntree = 1000)
Type of random forest: regression
Number of trees: 1000
No. of variables tried at each split: 3
Mean of squared residuals: 7.227598
% Var explained: 70.36
# Fit third random forestset.seed(200)model3 <-randomForest( y ~ .,data = simulated3,importance =TRUE,ntree =1000)# Variable importancerfImp3 <-varImp(model3, scale =FALSE)rfImp3
# Compare the importance of V1 across all three modelsdata.frame(Model =c("Model 1", "Model 2", "Model 3"),V1_Importance =c( rfImp1["V1", 1], rfImp2["V1", 1], rfImp3["V1", 1] ))
Model V1_Importance
1 Model 1 8.863298
2 Model 2 6.667840
3 Model 3 4.895570
##Answer:
After adding a predictor highly correlated with V1, the variable importance score for V1 decreased from 8.6254 to 6.6678. When another highly correlated predictor was added, the importance score decreased further to 4.8956. This suggests that the random forest model distributes predictive information among correlated predictors, reducing the individual importance score of V1. Therefore, adding highly correlated predictors can affect variable importance scores even when the original predictor remains informative.
c. Use the cforest function in the party package to fit a random forest model using conditional inference trees.
library(party)
Warning: package 'party' was built under R version 4.5.2
Loading required package: grid
Loading required package: mvtnorm
Loading required package: modeltools
Warning: package 'modeltools' was built under R version 4.5.2
Loading required package: stats4
Loading required package: strucchange
Warning: package 'strucchange' was built under R version 4.5.2
Loading required package: zoo
Warning: package 'zoo' was built under R version 4.5.2
Attaching package: 'zoo'
The following objects are masked from 'package:base':
as.Date, as.Date.numeric
Loading required package: sandwich
Warning: package 'sandwich' was built under R version 4.5.2
Model RandomForest Cforest_Traditional Cforest_Conditional
1 Model 1 8.863298 8.672661 5.604587
2 Model 2 6.667840 6.730145 3.135758
3 Model 3 4.895570 4.513619 1.707996
Interpretation
No, the cforest importance scores did not show exactly the same pattern as the traditional random forest. In the traditional random forest, V1’s importance decreased consistently as correlated predictors were added. However, both the traditional and conditional importance scores from cforest initially increased after adding the first correlated predictor and then decreased after adding the second. This suggests that correlated predictors can affect variable importance differently depending on the method used. Overall, all three methods showed a decrease in V1’s importance after adding two correlated predictors.
d. repeating with different tree model Boosted tree and Cubist tree
##install.packages('gbm')library(gbm)
Warning: package 'gbm' was built under R version 4.5.2
Loaded gbm 2.3.1
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
# Model 1: Original predictorsset.seed(200)gbmModel1 <-gbm( y ~ .,data = simulated,distribution ="gaussian",n.trees =1000,interaction.depth =3,shrinkage =0.01,verbose =FALSE)# Model 2: One correlated predictorset.seed(200)gbmModel2 <-gbm( y ~ .,data = simulated2,distribution ="gaussian",n.trees =1000,interaction.depth =3,shrinkage =0.01,verbose =FALSE)# Model 3: Two correlated predictorsset.seed(200)gbmModel3 <-gbm( y ~ .,data = simulated3,distribution ="gaussian",n.trees =1000,interaction.depth =3,shrinkage =0.01,verbose =FALSE)gbmImp1 <-summary(gbmModel1, plotit =FALSE)gbmImp2 <-summary(gbmModel2, plotit =FALSE)gbmImp3 <-summary(gbmModel3, plotit =FALSE)gbmImp1
Model GBM Cubist
1 Model 1 26.63777 71.0
2 Model 2 23.80413 71.0
3 Model 3 17.43017 57.5
Interpretation:
The boosted tree (GBM) and Cubist models did not show the same pattern as the traditional random forest. Both GBM and Cubist showed a similar pattern: the importance score of V1 increased from Model 1 to Model 2, then decreased in Model 3 after adding another highly correlated predictor. This suggests that correlated predictors affect variable importance differently depending on the modeling method used.
##Exercise 8.2
#install.packages('rpart')library(rpart)
Warning: package 'rpart' was built under R version 4.5.2
library(dplyr)
Warning: package 'dplyr' was built under R version 4.5.2
Attaching package: 'dplyr'
The following object is masked from 'package:party':
where
The following object is masked from 'package:randomForest':
combine
The following objects are masked from 'package:stats':
filter, lag
The following objects are masked from 'package:base':
intersect, setdiff, setequal, union
library(rpart)#library(rpart.plot)set.seed(200)n <-200simulated_tree <-data.frame(X1 =sample(1:2, n, replace =TRUE),X2 =sample(1:5, n, replace =TRUE),X3 =sample(1:20, n, replace =TRUE),X4 =runif(n))# Response independent of all predictorssimulated_tree$y <-rnorm(n)# Check the number of unique valuessapply(simulated_tree, function(x) length(unique(x)))
The simulation shows how different predictor granularities can affect variable importance in a decision tree. All predictors have the same sample size of 200 observations, but they have different numbers of unique values. Predictors with more unique values provide more possible splitting points, which can cause the decision tree to assign them higher importance even when they are not related to the response variable. This demonstrates how decision trees can be biased toward predictors with higher granularity.
Exercise 8.3
a. Why does the model on the right focus its importance on just the first few of predictors, whereas the model on the left spreads importance across more predictors?
The model on the right focuses on fewer predictors because it uses a higher learning rate and bagging fraction. The higher learning rate allows the model to make larger adjustments, while the higher bagging fraction uses more observations to build each tree. As a result, the model concentrates importance on a few predictors. The model on the left uses smaller values, which spreads importance across more predictors.
b. Which model do you think would be more predictive of other samples?
I would expect the model on the left to perform better on new samples because it uses a lower learning rate, allowing the model to learn more gradually and potentially reduce overfitting. The model on the right makes larger adjustments, which may increase the risk of overfitting. However, we would need to compare their prediction errors on test data to determine which model performs better.
c. How would increasing interaction depth affect the slope of predictor importance for either model in Fig. 8.24?
Increasing the interaction depth allows the boosted trees to capture more complex relationships between predictors. This may spread variable importance across more predictors and produce a flatter importance curve. However, if a few predictors dominate the interactions, the importance scores may remain concentrated among those predictors.
8.7
library(tidyverse)
Warning: package 'tibble' was built under R version 4.5.2
Warning: package 'tidyr' was built under R version 4.5.2
Warning: package 'readr' was built under R version 4.5.2
Warning: package 'purrr' was built under R version 4.5.2
Warning: package 'AppliedPredictiveModeling' was built under R version 4.5.2
library(caret)library(earth)
Warning: package 'earth' was built under R version 4.5.2
Loading required package: Formula
Loading required package: plotmo
Warning: package 'plotmo' was built under R version 4.5.2
Loading required package: plotrix
Warning: package 'plotrix' was built under R version 4.5.2
library(kernlab)
Attaching package: 'kernlab'
The following object is masked from 'package:purrr':
cross
The following object is masked from 'package:modeltools':
prior
The following object is masked from 'package:ggplot2':
alpha
library(nnet)
Warning: package 'nnet' was built under R version 4.5.2
'data.frame': 176 obs. of 58 variables:
$ Yield : num 38 42.4 42 41.4 42.5 ...
$ BiologicalMaterial01 : num 6.25 8.01 8.01 8.01 7.47 6.12 7.48 6.94 6.94 6.94 ...
$ BiologicalMaterial02 : num 49.6 61 61 61 63.3 ...
$ BiologicalMaterial03 : num 57 67.5 67.5 67.5 72.2 ...
$ BiologicalMaterial04 : num 12.7 14.7 14.7 14.7 14 ...
$ BiologicalMaterial05 : num 19.5 19.4 19.4 19.4 17.9 ...
$ BiologicalMaterial06 : num 43.7 53.1 53.1 53.1 54.7 ...
$ BiologicalMaterial07 : num 100 100 100 100 100 100 100 100 100 100 ...
$ BiologicalMaterial08 : num 16.7 19 19 19 18.2 ...
$ BiologicalMaterial09 : num 11.4 12.6 12.6 12.6 12.8 ...
$ BiologicalMaterial10 : num 3.46 3.46 3.46 3.46 3.05 3.78 3.04 3.85 3.85 3.85 ...
$ BiologicalMaterial11 : num 138 154 154 154 148 ...
$ BiologicalMaterial12 : num 18.8 21.1 21.1 21.1 21.1 ...
$ ManufacturingProcess01: num NA 0 0 0 10.7 12 11.5 12 12 12 ...
$ ManufacturingProcess02: num NA 0 0 0 0 0 0 0 0 0 ...
$ ManufacturingProcess03: num NA NA NA NA NA NA 1.56 1.55 1.56 1.55 ...
$ ManufacturingProcess04: num NA 917 912 911 918 924 933 929 928 938 ...
$ ManufacturingProcess05: num NA 1032 1004 1015 1028 ...
$ ManufacturingProcess06: num NA 210 207 213 206 ...
$ ManufacturingProcess07: num NA 177 178 177 178 178 177 178 177 177 ...
$ ManufacturingProcess08: num NA 178 178 177 178 178 178 178 177 177 ...
$ ManufacturingProcess09: num 43 46.6 45.1 44.9 45 ...
$ ManufacturingProcess10: num NA NA NA NA NA NA 11.6 10.2 9.7 10.1 ...
$ ManufacturingProcess11: num NA NA NA NA NA NA 11.5 11.3 11.1 10.2 ...
$ ManufacturingProcess12: num NA 0 0 0 0 0 0 0 0 0 ...
$ ManufacturingProcess13: num 35.5 34 34.8 34.8 34.6 34 32.4 33.6 33.9 34.3 ...
$ ManufacturingProcess14: num 4898 4869 4878 4897 4992 ...
$ ManufacturingProcess15: num 6108 6095 6087 6102 6233 ...
$ ManufacturingProcess16: num 4682 4617 4617 4635 4733 ...
$ ManufacturingProcess17: num 35.5 34 34.8 34.8 33.9 33.4 33.8 33.6 33.9 35.3 ...
$ ManufacturingProcess18: num 4865 4867 4877 4872 4886 ...
$ ManufacturingProcess19: num 6049 6097 6078 6073 6102 ...
$ ManufacturingProcess20: num 4665 4621 4621 4611 4659 ...
$ ManufacturingProcess21: num 0 0 0 0 -0.7 -0.6 1.4 0 0 1 ...
$ ManufacturingProcess22: num NA 3 4 5 8 9 1 2 3 4 ...
$ ManufacturingProcess23: num NA 0 1 2 4 1 1 2 3 1 ...
$ ManufacturingProcess24: num NA 3 4 5 18 1 1 2 3 4 ...
$ ManufacturingProcess25: num 4873 4869 4897 4892 4930 ...
$ ManufacturingProcess26: num 6074 6107 6116 6111 6151 ...
$ ManufacturingProcess27: num 4685 4630 4637 4630 4684 ...
$ ManufacturingProcess28: num 10.7 11.2 11.1 11.1 11.3 11.4 11.2 11.1 11.3 11.4 ...
$ ManufacturingProcess29: num 21 21.4 21.3 21.3 21.6 21.7 21.2 21.2 21.5 21.7 ...
$ ManufacturingProcess30: num 9.9 9.9 9.4 9.4 9 10.1 11.2 10.9 10.5 9.8 ...
$ ManufacturingProcess31: num 69.1 68.7 69.3 69.3 69.4 68.2 67.6 67.9 68 68.5 ...
$ ManufacturingProcess32: num 156 169 173 171 171 173 159 161 160 164 ...
$ ManufacturingProcess33: num 66 66 66 68 70 70 65 65 65 66 ...
$ ManufacturingProcess34: num 2.4 2.6 2.6 2.5 2.5 2.5 2.5 2.5 2.5 2.5 ...
$ ManufacturingProcess35: num 486 508 509 496 468 490 475 478 491 488 ...
$ ManufacturingProcess36: num 0.019 0.019 0.018 0.018 0.017 0.018 0.019 0.019 0.019 0.019 ...
$ ManufacturingProcess37: num 0.5 2 0.7 1.2 0.2 0.4 0.8 1 1.2 1.8 ...
$ ManufacturingProcess38: num 3 2 2 2 2 2 2 2 3 3 ...
$ ManufacturingProcess39: num 7.2 7.2 7.2 7.2 7.3 7.2 7.3 7.3 7.4 7.1 ...
$ ManufacturingProcess40: num NA 0.1 0 0 0 0 0 0 0 0 ...
$ ManufacturingProcess41: num NA 0.15 0 0 0 0 0 0 0 0 ...
$ ManufacturingProcess42: num 11.6 11.1 12 10.6 11 11.5 11.7 11.4 11.4 11.3 ...
$ ManufacturingProcess43: num 3 0.9 1 1.1 1.1 2.2 0.7 0.8 0.9 0.8 ...
$ ManufacturingProcess44: num 1.8 1.9 1.8 1.8 1.7 1.8 2 2 1.9 1.9 ...
$ ManufacturingProcess45: num 2.4 2.2 2.3 2.1 2.1 2 2.2 2.2 2.1 2.4 ...
summary(ChemicalManufacturingProcess)
Yield BiologicalMaterial01 BiologicalMaterial02 BiologicalMaterial03
Min. :35.25 Min. :4.580 Min. :46.87 Min. :56.97
1st Qu.:38.75 1st Qu.:5.978 1st Qu.:52.68 1st Qu.:64.98
Median :39.97 Median :6.305 Median :55.09 Median :67.22
Mean :40.18 Mean :6.411 Mean :55.69 Mean :67.70
3rd Qu.:41.48 3rd Qu.:6.870 3rd Qu.:58.74 3rd Qu.:70.43
Max. :46.34 Max. :8.810 Max. :64.75 Max. :78.25
BiologicalMaterial04 BiologicalMaterial05 BiologicalMaterial06
Min. : 9.38 Min. :13.24 Min. :40.60
1st Qu.:11.24 1st Qu.:17.23 1st Qu.:46.05
Median :12.10 Median :18.49 Median :48.46
Mean :12.35 Mean :18.60 Mean :48.91
3rd Qu.:13.22 3rd Qu.:19.90 3rd Qu.:51.34
Max. :23.09 Max. :24.85 Max. :59.38
BiologicalMaterial07 BiologicalMaterial08 BiologicalMaterial09
Min. :100.0 Min. :15.88 Min. :11.44
1st Qu.:100.0 1st Qu.:17.06 1st Qu.:12.60
Median :100.0 Median :17.51 Median :12.84
Mean :100.0 Mean :17.49 Mean :12.85
3rd Qu.:100.0 3rd Qu.:17.88 3rd Qu.:13.13
Max. :100.8 Max. :19.14 Max. :14.08
BiologicalMaterial10 BiologicalMaterial11 BiologicalMaterial12
Min. :1.770 Min. :135.8 Min. :18.35
1st Qu.:2.460 1st Qu.:143.8 1st Qu.:19.73
Median :2.710 Median :146.1 Median :20.12
Mean :2.801 Mean :147.0 Mean :20.20
3rd Qu.:2.990 3rd Qu.:149.6 3rd Qu.:20.75
Max. :6.870 Max. :158.7 Max. :22.21
ManufacturingProcess01 ManufacturingProcess02 ManufacturingProcess03
Min. : 0.00 Min. : 0.00 Min. :1.47
1st Qu.:10.80 1st Qu.:19.30 1st Qu.:1.53
Median :11.40 Median :21.00 Median :1.54
Mean :11.21 Mean :16.68 Mean :1.54
3rd Qu.:12.15 3rd Qu.:21.50 3rd Qu.:1.55
Max. :14.10 Max. :22.50 Max. :1.60
NA's :1 NA's :3 NA's :15
ManufacturingProcess04 ManufacturingProcess05 ManufacturingProcess06
Min. :911.0 Min. : 923.0 Min. :203.0
1st Qu.:928.0 1st Qu.: 986.8 1st Qu.:205.7
Median :934.0 Median : 999.2 Median :206.8
Mean :931.9 Mean :1001.7 Mean :207.4
3rd Qu.:936.0 3rd Qu.:1008.9 3rd Qu.:208.7
Max. :946.0 Max. :1175.3 Max. :227.4
NA's :1 NA's :1 NA's :2
ManufacturingProcess07 ManufacturingProcess08 ManufacturingProcess09
Min. :177.0 Min. :177.0 Min. :38.89
1st Qu.:177.0 1st Qu.:177.0 1st Qu.:44.89
Median :177.0 Median :178.0 Median :45.73
Mean :177.5 Mean :177.6 Mean :45.66
3rd Qu.:178.0 3rd Qu.:178.0 3rd Qu.:46.52
Max. :178.0 Max. :178.0 Max. :49.36
NA's :1 NA's :1
ManufacturingProcess10 ManufacturingProcess11 ManufacturingProcess12
Min. : 7.500 Min. : 7.500 Min. : 0.0
1st Qu.: 8.700 1st Qu.: 9.000 1st Qu.: 0.0
Median : 9.100 Median : 9.400 Median : 0.0
Mean : 9.179 Mean : 9.386 Mean : 857.8
3rd Qu.: 9.550 3rd Qu.: 9.900 3rd Qu.: 0.0
Max. :11.600 Max. :11.500 Max. :4549.0
NA's :9 NA's :10 NA's :1
ManufacturingProcess13 ManufacturingProcess14 ManufacturingProcess15
Min. :32.10 Min. :4701 Min. :5904
1st Qu.:33.90 1st Qu.:4828 1st Qu.:6010
Median :34.60 Median :4856 Median :6032
Mean :34.51 Mean :4854 Mean :6039
3rd Qu.:35.20 3rd Qu.:4882 3rd Qu.:6061
Max. :38.60 Max. :5055 Max. :6233
NA's :1
ManufacturingProcess16 ManufacturingProcess17 ManufacturingProcess18
Min. : 0 Min. :31.30 Min. : 0
1st Qu.:4561 1st Qu.:33.50 1st Qu.:4813
Median :4588 Median :34.40 Median :4835
Mean :4566 Mean :34.34 Mean :4810
3rd Qu.:4619 3rd Qu.:35.10 3rd Qu.:4862
Max. :4852 Max. :40.00 Max. :4971
ManufacturingProcess19 ManufacturingProcess20 ManufacturingProcess21
Min. :5890 Min. : 0 Min. :-1.8000
1st Qu.:6001 1st Qu.:4553 1st Qu.:-0.6000
Median :6022 Median :4582 Median :-0.3000
Mean :6028 Mean :4556 Mean :-0.1642
3rd Qu.:6050 3rd Qu.:4610 3rd Qu.: 0.0000
Max. :6146 Max. :4759 Max. : 3.6000
ManufacturingProcess22 ManufacturingProcess23 ManufacturingProcess24
Min. : 0.000 Min. :0.000 Min. : 0.000
1st Qu.: 3.000 1st Qu.:2.000 1st Qu.: 4.000
Median : 5.000 Median :3.000 Median : 8.000
Mean : 5.406 Mean :3.017 Mean : 8.834
3rd Qu.: 8.000 3rd Qu.:4.000 3rd Qu.:14.000
Max. :12.000 Max. :6.000 Max. :23.000
NA's :1 NA's :1 NA's :1
ManufacturingProcess25 ManufacturingProcess26 ManufacturingProcess27
Min. : 0 Min. : 0 Min. : 0
1st Qu.:4832 1st Qu.:6020 1st Qu.:4560
Median :4855 Median :6047 Median :4587
Mean :4828 Mean :6016 Mean :4563
3rd Qu.:4877 3rd Qu.:6070 3rd Qu.:4609
Max. :4990 Max. :6161 Max. :4710
NA's :5 NA's :5 NA's :5
ManufacturingProcess28 ManufacturingProcess29 ManufacturingProcess30
Min. : 0.000 Min. : 0.00 Min. : 0.000
1st Qu.: 0.000 1st Qu.:19.70 1st Qu.: 8.800
Median :10.400 Median :19.90 Median : 9.100
Mean : 6.592 Mean :20.01 Mean : 9.161
3rd Qu.:10.750 3rd Qu.:20.40 3rd Qu.: 9.700
Max. :11.500 Max. :22.00 Max. :11.200
NA's :5 NA's :5 NA's :5
ManufacturingProcess31 ManufacturingProcess32 ManufacturingProcess33
Min. : 0.00 Min. :143.0 Min. :56.00
1st Qu.:70.10 1st Qu.:155.0 1st Qu.:62.00
Median :70.80 Median :158.0 Median :64.00
Mean :70.18 Mean :158.5 Mean :63.54
3rd Qu.:71.40 3rd Qu.:162.0 3rd Qu.:65.00
Max. :72.50 Max. :173.0 Max. :70.00
NA's :5 NA's :5
ManufacturingProcess34 ManufacturingProcess35 ManufacturingProcess36
Min. :2.300 Min. :463.0 Min. :0.01700
1st Qu.:2.500 1st Qu.:490.0 1st Qu.:0.01900
Median :2.500 Median :495.0 Median :0.02000
Mean :2.494 Mean :495.6 Mean :0.01957
3rd Qu.:2.500 3rd Qu.:501.5 3rd Qu.:0.02000
Max. :2.600 Max. :522.0 Max. :0.02200
NA's :5 NA's :5 NA's :5
ManufacturingProcess37 ManufacturingProcess38 ManufacturingProcess39
Min. :0.000 Min. :0.000 Min. :0.000
1st Qu.:0.700 1st Qu.:2.000 1st Qu.:7.100
Median :1.000 Median :3.000 Median :7.200
Mean :1.014 Mean :2.534 Mean :6.851
3rd Qu.:1.300 3rd Qu.:3.000 3rd Qu.:7.300
Max. :2.300 Max. :3.000 Max. :7.500
ManufacturingProcess40 ManufacturingProcess41 ManufacturingProcess42
Min. :0.00000 Min. :0.00000 Min. : 0.00
1st Qu.:0.00000 1st Qu.:0.00000 1st Qu.:11.40
Median :0.00000 Median :0.00000 Median :11.60
Mean :0.01771 Mean :0.02371 Mean :11.21
3rd Qu.:0.00000 3rd Qu.:0.00000 3rd Qu.:11.70
Max. :0.10000 Max. :0.20000 Max. :12.10
NA's :1 NA's :1
ManufacturingProcess43 ManufacturingProcess44 ManufacturingProcess45
Min. : 0.0000 Min. :0.000 Min. :0.000
1st Qu.: 0.6000 1st Qu.:1.800 1st Qu.:2.100
Median : 0.8000 Median :1.900 Median :2.200
Mean : 0.9119 Mean :1.805 Mean :2.138
3rd Qu.: 1.0250 3rd Qu.:1.900 3rd Qu.:2.300
Max. :11.0000 Max. :2.100 Max. :2.600
The Cubist regression model showed the best overall test performance among the five tree-based models, with the lowest RMSE of 0.9540, MAE of 0.7306, and highest R² of 0.7949. Random Forest was the second-best model on the test set, followed by GBM, CART, and CTree. The cross-validation plot also showed that Cubist and GBM had relatively low prediction errors. Overall, Cubist appears to be the optimal tree-based model for predicting manufacturing Yield.
The Cubist model identified ManufacturingProcess32, ManufacturingProcess09, and ManufacturingProcess17 as the three most important predictors, with importance scores of 52.5, 35.0, and 30.0, respectively. Among the top 10 predictors, eight were manufacturing process variables and two were biological material variables. This suggests that manufacturing process variables play a major role in predicting Yield in the Cubist model. However, biological material variables also contribute to the prediction.
library(dplyr)# Import results from previous exercisestop10_linear <-read.csv("ridge_top10.csv")top10_nonlinear <-read.csv("svm_top10.csv")# Cubist vs Ridgecommon_linear <-intersect( top10_tree$Predictor, top10_linear$Predictor)# Cubist vs SVM Radialcommon_nonlinear <-intersect( top10_tree$Predictor, top10_nonlinear$Variable)common_linear
Warning: package 'rpart.plot' was built under R version 4.5.2
rpart.plot( cart_model$finalModel,#cubist_model$finalModel,type =2,extra =101,fallen.leaves =TRUE,main ="Regression Tree for Manufacturing Yield")
library(ggplot2)node_data <-data.frame(Yield = train_processed$Yield,Node =factor(cart_model$finalModel$where))ggplot(node_data, aes(x = Node, y = Yield)) +geom_boxplot(fill ="steelblue", alpha =0.7) +labs(title ="Distribution of Yield in Terminal Nodes",x ="Terminal Node",y ="Manufacturing Yield" ) +theme_minimal()
Interpretation:
Yes, the regression tree provides additional information about the relationships between biological material, manufacturing process predictors, and Yield. ManufacturingProcess32 is the first splitting variable, suggesting that it plays an important role in predicting Yield. Other process variables, including ManufacturingProcess17, ManufacturingProcess13, and ManufacturingProcess03, also contribute to the tree, while BiologicalMaterial03 and BiologicalMaterial05 indicate that biological characteristics are important as well.
The terminal-node boxplot shows that Yield differs across groups of observations. Node 5 has the lowest median Yield of approximately 37.3, while Node 15 has the highest median Yield of approximately 43.0. This suggests that certain combinations of biological and manufacturing process conditions are associated with higher manufacturing yields.
Overall, the regression tree provides additional insight by showing how different predictors interact to separate observations into groups with different Yield levels. However, these relationships are associations and do not necessarily indicate causation.