DATA624HW7
Magnus Skonberg
2021-11-06
library(dplyr)
library(forecast)
library(ggplot2)
library(caret)
library(VIM) #missing data visualization
library(tidyr)
library(mice)
library(corrplot)
library(MASS)
library(Boruta) #feature selection
#---User-defined function(s)---#
#Adapted correlation matrix used in EDA section:
plot_corr_matrix <- function(dataframe, significance_threshold){
title <- paste0('Correlation Matrix for significance > ',
significance_threshold)
df_cor <- dataframe %>% mutate_if(is.character, as.factor)
df_cor <- df_cor %>% mutate_if(is.factor, as.numeric)
#run a correlation and drop the insignificant ones
corr <- cor(df_cor, use = 'na.or.complete')
#prepare to drop duplicates and correlations of 1
corr[lower.tri(corr,diag=TRUE)] <- NA
#drop perfect correlations
corr[corr == 1] <- NA
#turn into a 3-column table
corr <- as.data.frame(as.table(corr))
#remove the NA values from above
corr <- na.omit(corr)
#select significant values
corr <- subset(corr, abs(Freq) > significance_threshold)
#sort by highest correlation
corr <- corr[order(-abs(corr$Freq)),]
#print table
# print(corr)
#turn corr back into matrix in order to plot with corrplot
mtx_corr <- reshape2::acast(corr, Var1~Var2, value.var="Freq")
#plot correlations visually
corrplot(mtx_corr,
title=title,
mar=c(0,0,1,0),
method='color',
tl.col="black",
na.label= " ",
addCoef.col = 'black',
number.cex = .9)
}Background
The purpose of this assignment was to explore Linear Regression exercises from Applied Predictive Modeling.
6.2
Developing a model to predict permeability (see Sect. 1.4) could save significant resources for a pharmaceutical company, while at the same time more rapidly identifying molecules that have a sufficient permeability to become a drug:
(a) Start R and use these commands to load the data:
library(AppliedPredictiveModeling)
data(package = "AppliedPredictiveModeling")$results[, "Item"]
data(permeability)The matrix fingerprints contains the 1,107 binary molecular predictors for the 165 compounds, while permeability contains permeability response.
(b) The fingerprint predictors indicate the presence or absence of substructures of a molecule and are often sparse meaning that relatively few of the molecules contain each substructure. Filter out the predictors that have low frequencies using the nearZeroVar function from the caret package. How many predictors are left for modeling?
The nearZeroVar function is used to identify predictors with very few unique values relative to the number of samples, these predictors are removed, and we then output how many predictors were removed v. how many remain:
#dim(fingerprints) 165 x 1107
#identify and remove low frequency predictors
low_freq <- nearZeroVar(fingerprints)
X <- fingerprints[,-low_freq]
#how many predictors were removed? how many are left?
(dim(fingerprints)[2] - dim(X)[2]) %>% paste(" predictors removed")## [1] "719 predictors removed"dim(X)[2] %>% paste(" predictors remain")## [1] "388 predictors remain"(c) Split the data into a training and a test set, pre-process the data, and tune a PLS model. How many latent variables are optimal and what is the corresponding resampled estimate of R^2?
set.seed(333)
#train test split - 75/25 split
sample_size = round(nrow(permeability)*.75)
index <- sample(seq_len(nrow(permeability)), size = sample_size)
#train-test split our data
X_train <- X[index, ]
X_test <- X[-index, ]
y_train <- permeability[index]
y_test <- permeability[-index]We apply a partial least squares function by mirroring the sample code from the course text and identifying our optimization metric as ‘Rsquared’, we then output model statistics:
plsTune <- train(X_train, y_train,
method='pls', metric='Rsquared',
tuneLength=20,
trControl=trainControl(method='cv'),
preProc=c('center', 'scale'))
#plsResult <- plsTune$results #necessary
plsTune## Partial Least Squares
##
## 124 samples
## 388 predictors
##
## Pre-processing: centered (388), scaled (388)
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 111, 112, 111, 111, 112, 112, ...
## Resampling results across tuning parameters:
##
## ncomp RMSE Rsquared MAE
## 1 11.598207 0.4516899 9.200876
## 2 9.894352 0.6085868 7.043954
## 3 9.656058 0.6212167 7.063028
## 4 9.847290 0.5982785 7.579568
## 5 9.789008 0.6079965 7.400446
## 6 9.644156 0.6248798 6.948888
## 7 9.611954 0.6337788 6.910518
## 8 9.592609 0.6314091 7.142761
## 9 10.004495 0.6138852 7.325439
## 10 9.999714 0.6162294 7.302291
## 11 10.009896 0.6112881 7.340993
## 12 9.980114 0.6076311 7.373643
## 13 10.117008 0.5934591 7.468921
## 14 10.350176 0.5803241 7.621719
## 15 10.512674 0.5714858 7.632762
## 16 10.693049 0.5569695 7.623068
## 17 10.677073 0.5608999 7.729981
## 18 10.697172 0.5629625 7.737699
## 19 10.876542 0.5516383 7.932201
## 20 10.930647 0.5483563 7.934937
##
## Rsquared was used to select the optimal model using the largest value.
## The final value used for the model was ncomp = 7.Our maximal R-squared 0.6260583 with an ncomp of 6.
(d) Predict the response for the test set. What is the test set estimate of R-squared.
We use the prediction() function in combination with postResample() to generate summary statistics for how our model performed on unseen, test data:
#predict test set response
plsPred <- predict(plsTune, newdata=X_test)
#test set estimate of Rsquared
postResample(pred=plsPred, obs=y_test)## RMSE Rsquared MAE
## 14.9605146 0.2536281 9.7955652Not well at all. An Rsquared of 0.2574 is low enough for us to rule out this partial least squares approach and consider another.
(e) Try building other models discussed in this chapter. Do any have better predictive performance?
LEFT OFF HERE
6.3
A chemical manufacturing process for a pharmaceutical product was discussed in section 1.4. In this problem, the objective is to understand the relationship between biological measurements of the raw materials (predictors), measurements of the manufacturing process (predictors), and the response of product yield. Biological predictors cannot be changed but can be used to assess the quality of the raw material before processing. On the other hand, manufacturing process predictors can be changed in the manufacturing process. Improving product yield by 1% will boost revenue by approximately one hundred thousand dollars per batch:
(a) Start R and use these comments to load the data:
library(AppliedPredictiveModeling)
data(package = "AppliedPredictiveModeling")$results[, "Item"]
data(ChemicalManufacturingProcess)The matrix processPredictors contains the 57 predictors (12 describing the input biological material and 45 describing the process predictors) for the 176 manufacturing runs. yield contains the percent yield for each run.
(b) A small percentage of cells in the predictor set contain missing values. Use an imputation function to fill in these missing values (e.g., see Sect. 3.8).
#convert data to matrix
cmp_matrix <- as.matrix(ChemicalManufacturingProcess)
#high level visualization of missing data
aggr(cmp_matrix, numbers=T)
From a high level, we see that the proportion of missing data is small and spread across numerous predictors. To deal with missing values I elect to use the pmm method (predictive mean matching) from the mice library.
Predictive mean matching calculates the predicted value for our target variable, and, for missing values, formulates a small set of “candidate donors” from complete cases that are closest to the predicted value for the missing entry. Donors are then chosen at random from candidates and imputed. To apply pmm we assume that the distribution is the same for missing cells as it is for observed data, and thus, the approach would be limited if the proportion of missing values were higher.
Once we’ve imputed missing values, we verify the success of the operation:
set.seed(333)
#impute missing values
#imputed_cmp_matrix <- preProcess(cmp_matrix, method = c("knnImpute"))
imputed_cmp_matrix <- mice(cmp_matrix, printFlag=F, method="pmm")
#verify no missing data
aggr(complete(imputed_cmp_matrix), numbers=T)
The fact that there are no proportion of missings bars and all full combinations confirms successful imputation.
(c) Split the data into a training and a test set, pre-process the data, and tune a model of your choice from this chapter. What is the optimal value of the performance metric?
With 57 predictors and 176 observations, it appears that feature exclusion will play an important role in preparing our data. I intend to twice filter the data to simplify the number of predictors we proceed to model building with, first by removing highly correlated variables and then utilizing the stepAIC() function (later) to remove low impact features.
We start by visiting the output of a custom-built correlation matrix function, where only variables with a correlation greater than 0.8 are shown:
cmp_comp <- complete(imputed_cmp_matrix)
#utilize custom-built correlation matrix generation function
plot_corr_matrix(cmp_comp, 0.8)
The correlation matrix above highlights that correlation between variables is indeed a concern and we can likely exclude a fair number of features to simplify our model. To do so, I lean on the findCorrelation() function with a threshold of 0.80. What this function does is search through the correlation matrix and return a vector of integers corresponding to columns to remove to reduce pair-wise correlations.
I then center and scale the data using R’s base scale() function, train test split the data using an 75/25 split (75% training), build a multi-linear regression model, and output the base model performance statistics:
highly_correlated = findCorrelation(cor(cmp_comp), 0.80)
cmp_comp1 = cmp_comp[, -highly_correlated]
#cmp_comp1
#center and scale
cmp_comp2 <- scale(cmp_comp1, center = TRUE, scale = TRUE)
#train test split - 75/25 split
sample_size = round(nrow(cmp_comp2)*.75)
index <- sample(seq_len(nrow(cmp_comp2)), size = sample_size)
train <- cmp_comp2[index, ]
test <- cmp_comp2[-index, ]
#build multi-linear regression model
lmFitAllPredictors <- lm(Yield ~ ., data = as.data.frame(train))
summary(lmFitAllPredictors)##
## Call:
## lm(formula = Yield ~ ., data = as.data.frame(train))
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.1925 -0.3229 -0.0120 0.3179 1.1339
##
## Coefficients: (1 not defined because of singularities)
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.009640 0.054561 0.177 0.86014
## BiologicalMaterial03 0.178093 0.132193 1.347 0.18122
## BiologicalMaterial05 0.077963 0.111974 0.696 0.48802
## BiologicalMaterial07 -0.049142 0.086397 -0.569 0.57088
## BiologicalMaterial09 -0.032002 0.133918 -0.239 0.81166
## BiologicalMaterial10 -0.114976 0.080688 -1.425 0.15756
## BiologicalMaterial11 0.004178 0.126613 0.033 0.97375
## ManufacturingProcess01 0.015917 0.104113 0.153 0.87883
## ManufacturingProcess02 -0.158583 0.164244 -0.966 0.33681
## ManufacturingProcess03 -0.057770 0.080293 -0.719 0.47366
## ManufacturingProcess04 0.177086 0.120073 1.475 0.14368
## ManufacturingProcess05 0.009864 0.071097 0.139 0.88996
## ManufacturingProcess06 0.097190 0.087135 1.115 0.26759
## ManufacturingProcess07 -0.087295 0.069814 -1.250 0.21433
## ManufacturingProcess08 -0.086231 0.078827 -1.094 0.27685
## ManufacturingProcess09 0.304707 0.164803 1.849 0.06768 .
## ManufacturingProcess10 -0.077350 0.135452 -0.571 0.56936
## ManufacturingProcess11 0.055640 0.256615 0.217 0.82883
## ManufacturingProcess12 0.043964 0.111148 0.396 0.69336
## ManufacturingProcess13 -0.170573 0.213330 -0.800 0.42602
## ManufacturingProcess16 0.006611 0.071945 0.092 0.92698
## ManufacturingProcess17 -0.097857 0.202503 -0.483 0.63008
## ManufacturingProcess19 0.114703 0.130544 0.879 0.38188
## ManufacturingProcess20 0.069457 0.090475 0.768 0.44463
## ManufacturingProcess21 NA NA NA NA
## ManufacturingProcess22 -0.050696 0.079295 -0.639 0.52420
## ManufacturingProcess23 0.011139 0.090527 0.123 0.90234
## ManufacturingProcess24 -0.084919 0.077679 -1.093 0.27716
## ManufacturingProcess26 0.068510 0.237713 0.288 0.77384
## ManufacturingProcess28 -0.173926 0.091472 -1.901 0.06038 .
## ManufacturingProcess30 -0.068663 0.324110 -0.212 0.83269
## ManufacturingProcess32 0.607762 0.157982 3.847 0.00022 ***
## ManufacturingProcess34 0.162768 0.067617 2.407 0.01807 *
## ManufacturingProcess35 -0.109014 0.117324 -0.929 0.35524
## ManufacturingProcess36 0.199705 0.168642 1.184 0.23938
## ManufacturingProcess37 -0.220737 0.076671 -2.879 0.00496 **
## ManufacturingProcess38 0.051956 0.087796 0.592 0.55545
## ManufacturingProcess39 0.152903 0.087820 1.741 0.08501 .
## ManufacturingProcess41 -0.024999 0.058010 -0.431 0.66752
## ManufacturingProcess43 0.084736 0.057983 1.461 0.14732
## ManufacturingProcess44 0.099937 0.128891 0.775 0.44011
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.58 on 92 degrees of freedom
## Multiple R-squared: 0.746, Adjusted R-squared: 0.6383
## F-statistic: 6.927 on 39 and 92 DF, p-value: 1.437e-14Our once filtered model has a promising R-squared value (0.7495). Additionally, it appears to only have 5 features that may be impertinent in addition to one with NAs (ManufacturingProcess21).
We proceed with set simplification via stepAIC():
aic_optimized_lm <- stepAIC(lmFitAllPredictors)## Start: AIC=-111.44
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial09 + BiologicalMaterial10 + BiologicalMaterial11 +
## ManufacturingProcess01 + ManufacturingProcess02 + ManufacturingProcess03 +
## ManufacturingProcess04 + ManufacturingProcess05 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess11 + ManufacturingProcess12 +
## ManufacturingProcess13 + ManufacturingProcess16 + ManufacturingProcess17 +
## ManufacturingProcess19 + ManufacturingProcess20 + ManufacturingProcess21 +
## ManufacturingProcess22 + ManufacturingProcess23 + ManufacturingProcess24 +
## ManufacturingProcess26 + ManufacturingProcess28 + ManufacturingProcess30 +
## ManufacturingProcess32 + ManufacturingProcess34 + ManufacturingProcess35 +
## ManufacturingProcess36 + ManufacturingProcess37 + ManufacturingProcess38 +
## ManufacturingProcess39 + ManufacturingProcess41 + ManufacturingProcess43 +
## ManufacturingProcess44
##
##
## Step: AIC=-111.44
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial09 + BiologicalMaterial10 + BiologicalMaterial11 +
## ManufacturingProcess01 + ManufacturingProcess02 + ManufacturingProcess03 +
## ManufacturingProcess04 + ManufacturingProcess05 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess11 + ManufacturingProcess12 +
## ManufacturingProcess13 + ManufacturingProcess16 + ManufacturingProcess17 +
## ManufacturingProcess19 + ManufacturingProcess20 + ManufacturingProcess22 +
## ManufacturingProcess23 + ManufacturingProcess24 + ManufacturingProcess26 +
## ManufacturingProcess28 + ManufacturingProcess30 + ManufacturingProcess32 +
## ManufacturingProcess34 + ManufacturingProcess35 + ManufacturingProcess36 +
## ManufacturingProcess37 + ManufacturingProcess38 + ManufacturingProcess39 +
## ManufacturingProcess41 + ManufacturingProcess43 + ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - BiologicalMaterial11 1 0.0004 30.954 -113.44
## - ManufacturingProcess16 1 0.0028 30.957 -113.43
## - ManufacturingProcess23 1 0.0051 30.959 -113.42
## - ManufacturingProcess05 1 0.0065 30.960 -113.41
## - ManufacturingProcess01 1 0.0079 30.962 -113.41
## - ManufacturingProcess30 1 0.0151 30.969 -113.38
## - ManufacturingProcess11 1 0.0158 30.970 -113.37
## - BiologicalMaterial09 1 0.0192 30.973 -113.36
## - ManufacturingProcess26 1 0.0279 30.982 -113.32
## - ManufacturingProcess12 1 0.0526 31.007 -113.22
## - ManufacturingProcess41 1 0.0625 31.016 -113.17
## - ManufacturingProcess17 1 0.0786 31.033 -113.11
## - BiologicalMaterial07 1 0.1089 31.063 -112.98
## - ManufacturingProcess10 1 0.1097 31.064 -112.97
## - ManufacturingProcess38 1 0.1178 31.072 -112.94
## - ManufacturingProcess22 1 0.1375 31.092 -112.85
## - BiologicalMaterial05 1 0.1631 31.117 -112.75
## - ManufacturingProcess03 1 0.1742 31.128 -112.70
## - ManufacturingProcess20 1 0.1983 31.152 -112.60
## - ManufacturingProcess44 1 0.2023 31.156 -112.58
## - ManufacturingProcess13 1 0.2151 31.169 -112.53
## - ManufacturingProcess19 1 0.2598 31.214 -112.34
## - ManufacturingProcess35 1 0.2905 31.244 -112.21
## - ManufacturingProcess02 1 0.3137 31.268 -112.11
## - ManufacturingProcess24 1 0.4021 31.356 -111.74
## - ManufacturingProcess08 1 0.4026 31.357 -111.73
## - ManufacturingProcess06 1 0.4186 31.373 -111.67
## - ManufacturingProcess36 1 0.4718 31.426 -111.44
## <none> 30.954 -111.44
## - ManufacturingProcess07 1 0.5260 31.480 -111.22
## - BiologicalMaterial03 1 0.6107 31.565 -110.86
## - BiologicalMaterial10 1 0.6832 31.637 -110.56
## - ManufacturingProcess43 1 0.7185 31.673 -110.41
## - ManufacturingProcess04 1 0.7318 31.686 -110.36
## - ManufacturingProcess39 1 1.0199 31.974 -109.16
## - ManufacturingProcess09 1 1.1502 32.104 -108.62
## - ManufacturingProcess28 1 1.2164 32.170 -108.35
## - ManufacturingProcess34 1 1.9496 32.904 -105.38
## - ManufacturingProcess37 1 2.7888 33.743 -102.05
## - ManufacturingProcess32 1 4.9794 35.933 -93.75
##
## Step: AIC=-113.44
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial09 + BiologicalMaterial10 + ManufacturingProcess01 +
## ManufacturingProcess02 + ManufacturingProcess03 + ManufacturingProcess04 +
## ManufacturingProcess05 + ManufacturingProcess06 + ManufacturingProcess07 +
## ManufacturingProcess08 + ManufacturingProcess09 + ManufacturingProcess10 +
## ManufacturingProcess11 + ManufacturingProcess12 + ManufacturingProcess13 +
## ManufacturingProcess16 + ManufacturingProcess17 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess22 + ManufacturingProcess23 +
## ManufacturingProcess24 + ManufacturingProcess26 + ManufacturingProcess28 +
## ManufacturingProcess30 + ManufacturingProcess32 + ManufacturingProcess34 +
## ManufacturingProcess35 + ManufacturingProcess36 + ManufacturingProcess37 +
## ManufacturingProcess38 + ManufacturingProcess39 + ManufacturingProcess41 +
## ManufacturingProcess43 + ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess16 1 0.0026 30.957 -115.43
## - ManufacturingProcess23 1 0.0055 30.960 -115.42
## - ManufacturingProcess01 1 0.0075 30.962 -115.41
## - ManufacturingProcess05 1 0.0076 30.962 -115.41
## - ManufacturingProcess30 1 0.0152 30.970 -115.37
## - ManufacturingProcess11 1 0.0161 30.971 -115.37
## - BiologicalMaterial09 1 0.0234 30.978 -115.34
## - ManufacturingProcess26 1 0.0280 30.982 -115.32
## - ManufacturingProcess12 1 0.0525 31.007 -115.21
## - ManufacturingProcess41 1 0.0623 31.017 -115.17
## - ManufacturingProcess17 1 0.0785 31.033 -115.10
## - BiologicalMaterial07 1 0.1091 31.063 -114.97
## - ManufacturingProcess10 1 0.1121 31.066 -114.96
## - ManufacturingProcess38 1 0.1221 31.077 -114.92
## - ManufacturingProcess22 1 0.1383 31.093 -114.85
## - ManufacturingProcess03 1 0.1770 31.131 -114.69
## - ManufacturingProcess20 1 0.1979 31.152 -114.60
## - ManufacturingProcess44 1 0.2070 31.161 -114.56
## - ManufacturingProcess13 1 0.2196 31.174 -114.50
## - BiologicalMaterial05 1 0.2282 31.183 -114.47
## - ManufacturingProcess19 1 0.2597 31.214 -114.33
## - ManufacturingProcess35 1 0.2945 31.249 -114.19
## - ManufacturingProcess02 1 0.3241 31.278 -114.06
## - ManufacturingProcess08 1 0.4040 31.358 -113.73
## - ManufacturingProcess24 1 0.4076 31.362 -113.71
## - ManufacturingProcess06 1 0.4193 31.374 -113.66
## <none> 30.954 -113.44
## - ManufacturingProcess36 1 0.4776 31.432 -113.42
## - ManufacturingProcess07 1 0.5273 31.482 -113.21
## - BiologicalMaterial03 1 0.6654 31.620 -112.63
## - BiologicalMaterial10 1 0.6828 31.637 -112.56
## - ManufacturingProcess43 1 0.7212 31.676 -112.40
## - ManufacturingProcess04 1 0.7573 31.712 -112.25
## - ManufacturingProcess39 1 1.0198 31.974 -111.16
## - ManufacturingProcess09 1 1.1591 32.113 -110.59
## - ManufacturingProcess28 1 1.3099 32.264 -109.97
## - ManufacturingProcess34 1 2.0171 32.972 -107.11
## - ManufacturingProcess37 1 2.8070 33.761 -103.98
## - ManufacturingProcess32 1 5.0306 35.985 -95.56
##
## Step: AIC=-115.43
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial09 + BiologicalMaterial10 + ManufacturingProcess01 +
## ManufacturingProcess02 + ManufacturingProcess03 + ManufacturingProcess04 +
## ManufacturingProcess05 + ManufacturingProcess06 + ManufacturingProcess07 +
## ManufacturingProcess08 + ManufacturingProcess09 + ManufacturingProcess10 +
## ManufacturingProcess11 + ManufacturingProcess12 + ManufacturingProcess13 +
## ManufacturingProcess17 + ManufacturingProcess19 + ManufacturingProcess20 +
## ManufacturingProcess22 + ManufacturingProcess23 + ManufacturingProcess24 +
## ManufacturingProcess26 + ManufacturingProcess28 + ManufacturingProcess30 +
## ManufacturingProcess32 + ManufacturingProcess34 + ManufacturingProcess35 +
## ManufacturingProcess36 + ManufacturingProcess37 + ManufacturingProcess38 +
## ManufacturingProcess39 + ManufacturingProcess41 + ManufacturingProcess43 +
## ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess01 1 0.0057 30.963 -117.402
## - ManufacturingProcess23 1 0.0063 30.963 -117.400
## - ManufacturingProcess05 1 0.0082 30.965 -117.392
## - ManufacturingProcess30 1 0.0223 30.979 -117.332
## - BiologicalMaterial09 1 0.0291 30.986 -117.303
## - ManufacturingProcess11 1 0.0356 30.993 -117.275
## - ManufacturingProcess26 1 0.0378 30.995 -117.266
## - ManufacturingProcess12 1 0.0571 31.014 -117.184
## - ManufacturingProcess41 1 0.0621 31.019 -117.162
## - ManufacturingProcess17 1 0.0837 31.041 -117.070
## - BiologicalMaterial07 1 0.1089 31.066 -116.963
## - ManufacturingProcess10 1 0.1138 31.071 -116.942
## - ManufacturingProcess38 1 0.1199 31.077 -116.916
## - ManufacturingProcess22 1 0.1430 31.100 -116.818
## - ManufacturingProcess03 1 0.1782 31.135 -116.669
## - ManufacturingProcess20 1 0.2066 31.164 -116.549
## - ManufacturingProcess44 1 0.2158 31.173 -116.510
## - ManufacturingProcess13 1 0.2197 31.177 -116.493
## - BiologicalMaterial05 1 0.2272 31.184 -116.462
## - ManufacturingProcess19 1 0.2911 31.248 -116.192
## - ManufacturingProcess35 1 0.3121 31.269 -116.103
## - ManufacturingProcess02 1 0.3360 31.293 -116.002
## - ManufacturingProcess08 1 0.4035 31.360 -115.717
## - ManufacturingProcess06 1 0.4184 31.375 -115.655
## - ManufacturingProcess24 1 0.4379 31.395 -115.573
## <none> 30.957 -115.427
## - ManufacturingProcess36 1 0.4955 31.452 -115.331
## - ManufacturingProcess07 1 0.5259 31.483 -115.203
## - BiologicalMaterial10 1 0.6835 31.640 -114.544
## - BiologicalMaterial03 1 0.7242 31.681 -114.375
## - ManufacturingProcess43 1 0.7262 31.683 -114.366
## - ManufacturingProcess04 1 0.7547 31.712 -114.247
## - ManufacturingProcess39 1 1.0195 31.976 -113.150
## - ManufacturingProcess09 1 1.2022 32.159 -112.398
## - ManufacturingProcess28 1 1.3449 32.302 -111.813
## - ManufacturingProcess34 1 2.0390 32.996 -109.007
## - ManufacturingProcess37 1 2.8613 33.818 -105.757
## - ManufacturingProcess32 1 5.0374 35.994 -97.526
##
## Step: AIC=-117.4
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial09 + BiologicalMaterial10 + ManufacturingProcess02 +
## ManufacturingProcess03 + ManufacturingProcess04 + ManufacturingProcess05 +
## ManufacturingProcess06 + ManufacturingProcess07 + ManufacturingProcess08 +
## ManufacturingProcess09 + ManufacturingProcess10 + ManufacturingProcess11 +
## ManufacturingProcess12 + ManufacturingProcess13 + ManufacturingProcess17 +
## ManufacturingProcess19 + ManufacturingProcess20 + ManufacturingProcess22 +
## ManufacturingProcess23 + ManufacturingProcess24 + ManufacturingProcess26 +
## ManufacturingProcess28 + ManufacturingProcess30 + ManufacturingProcess32 +
## ManufacturingProcess34 + ManufacturingProcess35 + ManufacturingProcess36 +
## ManufacturingProcess37 + ManufacturingProcess38 + ManufacturingProcess39 +
## ManufacturingProcess41 + ManufacturingProcess43 + ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess23 1 0.0084 30.971 -119.367
## - ManufacturingProcess05 1 0.0098 30.972 -119.361
## - ManufacturingProcess30 1 0.0233 30.986 -119.303
## - BiologicalMaterial09 1 0.0343 30.997 -119.257
## - ManufacturingProcess26 1 0.0398 31.003 -119.233
## - ManufacturingProcess11 1 0.0499 31.013 -119.190
## - ManufacturingProcess41 1 0.0691 31.032 -119.108
## - ManufacturingProcess12 1 0.0706 31.033 -119.102
## - ManufacturingProcess17 1 0.0841 31.047 -119.044
## - ManufacturingProcess10 1 0.1156 31.078 -118.911
## - BiologicalMaterial07 1 0.1169 31.080 -118.905
## - ManufacturingProcess22 1 0.1417 31.104 -118.800
## - ManufacturingProcess38 1 0.1421 31.105 -118.798
## - ManufacturingProcess03 1 0.1728 31.136 -118.668
## - ManufacturingProcess44 1 0.2107 31.173 -118.507
## - ManufacturingProcess20 1 0.2181 31.181 -118.476
## - BiologicalMaterial05 1 0.2218 31.185 -118.460
## - ManufacturingProcess13 1 0.2309 31.194 -118.422
## - ManufacturingProcess02 1 0.3305 31.293 -118.001
## - ManufacturingProcess19 1 0.3333 31.296 -117.989
## - ManufacturingProcess35 1 0.3586 31.321 -117.882
## - ManufacturingProcess06 1 0.4183 31.381 -117.631
## - ManufacturingProcess24 1 0.4378 31.400 -117.549
## - ManufacturingProcess08 1 0.4441 31.407 -117.522
## <none> 30.963 -117.402
## - ManufacturingProcess36 1 0.5190 31.482 -117.208
## - ManufacturingProcess07 1 0.5229 31.486 -117.192
## - BiologicalMaterial10 1 0.6779 31.641 -116.544
## - ManufacturingProcess43 1 0.7912 31.754 -116.072
## - BiologicalMaterial03 1 0.8355 31.798 -115.888
## - ManufacturingProcess39 1 1.0172 31.980 -115.135
## - ManufacturingProcess04 1 1.1484 32.111 -114.595
## - ManufacturingProcess09 1 1.3561 32.319 -113.744
## - ManufacturingProcess28 1 1.3693 32.332 -113.690
## - ManufacturingProcess34 1 2.0335 32.996 -111.006
## - ManufacturingProcess37 1 2.9643 33.927 -107.334
## - ManufacturingProcess32 1 5.0622 36.025 -99.414
##
## Step: AIC=-119.37
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial09 + BiologicalMaterial10 + ManufacturingProcess02 +
## ManufacturingProcess03 + ManufacturingProcess04 + ManufacturingProcess05 +
## ManufacturingProcess06 + ManufacturingProcess07 + ManufacturingProcess08 +
## ManufacturingProcess09 + ManufacturingProcess10 + ManufacturingProcess11 +
## ManufacturingProcess12 + ManufacturingProcess13 + ManufacturingProcess17 +
## ManufacturingProcess19 + ManufacturingProcess20 + ManufacturingProcess22 +
## ManufacturingProcess24 + ManufacturingProcess26 + ManufacturingProcess28 +
## ManufacturingProcess30 + ManufacturingProcess32 + ManufacturingProcess34 +
## ManufacturingProcess35 + ManufacturingProcess36 + ManufacturingProcess37 +
## ManufacturingProcess38 + ManufacturingProcess39 + ManufacturingProcess41 +
## ManufacturingProcess43 + ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess05 1 0.0071 30.978 -121.34
## - ManufacturingProcess30 1 0.0235 30.995 -121.27
## - BiologicalMaterial09 1 0.0387 31.010 -121.20
## - ManufacturingProcess26 1 0.0399 31.011 -121.20
## - ManufacturingProcess11 1 0.0494 31.020 -121.16
## - ManufacturingProcess41 1 0.0668 31.038 -121.08
## - ManufacturingProcess12 1 0.0727 31.044 -121.06
## - ManufacturingProcess17 1 0.0838 31.055 -121.01
## - ManufacturingProcess10 1 0.1148 31.086 -120.88
## - BiologicalMaterial07 1 0.1151 31.086 -120.88
## - ManufacturingProcess38 1 0.1382 31.109 -120.78
## - ManufacturingProcess22 1 0.1772 31.148 -120.61
## - ManufacturingProcess03 1 0.1893 31.160 -120.56
## - ManufacturingProcess44 1 0.2073 31.178 -120.49
## - ManufacturingProcess13 1 0.2237 31.195 -120.42
## - ManufacturingProcess20 1 0.2281 31.199 -120.40
## - BiologicalMaterial05 1 0.2442 31.215 -120.33
## - ManufacturingProcess02 1 0.3222 31.293 -120.00
## - ManufacturingProcess19 1 0.3392 31.310 -119.93
## - ManufacturingProcess35 1 0.3595 31.331 -119.84
## - ManufacturingProcess08 1 0.4394 31.410 -119.51
## - ManufacturingProcess06 1 0.4414 31.412 -119.50
## <none> 30.971 -119.37
## - ManufacturingProcess24 1 0.5069 31.478 -119.22
## - ManufacturingProcess07 1 0.5191 31.490 -119.17
## - ManufacturingProcess36 1 0.5243 31.495 -119.15
## - BiologicalMaterial10 1 0.6703 31.641 -118.54
## - ManufacturingProcess43 1 0.7933 31.764 -118.03
## - BiologicalMaterial03 1 0.8750 31.846 -117.69
## - ManufacturingProcess39 1 1.0559 32.027 -116.94
## - ManufacturingProcess04 1 1.1478 32.119 -116.56
## - ManufacturingProcess28 1 1.3610 32.332 -115.69
## - ManufacturingProcess09 1 1.4188 32.390 -115.45
## - ManufacturingProcess34 1 2.0372 33.008 -112.96
## - ManufacturingProcess37 1 3.1131 34.084 -108.72
## - ManufacturingProcess32 1 5.0700 36.041 -101.36
##
## Step: AIC=-121.34
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial09 + BiologicalMaterial10 + ManufacturingProcess02 +
## ManufacturingProcess03 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess11 + ManufacturingProcess12 +
## ManufacturingProcess13 + ManufacturingProcess17 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess22 + ManufacturingProcess24 +
## ManufacturingProcess26 + ManufacturingProcess28 + ManufacturingProcess30 +
## ManufacturingProcess32 + ManufacturingProcess34 + ManufacturingProcess35 +
## ManufacturingProcess36 + ManufacturingProcess37 + ManufacturingProcess38 +
## ManufacturingProcess39 + ManufacturingProcess41 + ManufacturingProcess43 +
## ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess30 1 0.0247 31.003 -123.23
## - BiologicalMaterial09 1 0.0388 31.017 -123.17
## - ManufacturingProcess26 1 0.0414 31.020 -123.16
## - ManufacturingProcess11 1 0.0531 31.031 -123.11
## - ManufacturingProcess41 1 0.0619 31.040 -123.07
## - ManufacturingProcess12 1 0.0691 31.047 -123.04
## - ManufacturingProcess17 1 0.0854 31.064 -122.97
## - BiologicalMaterial07 1 0.1130 31.091 -122.86
## - ManufacturingProcess10 1 0.1131 31.091 -122.86
## - ManufacturingProcess38 1 0.1346 31.113 -122.76
## - ManufacturingProcess22 1 0.1748 31.153 -122.59
## - ManufacturingProcess03 1 0.1825 31.161 -122.56
## - ManufacturingProcess44 1 0.2031 31.181 -122.47
## - ManufacturingProcess13 1 0.2204 31.199 -122.40
## - ManufacturingProcess20 1 0.2288 31.207 -122.36
## - BiologicalMaterial05 1 0.2518 31.230 -122.27
## - ManufacturingProcess02 1 0.3296 31.308 -121.94
## - ManufacturingProcess19 1 0.3366 31.315 -121.91
## - ManufacturingProcess35 1 0.3528 31.331 -121.84
## - ManufacturingProcess08 1 0.4337 31.412 -121.50
## - ManufacturingProcess06 1 0.4398 31.418 -121.48
## <none> 30.978 -121.34
## - ManufacturingProcess24 1 0.5048 31.483 -121.20
## - ManufacturingProcess36 1 0.5173 31.495 -121.15
## - ManufacturingProcess07 1 0.5316 31.510 -121.09
## - BiologicalMaterial10 1 0.6662 31.644 -120.53
## - ManufacturingProcess43 1 0.8018 31.780 -119.96
## - BiologicalMaterial03 1 0.8705 31.849 -119.68
## - ManufacturingProcess39 1 1.0520 32.030 -118.93
## - ManufacturingProcess04 1 1.1565 32.135 -118.50
## - ManufacturingProcess28 1 1.3547 32.333 -117.69
## - ManufacturingProcess09 1 1.4740 32.452 -117.20
## - ManufacturingProcess34 1 2.0327 33.011 -114.95
## - ManufacturingProcess37 1 3.1082 34.086 -110.72
## - ManufacturingProcess32 1 5.0632 36.041 -103.35
##
## Step: AIC=-123.23
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial09 + BiologicalMaterial10 + ManufacturingProcess02 +
## ManufacturingProcess03 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess11 + ManufacturingProcess12 +
## ManufacturingProcess13 + ManufacturingProcess17 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess22 + ManufacturingProcess24 +
## ManufacturingProcess26 + ManufacturingProcess28 + ManufacturingProcess32 +
## ManufacturingProcess34 + ManufacturingProcess35 + ManufacturingProcess36 +
## ManufacturingProcess37 + ManufacturingProcess38 + ManufacturingProcess39 +
## ManufacturingProcess41 + ManufacturingProcess43 + ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess11 1 0.0291 31.032 -125.11
## - BiologicalMaterial09 1 0.0358 31.039 -125.08
## - ManufacturingProcess26 1 0.0524 31.055 -125.01
## - ManufacturingProcess17 1 0.0656 31.068 -124.95
## - ManufacturingProcess41 1 0.0751 31.078 -124.91
## - ManufacturingProcess12 1 0.0753 31.078 -124.91
## - BiologicalMaterial07 1 0.1134 31.116 -124.75
## - ManufacturingProcess38 1 0.1366 31.140 -124.65
## - ManufacturingProcess10 1 0.1394 31.142 -124.64
## - ManufacturingProcess44 1 0.1840 31.187 -124.45
## - ManufacturingProcess03 1 0.1841 31.187 -124.45
## - ManufacturingProcess22 1 0.1951 31.198 -124.40
## - ManufacturingProcess20 1 0.2046 31.208 -124.36
## - BiologicalMaterial05 1 0.2326 31.236 -124.25
## - ManufacturingProcess13 1 0.2375 31.240 -124.22
## - ManufacturingProcess02 1 0.3170 31.320 -123.89
## - ManufacturingProcess35 1 0.3335 31.336 -123.82
## - ManufacturingProcess08 1 0.4375 31.440 -123.38
## - ManufacturingProcess06 1 0.4419 31.445 -123.36
## <none> 31.003 -123.23
## - ManufacturingProcess19 1 0.4897 31.493 -123.16
## - ManufacturingProcess36 1 0.4969 31.500 -123.13
## - ManufacturingProcess07 1 0.5219 31.525 -123.03
## - ManufacturingProcess24 1 0.5344 31.537 -122.97
## - BiologicalMaterial10 1 0.6745 31.677 -122.39
## - ManufacturingProcess43 1 0.7797 31.783 -121.95
## - BiologicalMaterial03 1 1.0175 32.020 -120.97
## - ManufacturingProcess39 1 1.1018 32.105 -120.62
## - ManufacturingProcess04 1 1.1765 32.179 -120.31
## - ManufacturingProcess28 1 1.3556 32.358 -119.58
## - ManufacturingProcess09 1 1.4654 32.468 -119.14
## - ManufacturingProcess34 1 2.0168 33.020 -116.91
## - ManufacturingProcess37 1 3.1023 34.105 -112.64
## - ManufacturingProcess32 1 5.0445 36.047 -105.33
##
## Step: AIC=-125.11
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial09 + BiologicalMaterial10 + ManufacturingProcess02 +
## ManufacturingProcess03 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess12 + ManufacturingProcess13 +
## ManufacturingProcess17 + ManufacturingProcess19 + ManufacturingProcess20 +
## ManufacturingProcess22 + ManufacturingProcess24 + ManufacturingProcess26 +
## ManufacturingProcess28 + ManufacturingProcess32 + ManufacturingProcess34 +
## ManufacturingProcess35 + ManufacturingProcess36 + ManufacturingProcess37 +
## ManufacturingProcess38 + ManufacturingProcess39 + ManufacturingProcess41 +
## ManufacturingProcess43 + ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - BiologicalMaterial09 1 0.0390 31.071 -126.94
## - ManufacturingProcess26 1 0.0549 31.087 -126.87
## - ManufacturingProcess12 1 0.0669 31.099 -126.82
## - ManufacturingProcess41 1 0.0782 31.110 -126.78
## - BiologicalMaterial07 1 0.1071 31.139 -126.65
## - ManufacturingProcess10 1 0.1110 31.143 -126.64
## - ManufacturingProcess03 1 0.1596 31.192 -126.43
## - ManufacturingProcess38 1 0.1633 31.195 -126.42
## - ManufacturingProcess44 1 0.1842 31.216 -126.33
## - ManufacturingProcess20 1 0.1971 31.229 -126.27
## - ManufacturingProcess17 1 0.2044 31.236 -126.24
## - ManufacturingProcess22 1 0.2090 31.241 -126.22
## - ManufacturingProcess13 1 0.2117 31.244 -126.21
## - BiologicalMaterial05 1 0.3328 31.365 -125.70
## - ManufacturingProcess35 1 0.3706 31.403 -125.54
## - ManufacturingProcess08 1 0.4354 31.467 -125.27
## <none> 31.032 -125.11
## - ManufacturingProcess06 1 0.4792 31.511 -125.08
## - ManufacturingProcess07 1 0.5110 31.543 -124.95
## - ManufacturingProcess36 1 0.5326 31.565 -124.86
## - ManufacturingProcess24 1 0.5434 31.575 -124.82
## - ManufacturingProcess02 1 0.5897 31.622 -124.62
## - ManufacturingProcess19 1 0.6284 31.660 -124.46
## - BiologicalMaterial10 1 0.6643 31.696 -124.31
## - ManufacturingProcess43 1 0.7802 31.812 -123.83
## - BiologicalMaterial03 1 1.0097 32.042 -122.88
## - ManufacturingProcess39 1 1.0998 32.132 -122.51
## - ManufacturingProcess04 1 1.1664 32.198 -122.24
## - ManufacturingProcess28 1 1.3270 32.359 -121.58
## - ManufacturingProcess34 1 1.9987 33.031 -118.87
## - ManufacturingProcess09 1 2.1424 33.174 -118.30
## - ManufacturingProcess37 1 3.1861 34.218 -114.21
## - ManufacturingProcess32 1 5.3311 36.363 -106.18
##
## Step: AIC=-126.94
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial10 + ManufacturingProcess02 + ManufacturingProcess03 +
## ManufacturingProcess04 + ManufacturingProcess06 + ManufacturingProcess07 +
## ManufacturingProcess08 + ManufacturingProcess09 + ManufacturingProcess10 +
## ManufacturingProcess12 + ManufacturingProcess13 + ManufacturingProcess17 +
## ManufacturingProcess19 + ManufacturingProcess20 + ManufacturingProcess22 +
## ManufacturingProcess24 + ManufacturingProcess26 + ManufacturingProcess28 +
## ManufacturingProcess32 + ManufacturingProcess34 + ManufacturingProcess35 +
## ManufacturingProcess36 + ManufacturingProcess37 + ManufacturingProcess38 +
## ManufacturingProcess39 + ManufacturingProcess41 + ManufacturingProcess43 +
## ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess26 1 0.0619 31.133 -128.68
## - ManufacturingProcess12 1 0.0721 31.143 -128.63
## - ManufacturingProcess41 1 0.0966 31.168 -128.53
## - BiologicalMaterial07 1 0.1074 31.178 -128.49
## - ManufacturingProcess10 1 0.1134 31.184 -128.46
## - ManufacturingProcess38 1 0.1494 31.220 -128.31
## - ManufacturingProcess44 1 0.1653 31.236 -128.24
## - ManufacturingProcess03 1 0.1823 31.253 -128.17
## - ManufacturingProcess17 1 0.2043 31.275 -128.08
## - ManufacturingProcess13 1 0.2128 31.284 -128.04
## - ManufacturingProcess22 1 0.2317 31.303 -127.96
## - ManufacturingProcess20 1 0.2403 31.311 -127.92
## - ManufacturingProcess35 1 0.3739 31.445 -127.36
## - ManufacturingProcess08 1 0.4395 31.511 -127.09
## <none> 31.071 -126.94
## - ManufacturingProcess06 1 0.4776 31.549 -126.93
## - ManufacturingProcess07 1 0.4879 31.559 -126.89
## - ManufacturingProcess24 1 0.5045 31.576 -126.81
## - BiologicalMaterial05 1 0.5560 31.627 -126.60
## - ManufacturingProcess36 1 0.5662 31.637 -126.56
## - ManufacturingProcess19 1 0.5897 31.661 -126.46
## - ManufacturingProcess43 1 0.8077 31.879 -125.55
## - ManufacturingProcess02 1 0.8379 31.909 -125.43
## - BiologicalMaterial10 1 0.9959 32.067 -124.78
## - ManufacturingProcess39 1 1.1029 32.174 -124.34
## - ManufacturingProcess04 1 1.1601 32.231 -124.10
## - ManufacturingProcess28 1 1.4634 32.534 -122.87
## - BiologicalMaterial03 1 1.4789 32.550 -122.80
## - ManufacturingProcess34 1 1.9626 33.034 -120.86
## - ManufacturingProcess09 1 2.1574 33.228 -120.08
## - ManufacturingProcess37 1 3.1723 34.243 -116.11
## - ManufacturingProcess32 1 6.4613 37.532 -104.00
##
## Step: AIC=-128.68
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial10 + ManufacturingProcess02 + ManufacturingProcess03 +
## ManufacturingProcess04 + ManufacturingProcess06 + ManufacturingProcess07 +
## ManufacturingProcess08 + ManufacturingProcess09 + ManufacturingProcess10 +
## ManufacturingProcess12 + ManufacturingProcess13 + ManufacturingProcess17 +
## ManufacturingProcess19 + ManufacturingProcess20 + ManufacturingProcess22 +
## ManufacturingProcess24 + ManufacturingProcess28 + ManufacturingProcess32 +
## ManufacturingProcess34 + ManufacturingProcess35 + ManufacturingProcess36 +
## ManufacturingProcess37 + ManufacturingProcess38 + ManufacturingProcess39 +
## ManufacturingProcess41 + ManufacturingProcess43 + ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess12 1 0.0795 31.212 -130.34
## - ManufacturingProcess41 1 0.0911 31.224 -130.29
## - BiologicalMaterial07 1 0.1039 31.237 -130.24
## - ManufacturingProcess10 1 0.1192 31.252 -130.17
## - ManufacturingProcess38 1 0.1274 31.260 -130.14
## - ManufacturingProcess03 1 0.1466 31.280 -130.06
## - ManufacturingProcess44 1 0.1819 31.315 -129.91
## - ManufacturingProcess17 1 0.2200 31.353 -129.75
## - ManufacturingProcess20 1 0.2232 31.356 -129.74
## - ManufacturingProcess13 1 0.2311 31.364 -129.70
## - ManufacturingProcess22 1 0.2489 31.382 -129.63
## - ManufacturingProcess35 1 0.3467 31.480 -129.22
## - ManufacturingProcess06 1 0.4709 31.604 -128.70
## <none> 31.133 -128.68
## - ManufacturingProcess24 1 0.4801 31.613 -128.66
## - ManufacturingProcess08 1 0.4854 31.618 -128.64
## - BiologicalMaterial05 1 0.5175 31.650 -128.50
## - ManufacturingProcess07 1 0.5441 31.677 -128.39
## - ManufacturingProcess36 1 0.6057 31.739 -128.13
## - ManufacturingProcess19 1 0.6452 31.778 -127.97
## - ManufacturingProcess43 1 0.7728 31.906 -127.44
## - ManufacturingProcess02 1 0.8996 32.032 -126.92
## - BiologicalMaterial10 1 0.9682 32.101 -126.64
## - ManufacturingProcess39 1 1.0572 32.190 -126.27
## - ManufacturingProcess04 1 1.1624 32.295 -125.84
## - ManufacturingProcess28 1 1.4263 32.559 -124.77
## - BiologicalMaterial03 1 1.4657 32.599 -124.61
## - ManufacturingProcess34 1 2.0221 33.155 -122.37
## - ManufacturingProcess09 1 2.0955 33.228 -122.08
## - ManufacturingProcess37 1 3.2215 34.354 -117.68
## - ManufacturingProcess32 1 6.5042 37.637 -105.64
##
## Step: AIC=-130.34
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial07 +
## BiologicalMaterial10 + ManufacturingProcess02 + ManufacturingProcess03 +
## ManufacturingProcess04 + ManufacturingProcess06 + ManufacturingProcess07 +
## ManufacturingProcess08 + ManufacturingProcess09 + ManufacturingProcess10 +
## ManufacturingProcess13 + ManufacturingProcess17 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess22 + ManufacturingProcess24 +
## ManufacturingProcess28 + ManufacturingProcess32 + ManufacturingProcess34 +
## ManufacturingProcess35 + ManufacturingProcess36 + ManufacturingProcess37 +
## ManufacturingProcess38 + ManufacturingProcess39 + ManufacturingProcess41 +
## ManufacturingProcess43 + ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - BiologicalMaterial07 1 0.0700 31.282 -132.05
## - ManufacturingProcess41 1 0.0737 31.286 -132.03
## - ManufacturingProcess03 1 0.1247 31.337 -131.82
## - ManufacturingProcess10 1 0.1578 31.370 -131.68
## - ManufacturingProcess44 1 0.1662 31.379 -131.64
## - ManufacturingProcess17 1 0.1926 31.405 -131.53
## - ManufacturingProcess38 1 0.1934 31.406 -131.53
## - ManufacturingProcess20 1 0.2586 31.471 -131.25
## - ManufacturingProcess22 1 0.2819 31.494 -131.16
## - ManufacturingProcess35 1 0.3844 31.597 -130.73
## - ManufacturingProcess13 1 0.3971 31.609 -130.67
## <none> 31.212 -130.34
## - ManufacturingProcess08 1 0.4812 31.694 -130.32
## - ManufacturingProcess24 1 0.4858 31.698 -130.30
## - ManufacturingProcess06 1 0.5489 31.761 -130.04
## - BiologicalMaterial05 1 0.5627 31.775 -129.98
## - ManufacturingProcess19 1 0.5662 31.779 -129.97
## - ManufacturingProcess36 1 0.6341 31.847 -129.69
## - ManufacturingProcess07 1 0.6462 31.859 -129.64
## - ManufacturingProcess43 1 0.7764 31.989 -129.10
## - ManufacturingProcess02 1 0.8202 32.033 -128.92
## - ManufacturingProcess39 1 1.1706 32.383 -127.48
## - ManufacturingProcess04 1 1.2164 32.429 -127.30
## - BiologicalMaterial10 1 1.3011 32.513 -126.95
## - ManufacturingProcess28 1 1.3652 32.578 -126.69
## - BiologicalMaterial03 1 1.7234 32.936 -125.25
## - ManufacturingProcess34 1 1.9551 33.167 -124.32
## - ManufacturingProcess09 1 2.0382 33.251 -123.99
## - ManufacturingProcess37 1 3.5003 34.713 -118.31
## - ManufacturingProcess32 1 6.7134 37.926 -106.63
##
## Step: AIC=-132.05
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess03 + ManufacturingProcess04 +
## ManufacturingProcess06 + ManufacturingProcess07 + ManufacturingProcess08 +
## ManufacturingProcess09 + ManufacturingProcess10 + ManufacturingProcess13 +
## ManufacturingProcess17 + ManufacturingProcess19 + ManufacturingProcess20 +
## ManufacturingProcess22 + ManufacturingProcess24 + ManufacturingProcess28 +
## ManufacturingProcess32 + ManufacturingProcess34 + ManufacturingProcess35 +
## ManufacturingProcess36 + ManufacturingProcess37 + ManufacturingProcess38 +
## ManufacturingProcess39 + ManufacturingProcess41 + ManufacturingProcess43 +
## ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess41 1 0.0635 31.346 -133.78
## - ManufacturingProcess03 1 0.1182 31.401 -133.55
## - ManufacturingProcess44 1 0.1625 31.445 -133.36
## - ManufacturingProcess10 1 0.1644 31.447 -133.35
## - ManufacturingProcess17 1 0.2180 31.500 -133.13
## - ManufacturingProcess38 1 0.2251 31.507 -133.10
## - ManufacturingProcess20 1 0.2645 31.547 -132.94
## - ManufacturingProcess22 1 0.3346 31.617 -132.64
## - ManufacturingProcess35 1 0.3757 31.658 -132.47
## - ManufacturingProcess13 1 0.3820 31.664 -132.44
## - ManufacturingProcess24 1 0.4571 31.739 -132.13
## - ManufacturingProcess08 1 0.4635 31.746 -132.10
## <none> 31.282 -132.05
## - ManufacturingProcess19 1 0.5499 31.832 -131.75
## - BiologicalMaterial05 1 0.5854 31.868 -131.60
## - ManufacturingProcess06 1 0.5893 31.872 -131.58
## - ManufacturingProcess36 1 0.6220 31.904 -131.45
## - ManufacturingProcess07 1 0.6231 31.905 -131.44
## - ManufacturingProcess43 1 0.8350 32.117 -130.57
## - ManufacturingProcess02 1 0.8516 32.134 -130.50
## - ManufacturingProcess39 1 1.1451 32.427 -129.30
## - BiologicalMaterial10 1 1.3199 32.602 -128.59
## - ManufacturingProcess04 1 1.3497 32.632 -128.47
## - ManufacturingProcess28 1 1.3603 32.643 -128.43
## - BiologicalMaterial03 1 1.7024 32.985 -127.05
## - ManufacturingProcess34 1 1.9697 33.252 -125.99
## - ManufacturingProcess09 1 2.0145 33.297 -125.81
## - ManufacturingProcess37 1 3.4797 34.762 -120.12
## - ManufacturingProcess32 1 6.8276 38.110 -107.99
##
## Step: AIC=-133.78
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess03 + ManufacturingProcess04 +
## ManufacturingProcess06 + ManufacturingProcess07 + ManufacturingProcess08 +
## ManufacturingProcess09 + ManufacturingProcess10 + ManufacturingProcess13 +
## ManufacturingProcess17 + ManufacturingProcess19 + ManufacturingProcess20 +
## ManufacturingProcess22 + ManufacturingProcess24 + ManufacturingProcess28 +
## ManufacturingProcess32 + ManufacturingProcess34 + ManufacturingProcess35 +
## ManufacturingProcess36 + ManufacturingProcess37 + ManufacturingProcess38 +
## ManufacturingProcess39 + ManufacturingProcess43 + ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess03 1 0.1082 31.454 -135.32
## - ManufacturingProcess44 1 0.1699 31.516 -135.07
## - ManufacturingProcess17 1 0.1827 31.529 -135.01
## - ManufacturingProcess10 1 0.1907 31.537 -134.98
## - ManufacturingProcess38 1 0.2368 31.583 -134.79
## - ManufacturingProcess20 1 0.2671 31.613 -134.66
## - ManufacturingProcess35 1 0.3359 31.682 -134.37
## - ManufacturingProcess22 1 0.3814 31.727 -134.18
## - ManufacturingProcess13 1 0.4532 31.799 -133.88
## - ManufacturingProcess08 1 0.4753 31.821 -133.79
## <none> 31.346 -133.78
## - ManufacturingProcess24 1 0.4829 31.829 -133.76
## - ManufacturingProcess06 1 0.5505 31.896 -133.48
## - BiologicalMaterial05 1 0.5626 31.908 -133.43
## - ManufacturingProcess36 1 0.5681 31.914 -133.41
## - ManufacturingProcess19 1 0.5883 31.934 -133.32
## - ManufacturingProcess07 1 0.6137 31.960 -133.22
## - ManufacturingProcess02 1 0.8177 32.164 -132.38
## - ManufacturingProcess43 1 0.8310 32.177 -132.32
## - ManufacturingProcess39 1 1.1008 32.447 -131.22
## - BiologicalMaterial10 1 1.2975 32.643 -130.43
## - ManufacturingProcess28 1 1.3749 32.721 -130.11
## - ManufacturingProcess04 1 1.5176 32.863 -129.54
## - BiologicalMaterial03 1 1.6464 32.992 -129.02
## - ManufacturingProcess09 1 2.1296 33.475 -127.10
## - ManufacturingProcess34 1 2.2396 33.586 -126.67
## - ManufacturingProcess37 1 3.4633 34.809 -121.95
## - ManufacturingProcess32 1 6.7709 38.117 -109.96
##
## Step: AIC=-135.32
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess13 + ManufacturingProcess17 +
## ManufacturingProcess19 + ManufacturingProcess20 + ManufacturingProcess22 +
## ManufacturingProcess24 + ManufacturingProcess28 + ManufacturingProcess32 +
## ManufacturingProcess34 + ManufacturingProcess35 + ManufacturingProcess36 +
## ManufacturingProcess37 + ManufacturingProcess38 + ManufacturingProcess39 +
## ManufacturingProcess43 + ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess17 1 0.1684 31.622 -136.62
## - ManufacturingProcess10 1 0.1685 31.623 -136.62
## - ManufacturingProcess44 1 0.1755 31.630 -136.59
## - ManufacturingProcess38 1 0.2001 31.654 -136.49
## - ManufacturingProcess20 1 0.2099 31.664 -136.45
## - ManufacturingProcess35 1 0.3065 31.761 -136.04
## - ManufacturingProcess22 1 0.3941 31.848 -135.68
## - ManufacturingProcess24 1 0.4541 31.908 -135.43
## <none> 31.454 -135.32
## - ManufacturingProcess13 1 0.4866 31.941 -135.30
## - BiologicalMaterial05 1 0.5011 31.955 -135.24
## - ManufacturingProcess08 1 0.5191 31.973 -135.16
## - ManufacturingProcess36 1 0.5560 32.010 -135.01
## - ManufacturingProcess19 1 0.5844 32.038 -134.89
## - ManufacturingProcess07 1 0.6323 32.086 -134.70
## - ManufacturingProcess06 1 0.6727 32.127 -134.53
## - ManufacturingProcess43 1 0.7324 32.186 -134.29
## - ManufacturingProcess02 1 0.7396 32.194 -134.26
## - ManufacturingProcess39 1 1.0816 32.536 -132.86
## - ManufacturingProcess28 1 1.3419 32.796 -131.81
## - BiologicalMaterial10 1 1.3880 32.842 -131.62
## - ManufacturingProcess04 1 1.4259 32.880 -131.47
## - BiologicalMaterial03 1 1.7615 33.216 -130.13
## - ManufacturingProcess09 1 2.0220 33.476 -129.10
## - ManufacturingProcess34 1 2.3707 33.825 -127.73
## - ManufacturingProcess37 1 3.6074 35.061 -122.99
## - ManufacturingProcess32 1 6.6818 38.136 -111.90
##
## Step: AIC=-136.62
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess13 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess22 + ManufacturingProcess24 +
## ManufacturingProcess28 + ManufacturingProcess32 + ManufacturingProcess34 +
## ManufacturingProcess35 + ManufacturingProcess36 + ManufacturingProcess37 +
## ManufacturingProcess38 + ManufacturingProcess39 + ManufacturingProcess43 +
## ManufacturingProcess44
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess44 1 0.1750 31.797 -137.89
## - ManufacturingProcess38 1 0.1933 31.816 -137.81
## - ManufacturingProcess35 1 0.3629 31.985 -137.11
## - ManufacturingProcess19 1 0.4661 32.089 -136.69
## - ManufacturingProcess24 1 0.4749 32.097 -136.65
## <none> 31.622 -136.62
## - ManufacturingProcess22 1 0.4841 32.107 -136.61
## - ManufacturingProcess08 1 0.4862 32.109 -136.60
## - ManufacturingProcess20 1 0.5207 32.143 -136.46
## - ManufacturingProcess02 1 0.6016 32.224 -136.13
## - ManufacturingProcess36 1 0.6084 32.231 -136.10
## - ManufacturingProcess07 1 0.6870 32.309 -135.78
## - ManufacturingProcess06 1 0.7017 32.324 -135.72
## - ManufacturingProcess43 1 0.7722 32.395 -135.44
## - ManufacturingProcess10 1 0.8832 32.506 -134.98
## - BiologicalMaterial05 1 0.9594 32.582 -134.67
## - ManufacturingProcess39 1 1.1652 32.788 -133.84
## - ManufacturingProcess04 1 1.3508 32.973 -133.10
## - ManufacturingProcess28 1 1.4223 33.045 -132.81
## - BiologicalMaterial10 1 1.5328 33.155 -132.37
## - BiologicalMaterial03 1 1.6703 33.293 -131.82
## - ManufacturingProcess13 1 1.6726 33.295 -131.82
## - ManufacturingProcess34 1 2.4697 34.092 -128.69
## - ManufacturingProcess09 1 3.1805 34.803 -125.97
## - ManufacturingProcess37 1 3.6975 35.320 -124.02
## - ManufacturingProcess32 1 6.7418 38.364 -113.11
##
## Step: AIC=-137.89
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess13 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess22 + ManufacturingProcess24 +
## ManufacturingProcess28 + ManufacturingProcess32 + ManufacturingProcess34 +
## ManufacturingProcess35 + ManufacturingProcess36 + ManufacturingProcess37 +
## ManufacturingProcess38 + ManufacturingProcess39 + ManufacturingProcess43
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess35 1 0.2917 32.089 -138.69
## - ManufacturingProcess38 1 0.3154 32.113 -138.59
## - ManufacturingProcess19 1 0.4538 32.251 -138.02
## <none> 31.797 -137.89
## - ManufacturingProcess36 1 0.4931 32.291 -137.86
## - ManufacturingProcess08 1 0.5080 32.305 -137.80
## - ManufacturingProcess24 1 0.5169 32.314 -137.76
## - ManufacturingProcess22 1 0.5246 32.322 -137.73
## - ManufacturingProcess20 1 0.5363 32.334 -137.68
## - ManufacturingProcess02 1 0.5546 32.352 -137.61
## - ManufacturingProcess07 1 0.6841 32.482 -137.08
## - ManufacturingProcess43 1 0.7325 32.530 -136.88
## - ManufacturingProcess06 1 0.7560 32.553 -136.79
## - ManufacturingProcess10 1 0.8255 32.623 -136.51
## - BiologicalMaterial05 1 0.8890 32.686 -136.25
## - ManufacturingProcess28 1 1.3847 33.182 -134.26
## - ManufacturingProcess04 1 1.4045 33.202 -134.19
## - ManufacturingProcess13 1 1.5616 33.359 -133.56
## - BiologicalMaterial10 1 1.7405 33.538 -132.86
## - BiologicalMaterial03 1 1.7879 33.585 -132.67
## - ManufacturingProcess39 1 2.0889 33.886 -131.49
## - ManufacturingProcess34 1 2.3107 34.108 -130.63
## - ManufacturingProcess09 1 3.4048 35.202 -126.46
## - ManufacturingProcess37 1 3.9843 35.782 -124.31
## - ManufacturingProcess32 1 6.5993 38.397 -115.00
##
## Step: AIC=-138.69
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess13 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess22 + ManufacturingProcess24 +
## ManufacturingProcess28 + ManufacturingProcess32 + ManufacturingProcess34 +
## ManufacturingProcess36 + ManufacturingProcess37 + ManufacturingProcess38 +
## ManufacturingProcess39 + ManufacturingProcess43
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess38 1 0.1403 32.229 -140.11
## - ManufacturingProcess36 1 0.2030 32.292 -139.85
## - ManufacturingProcess22 1 0.4702 32.559 -138.77
## <none> 32.089 -138.69
## - ManufacturingProcess24 1 0.5263 32.615 -138.54
## - ManufacturingProcess19 1 0.5311 32.620 -138.52
## - ManufacturingProcess43 1 0.5637 32.653 -138.39
## - ManufacturingProcess20 1 0.5956 32.685 -138.26
## - ManufacturingProcess08 1 0.6356 32.725 -138.10
## - ManufacturingProcess02 1 0.6625 32.752 -137.99
## - ManufacturingProcess06 1 0.7153 32.804 -137.78
## - BiologicalMaterial05 1 0.7518 32.841 -137.63
## - ManufacturingProcess07 1 0.8045 32.894 -137.42
## - ManufacturingProcess10 1 0.8567 32.946 -137.21
## - ManufacturingProcess28 1 1.3134 33.403 -135.39
## - ManufacturingProcess04 1 1.4003 33.489 -135.05
## - BiologicalMaterial10 1 1.5844 33.674 -134.32
## - BiologicalMaterial03 1 1.6842 33.773 -133.93
## - ManufacturingProcess13 1 1.8075 33.897 -133.45
## - ManufacturingProcess39 1 1.8935 33.983 -133.12
## - ManufacturingProcess34 1 2.3067 34.396 -131.52
## - ManufacturingProcess09 1 3.3224 35.412 -127.68
## - ManufacturingProcess37 1 3.9678 36.057 -125.30
## - ManufacturingProcess32 1 8.5265 40.616 -109.58
##
## Step: AIC=-140.11
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess13 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess22 + ManufacturingProcess24 +
## ManufacturingProcess28 + ManufacturingProcess32 + ManufacturingProcess34 +
## ManufacturingProcess36 + ManufacturingProcess37 + ManufacturingProcess39 +
## ManufacturingProcess43
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess36 1 0.2876 32.517 -140.94
## - ManufacturingProcess22 1 0.4041 32.634 -140.47
## - ManufacturingProcess19 1 0.4744 32.704 -140.18
## - ManufacturingProcess24 1 0.4831 32.713 -140.15
## <none> 32.229 -140.11
## - ManufacturingProcess43 1 0.5973 32.827 -139.69
## - ManufacturingProcess20 1 0.6138 32.843 -139.62
## - ManufacturingProcess08 1 0.6762 32.906 -139.37
## - ManufacturingProcess02 1 0.7166 32.946 -139.21
## - BiologicalMaterial05 1 0.7269 32.956 -139.17
## - ManufacturingProcess06 1 0.7282 32.958 -139.16
## - ManufacturingProcess07 1 0.8063 33.036 -138.85
## - ManufacturingProcess10 1 0.8289 33.058 -138.76
## - ManufacturingProcess04 1 1.2944 33.524 -136.91
## - ManufacturingProcess28 1 1.5092 33.739 -136.07
## - BiologicalMaterial10 1 1.5661 33.796 -135.85
## - BiologicalMaterial03 1 1.7099 33.939 -135.29
## - ManufacturingProcess13 1 1.7591 33.989 -135.09
## - ManufacturingProcess39 1 1.9563 34.186 -134.33
## - ManufacturingProcess34 1 2.2086 34.438 -133.36
## - ManufacturingProcess09 1 3.2741 35.504 -129.34
## - ManufacturingProcess37 1 4.1417 36.371 -126.15
## - ManufacturingProcess32 1 8.9974 41.227 -109.61
##
## Step: AIC=-140.94
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess13 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess22 + ManufacturingProcess24 +
## ManufacturingProcess28 + ManufacturingProcess32 + ManufacturingProcess34 +
## ManufacturingProcess37 + ManufacturingProcess39 + ManufacturingProcess43
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess22 1 0.4352 32.952 -141.18
## - ManufacturingProcess19 1 0.4532 32.970 -141.11
## - ManufacturingProcess24 1 0.4714 32.988 -141.04
## <none> 32.517 -140.94
## - ManufacturingProcess08 1 0.5062 33.023 -140.90
## - ManufacturingProcess07 1 0.6033 33.120 -140.51
## - ManufacturingProcess06 1 0.6085 33.126 -140.49
## - ManufacturingProcess02 1 0.6135 33.131 -140.47
## - ManufacturingProcess43 1 0.6264 33.143 -140.42
## - ManufacturingProcess20 1 0.7043 33.221 -140.11
## - BiologicalMaterial05 1 0.7430 33.260 -139.96
## - ManufacturingProcess10 1 0.7494 33.266 -139.93
## - ManufacturingProcess04 1 1.2329 33.750 -138.02
## - BiologicalMaterial10 1 1.4301 33.947 -137.26
## - ManufacturingProcess28 1 1.5804 34.097 -136.67
## - ManufacturingProcess13 1 1.6287 34.146 -136.49
## - BiologicalMaterial03 1 1.6358 34.153 -136.46
## - ManufacturingProcess34 1 2.1962 34.713 -134.31
## - ManufacturingProcess39 1 2.2260 34.743 -134.20
## - ManufacturingProcess09 1 3.5521 36.069 -129.25
## - ManufacturingProcess37 1 3.8559 36.373 -128.15
## - ManufacturingProcess32 1 12.3713 44.888 -100.38
##
## Step: AIC=-141.18
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess13 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess24 + ManufacturingProcess28 +
## ManufacturingProcess32 + ManufacturingProcess34 + ManufacturingProcess37 +
## ManufacturingProcess39 + ManufacturingProcess43
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess24 1 0.2806 33.233 -142.06
## - ManufacturingProcess19 1 0.3955 33.348 -141.61
## <none> 32.952 -141.18
## - ManufacturingProcess07 1 0.5592 33.512 -140.96
## - ManufacturingProcess08 1 0.5844 33.537 -140.86
## - ManufacturingProcess43 1 0.6453 33.598 -140.62
## - ManufacturingProcess06 1 0.6644 33.617 -140.55
## - BiologicalMaterial05 1 0.6707 33.623 -140.52
## - ManufacturingProcess10 1 0.7712 33.723 -140.13
## - ManufacturingProcess02 1 0.8357 33.788 -139.88
## - ManufacturingProcess20 1 0.8402 33.793 -139.86
## - BiologicalMaterial10 1 1.2516 34.204 -138.26
## - ManufacturingProcess04 1 1.4132 34.365 -137.64
## - ManufacturingProcess28 1 1.4437 34.396 -137.52
## - BiologicalMaterial03 1 1.7756 34.728 -136.25
## - ManufacturingProcess13 1 2.0537 35.006 -135.20
## - ManufacturingProcess39 1 2.1326 35.085 -134.90
## - ManufacturingProcess34 1 2.5464 35.499 -133.36
## - ManufacturingProcess09 1 3.1866 36.139 -131.00
## - ManufacturingProcess37 1 3.7936 36.746 -128.80
## - ManufacturingProcess32 1 12.0269 44.979 -102.11
##
## Step: AIC=-142.06
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess13 + ManufacturingProcess19 +
## ManufacturingProcess20 + ManufacturingProcess28 + ManufacturingProcess32 +
## ManufacturingProcess34 + ManufacturingProcess37 + ManufacturingProcess39 +
## ManufacturingProcess43
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess19 1 0.3696 33.602 -142.60
## - ManufacturingProcess08 1 0.4970 33.730 -142.10
## <none> 33.233 -142.06
## - ManufacturingProcess07 1 0.5899 33.823 -141.74
## - BiologicalMaterial05 1 0.6928 33.926 -141.34
## - ManufacturingProcess06 1 0.8201 34.053 -140.84
## - ManufacturingProcess43 1 0.8216 34.054 -140.84
## - ManufacturingProcess10 1 0.8806 34.113 -140.61
## - ManufacturingProcess02 1 0.9330 34.166 -140.41
## - ManufacturingProcess20 1 1.1353 34.368 -139.63
## - BiologicalMaterial10 1 1.2385 34.471 -139.23
## - ManufacturingProcess28 1 1.3569 34.590 -138.78
## - ManufacturingProcess04 1 1.5136 34.746 -138.18
## - BiologicalMaterial03 1 1.8612 35.094 -136.87
## - ManufacturingProcess39 1 1.8766 35.110 -136.81
## - ManufacturingProcess13 1 2.1498 35.383 -135.79
## - ManufacturingProcess34 1 2.6918 35.925 -133.78
## - ManufacturingProcess09 1 3.1209 36.354 -132.22
## - ManufacturingProcess37 1 3.8291 37.062 -129.67
## - ManufacturingProcess32 1 12.2380 45.471 -102.68
##
## Step: AIC=-142.6
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess08 + ManufacturingProcess09 +
## ManufacturingProcess10 + ManufacturingProcess13 + ManufacturingProcess20 +
## ManufacturingProcess28 + ManufacturingProcess32 + ManufacturingProcess34 +
## ManufacturingProcess37 + ManufacturingProcess39 + ManufacturingProcess43
##
## Df Sum of Sq RSS AIC
## - ManufacturingProcess08 1 0.4767 34.079 -142.743
## <none> 33.602 -142.603
## - ManufacturingProcess07 1 0.6024 34.205 -142.257
## - ManufacturingProcess10 1 0.8357 34.438 -141.360
## - ManufacturingProcess43 1 0.8736 34.476 -141.215
## - BiologicalMaterial05 1 0.9151 34.517 -141.056
## - ManufacturingProcess06 1 0.9764 34.579 -140.822
## - ManufacturingProcess20 1 1.1176 34.720 -140.284
## - ManufacturingProcess28 1 1.1461 34.749 -140.176
## - ManufacturingProcess02 1 1.3767 34.979 -139.303
## - BiologicalMaterial10 1 1.4117 35.014 -139.170
## - ManufacturingProcess04 1 1.5306 35.133 -138.723
## - BiologicalMaterial03 1 1.7347 35.337 -137.958
## - ManufacturingProcess39 1 1.7517 35.354 -137.895
## - ManufacturingProcess13 1 1.9883 35.591 -137.015
## - ManufacturingProcess34 1 2.4434 36.046 -135.337
## - ManufacturingProcess09 1 2.9044 36.507 -133.660
## - ManufacturingProcess37 1 3.6832 37.286 -130.873
## - ManufacturingProcess32 1 14.5389 48.141 -97.143
##
## Step: AIC=-142.74
## Yield ~ BiologicalMaterial03 + BiologicalMaterial05 + BiologicalMaterial10 +
## ManufacturingProcess02 + ManufacturingProcess04 + ManufacturingProcess06 +
## ManufacturingProcess07 + ManufacturingProcess09 + ManufacturingProcess10 +
## ManufacturingProcess13 + ManufacturingProcess20 + ManufacturingProcess28 +
## ManufacturingProcess32 + ManufacturingProcess34 + ManufacturingProcess37 +
## ManufacturingProcess39 + ManufacturingProcess43
##
## Df Sum of Sq RSS AIC
## <none> 34.079 -142.743
## - ManufacturingProcess10 1 0.7622 34.841 -141.824
## - ManufacturingProcess06 1 0.8386 34.918 -141.534
## - ManufacturingProcess28 1 0.9602 35.039 -141.075
## - BiologicalMaterial05 1 0.9903 35.069 -140.962
## - ManufacturingProcess43 1 0.9921 35.071 -140.955
## - ManufacturingProcess20 1 1.0774 35.157 -140.635
## - ManufacturingProcess07 1 1.1542 35.233 -140.347
## - ManufacturingProcess02 1 1.2014 35.281 -140.170
## - ManufacturingProcess04 1 1.4825 35.562 -139.122
## - ManufacturingProcess13 1 1.7168 35.796 -138.256
## - ManufacturingProcess39 1 1.7617 35.841 -138.090
## - BiologicalMaterial10 1 1.7673 35.846 -138.069
## - BiologicalMaterial03 1 1.9351 36.014 -137.453
## - ManufacturingProcess34 1 2.1692 36.248 -136.598
## - ManufacturingProcess37 1 3.2393 37.318 -132.757
## - ManufacturingProcess09 1 3.3915 37.471 -132.220
## - ManufacturingProcess32 1 14.8178 48.897 -97.087The Akaike information criterion (AIC) provides an estimate of out-of-sample prediction error and thereby relative model quality. Given a set of models, AIC estimates the quality of each model, relative to one another. What I was hoping to do in using the stepAIC() function was to automate the process of model optimization. To build the model with the fewest variables and the greatest predictive promise.
The result is a simplified model (only 17 features) with what appears to be high predictive promise. The optimal value of the performance metric is an AIC value of -156, the final model.
Next, we verify how the model performed for actual training Yield values:
#predict response of test set
train_predictions <- predict(aic_optimized_lm, as.data.frame(train))
#how did the model perform? (actual v. model)
data.frame( R2 = R2(train_predictions, as.data.frame(train)$Yield),
RMSE = RMSE(train_predictions, as.data.frame(train)$Yield),
MAE = MAE(train_predictions, as.data.frame(train)$Yield))## R2 RMSE MAE
## 1 0.7203241 0.5081098 0.4036883The high R-squared value and relatively low RMSE value reinforce the notion that we have a good model on our hands and so we proceed to verifying performance on the test set.
(d) Predict the response for the test set. What is the value of the performance metric and how does this compare with the resampled performance metric on the training set?
#predict response of test set
test_predictions <- predict(aic_optimized_lm, as.data.frame(test))
#how did the model perform? (actual v. model)
data.frame( R2 = R2(test_predictions, as.data.frame(test)$Yield),
RMSE = RMSE(test_predictions, as.data.frame(test)$Yield),
MAE = MAE(test_predictions, as.data.frame(test)$Yield))## R2 RMSE MAE
## 1 0.6913523 0.6186064 0.5095866When we shift to the test data, our R-squared value drops and RMSE value raises. What this indicates is that our model, while performing relatively well on unseen data, did not perform as strongly. Typically we want an R-squared greater than 0.7 and an RMSE between 0.2 and 0.5. With this in mind, our model’s performance may be considered sub optimal on verification data.
(e) Which predictors are most important in the model you have trained? Do either the biological or process predictors dominate the list?
In order to determine which predictors are most important for our trained model, I utilize the Boruta package. I select features based on our stepAIC optimize model, perform a Boruta search, and output the resulting variable importance ranking:
cmp_df <- as.data.frame(cmp_comp2) #convert to df
#select features from model
cmp_df_simplified <- cmp_df[, c('Yield','ManufacturingProcess10','ManufacturingProcess06','ManufacturingProcess28', 'BiologicalMaterial05', 'ManufacturingProcess43', 'ManufacturingProcess20', 'ManufacturingProcess07', 'ManufacturingProcess02', 'ManufacturingProcess04', 'ManufacturingProcess13', 'ManufacturingProcess39', 'BiologicalMaterial10', 'BiologicalMaterial03', 'ManufacturingProcess34', 'ManufacturingProcess37',
'ManufacturingProcess09', 'ManufacturingProcess32')]
#cmp_df_simplified #verify
#Perform Boruta search
boruta_output <- Boruta(Yield ~ ., data=cmp_df_simplified, doTrace=0, maxRuns = 1000)
#Get significant variables including tentatives
boruta_signif <- getSelectedAttributes(boruta_output, withTentative = TRUE)
#print(boruta_signif)
# Plot variable importance
plot(boruta_output, cex.axis=.7, las=2, xlab="", main="Variable Importance")
The Boruta function performs a top-down search for relevant features by comparing original attributes’ importance with importance achieveable at random, estimated via permutated copies, and progressively eliminating irrelevant features to stabilize. The resulting ourput show features with strongest variable importance from right to left. The features higher up on the plot (and in green) are our strongest predictors and those with the strongest correlation to the Yield.
We observe in the resulting plot that process predictors dominate the list with 14/17 of our model’s predictors and that our strongest predictor is ManufacturingProcess32.
(f) Explore the relationships between each of the top predictors and the response. How could this information be helpful in improving yield in future runs of the manufacturing process?
The Boruta output from above addresses this ask as well. We observe that process predictors dominate the list and that our top three predictors are ManufacturingProcess32, BiologicalMaterial03, and ManufacturingProcess13.
This information could be helpful inn improving yield in future runs of the manufacturing process by highlighting which parts of the process have the greatest impact on improving yield. Those with the highest importance factor (noted above) are the parts of the process Operations could focus on to minimize effort while maximizing Yield.