Linear Regression and its Cousins

KJ 6.3

A chemical manufacturing process for a pharmaceutical product was discussed in Sect. 1.4. In this problem, the objective is to understand the relationship between biological measurements of the raw materials (predictors), measurements of the manufacturing process (predictors), and the response of product yield. Biological predictors cannot be changed but can be used to assess the quality of the raw material before processing. On the other hand, manufacturing process predictors can be changed in the manufacturing process. Improving product yield by 1% will boost revenue by approximately one hundred thousand dollars per batch.

a

Start R and use these commands to load the data:

library(AppliedPredictiveModeling)
data("ChemicalManufacturingProcess")

df <- ChemicalManufacturingProcess

The matrix processPredictors contains the 57 predictors (12 describing the input biological material and 45 describing the process predictors) for the 176 manufacturing runs. yield contains the percent yield for each run.

b

A small percentage of cells in the predictor set contain missing values. Use an imputation function to fill in these missing values.

# bio materical
df[,2:13] %>%
  gather(variable, value) %>%
  ggplot(aes(x = value)) +
  geom_histogram() +
  facet_wrap(~variable, scales = 'free') +
  labs(x = element_blank(),
       y = element_blank(),
       title = 'Biological Materials')

# manufacturing process process
df[,14:ncol(ChemicalManufacturingProcess)] %>%
  gather(variable, value) %>%
  ggplot(aes(x = value)) +
  geom_histogram() +
  facet_wrap(~variable, scales = 'free') +
  labs(x = element_blank(),
       y = element_blank(),
       title = 'Manufacturing Processes')

trans <- preProcess(ChemicalManufacturingProcess, method = 'knnImpute')

trans
## Created from 152 samples and 58 variables
## 
## Pre-processing:
##   - centered (58)
##   - ignored (0)
##   - 5 nearest neighbor imputation (58)
##   - scaled (58)
df <- predict(trans, ChemicalManufacturingProcess)

c

Split the data into a training and a test set, pre-process the data, and tune a model of your choice from this chapter. What is the optimal value of the performance metric?

train_index <- createDataPartition(df$Yield, p = 0.8, list = F, times = 1)

train <- df[train_index,]
test <- df[-train_index,]

ctrl <- trainControl(method = 'cv', number = 10)

ridgeGrid <- data.frame(.lambda = seq(0, .1, length = 15))

set.seed(101)

ridgeRegFit <- train(df[,2:ncol(df)],
                     df$Yield,
                     method = 'ridge',
                     tuneGrid = ridgeGrid,
                     trControl = ctrl,
                     preProc = c('center','scale'))

ridgeRegFit
## Ridge Regression 
## 
## 176 samples
##  57 predictor
## 
## Pre-processing: centered (57), scaled (57) 
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 159, 159, 157, 158, 157, 159, ... 
## Resampling results across tuning parameters:
## 
##   lambda       RMSE       Rsquared   MAE      
##   0.000000000  0.7950651  0.5209711  0.5941800
##   0.007142857  1.0167030  0.5126568  0.6238809
##   0.014285714  1.0008464  0.5217466  0.6188789
##   0.021428571  0.9846256  0.5281832  0.6135598
##   0.028571429  0.9716471  0.5331961  0.6094121
##   0.035714286  0.9613062  0.5373123  0.6065129
##   0.042857143  0.9528905  0.5407885  0.6040632
##   0.050000000  0.9458799  0.5437667  0.6020693
##   0.057142857  0.9399151  0.5463359  0.6005539
##   0.064285714  0.9347475  0.5485590  0.5993244
##   0.071428571  0.9302026  0.5504842  0.5986839
##   0.078571429  0.9261553  0.5521511  0.5983155
##   0.085714286  0.9225142  0.5535932  0.5979392
##   0.092857143  0.9192117  0.5548391  0.5975622
##   0.100000000  0.9161964  0.5559139  0.5972448
## 
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was lambda = 0.

d

Predict the response for the test set. What is the value of the performance metric and how does it compare with the resampled performance metric on the training set?

test$pred <- predict(ridgeRegFit, test)

results <- test %>%
  select(obs = Yield, pred = pred)

defaultSummary(results)
##      RMSE  Rsquared       MAE 
## 0.4983870 0.8188203 0.4049606

The predictions on the test set actually show a lower RMSE (0.60 vs 0.85) and underperformed the fit on the training set.

e

Which predictors are most important in the model you have trained? Do either the biological or process predictors dominate the list?

variable_importance <- varImp(ridgeRegFit, scale = F)

variable_importance
## loess r-squared variable importance
## 
##   only 20 most important variables shown (out of 57)
## 
##                        Overall
## ManufacturingProcess32  0.3701
## ManufacturingProcess13  0.3332
## BiologicalMaterial06    0.3130
## ManufacturingProcess17  0.2772
## ManufacturingProcess36  0.2764
## BiologicalMaterial03    0.2722
## ManufacturingProcess09  0.2605
## BiologicalMaterial12    0.2516
## BiologicalMaterial02    0.2418
## ManufacturingProcess31  0.2367
## ManufacturingProcess06  0.2147
## ManufacturingProcess33  0.1814
## BiologicalMaterial11    0.1781
## ManufacturingProcess11  0.1769
## BiologicalMaterial04    0.1745
## BiologicalMaterial08    0.1551
## BiologicalMaterial01    0.1449
## ManufacturingProcess30  0.1290
## ManufacturingProcess12  0.1234
## BiologicalMaterial09    0.1200
plot(variable_importance, top = 20)

Based on the top 10 most important variables, there doesn’t seem to be a dominant variable type that holds the most importance.

f

Explore the relationships between each of the top predictors and the response. How could this information be helpful in improving yield in the future runs of the manufacturing process?

x <- 9

vi_df <- variable_importance$importance %>%
  data.frame() %>%
  top_n(x)

vi_df$var <- rownames(vi_df)

df_top_x <- df[,names(df) %in% vi_df$var] %>%
  bind_cols(Yield = df$Yield)

df_top_x %>%
  gather(variable, value, -Yield) %>%
  ggplot(aes(x = value, y = Yield)) +
  geom_point() +
  facet_wrap(~variable, scales = 'free_x') +
  labs(x = element_blank())

If we know the most important variables that affect yield and the relationship of each variable against yield, we can determine how to adjust each material or process to maximize results.