DATA_624_Homework_8

Author

Long Lin

7.2

7.2. Friedman (1991) introduced several benchmark data sets create by simulation. One of these simulations used the following nonlinear equation to create data: y = 10 sin(πx1x2) + 20(x3 − 0.5)2 + 10x4 + 5x5 + N (0, σ2) where the x values are random variables uniformly distributed between [0, 1] (there are also 5 other non-informative variables also created in the simulation). The package mlbench contains a function called mlbench.friedman1 that simulates these data:

library(mlbench)
library(caret)
Loading required package: ggplot2
Loading required package: lattice
library(earth)
Loading required package: Formula
Loading required package: plotmo
Loading required package: plotrix
library(kernlab)

Attaching package: 'kernlab'
The following object is masked from 'package:ggplot2':

    alpha
library(nnet)
set.seed(200)
trainingData <- mlbench.friedman1(200, sd = 1)
## We convert the 'x' data from a matrix to a data frame
## One reason is that this will give the columns names.
trainingData$x <- data.frame(trainingData$x)
## Look at the data using
featurePlot(trainingData$x, trainingData$y)

## or other methods.

## This creates a list with a vector 'y' and a matrix
## of predictors 'x'. Also simulate a large test set to
## estimate the true error rate with good precision:
testData <- mlbench.friedman1(5000, sd = 1)
testData$x <- data.frame(testData$x)

Tune several models on these data. For example:

KNN (K-Nearest Neighbors)

knnModel <- train(x = trainingData$x,
  y = trainingData$y,
  method = "knn",
  preProc = c("center", "scale"),
  tuneLength = 10)
knnModel
k-Nearest Neighbors 

200 samples
 10 predictor

Pre-processing: centered (10), scaled (10) 
Resampling: Bootstrapped (25 reps) 
Summary of sample sizes: 200, 200, 200, 200, 200, 200, ... 
Resampling results across tuning parameters:

  k   RMSE      Rsquared   MAE     
   5  3.466085  0.5121775  2.816838
   7  3.349428  0.5452823  2.727410
   9  3.264276  0.5785990  2.660026
  11  3.214216  0.6024244  2.603767
  13  3.196510  0.6176570  2.591935
  15  3.184173  0.6305506  2.577482
  17  3.183130  0.6425367  2.567787
  19  3.198752  0.6483184  2.592683
  21  3.188993  0.6611428  2.588787
  23  3.200458  0.6638353  2.604529

RMSE was used to select the optimal model using the smallest value.
The final value used for the model was k = 17.
knnPred <- predict(knnModel, newdata = testData$x)
## The function 'postResample' can be used to get the test set performance values
postResample(pred = knnPred, obs = testData$y)
     RMSE  Rsquared       MAE 
3.2040595 0.6819919 2.5683461 

Which models appear to give the best performance? Does MARS select the informative predictors (those named X1–X5)?

MARS (Multivariate Adaptive Regression Splines)

set.seed(200)
marsModel <- train(x = trainingData$x, 
                   y = trainingData$y,
                   method = "earth",
                   preProc = c("center", "scale"),
                   tuneGrid = expand.grid(degree = 1:2, nprune = 2:20))
marsModel
Multivariate Adaptive Regression Spline 

200 samples
 10 predictor

Pre-processing: centered (10), scaled (10) 
Resampling: Bootstrapped (25 reps) 
Summary of sample sizes: 200, 200, 200, 200, 200, 200, ... 
Resampling results across tuning parameters:

  degree  nprune  RMSE      Rsquared   MAE     
  1        2      4.447045  0.2249607  3.650128
  1        3      3.744821  0.4546610  3.019175
  1        4      2.828643  0.6892908  2.244131
  1        5      2.524326  0.7516356  2.027435
  1        6      2.406670  0.7747079  1.906733
  1        7      2.027113  0.8375721  1.594956
  1        8      1.874633  0.8618476  1.474219
  1        9      1.800794  0.8728377  1.411703
  1       10      1.810047  0.8721377  1.412023
  1       11      1.821314  0.8714221  1.427124
  1       12      1.831608  0.8700790  1.430044
  1       13      1.839717  0.8686550  1.440537
  1       14      1.849381  0.8672327  1.450876
  1       15      1.856211  0.8663787  1.452430
  1       16      1.857086  0.8661612  1.454255
  1       17      1.853742  0.8667095  1.452920
  1       18      1.853742  0.8667095  1.452920
  1       19      1.853742  0.8667095  1.452920
  1       20      1.853742  0.8667095  1.452920
  2        2      4.447780  0.2248695  3.650597
  2        3      3.737891  0.4543357  3.018103
  2        4      2.854288  0.6832049  2.259488
  2        5      2.513582  0.7550084  2.004730
  2        6      2.387478  0.7799585  1.889787
  2        7      2.044028  0.8354683  1.615415
  2        8      1.910896  0.8568917  1.500375
  2        9      1.810765  0.8703004  1.404288
  2       10      1.677078  0.8885385  1.321634
  2       11      1.561012  0.9045745  1.234778
  2       12      1.503867  0.9112625  1.183593
  2       13      1.507992  0.9112557  1.172444
  2       14      1.505298  0.9114749  1.171595
  2       15      1.527789  0.9091635  1.188885
  2       16      1.532851  0.9082380  1.188192
  2       17      1.551046  0.9061881  1.199236
  2       18      1.566120  0.9047152  1.213000
  2       19      1.574242  0.9036555  1.217297
  2       20      1.574242  0.9036555  1.217297

RMSE was used to select the optimal model using the smallest value.
The final values used for the model were nprune = 12 and degree = 2.
marsPred <- predict(marsModel, newdata = testData$x)
postResample(pred = marsPred, obs = testData$y)
     RMSE  Rsquared       MAE 
1.3227340 0.9291489 1.0524686 
varImp(marsModel)
earth variable importance

   Overall
X1  100.00
X4   75.40
X2   49.00
X5   15.72
X3    0.00
summary(marsModel$finalModel)
Call: earth(x=data.frame[200,10], y=c(18.46,16.1,17...), keepxy=TRUE, degree=2,
            nprune=12)

                                  coefficients
(Intercept)                          21.690154
h(0.507267-X1)                       -4.203744
h(X1-0.507267)                        3.072355
h(0.325504-X2)                       -5.314859
h(-0.216741-X3)                       3.320304
h(X3- -0.216741)                      2.321760
h(0.953812-X4)                       -2.775288
h(X4-0.953812)                        2.778320
h(1.17878-X5)                        -1.607769
h(X1-0.507267) * h(X2- -0.798188)    -3.199202
h(0.606835-X1) * h(0.325504-X2)       2.030856
h(0.325504-X2) * h(X3-0.795427)       1.369704

Selected 12 of 21 terms, and 5 of 10 predictors (nprune=12)
Termination condition: Reached nk 21
Importance: X1, X4, X2, X5, X3, X6-unused, X7-unused, X8-unused, X9-unused, ...
Number of terms at each degree of interaction: 1 8 3
GCV 1.842426    RSS 270.9495    GRSq 0.9251967    RSq 0.9444425

SVM (Support Vector Machine)

set.seed(200)
svmModel <- train(x = trainingData$x, 
                  y = trainingData$y,
                  method = "svmRadial",
                  preProc = c("center", "scale"),
                  tuneLength = 10)
svmModel
Support Vector Machines with Radial Basis Function Kernel 

200 samples
 10 predictor

Pre-processing: centered (10), scaled (10) 
Resampling: Bootstrapped (25 reps) 
Summary of sample sizes: 200, 200, 200, 200, 200, 200, ... 
Resampling results across tuning parameters:

  C       RMSE      Rsquared   MAE     
    0.25  2.635010  0.7685188  2.074977
    0.50  2.423373  0.7839086  1.902162
    1.00  2.284133  0.8001542  1.791776
    2.00  2.196624  0.8126474  1.713560
    4.00  2.143035  0.8209820  1.668024
    8.00  2.119156  0.8246312  1.649384
   16.00  2.117438  0.8248677  1.648568
   32.00  2.117438  0.8248677  1.648568
   64.00  2.117438  0.8248677  1.648568
  128.00  2.117438  0.8248677  1.648568

Tuning parameter 'sigma' was held constant at a value of 0.06299324
RMSE was used to select the optimal model using the smallest value.
The final values used for the model were sigma = 0.06299324 and C = 16.
svmPred <- predict(svmModel, newdata = testData$x)
postResample(pred = svmPred, obs = testData$y)
     RMSE  Rsquared       MAE 
2.0736997 0.8256573 1.5751967 

Neural Network

set.seed(200)
nnGrid <- expand.grid(decay = c(0.00, 0.01, .1), size = c(1:10))
nnModel <- train(x = trainingData$x, 
                   y = trainingData$y,
                   method = "nnet",
                   preProc = c("center", "scale"),
                   linout = TRUE,
                   trace = FALSE,
                   MaxNWts = 10 * (ncol(trainingData$x) + 1) + 10 + 1,
                   maxit = 500,
                   tuneGrid = nnGrid)
nnModel
Neural Network 

200 samples
 10 predictor

Pre-processing: centered (10), scaled (10) 
Resampling: Bootstrapped (25 reps) 
Summary of sample sizes: 200, 200, 200, 200, 200, 200, ... 
Resampling results across tuning parameters:

  decay  size  RMSE      Rsquared   MAE     
  0.00    1    2.716196  0.7067011  2.116773
  0.00    2    2.866107  0.6902232  2.248831
  0.00    3    2.723771  0.7170827  2.156878
  0.00    4    3.489200  0.6093775  2.513715
  0.00    5    5.058811  0.4934758  3.224009
  0.00    6    8.113801  0.3781921  4.393986
  0.00    7    8.037891  0.3549785  4.472322
  0.00    8    8.858799  0.3274475  4.746915
  0.00    9    7.140405  0.4086163  4.430303
  0.00   10    4.574844  0.4888619  3.329784
  0.01    1    2.693412  0.7146609  2.114155
  0.01    2    2.968614  0.6658772  2.335700
  0.01    3    2.894412  0.6856657  2.251480
  0.01    4    3.083548  0.6586936  2.440769
  0.01    5    3.374554  0.6200405  2.629490
  0.01    6    3.456404  0.6027993  2.713169
  0.01    7    4.039034  0.5345452  3.117428
  0.01    8    3.954688  0.5470789  3.115898
  0.01    9    4.051493  0.5096129  3.177828
  0.01   10    3.931815  0.5310510  3.125908
  0.10    1    2.607519  0.7347198  2.025451
  0.10    2    2.854740  0.6855983  2.242564
  0.10    3    2.767154  0.7087167  2.178078
  0.10    4    2.950244  0.6794368  2.334747
  0.10    5    3.022884  0.6752370  2.403152
  0.10    6    3.173633  0.6467267  2.542692
  0.10    7    3.340143  0.6201282  2.609767
  0.10    8    3.653923  0.5717183  2.877705
  0.10    9    3.430889  0.6066110  2.716133
  0.10   10    3.236111  0.6331641  2.542236

RMSE was used to select the optimal model using the smallest value.
The final values used for the model were size = 1 and decay = 0.1.
nnPred <- predict(nnModel, newdata = testData$x)
postResample(pred = nnPred, obs = testData$y)
     RMSE  Rsquared       MAE 
2.6492441 0.7177347 2.0294523 

The MARS model appears to give the best performance. According to the RMSE values, the MARS model had the lowest RMSE followed by the SVM model and then the Neural Network model. The example KNN model actually performed the worst out of the 4 models. The MARS model does select the informative predictors (those named X1–X5) according to the summary statistics above.

7.5

Exercise 6.3 describes data for a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several nonlinear regression models.

library(AppliedPredictiveModeling)
library(caret)
library(tidyverse)
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.2.0     ✔ readr     2.2.0
✔ forcats   1.0.1     ✔ stringr   1.6.0
✔ lubridate 1.9.5     ✔ tibble    3.3.1
✔ purrr     1.2.1     ✔ tidyr     1.3.2
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ kernlab::alpha() masks ggplot2::alpha()
✖ purrr::cross()   masks kernlab::cross()
✖ dplyr::filter()  masks stats::filter()
✖ dplyr::lag()     masks stats::lag()
✖ purrr::lift()    masks caret::lift()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(RANN)
library(earth)
library(kernlab)
library(nnet)
data(ChemicalManufacturingProcess)

impute_model <- preProcess(ChemicalManufacturingProcess, method = "knnImpute")

chem_imputed <- predict(impute_model, ChemicalManufacturingProcess)

nzv_predictors <- nearZeroVar(chem_imputed)
if(length(nzv_predictors) > 0){
  chem_imputed <- chem_imputed[, -nzv_predictors]
}

set.seed(8585)
index <- createDataPartition(chem_imputed$Yield, p = 0.8, list = FALSE)

training_data <- chem_imputed[index, ]
test_data  <- chem_imputed[-index, ]

training_x <- training_data[, -1]
training_y <- training_data$Yield
test_x <- test_data[, -1]
test_y <- test_data$Yield

ctrl <- trainControl(method = "cv", number = 10)

(a) Which nonlinear regression model gives the optimal resampling and test set performance?

SVM Model

set.seed(200)
svm_model <- train(x = training_x, y = training_y, 
                 method = "svmRadial", 
                 tuneLength = 10, 
                 trControl = ctrl)
svm_model
Support Vector Machines with Radial Basis Function Kernel 

144 samples
 56 predictor

No pre-processing
Resampling: Cross-Validated (10 fold) 
Summary of sample sizes: 130, 129, 129, 130, 131, 131, ... 
Resampling results across tuning parameters:

  C       RMSE       Rsquared   MAE      
    0.25  0.7907526  0.5150952  0.6459562
    0.50  0.7308700  0.5555113  0.6006713
    1.00  0.6906014  0.5855178  0.5631447
    2.00  0.6597609  0.6101799  0.5340240
    4.00  0.6473585  0.6163341  0.5179436
    8.00  0.6301217  0.6339842  0.5056210
   16.00  0.6283767  0.6362071  0.5041480
   32.00  0.6283767  0.6362071  0.5041480
   64.00  0.6283767  0.6362071  0.5041480
  128.00  0.6283767  0.6362071  0.5041480

Tuning parameter 'sigma' was held constant at a value of 0.01473086
RMSE was used to select the optimal model using the smallest value.
The final values used for the model were sigma = 0.01473086 and C = 16.
svm_pred <- predict(svm_model, newdata = test_x)
postResample(pred = svm_pred, obs = test_y)
     RMSE  Rsquared       MAE 
0.4337143 0.8087388 0.3160553 

MARS Model

set.seed(200)
mars_model <- train(x = training_x, y = training_y, 
                  method = "earth", 
                  tuneGrid = expand.grid(degree = 1:2, nprune = 2:20), 
                  trControl = ctrl)
mars_model
Multivariate Adaptive Regression Spline 

144 samples
 56 predictor

No pre-processing
Resampling: Cross-Validated (10 fold) 
Summary of sample sizes: 130, 129, 129, 130, 131, 131, ... 
Resampling results across tuning parameters:

  degree  nprune  RMSE       Rsquared   MAE      
  1        2      0.7784960  0.4521123  0.6110095
  1        3      0.6797222  0.5686327  0.5484654
  1        4      0.7693735  0.4824164  0.5886652
  1        5      0.7809390  0.4718923  0.5993890
  1        6      0.7663924  0.4876473  0.5883228
  1        7      0.8024829  0.4476053  0.6117258
  1        8      0.7641791  0.4837913  0.5941973
  1        9      0.7505538  0.5003040  0.5853296
  1       10      0.7923358  0.4568956  0.6040740
  1       11      0.7869087  0.4594732  0.5998450
  1       12      0.7982217  0.4615801  0.6125163
  1       13      0.7812971  0.4728658  0.6040598
  1       14      0.7769953  0.4796635  0.5986975
  1       15      0.7718506  0.4857815  0.5929375
  1       16      0.7756903  0.4831118  0.5990545
  1       17      0.7835119  0.4721257  0.6075582
  1       18      0.7831845  0.4737361  0.6099487
  1       19      0.7831845  0.4737361  0.6099487
  1       20      0.7831845  0.4737361  0.6099487
  2        2      0.8258607  0.3929132  0.6526192
  2        3      0.6842036  0.5611024  0.5595000
  2        4      0.6710648  0.5810654  0.5412634
  2        5      0.7602456  0.4934500  0.5924034
  2        6      0.8632648  0.4219467  0.6563768
  2        7      0.8586593  0.4388657  0.6502123
  2        8      0.7743669  0.5338320  0.5970172
  2        9      0.8173317  0.4930046  0.6377390
  2       10      0.8081748  0.4922890  0.6387744
  2       11      0.8568980  0.4523071  0.6600723
  2       12      0.8254677  0.5065480  0.6390540
  2       13      0.8106102  0.5272095  0.6247974
  2       14      0.8301089  0.5205785  0.6447048
  2       15      0.8503671  0.5037697  0.6576410
  2       16      0.8654722  0.4887383  0.6685763
  2       17      0.8447461  0.4906747  0.6624287
  2       18      0.8715130  0.4915681  0.6694883
  2       19      0.9011223  0.4712341  0.6909122
  2       20      1.4939234  0.4268067  0.8654001

RMSE was used to select the optimal model using the smallest value.
The final values used for the model were nprune = 4 and degree = 2.
mars_pred <- predict(mars_model, newdata = test_x)
postResample(pred = mars_pred, obs = test_y)
     RMSE  Rsquared       MAE 
0.5567879 0.6845715 0.4351563 

KNN Model

set.seed(200)
knn_model <- train(x = training_x, y = training_y, 
                 method = "knn", 
                 tuneLength = 10, 
                 trControl = ctrl)
knn_model
k-Nearest Neighbors 

144 samples
 56 predictor

No pre-processing
Resampling: Cross-Validated (10 fold) 
Summary of sample sizes: 130, 129, 129, 130, 131, 131, ... 
Resampling results across tuning parameters:

  k   RMSE       Rsquared   MAE      
   5  0.7582754  0.4689249  0.5958651
   7  0.7567786  0.4860430  0.5988680
   9  0.7656574  0.4791745  0.6129237
  11  0.7676219  0.4867971  0.6218451
  13  0.7671042  0.4938682  0.6143599
  15  0.7772014  0.4762563  0.6203014
  17  0.7932353  0.4571005  0.6377444
  19  0.7895134  0.4608770  0.6348365
  21  0.7889588  0.4668767  0.6347396
  23  0.7975105  0.4598221  0.6427061

RMSE was used to select the optimal model using the smallest value.
The final value used for the model was k = 7.
knn_pred <- predict(knn_model, newdata = test_x)
postResample(pred = knn_pred, obs = test_y)
     RMSE  Rsquared       MAE 
0.6106433 0.6013080 0.5147434 

Neural Network Model

set.seed(200)
nn_grid <- expand.grid(size = c(1, 3, 5, 7), decay = c(0, 0.01, 0.1))
nn_model <- train(x = training_x, y = training_y, 
                  method = "nnet", 
                  tuneGrid = nn_grid, 
                  trControl = ctrl, 
                  linout = TRUE, 
                  trace = FALSE,
                  maxit = 500)
nn_model
Neural Network 

144 samples
 56 predictor

No pre-processing
Resampling: Cross-Validated (10 fold) 
Summary of sample sizes: 130, 129, 129, 130, 131, 131, ... 
Resampling results across tuning parameters:

  size  decay  RMSE       Rsquared   MAE      
  1     0.00   1.0241433  0.3094811  0.8174104
  1     0.01   0.9245402  0.3926698  0.7321303
  1     0.10   0.7426006  0.5226842  0.5945290
  3     0.00   1.0465000  0.3946573  0.8502274
  3     0.01   1.0371149  0.3298719  0.8325435
  3     0.10   0.8522663  0.4876265  0.6917522
  5     0.00   1.2264561  0.2575453  1.0005826
  5     0.01   0.9588326  0.3752334  0.7580594
  5     0.10   0.7674455  0.5354913  0.6088145
  7     0.00   1.0987498  0.3654437  0.8964060
  7     0.01   0.7545268  0.5274101  0.6143798
  7     0.10   0.7474360  0.5398598  0.5943122

RMSE was used to select the optimal model using the smallest value.
The final values used for the model were size = 1 and decay = 0.1.
nn_pred <- predict(nn_model, newdata = test_x)
postResample(pred = nn_pred, obs = test_y)
     RMSE  Rsquared       MAE 
0.5674081 0.6676520 0.4787453 
resamps <- resamples(list(SVM = svm_model, MARS = mars_model, KNN = knn_model, NN = nn_model))
summary(resamps)

Call:
summary.resamples(object = resamps)

Models: SVM, MARS, KNN, NN 
Number of resamples: 10 

MAE 
          Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
SVM  0.3779863 0.4297056 0.4962653 0.5041480 0.5741598 0.6677082    0
MARS 0.3982727 0.4786543 0.5299589 0.5412634 0.5758744 0.8070333    0
KNN  0.4299905 0.5192368 0.5889512 0.5988680 0.6600127 0.7662737    0
NN   0.4104028 0.4701289 0.6019592 0.5945290 0.6669310 0.8840452    0

RMSE 
          Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
SVM  0.4701273 0.5624376 0.6163458 0.6283767 0.7164603 0.8132034    0
MARS 0.4823963 0.6314224 0.6804129 0.6710648 0.7057017 0.9084705    0
KNN  0.5736776 0.7081903 0.7480029 0.7567786 0.8249372 0.9086932    0
NN   0.4935170 0.6020204 0.7315897 0.7426006 0.8035316 1.0854727    0

Rsquared 
           Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
SVM  0.36239724 0.6000560 0.6541328 0.6362071 0.7267933 0.8821837    0
MARS 0.31826066 0.5490608 0.6043979 0.5810654 0.6537452 0.7732053    0
KNN  0.15458364 0.4554774 0.4955779 0.4860430 0.6102074 0.6714464    0
NN   0.05798509 0.4241956 0.5180183 0.5226842 0.6834319 0.8204222    0

Based on the results, the SVM model performed the best in terms of having the lowest RMSE value and the highest Rsquared value. Next was the MARS model and then the Neural Network model and finally the KNN model.

(b) Which predictors are most important in the optimal nonlinear regression model? Do either the biological or process variables dominate the list? How do the top ten important predictors compare to the top ten predictors from the optimal linear model?

svm_imp <- varImp(svm_model)
plot(svm_imp, top = 10, main = "Top 10 Important Variables (SVM)")

set.seed(8585)
pls_mod <- train(x = training_x, y = training_y, method = "pls", tuneLength = 10, trControl = ctrl)
plot(varImp(pls_mod), top = 10, main = "Top 10 Important Variables (PLS)")

Attaching package: 'pls'
The following object is masked from 'package:caret':

    R2
The following object is masked from 'package:stats':

    loadings

The top five most important predictors in the SVM model were the ManufacturingProcess32, ManufacturingProcess13, BiologicalMaterial06, ManufacturingProcess17, and ManufacturingProcess36. The ManufactoringProcess variables dominate the list with only one BiologicalMaterial variable in the top 5. The optimal linear model from Exercise 6.3 was the PLS model and that was dominated by ManufactoringProcess predictors. The two models share 8 of the same predictors in the top 10.

(c) Explore the relationships between the top predictors and the response for the predictors that are unique to the optimal nonlinear regression model. Do these plots reveal intuition about the biological or process predictors and their relationship with yield?

Between the top 10 predictors for the linear and nonlinear regression models, there were two that were unique to the SVM model. The two predictors were the ManufacturingProcess31 and BiologicalMaterial12.

library(ggplot2)

ggplot(training_data, aes(x = ManufacturingProcess31, y = Yield)) +
  geom_point(alpha = 0.5) +
  geom_smooth(method = "loess", color = "blue", se = FALSE) +
  theme_minimal() +
  labs(title = "Yield vs ManufacturingProcess31")
`geom_smooth()` using formula = 'y ~ x'

Looking at the plot above, the nonlinear regression model was able to capture the relationship between ManufacturingProcess31 and Yield while a linear regression model would probably just go straight through the plot and capture nothing.

library(ggplot2)

ggplot(training_data, aes(x = BiologicalMaterial12, y = Yield)) +
  geom_point(alpha = 0.5) +
  geom_smooth(method = "loess", color = "blue", se = FALSE) +
  theme_minimal() +
  labs(title = "Yield vs BiologicalMaterial12")
`geom_smooth()` using formula = 'y ~ x'

Looking at the plot above, the nonlinear regression model does a good job at capturing the relationship between the Yield and BiologicalMaterial12 because it is able to capture the upswing in the middle and then the downward curve at the end. A linear model would be able to capture some of the relationship but would not do as well of a job compared to the nonlinear model.