Data624_Assignment9
Christian Uriostegui
2024-11-12
Exercise 8.1
Recreate the simulated data from Exercise 7.2:
set.seed(200)
simulated <- mlbench.friedman1(200, sd = 1)
simulated <- cbind(simulated$x, simulated$y)
simulated <- as.data.frame(simulated)
colnames(simulated)[ncol(simulated)] <- "y"Part A
Fit a random forest model to all of the predictors, then estimate the variable importance scores:
model1 <- randomForest(y ~ ., data = simulated,
importance = TRUE,
ntree = 1000)
rfImp1 <- varImp(model1, scale = FALSE)rfImp1## Overall
## V1 8.732235404
## V2 6.415369387
## V3 0.763591825
## V4 7.615118809
## V5 2.023524577
## V6 0.165111172
## V7 -0.005961659
## V8 -0.166362581
## V9 -0.095292651
## V10 -0.074944788Did the random forest model significantly use the uninformative predictors (V6– V10)?
Response
The model did not significantly use predictors V6-V10. V6 scored a low significance number (0.165111172), while variables V7-V10 scored a negative number - which suggests that they can hurt accuracy of the model.
Part B
Now add an additional predictor that is highly correlated with one of the informative predictors. For example:
simulated$duplicate1 <- simulated$V1 + rnorm(200) * .1
cor(simulated$duplicate1, simulated$V1)## [1] 0.9460206model2 <- randomForest(y ~ ., data = simulated,
importance = TRUE,
ntree = 1000)
rfImp2 <- varImp(model2, scale = FALSE)rfImp2## Overall
## V1 5.69119973
## V2 6.06896061
## V3 0.62970218
## V4 7.04752238
## V5 1.87238438
## V6 0.13569065
## V7 -0.01345645
## V8 -0.04370565
## V9 0.00840438
## V10 0.02894814
## duplicate1 4.28331581# adding another var also correlated
simulated$duplicate2 <- simulated$V1 + rnorm(200) * .1
cor(simulated$duplicate2, simulated$V1)## [1] 0.9408631model3 <- randomForest(y ~ ., data = simulated,
importance = TRUE,
ntree = 1000)
rfImp3 <- varImp(model3, scale = FALSE)rfImp3## Overall
## V1 4.91687329
## V2 6.52816504
## V3 0.58711552
## V4 7.04870917
## V5 2.03115561
## V6 0.14213148
## V7 0.10991985
## V8 -0.08405687
## V9 -0.01075028
## V10 0.09230576
## duplicate1 3.80068234
## duplicate2 1.87721959Fit another random forest model to these data. Did the importance score for V1 change? What happens when you add another predictor that is also highly correlated with V1?
Response
Everytime we introduce a variable that is highly correlated with V1, the importance score for V1 drops. This negatively impacts the score because we are introducing redundancy to the model. V1, duplicate1 and duplicate2 explain similar information, leading to a drop in the importance score to V1.
Part C
Use the cforest function in the party package to fit a random forest model using conditional inference trees. The party package function varimp can calculate predictor importance. The conditional argument of that function toggles between the traditional importance measure and the modified version described in Strobl et al. (2007). Do these importances show the same pattern as the traditional random forest model?
cforest_model <- cforest(y ~ ., data = simulated,
controls = cforest_unbiased(ntree = 1000))# Traditional variable importance
traditional_importance <- varimp(cforest_model, conditional = FALSE)
print("Traditional Variable Importance:")## [1] "Traditional Variable Importance:"print(traditional_importance)## V1 V2 V3 V4 V5 V6
## 4.389829273 5.895994513 0.018773326 7.148348396 1.639542658 -0.019720513
## V7 V8 V9 V10 duplicate1 duplicate2
## 0.037680863 -0.026851969 -0.005669536 -0.022708935 4.225800961 1.159922187# Conditional variable importance
conditional_importance <- varimp(cforest_model, conditional = TRUE)
print("Conditional Variable Importance:")## [1] "Conditional Variable Importance:"print(conditional_importance)## V1 V2 V3 V4 V5 V6
## 1.754849270 4.715103754 0.010612108 5.736625783 0.988062464 -0.009498226
## V7 V8 V9 V10 duplicate1 duplicate2
## -0.008040813 -0.001296254 0.008023805 0.004988622 1.484276741 0.322110747Response
For both the original random forest model and the traditional importance measure model they assign high importance values to V1, V2 and V4. The values are slightly different. For ex V1 in the original RF modell is 4.91687329, while V1 in the traditional importance model is 4.389829273.
The patterns in the conditional importance varies both the other two models. The importance scores for V1, V2, V4, as well as the duplicates have decreased. This is because the conditional importance model reduces the importance of correlated variables.
Part D
Repeat this process with different tree models, such as boosted trees and Cubist. Does the same pattern occur?
Response
predictors <- as.matrix(simulated[, setdiff(names(simulated), "y")])
response <- simulated$y
dtrain <- xgb.DMatrix(data = predictors, label = simulated$y)
xgb_model <- xgboost(data = dtrain, max_depth = 6, nrounds = 100, objective = "reg:squarederror", verbose = 0)xgb_importance <- xgb.importance(model = xgb_model)
print("Boosted Trees Variable Importance:")## [1] "Boosted Trees Variable Importance:"print(xgb_importance)## Feature Gain Cover Frequency
## 1: V1 0.298329836 0.07210476 0.15091384
## 2: V4 0.261633373 0.11697020 0.10026110
## 3: V2 0.257249710 0.12572691 0.12793734
## 4: V5 0.054046868 0.10047108 0.09086162
## 5: duplicate1 0.050515894 0.06358473 0.05274151
## 6: V3 0.038116208 0.13428076 0.10757180
## 7: V7 0.015941277 0.07773971 0.08198433
## 8: V6 0.009808250 0.04973403 0.06579634
## 9: V10 0.003948043 0.07692828 0.05587467
## 10: V8 0.003890484 0.04639814 0.05848564
## 11: V9 0.003595885 0.07194699 0.06161880
## 12: duplicate2 0.002924171 0.06411441 0.04595300predictors <- simulated[, setdiff(names(simulated), "y")]
cubist_model <- cubist(x = predictors, y = response)cubist_importance <- summary(cubist_model)
print("Cubist Variable Importance:")## [1] "Cubist Variable Importance:"print(cubist_importance)##
## Call:
## cubist.default(x = predictors, y = response)
##
##
## Cubist [Release 2.07 GPL Edition] Wed Nov 13 23:21:24 2024
## ---------------------------------
##
## Target attribute `outcome'
##
## Read 200 cases (13 attributes) from undefined.data
##
## Model:
##
## Rule 1: [200 cases, mean 14.416183, range 3.55596 to 28.38167, est err 1.936506]
##
## outcome = 0.269253 + 8.9 V4 + 7.1 V2 + 5.1 V5 + 4.8 V1 + 3.2 duplicate1
##
##
## Evaluation on training data (200 cases):
##
## Average |error| 2.012236
## Relative |error| 0.50
## Correlation coefficient 0.87
##
##
## Attribute usage:
## Conds Model
##
## 100% V1
## 100% V2
## 100% V4
## 100% V5
## 100% duplicate1
##
##
## Time: 0.0 secsThe XGBoost model is similar to the original random forest model because it’s top variables are V4,V2, and V1. Unlike the random forest model, the XG Boost model gives duplicate variables lower significance.
The cubist model also assigns high significance to V4, V2 and V1. However, it assigns a low significance to duplicate1 and ignores duplicate2. This model is stronger at handling noise variables.
Exercise 8.2
Use a simulation to show tree bias with different granularities.
Response
Below I created a simulation which involves two predictors:
X1 a categorical variable, with two levels, that is related to our target variable Y.
X2 is a continuous variable that is not related to the target variable Y.
When running 100 simulations of one-split decision tree, the model chose X2 100% of the time. This demonstrates a common problem with decision trees. They will display tree bias towards continuous or more granular variable despite not having a relationship with the response.
# Set up the simulation
set.seed(102)
X1 <- rep(1:2, each = 100)
Y <- X1 + rnorm(200, mean = 0, sd = 4)
set.seed(103)
X2 <- rnorm(200, mean = 0, sd = 2)
simData <- data.frame(Y = Y, X1 = as.factor(X1), X2 = X2)
# Function to perform a one-split tree
one_split_tree <- function(data) {
tree_model <- rpart(Y ~ X1 + X2, data = data, control = rpart.control(maxdepth = 1))
return(as.character(tree_model$frame$var[1])) # Return the variable used for the split
}
# Run the simulation 100 times
set.seed(104)
num_simulations <- 100
split_results <- replicate(num_simulations, one_split_tree(simData))
# Tally the results
table(split_results)## split_results
## X2
## 100Exercise 8.3
In stochastic gradient boosting the bagging fraction and learning rate will govern the construction of the trees as they are guided by the gradient. Although the optimal values of these parameters should be obtained through the tuning process, it is helpful to understand how the magnitudes of these parameters affect magnitudes of variable importance. Figure 8.24 provides the variable importance plots for boosting using two extreme values for the bagging fraction (0.1 and 0.9) and the learning rate (0.1 and 0.9) for the solubility data.The left-hand plot has both parameters set to 0.1, and the right-hand plot has both set to 0.9:
Figure 8.24
Part A
Why does the model on the right focus its importance on just the first few of predictors,where as the model on the left spreads importance across more predictors?
Response
The difference in how importance is assigned to values is due to the tunning parameters. When the parameters are set to 0.1 like in the chart to the left, it reduces the model’s sensitivity and will be less likely to amplify certain variables. The model with the higher bagging rate of 0.9, the model uses a larger portion of data and has a higher learning rate. This allows it to adjust towards the most important features while filtering out less important features.
Part B
Which model do you think would be more predictive of other samples?
Response
The left model would be best because it avoids overfitting and will be able to better learn from other samples. The lower bagging fraction allows more randomness and looks at more predictors. The model on the right places it’s emphasis on a few variables. So while it may perform well on it’s data, it will perform poorly on data it hasn’t seen before.
Part C
How would increasing interaction depth affect the slope of predictor importance for either model in Fig.8.24?
Response
Increasing interaction depth can increase the interactions and patterns seen among variables.
For the model with the lower bagging fraction (left plot) it will have a similar pattern. It will likely increase the significance of top values while also evenly spreading importance across predictors. The higher bagging fraction model will focus even more on dominant predictors because it will find more complex relationships with them.
Exercise 8.7
Refer to Exercises 6.3 and 7.5 which describe a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several tree-based models:
data("ChemicalManufacturingProcess")cmp <- ChemicalManufacturingProcess %>%
mutate(across(where(is.numeric), ~ ifelse(is.na(.), mean(., na.rm = TRUE), .)))set.seed(123)
trainIndex <- createDataPartition(cmp$Yield, p = 0.75, list = FALSE)
trainData <- cmp[trainIndex, ]
testData <- cmp[-trainIndex, ]Part A
Which tree-based regression model gives the optimal resampling and test set performance?
Response
train_control <- trainControl(method = "cv", number = 5)
set.seed(123)
# Decision Tree
model_rpart <- train(Yield ~ ., data = trainData, method = "rpart", trControl = train_control)## Warning in nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,
## : There were missing values in resampled performance measures.# Random Forest
model_rf <- train(Yield ~ ., data = trainData, method = "rf", trControl = train_control)
# Gradient Boosting Machine
model_gbm <- train(Yield ~ ., data = trainData, method = "gbm", trControl = train_control, verbose = FALSE)
# Extreme Gradient Boosting
model_xgb <- train(Yield ~ ., data = trainData, method = "xgbTree", trControl = train_control)## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [23:21:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.model_rpart## CART
##
## 132 samples
## 57 predictor
##
## No pre-processing
## Resampling: Cross-Validated (5 fold)
## Summary of sample sizes: 105, 107, 106, 106, 104
## Resampling results across tuning parameters:
##
## cp RMSE Rsquared MAE
## 0.07805215 1.557288 0.3794076 1.194230
## 0.10411810 1.588786 0.3421578 1.242926
## 0.40046602 1.686062 0.3245573 1.356610
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was cp = 0.07805215.model_rf## Random Forest
##
## 132 samples
## 57 predictor
##
## No pre-processing
## Resampling: Cross-Validated (5 fold)
## Summary of sample sizes: 106, 105, 107, 105, 105
## Resampling results across tuning parameters:
##
## mtry RMSE Rsquared MAE
## 2 1.281401 0.6266450 1.0067497
## 29 1.142061 0.6609793 0.8854347
## 57 1.164003 0.6371759 0.8840761
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was mtry = 29.model_gbm## Stochastic Gradient Boosting
##
## 132 samples
## 57 predictor
##
## No pre-processing
## Resampling: Cross-Validated (5 fold)
## Summary of sample sizes: 107, 106, 105, 105, 105
## Resampling results across tuning parameters:
##
## interaction.depth n.trees RMSE Rsquared MAE
## 1 50 1.170115 0.6426067 0.9054975
## 1 100 1.123112 0.6694031 0.8776542
## 1 150 1.120073 0.6721615 0.8735889
## 2 50 1.142645 0.6510835 0.8949599
## 2 100 1.109999 0.6680089 0.8732209
## 2 150 1.106086 0.6714376 0.8629886
## 3 50 1.172134 0.6370956 0.9117984
## 3 100 1.136787 0.6507049 0.8977929
## 3 150 1.128993 0.6571819 0.9019076
##
## Tuning parameter 'shrinkage' was held constant at a value of 0.1
##
## Tuning parameter 'n.minobsinnode' was held constant at a value of 10
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were n.trees = 150, interaction.depth =
## 2, shrinkage = 0.1 and n.minobsinnode = 10.model_xgb## eXtreme Gradient Boosting
##
## 132 samples
## 57 predictor
##
## No pre-processing
## Resampling: Cross-Validated (5 fold)
## Summary of sample sizes: 105, 107, 105, 105, 106
## Resampling results across tuning parameters:
##
## eta max_depth colsample_bytree subsample nrounds RMSE Rsquared
## 0.3 1 0.6 0.50 50 1.181356 0.6237731
## 0.3 1 0.6 0.50 100 1.168951 0.6343349
## 0.3 1 0.6 0.50 150 1.148587 0.6459149
## 0.3 1 0.6 0.75 50 1.173754 0.6157726
## 0.3 1 0.6 0.75 100 1.140930 0.6443032
## 0.3 1 0.6 0.75 150 1.124863 0.6520300
## 0.3 1 0.6 1.00 50 1.176068 0.6160754
## 0.3 1 0.6 1.00 100 1.133476 0.6463817
## 0.3 1 0.6 1.00 150 1.131943 0.6465826
## 0.3 1 0.8 0.50 50 1.176898 0.6182034
## 0.3 1 0.8 0.50 100 1.193562 0.6143945
## 0.3 1 0.8 0.50 150 1.191320 0.6117701
## 0.3 1 0.8 0.75 50 1.212066 0.5957406
## 0.3 1 0.8 0.75 100 1.154887 0.6305249
## 0.3 1 0.8 0.75 150 1.163151 0.6267993
## 0.3 1 0.8 1.00 50 1.211320 0.5918165
## 0.3 1 0.8 1.00 100 1.153378 0.6296102
## 0.3 1 0.8 1.00 150 1.134325 0.6426135
## 0.3 2 0.6 0.50 50 1.229788 0.5740206
## 0.3 2 0.6 0.50 100 1.234011 0.5747844
## 0.3 2 0.6 0.50 150 1.226940 0.5799946
## 0.3 2 0.6 0.75 50 1.199875 0.5965787
## 0.3 2 0.6 0.75 100 1.188070 0.6072742
## 0.3 2 0.6 0.75 150 1.188065 0.6069832
## 0.3 2 0.6 1.00 50 1.142583 0.6405299
## 0.3 2 0.6 1.00 100 1.133575 0.6488273
## 0.3 2 0.6 1.00 150 1.131205 0.6505468
## 0.3 2 0.8 0.50 50 1.204159 0.5969489
## 0.3 2 0.8 0.50 100 1.214536 0.5868530
## 0.3 2 0.8 0.50 150 1.211579 0.5875720
## 0.3 2 0.8 0.75 50 1.204494 0.6006770
## 0.3 2 0.8 0.75 100 1.188927 0.6098305
## 0.3 2 0.8 0.75 150 1.192383 0.6093147
## 0.3 2 0.8 1.00 50 1.173357 0.6243697
## 0.3 2 0.8 1.00 100 1.165150 0.6319064
## 0.3 2 0.8 1.00 150 1.162299 0.6341742
## 0.3 3 0.6 0.50 50 1.255806 0.5637611
## 0.3 3 0.6 0.50 100 1.244048 0.5752078
## 0.3 3 0.6 0.50 150 1.242932 0.5760729
## 0.3 3 0.6 0.75 50 1.209926 0.5897123
## 0.3 3 0.6 0.75 100 1.207285 0.5930281
## 0.3 3 0.6 0.75 150 1.207402 0.5929675
## 0.3 3 0.6 1.00 50 1.214459 0.5950847
## 0.3 3 0.6 1.00 100 1.213563 0.5961953
## 0.3 3 0.6 1.00 150 1.213387 0.5963421
## 0.3 3 0.8 0.50 50 1.185787 0.6054091
## 0.3 3 0.8 0.50 100 1.176461 0.6122029
## 0.3 3 0.8 0.50 150 1.173962 0.6138344
## 0.3 3 0.8 0.75 50 1.155211 0.6331169
## 0.3 3 0.8 0.75 100 1.156973 0.6333864
## 0.3 3 0.8 0.75 150 1.156735 0.6334789
## 0.3 3 0.8 1.00 50 1.108322 0.6592853
## 0.3 3 0.8 1.00 100 1.110208 0.6573929
## 0.3 3 0.8 1.00 150 1.110092 0.6575213
## 0.4 1 0.6 0.50 50 1.273645 0.5553378
## 0.4 1 0.6 0.50 100 1.312831 0.5400895
## 0.4 1 0.6 0.50 150 1.312994 0.5444202
## 0.4 1 0.6 0.75 50 1.225996 0.5653699
## 0.4 1 0.6 0.75 100 1.203627 0.5881694
## 0.4 1 0.6 0.75 150 1.184045 0.6017211
## 0.4 1 0.6 1.00 50 1.183232 0.6209807
## 0.4 1 0.6 1.00 100 1.160228 0.6378279
## 0.4 1 0.6 1.00 150 1.138113 0.6497670
## 0.4 1 0.8 0.50 50 1.191168 0.6120185
## 0.4 1 0.8 0.50 100 1.193489 0.6221821
## 0.4 1 0.8 0.50 150 1.178936 0.6315083
## 0.4 1 0.8 0.75 50 1.225785 0.5814910
## 0.4 1 0.8 0.75 100 1.219732 0.5875352
## 0.4 1 0.8 0.75 150 1.203572 0.6002071
## 0.4 1 0.8 1.00 50 1.158814 0.6241500
## 0.4 1 0.8 1.00 100 1.124138 0.6476678
## 0.4 1 0.8 1.00 150 1.115937 0.6532791
## 0.4 2 0.6 0.50 50 1.362439 0.4961261
## 0.4 2 0.6 0.50 100 1.397889 0.4824297
## 0.4 2 0.6 0.50 150 1.394368 0.4856973
## 0.4 2 0.6 0.75 50 1.192485 0.6069483
## 0.4 2 0.6 0.75 100 1.183733 0.6148032
## 0.4 2 0.6 0.75 150 1.180831 0.6163853
## 0.4 2 0.6 1.00 50 1.136825 0.6652141
## 0.4 2 0.6 1.00 100 1.140763 0.6640559
## 0.4 2 0.6 1.00 150 1.141623 0.6633641
## 0.4 2 0.8 0.50 50 1.196638 0.5935863
## 0.4 2 0.8 0.50 100 1.188030 0.6019289
## 0.4 2 0.8 0.50 150 1.186001 0.6024689
## 0.4 2 0.8 0.75 50 1.214517 0.5965359
## 0.4 2 0.8 0.75 100 1.212979 0.6017327
## 0.4 2 0.8 0.75 150 1.214064 0.6015942
## 0.4 2 0.8 1.00 50 1.220698 0.5901829
## 0.4 2 0.8 1.00 100 1.221567 0.5886467
## 0.4 2 0.8 1.00 150 1.220036 0.5892447
## 0.4 3 0.6 0.50 50 1.376537 0.5003929
## 0.4 3 0.6 0.50 100 1.372466 0.5025957
## 0.4 3 0.6 0.50 150 1.371119 0.5033218
## 0.4 3 0.6 0.75 50 1.296697 0.5419044
## 0.4 3 0.6 0.75 100 1.294449 0.5429898
## 0.4 3 0.6 0.75 150 1.294439 0.5430211
## 0.4 3 0.6 1.00 50 1.249714 0.5674369
## 0.4 3 0.6 1.00 100 1.250033 0.5676398
## 0.4 3 0.6 1.00 150 1.250012 0.5676534
## 0.4 3 0.8 0.50 50 1.329726 0.5235551
## 0.4 3 0.8 0.50 100 1.335130 0.5194780
## 0.4 3 0.8 0.50 150 1.335073 0.5194875
## 0.4 3 0.8 0.75 50 1.237696 0.5727673
## 0.4 3 0.8 0.75 100 1.235870 0.5745388
## 0.4 3 0.8 0.75 150 1.235921 0.5745020
## 0.4 3 0.8 1.00 50 1.205431 0.5888824
## 0.4 3 0.8 1.00 100 1.203848 0.5899591
## 0.4 3 0.8 1.00 150 1.203845 0.5899619
## MAE
## 0.9139607
## 0.9227520
## 0.9027262
## 0.9132120
## 0.8969261
## 0.8800168
## 0.9096824
## 0.8723558
## 0.8702538
## 0.9326941
## 0.9599580
## 0.9509310
## 0.9129142
## 0.8710642
## 0.8621075
## 0.9335781
## 0.8864350
## 0.8763126
## 0.9042167
## 0.8960185
## 0.8936234
## 0.9017656
## 0.8884023
## 0.8867278
## 0.8657738
## 0.8478229
## 0.8426987
## 0.9646138
## 0.9631510
## 0.9656707
## 0.9174680
## 0.9040477
## 0.9058386
## 0.9215051
## 0.9100411
## 0.9056909
## 0.9627904
## 0.9459640
## 0.9452663
## 0.9308769
## 0.9320468
## 0.9324059
## 0.9544676
## 0.9543435
## 0.9543372
## 0.9323229
## 0.9286747
## 0.9263236
## 0.9047708
## 0.9081723
## 0.9079763
## 0.8422532
## 0.8428436
## 0.8426737
## 1.0299965
## 1.0669286
## 1.0562726
## 0.9469377
## 0.9319392
## 0.9322490
## 0.9131149
## 0.8925862
## 0.8760784
## 0.9400634
## 0.9477946
## 0.9308371
## 0.9686298
## 0.9672414
## 0.9479300
## 0.9015800
## 0.8772470
## 0.8711224
## 1.0352853
## 1.0784642
## 1.0767212
## 0.9012824
## 0.8962972
## 0.8948358
## 0.8636662
## 0.8644532
## 0.8656008
## 0.9224185
## 0.9340520
## 0.9358985
## 0.8787713
## 0.8854166
## 0.8850485
## 0.9438403
## 0.9372577
## 0.9362009
## 1.0726698
## 1.0741282
## 1.0727059
## 1.0002515
## 0.9989612
## 0.9989481
## 0.9400993
## 0.9396816
## 0.9396773
## 1.0324791
## 1.0346560
## 1.0344134
## 0.9570562
## 0.9559948
## 0.9560250
## 0.9128135
## 0.9114936
## 0.9114996
##
## Tuning parameter 'gamma' was held constant at a value of 0
## Tuning
## parameter 'min_child_weight' was held constant at a value of 1
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nrounds = 50, max_depth = 3, eta
## = 0.3, gamma = 0, colsample_bytree = 0.8, min_child_weight = 1 and subsample
## = 1.dtPred <- predict(model_rpart, newdata = testData)
dtResults <- postResample(pred = dtPred, obs = testData$Yield)
print(dtResults)## RMSE Rsquared MAE
## 1.5623205 0.2653869 1.1994155rfPred <- predict(model_rf, newdata = testData)
rfResults <- postResample(pred = rfPred, obs = testData$Yield)
print(rfResults)## RMSE Rsquared MAE
## 1.1920723 0.5421700 0.8654828gbmPred <- predict(model_gbm, newdata = testData)
gbmResults <- postResample(pred = gbmPred, obs = testData$Yield)
print(gbmResults)## RMSE Rsquared MAE
## 1.1649993 0.5667080 0.9234405xgbPred <- predict(model_xgb, newdata = testData)
xgbResults <- postResample(pred = xgbPred, obs = testData$Yield)
print(xgbResults)## RMSE Rsquared MAE
## 1.1087330 0.5978429 0.8620709When looking at cross-validation performance the Gradient Boosting Machine (GBM) performs the best. It has the lowest RMSE (1.1061) and highest R-squared (0.6714).
When observing for the best train set performance the Extreme Gradient Boosting (XGBoost) model has the lowest RMSE (1.1087330) and highest R-squared (0.5978429).
The GBM Model has the best cross-validation performance and second best train set performance. For this reason, I’ll select this as our optimal tree-based regression model.
Part B
Which predictors are most important in the optimal tree-based regression model? Do either the biological or process variables dominate the list?
Response
importance <- varImp(model_gbm, scale = FALSE)
print(importance)## gbm variable importance
##
## only 20 most important variables shown (out of 57)
##
## Overall
## ManufacturingProcess32 501.91
## BiologicalMaterial03 130.69
## ManufacturingProcess17 118.04
## ManufacturingProcess30 79.74
## ManufacturingProcess13 78.28
## ManufacturingProcess06 73.92
## ManufacturingProcess15 59.89
## BiologicalMaterial09 55.87
## BiologicalMaterial12 55.43
## BiologicalMaterial05 49.51
## ManufacturingProcess31 48.98
## ManufacturingProcess09 41.58
## BiologicalMaterial06 37.11
## ManufacturingProcess35 35.12
## ManufacturingProcess04 32.77
## ManufacturingProcess37 31.22
## ManufacturingProcess27 26.65
## ManufacturingProcess05 26.14
## BiologicalMaterial10 24.28
## BiologicalMaterial08 23.60The ManufacturingProcess variables are make up four of the top five most important predictors. ManufacturingProcess32 is the most significant predictor. In this model, the manufacturing processes play a more critical role in predicting Yield than the biological material variables.
Part C
Plot the optimal single tree with the distribution of yield in the terminal nodes. Does this view of the data provide additional knowledge about the biological or process predictors and their relationship with yield?
Response
set.seed(123)
single_tree_model <- rpart(Yield ~ ., data = trainData, control = rpart.control(cp = 0.07805215))
rpart.plot(single_tree_model, type = 3, box.palette = "Blues", fallen.leaves = TRUE, cex = 0.6)
In this alternative view, we still see that the Manufacturing Processes have a major impact in determining yield outcomes. There is however, additional knowledge provided with the Biological variables. Biological variables also have influence over the yield in specific scenarios, something which isn’t as obvious when looking solely at the importance values.