Code
knitr::opts_chunk$set(echo = TRUE, warning = FALSE, error = FALSE)DATA 624 Homework9
Running Code
Do problems 8.1, 8.2, 8.3, and 8.7 in Kuhn and Johnson.
- 8.1. Recreate the simulated data from Exercise 7.2:
Code
set.seed(200)
simulated <- mlbench.friedman1(200, sd = 1)
simulated <- cbind(simulated$x, simulated$y)
simulated <- as.data.frame(simulated)
colnames(simulated)[ncol(simulated)] <- "y"- Fit a random forest model to all of the predictors, then estimate the variable importance scores:
Did the random forest model significantly use the uninformative predictors (V6 – V10)?
ANSWER:The variable importance values show that these predictors were not highly scored since they are either close to zero or minimum negative value.
Code
rf1 <- randomForest(y ~ ., data = simulated, importance = TRUE, ntree = 1000)
rimp <- varImp(rf1, scale = FALSE)
rimp %>%
as.data.frame() %>%
rownames_to_column() %>%
rename(Variable = rowname) %>%
arrange(desc(Overall)) %>%
kable() %>%
kable_styling()| Variable | Overall |
|---|---|
| V1 | 8.7322354 |
| V4 | 7.6151188 |
| V2 | 6.4153694 |
| V5 | 2.0235246 |
| V3 | 0.7635918 |
| V6 | 0.1651112 |
| V7 | -0.0059617 |
| V10 | -0.0749448 |
| V9 | -0.0952927 |
| V8 | -0.1663626 |
- Now add an additional predictor that is highly correlated with one of the informative predictors. Fit another random forest model to these data.
Did the importance score for V1 change?
What happens when you add another predictor that is also highly correlated with V1?
ANSWER: We can see that V1 score decreased. As given in the book, the between-predictor correlations can cause and dilute the importance of key predictors. If we add another correlated predictor, the combined importance of these 2 predictors will be lower. Therefore, we need to be careful when adding additional highly correlated value to an existing already important variable which can be detrimental.
Code
set.seed(100)
simulated$duplicate1 <- simulated$V1 + rnorm(200) * .1
cor(simulated$duplicate1, simulated$V1)[1] 0.9509187Code
rf2 <- randomForest(y ~ ., data = simulated, importance = TRUE, ntree = 1000)
rimp2 <- varImp(rf2, scale = FALSE)
rimp2 %>%
as.data.frame() %>%
rownames_to_column() %>%
rename(Variable = rowname) %>%
arrange(desc(Overall)) %>%
kable() %>%
kable_styling()| Variable | Overall |
|---|---|
| V4 | 7.2020404 |
| V1 | 6.4930607 |
| V2 | 6.2158219 |
| duplicate1 | 4.1642335 |
| V5 | 2.1352447 |
| V3 | 0.6483959 |
| V6 | 0.1402481 |
| V10 | 0.0409874 |
| V7 | -0.0498396 |
| V9 | -0.1141065 |
| V8 | -0.1925104 |
- Use the cforest function in the party package to fit a random forest model using conditional inference trees. The party package function varimp can calculate predictor importance. The conditional argument of that function toggles between the traditional importance measure and the modified version described in Strobl et al. (2007).
Do these importances show the same pattern as the traditional random forest model?
ANSWER: The order of the top overall important variables have changed below. V1 has lost further significance in this function.V3 and V10 have lesser importance values here in this model. Also V1 and duplicate which were correlated have lesser importance values on both than the earlier model causing lesser bias in the cforest model.
Code
set.seed(100)
rf3 <- cforest(y ~ ., data = simulated)
rimp_3 <- varImp(rf3, conditional = TRUE)
rimp_3 %>%
as.data.frame() %>%
rownames_to_column() %>%
rename(Variable = rowname) %>%
arrange(desc(Overall)) %>%
kable() %>%
kable_styling()| Variable | Overall |
|---|---|
| V4 | 6.0827970 |
| V2 | 4.9191405 |
| V1 | 3.1948232 |
| V5 | 1.1922204 |
| duplicate1 | 1.0183557 |
| V7 | 0.0522786 |
| V3 | 0.0252888 |
| V6 | 0.0194944 |
| V10 | -0.0031924 |
| V9 | -0.0057159 |
| V8 | -0.0064838 |
- Repeat this process with different tree models, such as boosted trees and Cubist.
BOOSTED TREES
Does the same pattern occur?
ANSWER: BOOSTED: when trained without the duplicate values, the same set of top overall important variables remain there still. V1 is the highest here.
```{r}
set.seed(100)
gbmgrid <- expand.grid(.interaction.depth = seq(1, 7, by = 2),
.n.trees = seq(100, 1000, by = 100),
.shrinkage = c(0.01, 0.1),
.n.minobsinnode = 10)
gbm_1 <- train(y ~ ., data = dplyr::select(simulated, -duplicate1), method="gbm", tuneGrid = gbmgrid, verbose = FALSE)
gbmimp_1 <- varImp(gbm_1)
gbmimp_1$importance %>%
as.data.frame() %>%
rownames_to_column() %>%
rename(Variable = rowname) %>%
arrange(desc(Overall)) %>%
kable() %>%
kable_styling()
```ANSWER: BOOSTED: when trained with the duplicate values, V1’s importance is decreased and duplicate1 is also important in the set of predictors due to correlation with V1.
Code
set.seed(100)
gbmgrid <- expand.grid(.interaction.depth = seq(1, 7, by = 2),
.n.trees = seq(100, 1000, by = 100),
.shrinkage = c(0.01, 0.1),
.n.minobsinnode = 10)
gbm_2 <- train(y ~ ., data = simulated, method="gbm", tuneGrid = gbmgrid, verbose = FALSE)
gbm_imp_2 <- varImp(gbm_2)
gbm_imp_2$importance %>%
as.data.frame() %>%
rownames_to_column() %>%
rename(Variable = rowname) %>%
arrange(desc(Overall)) %>%
kable() %>%
kable_styling()| Variable | Overall |
|---|---|
| V2 | 100.0000000 |
| V4 | 98.3633787 |
| V1 | 67.0372905 |
| V5 | 38.9461972 |
| duplicate1 | 37.7218395 |
| V3 | 21.8366583 |
| V7 | 4.9593253 |
| V6 | 3.8566801 |
| V9 | 0.2547942 |
| V10 | 0.1998168 |
| V8 | 0.0000000 |
CUBIST without duplicate: ANSWER: The model cubist keeps the same set of predictors on overall importance similar to other models.
Code
set.seed(100)
cubist_1 <- train(y ~ ., data = dplyr::select(simulated, -duplicate1), method="cubist")
cubist_imp_1 <- varImp(cubist_1)
cubist_imp_1$importance %>%
as.data.frame() %>%
rownames_to_column() %>%
rename(Variable = rowname) %>%
arrange(desc(Overall)) %>%
kable() %>%
kable_styling()| Variable | Overall |
|---|---|
| V1 | 100.00000 |
| V2 | 75.69444 |
| V4 | 68.05556 |
| V3 | 58.33333 |
| V5 | 55.55556 |
| V6 | 15.27778 |
| V7 | 0.00000 |
| V8 | 0.00000 |
| V9 | 0.00000 |
| V10 | 0.00000 |
CUBIST with duplicate:
ANSWER: The model cubist seems to behave a bit differently when it comes to below model. It has kept V1 the same importance without reducing its importance and included a small component of importance of the duplicate variable, which seems to indicate some noise that is not necessary and optimal from a performance perpective.
Code
set.seed(100)
cubist_2 <- train(y ~ ., data = simulated, method="cubist")
cubist_imp_2 <- varImp(cubist_2)
cubist_imp_2$importance %>%
as.data.frame() %>%
rownames_to_column() %>%
rename(Variable = rowname) %>%
arrange(desc(Overall)) %>%
kable() %>%
kable_styling()| Variable | Overall |
|---|---|
| V1 | 100.000000 |
| V2 | 79.861111 |
| V4 | 68.055556 |
| V3 | 58.333333 |
| V5 | 51.388889 |
| V6 | 25.694444 |
| duplicate1 | 4.166667 |
| V7 | 0.000000 |
| V8 | 0.000000 |
| V9 | 0.000000 |
| V10 | 0.000000 |
8.2 Use a simulation to show tree bias with different granularities.
ANWER: We can simulate a function which is non-linear. Then we can check the MSE of the models. We can see below as the tree depth increases and granularity increases, the MSE decreases, however, at one point, it plateaus out and shows that the optimum depth is achieved and beyond that the model is not overfitting the data and says at the same MSE value.
Code
set.seed(100)
n_sample <- 400
nonlinear_function <- function(x){
sin(1.50 * x) + 2 * cos(.30*x)
}
x <- runif(n_sample, 1, 25)
f_of_x <- nonlinear_function(x)
noise <- rnorm(n_sample, 0, 2)
y <- f_of_x + noise
df1 <- data.frame(y=y, x=x)
in_t <- createDataPartition(df1$y, p = .8, list = FALSE, times = 1)
train_df <- df1[in_t,]
test_df <- df1[-in_t,]
results <- data.frame(Granularity = c(NA), MSE = c(NA), data = c(NA)) %>% na.omit()
get_mse <- function(model, data){
y_hat <- predict(model, data)
mse <- mean((y_hat - data$y)^2)
return(mse)
}
for(depth in seq(1:10)){
rtree_model <- rpart(y ~ x, data = train_df, control=rpart.control(maxdepth=depth))
results <- rbind(results, data.frame(Granularity = depth, MSE = get_mse(rtree_model, train_df), data = "Training"))
results <- rbind(results, data.frame(Granularity = depth, MSE = get_mse(rtree_model, test_df), data = "Test"))
}
ggplot(results, aes(Granularity, MSE, color = data, group = data)) +
geom_line() +
geom_point() +
scale_color_brewer(palette = "Set1") +
theme(legend.position = "bottom", legend.title = element_blank())
8.3. In stochastic gradient boosting the bagging fraction and learning rate will govern the construction of the trees as they are guided by the gradient. Although the optimal values of these parameters should be obtained through the tuning process, it is helpful to understand how the magnitudes of these parameters affect magnitudes of variable importance. Figure 8.24 provides the variable importance plots for boosting using two extreme values for the bagging fraction (0.1 and 0.9) and the learning rate (0.1 and 0.9) for the solubility data. The left-hand plot has both parameters set to 0.1, and the right-hand plot has both set to 0.9:
- Why does the model on the right focus its importance on just the first few of predictors, whereas the model on the left spreads importance across more predictors?
The bagging fraction and learning rate are both parameters that can be used in boosting method. Friedman recommended a fraction of 0.5. The learning rate is ideally smaller values works best. When the learning rate is small, then we see that the weak learner is optimal at that level, however, the bagging fraction is also lower than 0.5. The model on the left works to find the best fit including these 2 parameters. The one on the right, has a higher learning rate and higher than recommended bagging fraction, therefore it selects fewer predictors and probably overfits the data and has higher computational complexity at the same tree depth.
- Which model do you think would be more predictive of other samples?
ANSWER: The one on the left has better fraction and learning rate, therefore,I would select that one to tune the model.
- How would increasing interaction depth affect the slope of predictor importance for either model in Fig. 8.24?
ANSWER: Increasing interaction depth impacts steeper importance profile on boosted trees, because more of the same predictors are selected across the trees therfore contributing more to the importance metric for the predictor.
8.7. Refer to Exercises 6.3 and 7.5 which describe a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several tree-based models:
- Which tree-based regression model gives the optimal resampling and test set performance?
ANSWER: The tree based models have a better score on the Rsquared. Also, the RMSE is lower on the boosted tree model, which indicates better performance on the models. We do have to be aware of the complexity and over fitting. In the case of the boosting model the number of trees was 1000 and the interaction depth was 25 which is a larger model. The boosted tree model had the highest rsquared and lowest rmse on the in sample test set.
Code
# load data
library(AppliedPredictiveModeling)
data(ChemicalManufacturingProcess)
set.seed(52)
knn_model <- preProcess(ChemicalManufacturingProcess, "knnImpute")
df <- predict(knn_model, ChemicalManufacturingProcess)
df1 <- df %>%
select_at(vars(-one_of(nearZeroVar(., names = TRUE))))
in_train <- createDataPartition(df1$Yield, times = 1, p = 0.8, list = FALSE)
train_df <- df1[in_train, ]
test_df <- df1[-in_train, ]
pls_model <- train(
Yield ~ ., data = train_df, method = "pls",
center = TRUE,
scale = TRUE,
trControl = trainControl("cv", number = 10),
tuneLength = 25
)
pls_predictions <- predict(pls_model, test_df)
pls_in_sample <- pls_model$results[pls_model$results$ncomp == pls_model$bestTune$ncomp,]
results <- data.frame(t(postResample(pred = pls_predictions, obs = test_df$Yield))) %>%
mutate("In Sample RMSE" = pls_in_sample$RMSE,
"In Sample Rsquared" = pls_in_sample$Rsquared,
"In Sample MAE" = pls_in_sample$MAE,
"Model"= "PLS")
pls_modelPartial Least Squares
144 samples
56 predictor
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 128, 130, 129, 129, 129, 132, ...
Resampling results across tuning parameters:
ncomp RMSE Rsquared MAE
1 0.7588610 0.4602386 0.6320106
2 0.8282858 0.5041334 0.5974373
3 0.6527136 0.5979023 0.5296489
4 0.7160534 0.5800913 0.5512772
5 0.7913601 0.5548396 0.5692188
6 0.8153282 0.5515141 0.5813398
7 0.7631424 0.5562349 0.5824913
8 0.7904059 0.5479227 0.5926194
9 0.7944752 0.5431205 0.5954466
10 0.8128817 0.5379553 0.6058177
11 0.8378909 0.5293325 0.6228237
12 0.8659149 0.5198728 0.6360241
13 0.9254579 0.5076041 0.6586479
14 0.9705785 0.5045896 0.6706695
15 1.0432300 0.5010992 0.6890141
16 1.1189361 0.4955041 0.7079274
17 1.1881381 0.4898339 0.7269274
18 1.2407903 0.4893460 0.7408979
19 1.3504351 0.4838987 0.7694120
20 1.4318285 0.4803213 0.7904950
21 1.5502310 0.4766268 0.8233676
22 1.6805618 0.4725997 0.8629514
23 1.7785534 0.4635749 0.9002786
24 1.8315949 0.4565407 0.9237536
25 1.8940851 0.4526206 0.9494982
RMSE was used to select the optimal model using the smallest value.
The final value used for the model was ncomp = 3.Code
set.seed(50)
bag1 = bagControl(fit = ctreeBag$fit, predict = ctreeBag$pred, aggregate = ctreeBag$aggregate)
bag_model <- train(Yield ~ ., data = train_df, method="bag", bagControl = bag1,
center = TRUE,
scale = TRUE,
trControl = trainControl("cv", number = 10),
tuneLength = 25)
bag_predictions <- predict(bag_model, test_df)
bag_in_sample <- merge(bag_model$results, bag_model$bestTune)
results <- data.frame(t(postResample(pred = bag_predictions, obs = test_df$Yield))) %>%
mutate("In Sample RMSE" = bag_in_sample$RMSE,
"In Sample Rsquared" = bag_in_sample$Rsquared,
"In Sample MAE" = bag_in_sample$MAE,
"Model"= "Bagged Tree") %>%
rbind(results)
bag_modelBagged Model
144 samples
56 predictor
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 129, 129, 130, 129, 131, 131, ...
Resampling results:
RMSE Rsquared MAE
0.7288567 0.5222249 0.5902326
Tuning parameter 'vars' was held constant at a value of 56Code
set.seed(50)
gbm1 <- train(Yield ~ ., data = train_df, method="gbm", verbose = FALSE,
trControl = trainControl("cv", number = 10),
tuneLength = 25)
gbm_predictions <- predict(gbm1, test_df)
gbm_in_sample <- merge(gbm1$results, gbm1$bestTune)
results <- data.frame(t(postResample(pred = gbm_predictions, obs = test_df$Yield))) %>%
mutate("In Sample RMSE" = gbm_in_sample$RMSE,
"In Sample Rsquared" = gbm_in_sample$Rsquared,
"In Sample MAE" = gbm_in_sample$MAE,
"Model"= "Boosted Tree") %>%
rbind(results)
gbm1Stochastic Gradient Boosting
144 samples
56 predictor
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 129, 129, 130, 129, 131, 131, ...
Resampling results across tuning parameters:
interaction.depth n.trees RMSE Rsquared MAE
1 50 0.6658771 0.6076118 0.5156863
1 100 0.6480929 0.6178329 0.5026019
1 150 0.6521267 0.6214618 0.5093340
1 200 0.6602461 0.6095122 0.5208637
1 250 0.6608882 0.6090869 0.5233967
1 300 0.6609715 0.6089046 0.5267200
1 350 0.6598821 0.6091554 0.5250823
1 400 0.6611865 0.6062903 0.5239535
1 450 0.6650426 0.6016924 0.5261714
1 500 0.6661135 0.6021032 0.5262585
1 550 0.6708525 0.5973289 0.5320108
1 600 0.6731261 0.5953547 0.5335536
1 650 0.6723260 0.5937803 0.5338859
1 700 0.6748081 0.5938551 0.5367490
1 750 0.6751008 0.5915194 0.5371940
1 800 0.6715438 0.5966190 0.5323865
1 850 0.6734561 0.5942363 0.5347708
1 900 0.6749715 0.5947333 0.5372078
1 950 0.6762783 0.5939449 0.5372847
1 1000 0.6724175 0.5966979 0.5330613
1 1050 0.6731340 0.5972850 0.5329174
1 1100 0.6711356 0.5998748 0.5309498
1 1150 0.6743453 0.5959950 0.5336098
1 1200 0.6763539 0.5938292 0.5352138
1 1250 0.6773485 0.5931282 0.5352943
2 50 0.6323804 0.6372195 0.4868686
2 100 0.6229595 0.6498941 0.4826181
2 150 0.6214577 0.6560514 0.4773672
2 200 0.6126572 0.6640127 0.4724291
2 250 0.6130859 0.6645138 0.4735478
2 300 0.6109182 0.6671505 0.4722638
2 350 0.6113258 0.6667449 0.4722960
2 400 0.6121150 0.6658309 0.4742726
2 450 0.6143044 0.6637094 0.4770173
2 500 0.6161098 0.6629597 0.4785274
2 550 0.6162701 0.6621466 0.4788790
2 600 0.6150743 0.6633184 0.4773131
2 650 0.6159071 0.6629090 0.4793213
2 700 0.6160713 0.6631103 0.4793863
2 750 0.6159454 0.6631845 0.4788571
2 800 0.6163159 0.6631819 0.4789267
2 850 0.6162417 0.6636161 0.4786614
2 900 0.6154671 0.6643867 0.4780989
2 950 0.6151443 0.6650220 0.4776551
2 1000 0.6152376 0.6650790 0.4776277
2 1050 0.6148193 0.6657087 0.4771092
2 1100 0.6148280 0.6658062 0.4772242
2 1150 0.6146770 0.6658485 0.4771229
2 1200 0.6146978 0.6658354 0.4770999
2 1250 0.6146087 0.6660150 0.4769995
3 50 0.6352718 0.6352804 0.5025100
3 100 0.6300023 0.6367192 0.5010037
3 150 0.6292878 0.6389098 0.5042718
3 200 0.6268944 0.6409488 0.5031280
3 250 0.6245620 0.6454570 0.4992228
3 300 0.6241298 0.6471252 0.4966624
3 350 0.6219960 0.6502330 0.4964283
3 400 0.6207957 0.6511997 0.4960958
3 450 0.6206732 0.6516843 0.4965343
3 500 0.6205168 0.6522394 0.4961079
3 550 0.6205486 0.6521374 0.4965039
3 600 0.6205194 0.6522669 0.4964744
3 650 0.6206693 0.6522014 0.4966702
3 700 0.6205786 0.6524486 0.4966132
3 750 0.6203727 0.6526182 0.4965538
3 800 0.6202708 0.6528732 0.4964925
3 850 0.6203177 0.6527609 0.4964907
3 900 0.6202194 0.6529066 0.4964309
3 950 0.6202913 0.6529241 0.4965703
3 1000 0.6203288 0.6529041 0.4966478
3 1050 0.6202581 0.6529585 0.4966226
3 1100 0.6202003 0.6530275 0.4965650
3 1150 0.6202061 0.6530442 0.4966087
3 1200 0.6201971 0.6530459 0.4966283
3 1250 0.6201963 0.6530688 0.4966348
4 50 0.6465346 0.6274878 0.5075889
4 100 0.6374716 0.6378478 0.5067489
4 150 0.6260815 0.6513254 0.4964974
4 200 0.6200602 0.6572357 0.4935744
4 250 0.6189276 0.6586816 0.4904474
4 300 0.6171230 0.6603259 0.4893893
4 350 0.6144585 0.6632559 0.4872706
4 400 0.6134540 0.6640714 0.4869912
4 450 0.6120918 0.6652703 0.4858017
4 500 0.6120635 0.6653348 0.4856541
4 550 0.6118028 0.6656497 0.4858174
4 600 0.6114066 0.6659599 0.4855529
4 650 0.6111649 0.6662778 0.4854118
4 700 0.6105899 0.6668890 0.4852619
4 750 0.6103929 0.6670745 0.4852722
4 800 0.6102519 0.6670907 0.4851811
4 850 0.6099558 0.6674053 0.4850551
4 900 0.6098695 0.6674693 0.4850031
4 950 0.6097507 0.6675585 0.4849697
4 1000 0.6097607 0.6675541 0.4850067
4 1050 0.6096486 0.6676225 0.4850104
4 1100 0.6096172 0.6677026 0.4849981
4 1150 0.6096017 0.6677154 0.4849987
4 1200 0.6095536 0.6677569 0.4849529
4 1250 0.6095623 0.6677498 0.4849883
5 50 0.6447665 0.6218846 0.5151245
5 100 0.6352173 0.6384244 0.5101078
5 150 0.6281540 0.6470187 0.5037302
5 200 0.6246674 0.6525461 0.4998716
5 250 0.6240824 0.6514726 0.4994844
5 300 0.6247367 0.6517870 0.5000373
5 350 0.6236888 0.6530613 0.4982807
5 400 0.6214734 0.6561941 0.4961570
5 450 0.6206068 0.6572210 0.4954744
5 500 0.6195116 0.6588175 0.4946671
5 550 0.6198529 0.6583723 0.4947576
5 600 0.6195378 0.6588909 0.4947660
5 650 0.6190730 0.6593057 0.4942814
5 700 0.6193038 0.6590590 0.4944522
5 750 0.6192505 0.6593290 0.4945057
5 800 0.6192821 0.6592651 0.4944588
5 850 0.6193500 0.6591118 0.4945493
5 900 0.6193087 0.6592343 0.4945710
5 950 0.6192678 0.6593077 0.4945713
5 1000 0.6193969 0.6591867 0.4946803
5 1050 0.6194312 0.6591307 0.4947130
5 1100 0.6194528 0.6590976 0.4948092
5 1150 0.6195454 0.6590447 0.4949145
5 1200 0.6194956 0.6590940 0.4948831
5 1250 0.6194445 0.6591548 0.4948220
6 50 0.6032358 0.6762816 0.4708649
6 100 0.5981644 0.6787256 0.4666171
6 150 0.5929715 0.6831696 0.4629702
6 200 0.5901683 0.6860182 0.4614689
6 250 0.5896816 0.6859785 0.4614771
6 300 0.5869001 0.6899448 0.4590856
6 350 0.5854199 0.6917370 0.4574679
6 400 0.5853056 0.6922191 0.4576023
6 450 0.5842552 0.6937987 0.4570216
6 500 0.5835167 0.6946466 0.4564906
6 550 0.5832880 0.6949733 0.4563552
6 600 0.5832773 0.6950357 0.4561864
6 650 0.5829007 0.6955985 0.4557966
6 700 0.5829963 0.6955117 0.4556468
6 750 0.5828587 0.6956740 0.4557227
6 800 0.5827989 0.6957545 0.4556157
6 850 0.5826174 0.6959084 0.4554695
6 900 0.5825857 0.6959219 0.4554509
6 950 0.5826607 0.6958348 0.4554576
6 1000 0.5826293 0.6959407 0.4554585
6 1050 0.5826381 0.6959361 0.4555571
6 1100 0.5826608 0.6959251 0.4556157
6 1150 0.5827306 0.6958349 0.4556631
6 1200 0.5827436 0.6958376 0.4557009
6 1250 0.5827341 0.6958488 0.4556943
7 50 0.6046085 0.6700724 0.4665491
7 100 0.5993507 0.6748706 0.4724635
7 150 0.5879991 0.6917751 0.4662925
7 200 0.5803729 0.6987041 0.4581599
7 250 0.5788004 0.7020600 0.4581822
7 300 0.5751207 0.7071273 0.4558430
7 350 0.5725350 0.7095157 0.4544543
7 400 0.5722865 0.7100919 0.4542607
7 450 0.5712031 0.7111779 0.4542526
7 500 0.5711597 0.7110912 0.4543717
7 550 0.5715496 0.7108313 0.4548160
7 600 0.5710228 0.7111557 0.4544906
7 650 0.5711562 0.7110007 0.4545898
7 700 0.5709096 0.7112768 0.4545356
7 750 0.5709021 0.7112402 0.4546730
7 800 0.5709155 0.7112873 0.4548405
7 850 0.5708596 0.7112712 0.4548029
7 900 0.5706825 0.7114498 0.4547880
7 950 0.5707348 0.7114063 0.4548733
7 1000 0.5707555 0.7113956 0.4549329
7 1050 0.5708762 0.7112991 0.4551331
7 1100 0.5708368 0.7113319 0.4551291
7 1150 0.5708860 0.7112880 0.4551987
7 1200 0.5709310 0.7112533 0.4552587
7 1250 0.5709202 0.7112709 0.4552640
8 50 0.6307438 0.6541374 0.4923318
8 100 0.6115205 0.6747954 0.4787321
8 150 0.6057625 0.6829463 0.4759356
8 200 0.6035484 0.6869472 0.4761999
8 250 0.6048121 0.6868057 0.4767126
8 300 0.5997311 0.6923605 0.4736060
8 350 0.5989311 0.6932693 0.4736133
8 400 0.5970918 0.6952876 0.4720541
8 450 0.5975226 0.6950192 0.4722325
8 500 0.5968384 0.6956798 0.4718402
8 550 0.5967981 0.6955852 0.4718844
8 600 0.5964302 0.6960042 0.4712361
8 650 0.5964675 0.6959463 0.4711457
8 700 0.5964634 0.6960756 0.4710832
8 750 0.5964451 0.6960809 0.4710445
8 800 0.5964556 0.6960393 0.4709956
8 850 0.5964525 0.6960528 0.4710724
8 900 0.5964244 0.6960561 0.4709794
8 950 0.5963261 0.6961273 0.4708918
8 1000 0.5963369 0.6961187 0.4709548
8 1050 0.5962456 0.6962387 0.4708917
8 1100 0.5961612 0.6963387 0.4708059
8 1150 0.5961174 0.6963894 0.4708010
8 1200 0.5960876 0.6964203 0.4707832
8 1250 0.5960241 0.6964692 0.4707040
9 50 0.6306005 0.6443434 0.5079906
9 100 0.6091950 0.6619871 0.4944600
9 150 0.6002807 0.6731126 0.4894069
9 200 0.5938829 0.6803880 0.4846796
9 250 0.5948338 0.6809885 0.4852801
9 300 0.5935818 0.6820477 0.4851929
9 350 0.5924271 0.6841334 0.4845214
9 400 0.5930700 0.6833043 0.4848393
9 450 0.5912966 0.6855365 0.4832038
9 500 0.5919179 0.6852024 0.4844722
9 550 0.5916342 0.6858866 0.4843174
9 600 0.5911720 0.6865298 0.4840675
9 650 0.5911675 0.6865898 0.4840518
9 700 0.5913962 0.6865023 0.4841596
9 750 0.5914327 0.6865365 0.4840976
9 800 0.5912621 0.6867682 0.4839628
9 850 0.5915709 0.6865460 0.4842227
9 900 0.5915647 0.6865465 0.4842021
9 950 0.5914195 0.6867429 0.4840498
9 1000 0.5913940 0.6867366 0.4839522
9 1050 0.5914241 0.6867343 0.4839633
9 1100 0.5915544 0.6866201 0.4840741
9 1150 0.5915640 0.6866360 0.4840564
9 1200 0.5916038 0.6865638 0.4841394
9 1250 0.5915942 0.6866021 0.4841382
10 50 0.6086727 0.6552207 0.4873221
10 100 0.6042680 0.6603426 0.4870945
10 150 0.6028962 0.6608075 0.4863969
10 200 0.5996710 0.6656893 0.4804635
10 250 0.5996538 0.6661200 0.4806329
10 300 0.5957531 0.6710592 0.4778370
10 350 0.5932085 0.6743518 0.4763412
10 400 0.5918639 0.6756721 0.4750306
10 450 0.5906671 0.6771332 0.4739254
10 500 0.5892375 0.6785403 0.4724610
10 550 0.5887914 0.6791832 0.4723072
10 600 0.5886613 0.6792535 0.4721647
10 650 0.5882934 0.6796672 0.4719162
10 700 0.5880294 0.6800740 0.4718385
10 750 0.5879910 0.6800299 0.4720612
10 800 0.5879754 0.6800913 0.4722094
10 850 0.5879147 0.6801961 0.4723438
10 900 0.5878930 0.6802688 0.4724309
10 950 0.5877280 0.6803747 0.4723792
10 1000 0.5876150 0.6805357 0.4723729
10 1050 0.5875358 0.6806099 0.4723779
10 1100 0.5874206 0.6806937 0.4723363
10 1150 0.5874874 0.6806397 0.4724467
10 1200 0.5874942 0.6806623 0.4724939
10 1250 0.5874946 0.6806628 0.4725226
11 50 0.6469583 0.6214635 0.5071461
11 100 0.6407726 0.6333025 0.5104839
11 150 0.6260206 0.6506828 0.5029527
11 200 0.6242389 0.6537536 0.5025663
11 250 0.6233805 0.6544049 0.5004781
11 300 0.6208774 0.6584952 0.4985216
11 350 0.6200443 0.6590470 0.4973263
11 400 0.6182350 0.6616717 0.4954718
11 450 0.6182362 0.6622873 0.4950870
11 500 0.6171843 0.6633915 0.4938810
11 550 0.6169638 0.6636348 0.4939154
11 600 0.6171270 0.6634656 0.4939673
11 650 0.6169955 0.6635797 0.4939455
11 700 0.6169478 0.6638530 0.4939270
11 750 0.6169477 0.6639677 0.4938949
11 800 0.6168482 0.6642002 0.4938753
11 850 0.6165542 0.6645881 0.4936700
11 900 0.6165215 0.6646528 0.4936491
11 950 0.6163687 0.6648255 0.4934883
11 1000 0.6162659 0.6650314 0.4934198
11 1050 0.6162582 0.6650770 0.4934503
11 1100 0.6162748 0.6650668 0.4934737
11 1150 0.6163296 0.6650689 0.4935144
11 1200 0.6162594 0.6651298 0.4934497
11 1250 0.6161522 0.6652572 0.4933190
12 50 0.6048908 0.6585029 0.4906486
12 100 0.5860820 0.6835540 0.4776855
12 150 0.5737430 0.6976209 0.4673844
12 200 0.5700714 0.7014910 0.4662272
12 250 0.5678868 0.7036630 0.4616102
12 300 0.5655462 0.7058906 0.4601785
12 350 0.5640256 0.7075817 0.4588761
12 400 0.5630097 0.7087578 0.4587631
12 450 0.5625244 0.7091479 0.4588339
12 500 0.5622287 0.7094033 0.4588054
12 550 0.5624524 0.7090957 0.4590912
12 600 0.5625164 0.7089843 0.4589437
12 650 0.5626884 0.7087097 0.4592683
12 700 0.5623187 0.7090307 0.4590719
12 750 0.5625504 0.7088823 0.4593614
12 800 0.5625949 0.7088568 0.4593220
12 850 0.5624321 0.7089688 0.4592323
12 900 0.5624833 0.7088327 0.4592455
12 950 0.5624998 0.7088526 0.4591693
12 1000 0.5624437 0.7089120 0.4591216
12 1050 0.5623968 0.7089468 0.4590643
12 1100 0.5624248 0.7089276 0.4591150
12 1150 0.5624357 0.7089285 0.4591230
12 1200 0.5624462 0.7089336 0.4591594
12 1250 0.5623973 0.7089670 0.4591342
13 50 0.6249201 0.6589457 0.4916246
13 100 0.6060384 0.6755682 0.4820170
13 150 0.5993754 0.6880409 0.4780219
13 200 0.5963328 0.6898661 0.4749589
13 250 0.5952598 0.6901034 0.4747026
13 300 0.5933261 0.6920355 0.4730187
13 350 0.5941051 0.6909369 0.4731635
13 400 0.5928677 0.6924350 0.4717661
13 450 0.5914826 0.6938854 0.4708669
13 500 0.5912468 0.6943130 0.4709682
13 550 0.5912502 0.6943816 0.4715070
13 600 0.5910123 0.6946883 0.4711042
13 650 0.5905004 0.6950990 0.4709002
13 700 0.5906997 0.6949771 0.4710327
13 750 0.5906629 0.6949677 0.4710439
13 800 0.5907453 0.6949113 0.4711452
13 850 0.5905046 0.6952379 0.4709634
13 900 0.5905219 0.6951963 0.4709771
13 950 0.5904680 0.6952353 0.4709727
13 1000 0.5903979 0.6953660 0.4708475
13 1050 0.5903617 0.6954056 0.4708748
13 1100 0.5902515 0.6955262 0.4707789
13 1150 0.5903149 0.6954558 0.4709172
13 1200 0.5903033 0.6954508 0.4709179
13 1250 0.5903194 0.6954435 0.4709218
14 50 0.6347983 0.6353294 0.5034317
14 100 0.6276955 0.6467263 0.5053487
14 150 0.6184020 0.6578093 0.4958389
14 200 0.6129389 0.6662981 0.4906523
14 250 0.6096112 0.6705528 0.4902822
14 300 0.6047045 0.6744077 0.4859390
14 350 0.6052626 0.6745964 0.4868171
14 400 0.6037003 0.6762062 0.4852204
14 450 0.6024292 0.6779640 0.4842230
14 500 0.6017230 0.6787579 0.4833910
14 550 0.6010933 0.6795197 0.4828767
14 600 0.6008374 0.6795992 0.4823814
14 650 0.6004812 0.6800440 0.4821228
14 700 0.6000079 0.6804568 0.4816286
14 750 0.6000086 0.6806414 0.4815093
14 800 0.5996918 0.6809222 0.4811060
14 850 0.5998264 0.6807545 0.4811843
14 900 0.5996994 0.6808051 0.4811404
14 950 0.5995097 0.6810433 0.4809205
14 1000 0.5995453 0.6809732 0.4809339
14 1050 0.5995959 0.6809562 0.4809036
14 1100 0.5996474 0.6809134 0.4809308
14 1150 0.5996611 0.6809417 0.4809555
14 1200 0.5996483 0.6809297 0.4808988
14 1250 0.5997062 0.6809293 0.4809025
15 50 0.6334989 0.6398025 0.5019600
15 100 0.6183567 0.6601172 0.4917846
15 150 0.6119174 0.6653111 0.4883722
15 200 0.6090616 0.6679938 0.4874494
15 250 0.6041218 0.6736493 0.4832738
15 300 0.6032696 0.6736591 0.4813161
15 350 0.6005726 0.6768803 0.4790594
15 400 0.5995502 0.6776221 0.4778072
15 450 0.5978759 0.6793054 0.4764452
15 500 0.5966600 0.6805415 0.4754431
15 550 0.5960117 0.6811270 0.4749141
15 600 0.5958386 0.6810520 0.4742842
15 650 0.5958464 0.6811185 0.4743725
15 700 0.5955530 0.6815333 0.4738812
15 750 0.5952982 0.6817782 0.4735658
15 800 0.5950617 0.6819927 0.4734036
15 850 0.5948488 0.6821384 0.4732322
15 900 0.5946668 0.6822252 0.4730407
15 950 0.5945322 0.6823487 0.4729210
15 1000 0.5944900 0.6823850 0.4728742
15 1050 0.5944988 0.6823720 0.4729512
15 1100 0.5945059 0.6822678 0.4729629
15 1150 0.5943716 0.6823876 0.4728521
15 1200 0.5942714 0.6824924 0.4727484
15 1250 0.5942612 0.6824801 0.4727550
16 50 0.6341264 0.6298137 0.4953612
16 100 0.6264258 0.6404451 0.4975088
16 150 0.6211893 0.6487324 0.4987269
16 200 0.6158703 0.6534762 0.4951865
16 250 0.6140993 0.6551248 0.4932260
16 300 0.6101390 0.6597874 0.4911832
16 350 0.6054016 0.6650097 0.4873227
16 400 0.6043623 0.6664530 0.4871212
16 450 0.6037322 0.6672832 0.4874287
16 500 0.6027434 0.6687552 0.4869702
16 550 0.6016288 0.6701513 0.4862017
16 600 0.6013574 0.6704936 0.4860468
16 650 0.6006951 0.6710867 0.4857729
16 700 0.6005253 0.6712453 0.4857093
16 750 0.6003603 0.6714311 0.4856229
16 800 0.6002723 0.6716529 0.4854878
16 850 0.6001507 0.6718075 0.4853516
16 900 0.6001999 0.6717537 0.4854869
16 950 0.5999359 0.6721320 0.4852241
16 1000 0.5999109 0.6721929 0.4851952
16 1050 0.5998986 0.6721938 0.4852673
16 1100 0.5997596 0.6723883 0.4851269
16 1150 0.5997532 0.6724062 0.4851968
16 1200 0.5997736 0.6723922 0.4852098
16 1250 0.5997938 0.6723876 0.4851800
17 50 0.6442916 0.6264271 0.5066966
17 100 0.6240671 0.6419534 0.4943208
17 150 0.6143866 0.6522959 0.4891908
17 200 0.6075846 0.6586134 0.4804249
17 250 0.6007850 0.6663239 0.4770740
17 300 0.6003304 0.6680938 0.4769595
17 350 0.6004983 0.6681801 0.4766131
17 400 0.5985421 0.6700299 0.4749971
17 450 0.5984537 0.6702797 0.4747411
17 500 0.5986463 0.6700816 0.4747634
17 550 0.5987854 0.6697136 0.4751464
17 600 0.5987157 0.6699357 0.4747918
17 650 0.5985623 0.6700879 0.4746007
17 700 0.5982515 0.6704929 0.4747104
17 750 0.5982305 0.6705993 0.4748839
17 800 0.5982734 0.6706456 0.4750269
17 850 0.5982242 0.6706444 0.4751951
17 900 0.5981319 0.6708300 0.4750770
17 950 0.5982675 0.6707568 0.4752553
17 1000 0.5983488 0.6706174 0.4753381
17 1050 0.5983675 0.6706184 0.4753914
17 1100 0.5983412 0.6706540 0.4754394
17 1150 0.5983349 0.6706973 0.4754244
17 1200 0.5983228 0.6707162 0.4754384
17 1250 0.5983436 0.6707130 0.4755035
18 50 0.6346912 0.6459982 0.5070992
18 100 0.6262617 0.6511365 0.5049423
18 150 0.6170782 0.6595619 0.4965010
18 200 0.6166726 0.6603157 0.4919010
18 250 0.6141941 0.6630532 0.4879261
18 300 0.6114540 0.6663258 0.4854699
18 350 0.6098374 0.6679587 0.4840043
18 400 0.6086503 0.6692595 0.4820774
18 450 0.6089632 0.6690399 0.4829857
18 500 0.6081624 0.6702137 0.4826138
18 550 0.6074018 0.6709710 0.4818718
18 600 0.6072580 0.6712524 0.4815755
18 650 0.6071340 0.6712755 0.4815474
18 700 0.6069835 0.6715392 0.4814161
18 750 0.6069921 0.6715682 0.4813269
18 800 0.6067890 0.6718735 0.4810730
18 850 0.6067163 0.6718859 0.4810644
18 900 0.6066135 0.6720901 0.4809139
18 950 0.6065848 0.6721448 0.4808207
18 1000 0.6065183 0.6722390 0.4807317
18 1050 0.6064096 0.6723855 0.4806788
18 1100 0.6064004 0.6723816 0.4806198
18 1150 0.6063118 0.6724796 0.4805625
18 1200 0.6063735 0.6724067 0.4806335
18 1250 0.6063779 0.6723796 0.4806562
19 50 0.6360828 0.6353009 0.5056253
19 100 0.6306396 0.6388294 0.5089426
19 150 0.6263851 0.6464934 0.5050485
19 200 0.6216174 0.6530169 0.5021487
19 250 0.6202153 0.6557822 0.5007263
19 300 0.6179185 0.6576042 0.4986360
19 350 0.6162402 0.6593836 0.4975794
19 400 0.6162662 0.6593665 0.4973089
19 450 0.6159412 0.6596908 0.4970661
19 500 0.6160872 0.6595771 0.4968170
19 550 0.6160878 0.6596952 0.4965303
19 600 0.6162729 0.6594065 0.4967175
19 650 0.6162805 0.6594616 0.4966188
19 700 0.6163562 0.6594630 0.4965910
19 750 0.6164022 0.6595201 0.4967129
19 800 0.6165677 0.6593908 0.4968284
19 850 0.6167499 0.6592129 0.4968568
19 900 0.6167636 0.6591735 0.4969080
19 950 0.6166956 0.6592535 0.4968653
19 1000 0.6166686 0.6592361 0.4968029
19 1050 0.6167415 0.6591964 0.4968396
19 1100 0.6167587 0.6591786 0.4968567
19 1150 0.6167290 0.6591526 0.4968421
19 1200 0.6168067 0.6590631 0.4969281
19 1250 0.6168006 0.6590914 0.4968860
20 50 0.6393043 0.6272292 0.5064711
20 100 0.6158031 0.6561712 0.4806579
20 150 0.6053851 0.6695768 0.4782662
20 200 0.6016993 0.6752503 0.4760467
20 250 0.5991123 0.6789517 0.4733683
20 300 0.5986646 0.6791118 0.4736841
20 350 0.5975858 0.6809013 0.4742335
20 400 0.5967021 0.6821341 0.4735786
20 450 0.5965369 0.6826920 0.4735626
20 500 0.5966341 0.6828534 0.4736492
20 550 0.5959364 0.6836607 0.4733691
20 600 0.5956338 0.6840719 0.4731504
20 650 0.5956502 0.6842783 0.4733594
20 700 0.5952032 0.6847271 0.4732594
20 750 0.5952618 0.6848024 0.4736700
20 800 0.5952497 0.6849535 0.4738530
20 850 0.5951275 0.6851381 0.4737484
20 900 0.5950545 0.6853542 0.4739273
20 950 0.5950634 0.6854079 0.4740176
20 1000 0.5949297 0.6855809 0.4739961
20 1050 0.5949060 0.6856514 0.4740281
20 1100 0.5949161 0.6856707 0.4741613
20 1150 0.5947659 0.6858667 0.4740931
20 1200 0.5947133 0.6858932 0.4741255
20 1250 0.5946917 0.6859690 0.4741194
21 50 0.6298545 0.6338366 0.4987605
21 100 0.6121012 0.6494822 0.4874314
21 150 0.6028798 0.6593412 0.4787767
21 200 0.5966352 0.6667617 0.4708064
21 250 0.5945292 0.6689641 0.4684073
21 300 0.5892673 0.6732715 0.4631385
21 350 0.5876096 0.6742193 0.4614882
21 400 0.5864157 0.6750828 0.4606527
21 450 0.5847880 0.6769216 0.4589279
21 500 0.5837196 0.6778528 0.4580316
21 550 0.5836931 0.6776391 0.4576426
21 600 0.5836239 0.6776203 0.4575351
21 650 0.5829795 0.6782581 0.4572626
21 700 0.5828355 0.6783064 0.4571530
21 750 0.5828693 0.6781767 0.4570992
21 800 0.5828607 0.6781033 0.4569551
21 850 0.5829106 0.6779084 0.4569551
21 900 0.5828121 0.6780165 0.4568384
21 950 0.5828453 0.6779945 0.4568553
21 1000 0.5828341 0.6779221 0.4568378
21 1050 0.5829463 0.6779000 0.4569351
21 1100 0.5829753 0.6778201 0.4568837
21 1150 0.5829802 0.6777929 0.4568698
21 1200 0.5830422 0.6776920 0.4569362
21 1250 0.5830529 0.6776730 0.4569451
22 50 0.6271314 0.6567355 0.4932729
22 100 0.6130760 0.6683818 0.4824777
22 150 0.6131171 0.6649369 0.4802090
22 200 0.6042633 0.6755230 0.4743092
22 250 0.6054203 0.6735601 0.4762352
22 300 0.6023080 0.6764959 0.4736372
22 350 0.6015929 0.6773458 0.4737798
22 400 0.5995664 0.6798302 0.4718273
22 450 0.5988794 0.6805564 0.4715995
22 500 0.5987417 0.6808740 0.4707719
22 550 0.5987427 0.6809643 0.4707380
22 600 0.5984274 0.6811878 0.4708386
22 650 0.5984516 0.6813670 0.4708429
22 700 0.5983326 0.6813556 0.4708446
22 750 0.5982757 0.6813538 0.4707725
22 800 0.5981261 0.6816242 0.4706584
22 850 0.5981609 0.6815221 0.4706136
22 900 0.5981456 0.6815979 0.4706972
22 950 0.5982970 0.6813996 0.4708754
22 1000 0.5983644 0.6813821 0.4708916
22 1050 0.5984854 0.6812789 0.4710374
22 1100 0.5984718 0.6812893 0.4709554
22 1150 0.5985354 0.6812815 0.4710393
22 1200 0.5984932 0.6813695 0.4710443
22 1250 0.5984825 0.6813924 0.4710457
23 50 0.6062560 0.6657945 0.4765715
23 100 0.5939231 0.6747011 0.4774864
23 150 0.5845075 0.6831029 0.4681894
23 200 0.5793742 0.6915965 0.4638564
23 250 0.5760582 0.6963723 0.4611340
23 300 0.5738503 0.6993406 0.4586839
23 350 0.5714230 0.7023866 0.4568008
23 400 0.5710127 0.7029175 0.4568520
23 450 0.5702284 0.7038325 0.4562015
23 500 0.5686264 0.7054561 0.4553975
23 550 0.5682258 0.7057861 0.4548842
23 600 0.5679343 0.7063373 0.4545522
23 650 0.5675263 0.7068088 0.4542851
23 700 0.5672464 0.7072373 0.4539121
23 750 0.5674613 0.7069840 0.4540855
23 800 0.5675125 0.7069848 0.4539891
23 850 0.5674647 0.7070624 0.4540194
23 900 0.5672508 0.7073170 0.4538355
23 950 0.5671983 0.7073773 0.4538131
23 1000 0.5671703 0.7074044 0.4538347
23 1050 0.5671797 0.7073933 0.4538198
23 1100 0.5670895 0.7075594 0.4537901
23 1150 0.5670894 0.7075626 0.4537859
23 1200 0.5670406 0.7076199 0.4537402
23 1250 0.5670751 0.7075815 0.4538126
24 50 0.6322423 0.6501994 0.4927172
24 100 0.6325136 0.6560010 0.4961598
24 150 0.6265079 0.6656859 0.4930201
24 200 0.6217863 0.6720372 0.4889889
24 250 0.6180777 0.6746360 0.4848876
24 300 0.6145026 0.6788581 0.4812016
24 350 0.6137681 0.6795356 0.4801706
24 400 0.6128551 0.6813281 0.4791128
24 450 0.6128902 0.6818506 0.4791415
24 500 0.6116495 0.6832734 0.4776770
24 550 0.6114982 0.6833382 0.4773126
24 600 0.6114923 0.6835472 0.4773332
24 650 0.6114284 0.6835031 0.4774549
24 700 0.6108634 0.6841272 0.4765968
24 750 0.6109060 0.6840993 0.4766371
24 800 0.6107687 0.6843034 0.4764386
24 850 0.6106720 0.6845979 0.4763630
24 900 0.6107301 0.6845271 0.4764282
24 950 0.6106173 0.6846485 0.4763145
24 1000 0.6105554 0.6847416 0.4762231
24 1050 0.6104846 0.6848681 0.4761405
24 1100 0.6103888 0.6849223 0.4760634
24 1150 0.6103790 0.6849478 0.4760139
24 1200 0.6103422 0.6849618 0.4759261
24 1250 0.6103311 0.6850492 0.4759291
25 50 0.5922564 0.6892464 0.4738175
25 100 0.5794845 0.6973421 0.4677167
25 150 0.5801734 0.6975618 0.4705391
25 200 0.5739089 0.7021855 0.4655251
25 250 0.5692089 0.7078993 0.4613880
25 300 0.5654349 0.7124267 0.4586893
25 350 0.5634377 0.7143808 0.4573831
25 400 0.5624128 0.7158583 0.4559508
25 450 0.5619968 0.7163074 0.4563456
25 500 0.5618370 0.7166728 0.4561748
25 550 0.5614977 0.7172825 0.4561541
25 600 0.5613529 0.7175601 0.4562714
25 650 0.5608309 0.7179985 0.4563884
25 700 0.5608731 0.7182834 0.4564952
25 750 0.5606044 0.7187293 0.4564090
25 800 0.5605476 0.7187883 0.4564591
25 850 0.5605270 0.7188082 0.4564905
25 900 0.5605337 0.7188219 0.4565267
25 950 0.5606569 0.7187305 0.4566571
25 1000 0.5605178 0.7189074 0.4565679
25 1050 0.5606036 0.7188923 0.4566889
25 1100 0.5606642 0.7187989 0.4567645
25 1150 0.5605386 0.7189424 0.4566585
25 1200 0.5606308 0.7188560 0.4567531
25 1250 0.5606716 0.7188334 0.4568109
Tuning parameter 'shrinkage' was held constant at a value of 0.1
Tuning parameter 'n.minobsinnode' was held constant at a value of 10
RMSE was used to select the optimal model using the smallest value.
The final values used for the model were n.trees = 1000, interaction.depth
= 25, shrinkage = 0.1 and n.minobsinnode = 10.Code
set.seed(50)
rf <- train(Yield ~ ., data = train_df, method = "ranger",
scale = TRUE,
trControl = trainControl("cv", number = 10),
tuneLength = 25)
rf_predictions <- predict(rf, test_df)
rf_in_sample <- merge(rf$results, rf$bestTune)
results <- data.frame(t(postResample(pred = rf_predictions, obs = test_df$Yield))) %>%
mutate("In Sample RMSE" = rf_in_sample$RMSE,
"In Sample Rsquared" = rf_in_sample$Rsquared,
"In Sample MAE" = rf_in_sample$MAE,
"Model"= "Random Forest") %>%
rbind(results)
rfRandom Forest
144 samples
56 predictor
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 129, 129, 130, 129, 131, 131, ...
Resampling results across tuning parameters:
mtry splitrule RMSE Rsquared MAE
2 variance 0.6836602 0.6603966 0.5638955
2 extratrees 0.7404371 0.6283589 0.6147323
4 variance 0.6470821 0.6894120 0.5254556
4 extratrees 0.6940089 0.6660007 0.5703897
6 variance 0.6328935 0.6981419 0.5066383
6 extratrees 0.6701319 0.6848446 0.5490363
8 variance 0.6181224 0.7037927 0.4922997
8 extratrees 0.6604988 0.6895982 0.5424348
11 variance 0.6181320 0.6983780 0.4920936
11 extratrees 0.6443363 0.7070844 0.5249113
13 variance 0.6145744 0.6970122 0.4896856
13 extratrees 0.6419789 0.6970493 0.5235445
15 variance 0.6146263 0.6939432 0.4889142
15 extratrees 0.6415920 0.6949650 0.5229583
17 variance 0.6124678 0.6946846 0.4868622
17 extratrees 0.6329865 0.7044442 0.5153290
20 variance 0.6093918 0.6922849 0.4807739
20 extratrees 0.6289689 0.7064541 0.5099941
22 variance 0.6130025 0.6870915 0.4826229
22 extratrees 0.6278733 0.7086465 0.5092308
24 variance 0.6073355 0.6945201 0.4786866
24 extratrees 0.6326962 0.6961471 0.5111293
26 variance 0.6127645 0.6860286 0.4797881
26 extratrees 0.6252118 0.7052241 0.5041821
29 variance 0.6058078 0.6955684 0.4754294
29 extratrees 0.6295666 0.6947297 0.5103248
31 variance 0.6174034 0.6778964 0.4827147
31 extratrees 0.6206464 0.7074282 0.4996916
33 variance 0.6117603 0.6845685 0.4789453
33 extratrees 0.6212273 0.7040009 0.4976389
35 variance 0.6122260 0.6862440 0.4787326
35 extratrees 0.6206798 0.7043025 0.4998583
38 variance 0.6116932 0.6805848 0.4761453
38 extratrees 0.6173992 0.7053132 0.4966423
40 variance 0.6160921 0.6770483 0.4795322
40 extratrees 0.6310077 0.6823063 0.5038169
42 variance 0.6189327 0.6731705 0.4822206
42 extratrees 0.6208384 0.6972107 0.4983777
44 variance 0.6163102 0.6744757 0.4781859
44 extratrees 0.6238461 0.6949696 0.4963412
47 variance 0.6175443 0.6729756 0.4775111
47 extratrees 0.6256555 0.6896276 0.5025891
49 variance 0.6201127 0.6689257 0.4784543
49 extratrees 0.6153936 0.7025415 0.4952185
51 variance 0.6192791 0.6676023 0.4831584
51 extratrees 0.6220058 0.6971223 0.4978156
53 variance 0.6205873 0.6681147 0.4791386
53 extratrees 0.6211123 0.6972752 0.4960795
56 variance 0.6247335 0.6615175 0.4835692
56 extratrees 0.6187092 0.6960054 0.4959025
Tuning parameter 'min.node.size' was held constant at a value of 5
RMSE was used to select the optimal model using the smallest value.
The final values used for the model were mtry = 29, splitrule = variance
and min.node.size = 5.Code
set.seed(50)
crf <- train(Yield ~ ., data = train_df, method = "cforest",
trControl = trainControl("cv", number = 10),
tuneLength = 25)
crf_predictions <- predict(crf, test_df)
crf_in_sample <- merge(crf$results, crf$bestTune)
results <- data.frame(t(postResample(pred = crf_predictions, obs = test_df$Yield))) %>%
mutate("In Sample RMSE" = crf_in_sample$RMSE,
"In Sample Rsquared" = crf_in_sample$Rsquared,
"In Sample MAE" = crf_in_sample$MAE,
"Model"= "Conditional Random Forest") %>%
rbind(results)
crfConditional Inference Random Forest
144 samples
56 predictor
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 129, 129, 130, 129, 131, 131, ...
Resampling results across tuning parameters:
mtry RMSE Rsquared MAE
2 0.8175879 0.5548598 0.6728052
4 0.7296286 0.6082621 0.5943835
6 0.6984359 0.6236242 0.5653711
8 0.6853913 0.6261895 0.5498512
11 0.6747256 0.6304921 0.5364053
13 0.6687738 0.6319654 0.5291691
15 0.6688481 0.6308462 0.5280941
17 0.6667166 0.6224298 0.5305037
20 0.6664660 0.6215283 0.5279802
22 0.6673760 0.6170557 0.5273973
24 0.6683276 0.6153035 0.5281188
26 0.6671232 0.6138167 0.5268977
29 0.6652622 0.6144598 0.5241412
31 0.6718634 0.6041957 0.5292660
33 0.6735270 0.6008782 0.5292595
35 0.6751449 0.5987309 0.5292281
38 0.6747555 0.5972028 0.5280295
40 0.6773334 0.5917736 0.5329994
42 0.6753576 0.5968210 0.5302687
44 0.6772375 0.5934633 0.5314206
47 0.6787163 0.5889502 0.5345653
49 0.6841916 0.5834709 0.5372852
51 0.6814431 0.5850231 0.5373296
53 0.6829259 0.5821680 0.5401076
56 0.6813225 0.5862141 0.5373336
RMSE was used to select the optimal model using the smallest value.
The final value used for the model was mtry = 29.Code
results %>%
arrange(RMSE) %>%
kable() %>%
kable_styling()| RMSE | Rsquared | MAE | In Sample RMSE | In Sample Rsquared | In Sample MAE | Model |
|---|---|---|---|---|---|---|
| 0.4473759 | 0.7327923 | 0.3833728 | 0.6058078 | 0.6955684 | 0.4754294 | Random Forest |
| 0.5128044 | 0.6447513 | 0.4313862 | 0.6652622 | 0.6144598 | 0.5241412 | Conditional Random Forest |
| 0.5658495 | 0.6353186 | 0.4775814 | 0.5605178 | 0.7189074 | 0.4565679 | Boosted Tree |
| 0.5720979 | 0.5545634 | 0.4497440 | 0.7288567 | 0.5222249 | 0.5902326 | Bagged Tree |
| 0.9530747 | 0.4514444 | 0.6910022 | 0.6527136 | 0.5979023 | 0.5296489 | PLS |
- Which predictors are most important in the optimal tree-based regression model? Do either the biological or process variables dominate the list?
How do the top 10 important predictors compare to the top 10 predictors from the optimal linear and nonlinear models?
For the PLS linear model , there are 5 manufacturing process variables that are highly important. However, for the tree models, it seems to be a mix of both manufacturing and a few biological variables that are considered important on the first 5 set of variables.
Code
varImp(bag_model)loess r-squared variable importance
only 20 most important variables shown (out of 56)
Overall
ManufacturingProcess32 100.00
ManufacturingProcess13 83.85
BiologicalMaterial06 83.27
BiologicalMaterial12 77.07
ManufacturingProcess17 73.83
ManufacturingProcess36 72.58
BiologicalMaterial03 71.24
ManufacturingProcess09 70.86
ManufacturingProcess31 61.18
ManufacturingProcess06 60.11
BiologicalMaterial02 58.31
BiologicalMaterial11 54.80
ManufacturingProcess11 47.66
BiologicalMaterial08 46.48
ManufacturingProcess33 44.51
BiologicalMaterial04 41.36
ManufacturingProcess29 41.32
BiologicalMaterial09 36.86
ManufacturingProcess12 34.10
BiologicalMaterial01 33.86Code
varImp(gbm1)gbm variable importance
only 20 most important variables shown (out of 56)
Overall
ManufacturingProcess32 100.000
ManufacturingProcess06 26.984
BiologicalMaterial12 18.439
BiologicalMaterial03 17.535
ManufacturingProcess17 16.750
ManufacturingProcess09 12.475
BiologicalMaterial05 10.668
BiologicalMaterial11 9.154
ManufacturingProcess13 7.866
ManufacturingProcess16 6.846
ManufacturingProcess31 6.785
ManufacturingProcess04 6.651
ManufacturingProcess15 6.570
BiologicalMaterial06 6.555
ManufacturingProcess26 6.503
ManufacturingProcess37 5.675
BiologicalMaterial09 5.595
ManufacturingProcess22 5.572
ManufacturingProcess01 5.349
ManufacturingProcess43 5.265Code
varImp(pls_model)pls variable importance
only 20 most important variables shown (out of 56)
Overall
ManufacturingProcess32 100.00
ManufacturingProcess36 81.33
ManufacturingProcess13 80.83
ManufacturingProcess09 76.85
ManufacturingProcess17 73.30
ManufacturingProcess06 69.05
BiologicalMaterial02 58.00
BiologicalMaterial08 57.85
ManufacturingProcess33 57.38
BiologicalMaterial06 55.70
BiologicalMaterial12 52.46
ManufacturingProcess12 52.04
BiologicalMaterial11 51.92
BiologicalMaterial03 50.19
BiologicalMaterial01 49.09
BiologicalMaterial04 48.43
ManufacturingProcess11 48.00
ManufacturingProcess34 43.23
ManufacturingProcess28 40.62
ManufacturingProcess04 39.57- Plot the optimal single tree with the distribution of yield in the terminal nodes.
Does this view of the data provide additional knowledge about the biological or process predictors and their relationship with yield?
ANSWER: yes, the rules on the decision tree include both the biological and process predictors. The value determined as the top of the tree to split on is Manuprocess32. Once the samples traverse the tree, the terminal nodes give us the final outcome of the value of the target variable or yield. For instance at terminal node 4, we have a yield of -0.97.
Code
set.seed(100)
ctrl = rpart.control(maxdepth=7)
cart_model <- train(Yield ~ ., data = train_df, method = "rpart",control=ctrl)
#,
# trControl = trainControl("cv", number = 10),
# tuneLength = 25)
cart_model$finalModeln= 144
node), split, n, deviance, yval
* denotes terminal node
1) root 144 151.43780 -0.008017689
2) ManufacturingProcess32< 0.191596 83 42.82953 -0.565716800
4) BiologicalMaterial11< -0.3896263 41 13.66912 -0.974040400 *
5) BiologicalMaterial11>=-0.3896263 42 15.65146 -0.167115200 *
3) ManufacturingProcess32>=0.191596 61 47.66711 0.750818800 *Code
fancyRpartPlot(cart_model$finalModel)