Lab 2
Penalized Regression
Assigned 4/21/20, Due 4/28/20
Read in the train.csv data. Please feel free to use sample_frac() if you find that the data file is too large for your machine.
data <- import(path(git, "data/train.csv")) %>%
sample_frac(.2) %>%
mutate(ncessch = as.factor(ncessch),
enrl_grd = as.factor(enrl_grd))1. Initial Split
Set a seed and split the data into a training set and a testing set as two named objects.
set.seed(3000)
data_split <- initial_split(data)
data_train <- training(data_split)
data_test <- testing(data_split)
rm(data)2. Resample
Set a seed and use 10-fold cross-validation to resample the traning data.
set.seed(3000)
(cv_splits <- vfold_cv(data_train))## # 10-fold cross-validation
## # A tibble: 10 x 2
## splits id
## <list> <chr>
## 1 <split [25.6K/2.8K]> Fold01
## 2 <split [25.6K/2.8K]> Fold02
## 3 <split [25.6K/2.8K]> Fold03
## 4 <split [25.6K/2.8K]> Fold04
## 5 <split [25.6K/2.8K]> Fold05
## 6 <split [25.6K/2.8K]> Fold06
## 7 <split [25.6K/2.8K]> Fold07
## 8 <split [25.6K/2.8K]> Fold08
## 9 <split [25.6K/2.8K]> Fold09
## 10 <split [25.6K/2.8K]> Fold103. Preprocess
Complete the code maze below by filling in the blanks (____) to create a recipe object that includes: * a formula model with score predicted by 4 predictors * be sure there are no missing data in your predictors (try step_naomit()) * center and scale all numeric predictors * dummy code all nominal predictors
lasso4_rec <-
recipe(
formula = score ~ ethnic_cd + enrl_grd + econ_dsvntg + ncessch,
data = data_train #use your training set here
) %>%
step_naomit(everything(), skip = TRUE) %>%
step_string2factor(ethnic_cd, econ_dsvntg) %>% #may not need, depending on your formula
step_dummy(ethnic_cd, enrl_grd, econ_dsvntg, ncessch) #may not need, depending on your formula
# step_normalize(ethnic_cd, enrl_grd, econ_dsvntg, ncessch) #may not need, depending on your formula4. Parsnip model
Create a {parsnip} lasso model where the penalty hyperparameter is set to be tuned.
mod_lasso <- linear_reg() %>%
set_engine("glmnet") %>%
set_mode("regression") %>%
set_args(penalty = tune(),
mixture = 1)5. Fit a tuned lasso model
Complete the code maze below to fit a tuned lasso model.
lasso_grid <- grid_regular(penalty(), levels = 10)
lasso4_fit_1 <- tune_grid(
mod_lasso,
preprocessor = lasso4_rec,
resamples = cv_splits,
grid = lasso_grid,
control = tune::control_resamples(verbose = TRUE,
save_pred = TRUE)
)## i Fold01: recipe## ✓ Fold01: recipe## i Fold01: model 1/1## ✓ Fold01: model 1/1## i Fold01: model 1/1 (predictions)## ! Fold01: model 1/1 (predictions): There are new levels in a factor: NA## i Fold02: recipe## ✓ Fold02: recipe## i Fold02: model 1/1## ✓ Fold02: model 1/1## i Fold02: model 1/1 (predictions)## ! Fold02: model 1/1 (predictions): There are new levels in a factor: NA## i Fold03: recipe## ✓ Fold03: recipe## i Fold03: model 1/1## ✓ Fold03: model 1/1## i Fold03: model 1/1 (predictions)## ! Fold03: model 1/1 (predictions): There are new levels in a factor: NA## i Fold04: recipe## ✓ Fold04: recipe## i Fold04: model 1/1## ✓ Fold04: model 1/1## i Fold04: model 1/1 (predictions)## ! Fold04: model 1/1 (predictions): There are new levels in a factor: NA## i Fold05: recipe## ✓ Fold05: recipe## i Fold05: model 1/1## ✓ Fold05: model 1/1## i Fold05: model 1/1 (predictions)## ! Fold05: model 1/1 (predictions): There are new levels in a factor: NA## i Fold06: recipe## ✓ Fold06: recipe## i Fold06: model 1/1## ✓ Fold06: model 1/1## i Fold06: model 1/1 (predictions)## ! Fold06: model 1/1 (predictions): There are new levels in a factor: NA## i Fold07: recipe## ✓ Fold07: recipe## i Fold07: model 1/1## ✓ Fold07: model 1/1## i Fold07: model 1/1 (predictions)## ! Fold07: model 1/1 (predictions): There are new levels in a factor: NA## i Fold08: recipe## ✓ Fold08: recipe## i Fold08: model 1/1## ✓ Fold08: model 1/1## i Fold08: model 1/1 (predictions)## ! Fold08: model 1/1 (predictions): There are new levels in a factor: NA## i Fold09: recipe## ✓ Fold09: recipe## i Fold09: model 1/1## ✓ Fold09: model 1/1## i Fold09: model 1/1 (predictions)## ! Fold09: model 1/1 (predictions): There are new levels in a factor: NA## i Fold10: recipe## ✓ Fold10: recipe## i Fold10: model 1/1## ✓ Fold10: model 1/1## i Fold10: model 1/1 (predictions)## ! Fold10: model 1/1 (predictions): There are new levels in a factor: NAQuestion A
- How many models were fit to each fold of
lasso4_fit_1? (Please provide a numeric answer, and use code to corroborate your answer.)
# 10 models per fold
lasso4_fit_1 %>%
collect_metrics(summarize = FALSE) %>%
select(id, penalty, .config)## # A tibble: 200 x 3
## id penalty .config
## <chr> <dbl> <chr>
## 1 Fold01 0.0000000001 Model01
## 2 Fold01 0.00000000129 Model02
## 3 Fold01 0.0000000167 Model03
## 4 Fold01 0.000000215 Model04
## 5 Fold01 0.00000278 Model05
## 6 Fold01 0.0000359 Model06
## 7 Fold01 0.000464 Model07
## 8 Fold01 0.00599 Model08
## 9 Fold01 0.0774 Model09
## 10 Fold01 1 Model10
## # … with 190 more rows- Use code to list the different values of
penalty()that were used.
lasso4_fit_1 %>%
collect_metrics(summarize = FALSE) %>%
filter(.metric == "rmse") %>%
select(penalty, .config)## # A tibble: 100 x 2
## penalty .config
## <dbl> <chr>
## 1 0.0000000001 Model01
## 2 0.00000000129 Model02
## 3 0.0000000167 Model03
## 4 0.000000215 Model04
## 5 0.00000278 Model05
## 6 0.0000359 Model06
## 7 0.000464 Model07
## 8 0.00599 Model08
## 9 0.0774 Model09
## 10 1 Model10
## # … with 90 more rows6. Fit another tuned lasso model
Use your code from (5) above to complete the code maze below to fit a second tuned lasso model, using the same parsnip model, recipe, and resampled object you used before.
lasso4_fit_2 <- tune_grid(
mod_lasso,
preprocessor = lasso4_rec,
resamples = cv_splits,
control = tune::control_resamples(verbose = TRUE,
save_pred = TRUE)
)## i Fold01: recipe## ✓ Fold01: recipe## i Fold01: model 1/1## ✓ Fold01: model 1/1## i Fold01: model 1/1 (predictions)## ! Fold01: model 1/1 (predictions): There are new levels in a factor: NA## i Fold02: recipe## ✓ Fold02: recipe## i Fold02: model 1/1## ✓ Fold02: model 1/1## i Fold02: model 1/1 (predictions)## ! Fold02: model 1/1 (predictions): There are new levels in a factor: NA## i Fold03: recipe## ✓ Fold03: recipe## i Fold03: model 1/1## ✓ Fold03: model 1/1## i Fold03: model 1/1 (predictions)## ! Fold03: model 1/1 (predictions): There are new levels in a factor: NA## i Fold04: recipe## ✓ Fold04: recipe## i Fold04: model 1/1## ✓ Fold04: model 1/1## i Fold04: model 1/1 (predictions)## ! Fold04: model 1/1 (predictions): There are new levels in a factor: NA## i Fold05: recipe## ✓ Fold05: recipe## i Fold05: model 1/1## ✓ Fold05: model 1/1## i Fold05: model 1/1 (predictions)## ! Fold05: model 1/1 (predictions): There are new levels in a factor: NA## i Fold06: recipe## ✓ Fold06: recipe## i Fold06: model 1/1## ✓ Fold06: model 1/1## i Fold06: model 1/1 (predictions)## ! Fold06: model 1/1 (predictions): There are new levels in a factor: NA## i Fold07: recipe## ✓ Fold07: recipe## i Fold07: model 1/1## ✓ Fold07: model 1/1## i Fold07: model 1/1 (predictions)## ! Fold07: model 1/1 (predictions): There are new levels in a factor: NA## i Fold08: recipe## ✓ Fold08: recipe## i Fold08: model 1/1## ✓ Fold08: model 1/1## i Fold08: model 1/1 (predictions)## ! Fold08: model 1/1 (predictions): There are new levels in a factor: NA## i Fold09: recipe## ✓ Fold09: recipe## i Fold09: model 1/1## ✓ Fold09: model 1/1## i Fold09: model 1/1 (predictions)## ! Fold09: model 1/1 (predictions): There are new levels in a factor: NA## i Fold10: recipe## ✓ Fold10: recipe## i Fold10: model 1/1## ✓ Fold10: model 1/1## i Fold10: model 1/1 (predictions)## ! Fold10: model 1/1 (predictions): There are new levels in a factor: NAQuestion B
- How many models were fit to each fold of
lasso4_fit_2? (Please provide a numeric answer, and use code to corroborate your answer.)
# 10 models per fold
lasso4_fit_2 %>%
collect_metrics(summarize = FALSE) %>%
filter(.metric == "rmse") %>%
select(id, penalty, .config)## # A tibble: 100 x 3
## id penalty .config
## <chr> <dbl> <chr>
## 1 Fold01 1.10e-10 Model01
## 2 Fold01 4.19e- 9 Model02
## 3 Fold01 1.36e- 8 Model03
## 4 Fold01 9.72e- 7 Model04
## 5 Fold01 2.64e- 6 Model05
## 6 Fold01 1.13e- 5 Model06
## 7 Fold01 5.95e- 4 Model07
## 8 Fold01 4.60e- 3 Model08
## 9 Fold01 7.87e- 2 Model09
## 10 Fold01 3.74e- 1 Model10
## # … with 90 more rows- If this is different than the number of models of
lasso4_fit_1, please explain why. # It is not different - Use code to list the different values of
penalty()that were used for lasso4_fit_2.
lasso4_fit_2 %>%
collect_metrics() %>%
filter(.metric == "rmse") %>%
select(penalty, .config)## # A tibble: 10 x 2
## penalty .config
## <dbl> <chr>
## 1 1.10e-10 Model01
## 2 4.19e- 9 Model02
## 3 1.36e- 8 Model03
## 4 9.72e- 7 Model04
## 5 2.64e- 6 Model05
## 6 1.13e- 5 Model06
## 7 5.95e- 4 Model07
## 8 4.60e- 3 Model08
## 9 7.87e- 2 Model09
## 10 3.74e- 1 Model107. Complete the necessary steps to create and fit a tuned lasso model that has seven or more predictors (use any tuning grid you like). Note that you will need to create a new recipe as well.
data_train <- data_train %>%
mutate(attnd_schl_inst_id = as.factor(attnd_schl_inst_id))
lasso7_rec <-
recipe(
formula = score ~ ncessch + ethnic_cd + sp_ed_fg + econ_dsvntg + lat + lon + enrl_grd,
data = data_train
) %>%
step_naomit(everything(), skip = TRUE) %>%
step_string2factor(ethnic_cd, sp_ed_fg, econ_dsvntg) %>%
step_dummy(ethnic_cd, sp_ed_fg, econ_dsvntg, ncessch, enrl_grd) %>%
step_interact(terms = ~ lat:lon)
lasso_grid <- grid_regular(penalty(), levels = 10)
lasso7_fit_1 <- tune_grid(
mod_lasso,
preprocessor = lasso7_rec,
resamples = cv_splits,
grid = lasso_grid,
control = tune::control_resamples(verbose = TRUE,
save_pred = TRUE)
)## i Fold01: recipe## ✓ Fold01: recipe## i Fold01: model 1/1## ✓ Fold01: model 1/1## i Fold01: model 1/1 (predictions)## ! Fold01: model 1/1 (predictions): There are new levels in a factor: NA## i Fold02: recipe## ✓ Fold02: recipe## i Fold02: model 1/1## ✓ Fold02: model 1/1## i Fold02: model 1/1 (predictions)## ! Fold02: model 1/1 (predictions): There are new levels in a factor: NA## i Fold03: recipe## ✓ Fold03: recipe## i Fold03: model 1/1## ✓ Fold03: model 1/1## i Fold03: model 1/1 (predictions)## ! Fold03: model 1/1 (predictions): There are new levels in a factor: NA## i Fold04: recipe## ✓ Fold04: recipe## i Fold04: model 1/1## ✓ Fold04: model 1/1## i Fold04: model 1/1 (predictions)## ! Fold04: model 1/1 (predictions): There are new levels in a factor: NA## i Fold05: recipe## ✓ Fold05: recipe## i Fold05: model 1/1## ✓ Fold05: model 1/1## i Fold05: model 1/1 (predictions)## ! Fold05: model 1/1 (predictions): There are new levels in a factor: NA## i Fold06: recipe## ✓ Fold06: recipe## i Fold06: model 1/1## ✓ Fold06: model 1/1## i Fold06: model 1/1 (predictions)## ! Fold06: model 1/1 (predictions): There are new levels in a factor: NA## i Fold07: recipe## ✓ Fold07: recipe## i Fold07: model 1/1## ✓ Fold07: model 1/1## i Fold07: model 1/1 (predictions)## ! Fold07: model 1/1 (predictions): There are new levels in a factor: NA## i Fold08: recipe## ✓ Fold08: recipe## i Fold08: model 1/1## ✓ Fold08: model 1/1## i Fold08: model 1/1 (predictions)## ! Fold08: model 1/1 (predictions): There are new levels in a factor: NA## i Fold09: recipe## ✓ Fold09: recipe## i Fold09: model 1/1## ✓ Fold09: model 1/1## i Fold09: model 1/1 (predictions)## ! Fold09: model 1/1 (predictions): There are new levels in a factor: NA## i Fold10: recipe## ✓ Fold10: recipe## i Fold10: model 1/1## ✓ Fold10: model 1/1## i Fold10: model 1/1 (predictions)## ! Fold10: model 1/1 (predictions): There are new levels in a factor: NAlasso7_fit_1 %>%
collect_metrics(summarize = FALSE)## # A tibble: 200 x 6
## id penalty .metric .estimator .estimate .config
## <chr> <dbl> <chr> <chr> <dbl> <chr>
## 1 Fold01 0.0000000001 rmse standard 90.8 Model01
## 2 Fold01 0.00000000129 rmse standard 90.8 Model02
## 3 Fold01 0.0000000167 rmse standard 90.8 Model03
## 4 Fold01 0.000000215 rmse standard 90.8 Model04
## 5 Fold01 0.00000278 rmse standard 90.8 Model05
## 6 Fold01 0.0000359 rmse standard 90.8 Model06
## 7 Fold01 0.000464 rmse standard 90.8 Model07
## 8 Fold01 0.00599 rmse standard 90.8 Model08
## 9 Fold01 0.0774 rmse standard 90.6 Model09
## 10 Fold01 1 rmse standard 91.3 Model10
## # … with 190 more rowsshow_best(lasso7_fit_1, metric = "rmse", n = 10)## # A tibble: 10 x 7
## penalty .metric .estimator mean n std_err .config
## <dbl> <chr> <chr> <dbl> <int> <dbl> <chr>
## 1 0.0774 rmse standard 92.1 10 0.352 Model09
## 2 0.0000000001 rmse standard 92.3 10 0.344 Model01
## 3 0.00000000129 rmse standard 92.3 10 0.344 Model02
## 4 0.0000000167 rmse standard 92.3 10 0.344 Model03
## 5 0.000000215 rmse standard 92.3 10 0.344 Model04
## 6 0.00000278 rmse standard 92.3 10 0.344 Model05
## 7 0.0000359 rmse standard 92.3 10 0.344 Model06
## 8 0.000464 rmse standard 92.3 10 0.344 Model07
## 9 0.00599 rmse standard 92.3 10 0.344 Model08
## 10 1 rmse standard 92.9 10 0.398 Model10show_best(lasso4_fit_1, metric = "rmse", n = 10)## # A tibble: 10 x 7
## penalty .metric .estimator mean n std_err .config
## <dbl> <chr> <chr> <dbl> <int> <dbl> <chr>
## 1 0.0774 rmse standard 97.1 10 0.397 Model09
## 2 0.0000000001 rmse standard 97.3 10 0.394 Model01
## 3 0.00000000129 rmse standard 97.3 10 0.394 Model02
## 4 0.0000000167 rmse standard 97.3 10 0.394 Model03
## 5 0.000000215 rmse standard 97.3 10 0.394 Model04
## 6 0.00000278 rmse standard 97.3 10 0.394 Model05
## 7 0.0000359 rmse standard 97.3 10 0.394 Model06
## 8 0.000464 rmse standard 97.3 10 0.394 Model07
## 9 0.00599 rmse standard 97.3 10 0.394 Model08
## 10 1 rmse standard 97.8 10 0.402 Model108. Compare the metrics from the best lasso model with 4 predictors to the best lasso model with 7+ predicors. Which is best?
The model with 7+ predictors is a better fit
lasso4fit <- lasso4_fit_1 %>%
mutate(model = "lasso4")
lasso7fit <- lasso7_fit_1 %>%
mutate(model = "lasso7")
model_fits <- rbind(lasso4fit, lasso7fit)
show_best(model_fits, metric = "rmse", n = 10)## # A tibble: 10 x 8
## penalty model .metric .estimator mean n std_err .config
## <dbl> <chr> <chr> <chr> <dbl> <int> <dbl> <chr>
## 1 0.0774 lasso7 rmse standard 92.1 10 0.352 Model09
## 2 0.0000000001 lasso7 rmse standard 92.3 10 0.344 Model01
## 3 0.00000000129 lasso7 rmse standard 92.3 10 0.344 Model02
## 4 0.0000000167 lasso7 rmse standard 92.3 10 0.344 Model03
## 5 0.000000215 lasso7 rmse standard 92.3 10 0.344 Model04
## 6 0.00000278 lasso7 rmse standard 92.3 10 0.344 Model05
## 7 0.0000359 lasso7 rmse standard 92.3 10 0.344 Model06
## 8 0.000464 lasso7 rmse standard 92.3 10 0.344 Model07
## 9 0.00599 lasso7 rmse standard 92.3 10 0.344 Model08
## 10 1 lasso7 rmse standard 92.9 10 0.398 Model109. Fit a tuned elastic net model with the same predictors from (7).
- Create a new
{parsnip}elastic net model - Use the same recipe from (7) above
- Create and apply a regular grid for the elastic net model
- Compare the metrics from the elastic net model to the best lasso model from (8). Which would you choose for your final model? What are the best hyperparameters for that model?
Elastic Net model demonstrates the best fit. The best hyperparameters are a penalty of 1 and mixture of .22.
mod_enet <- linear_reg() %>%
set_engine("glmnet") %>%
set_mode("regression") %>%
set_args(penalty = tune(),
mixture = tune())
enet_params <- parameters(penalty(), mixture())
(enet_grid <- grid_regular(enet_params, levels = c(10, 10)))## # A tibble: 100 x 2
## penalty mixture
## <dbl> <dbl>
## 1 0.0000000001 0
## 2 0.00000000129 0
## 3 0.0000000167 0
## 4 0.000000215 0
## 5 0.00000278 0
## 6 0.0000359 0
## 7 0.000464 0
## 8 0.00599 0
## 9 0.0774 0
## 10 1 0
## # … with 90 more rowsenet7_fit_1 <- tune_grid(
mod_enet,
preprocessor = lasso7_rec,
resamples = cv_splits,
grid = enet_grid,
control = tune::control_resamples(verbose = TRUE,
save_pred = TRUE)
)## i Fold01: recipe## ✓ Fold01: recipe## i Fold01: model 1/10## ✓ Fold01: model 1/10## i Fold01: model 1/10 (predictions)## ! Fold01: model 1/10 (predictions): There are new levels in a factor: NA## i Fold01: model 2/10## ✓ Fold01: model 2/10## i Fold01: model 2/10 (predictions)## ! Fold01: model 2/10 (predictions): There are new levels in a factor: NA## i Fold01: model 3/10## ✓ Fold01: model 3/10## i Fold01: model 3/10 (predictions)## ! Fold01: model 3/10 (predictions): There are new levels in a factor: NA## i Fold01: model 4/10## ✓ Fold01: model 4/10## i Fold01: model 4/10 (predictions)## ! Fold01: model 4/10 (predictions): There are new levels in a factor: NA## i Fold01: model 5/10## ✓ Fold01: model 5/10## i Fold01: model 5/10 (predictions)## ! Fold01: model 5/10 (predictions): There are new levels in a factor: NA## i Fold01: model 6/10## ✓ Fold01: model 6/10## i Fold01: model 6/10 (predictions)## ! Fold01: model 6/10 (predictions): There are new levels in a factor: NA## i Fold01: model 7/10## ✓ Fold01: model 7/10## i Fold01: model 7/10 (predictions)## ! Fold01: model 7/10 (predictions): There are new levels in a factor: NA## i Fold01: model 8/10## ✓ Fold01: model 8/10## i Fold01: model 8/10 (predictions)## ! Fold01: model 8/10 (predictions): There are new levels in a factor: NA## i Fold01: model 9/10## ✓ Fold01: model 9/10## i Fold01: model 9/10 (predictions)## ! Fold01: model 9/10 (predictions): There are new levels in a factor: NA## i Fold01: model 10/10## ✓ Fold01: model 10/10## i Fold01: model 10/10 (predictions)## ! Fold01: model 10/10 (predictions): There are new levels in a factor: NA## i Fold02: recipe## ✓ Fold02: recipe## i Fold02: model 1/10## ✓ Fold02: model 1/10## i Fold02: model 1/10 (predictions)## ! Fold02: model 1/10 (predictions): There are new levels in a factor: NA## i Fold02: model 2/10## ✓ Fold02: model 2/10## i Fold02: model 2/10 (predictions)## ! Fold02: model 2/10 (predictions): There are new levels in a factor: NA## i Fold02: model 3/10## ✓ Fold02: model 3/10## i Fold02: model 3/10 (predictions)## ! Fold02: model 3/10 (predictions): There are new levels in a factor: NA## i Fold02: model 4/10## ✓ Fold02: model 4/10## i Fold02: model 4/10 (predictions)## ! Fold02: model 4/10 (predictions): There are new levels in a factor: NA## i Fold02: model 5/10## ✓ Fold02: model 5/10## i Fold02: model 5/10 (predictions)## ! Fold02: model 5/10 (predictions): There are new levels in a factor: NA## i Fold02: model 6/10## ✓ Fold02: model 6/10## i Fold02: model 6/10 (predictions)## ! Fold02: model 6/10 (predictions): There are new levels in a factor: NA## i Fold02: model 7/10## ✓ Fold02: model 7/10## i Fold02: model 7/10 (predictions)## ! Fold02: model 7/10 (predictions): There are new levels in a factor: NA## i Fold02: model 8/10## ✓ Fold02: model 8/10## i Fold02: model 8/10 (predictions)## ! Fold02: model 8/10 (predictions): There are new levels in a factor: NA## i Fold02: model 9/10## ✓ Fold02: model 9/10## i Fold02: model 9/10 (predictions)## ! Fold02: model 9/10 (predictions): There are new levels in a factor: NA## i Fold02: model 10/10## ✓ Fold02: model 10/10## i Fold02: model 10/10 (predictions)## ! Fold02: model 10/10 (predictions): There are new levels in a factor: NA## i Fold03: recipe## ✓ Fold03: recipe## i Fold03: model 1/10## ✓ Fold03: model 1/10## i Fold03: model 1/10 (predictions)## ! Fold03: model 1/10 (predictions): There are new levels in a factor: NA## i Fold03: model 2/10## ✓ Fold03: model 2/10## i Fold03: model 2/10 (predictions)## ! Fold03: model 2/10 (predictions): There are new levels in a factor: NA## i Fold03: model 3/10## ✓ Fold03: model 3/10## i Fold03: model 3/10 (predictions)## ! Fold03: model 3/10 (predictions): There are new levels in a factor: NA## i Fold03: model 4/10## ✓ Fold03: model 4/10## i Fold03: model 4/10 (predictions)## ! Fold03: model 4/10 (predictions): There are new levels in a factor: NA## i Fold03: model 5/10## ✓ Fold03: model 5/10## i Fold03: model 5/10 (predictions)## ! Fold03: model 5/10 (predictions): There are new levels in a factor: NA## i Fold03: model 6/10## ✓ Fold03: model 6/10## i Fold03: model 6/10 (predictions)## ! Fold03: model 6/10 (predictions): There are new levels in a factor: NA## i Fold03: model 7/10## ✓ Fold03: model 7/10## i Fold03: model 7/10 (predictions)## ! Fold03: model 7/10 (predictions): There are new levels in a factor: NA## i Fold03: model 8/10## ✓ Fold03: model 8/10## i Fold03: model 8/10 (predictions)## ! Fold03: model 8/10 (predictions): There are new levels in a factor: NA## i Fold03: model 9/10## ✓ Fold03: model 9/10## i Fold03: model 9/10 (predictions)## ! Fold03: model 9/10 (predictions): There are new levels in a factor: NA## i Fold03: model 10/10## ✓ Fold03: model 10/10## i Fold03: model 10/10 (predictions)## ! Fold03: model 10/10 (predictions): There are new levels in a factor: NA## i Fold04: recipe## ✓ Fold04: recipe## i Fold04: model 1/10## ✓ Fold04: model 1/10## i Fold04: model 1/10 (predictions)## ! Fold04: model 1/10 (predictions): There are new levels in a factor: NA## i Fold04: model 2/10## ✓ Fold04: model 2/10## i Fold04: model 2/10 (predictions)## ! Fold04: model 2/10 (predictions): There are new levels in a factor: NA## i Fold04: model 3/10## ✓ Fold04: model 3/10## i Fold04: model 3/10 (predictions)## ! Fold04: model 3/10 (predictions): There are new levels in a factor: NA## i Fold04: model 4/10## ✓ Fold04: model 4/10## i Fold04: model 4/10 (predictions)## ! Fold04: model 4/10 (predictions): There are new levels in a factor: NA## i Fold04: model 5/10## ✓ Fold04: model 5/10## i Fold04: model 5/10 (predictions)## ! Fold04: model 5/10 (predictions): There are new levels in a factor: NA## i Fold04: model 6/10## ✓ Fold04: model 6/10## i Fold04: model 6/10 (predictions)## ! Fold04: model 6/10 (predictions): There are new levels in a factor: NA## i Fold04: model 7/10## ✓ Fold04: model 7/10## i Fold04: model 7/10 (predictions)## ! Fold04: model 7/10 (predictions): There are new levels in a factor: NA## i Fold04: model 8/10## ✓ Fold04: model 8/10## i Fold04: model 8/10 (predictions)## ! Fold04: model 8/10 (predictions): There are new levels in a factor: NA## i Fold04: model 9/10## ✓ Fold04: model 9/10## i Fold04: model 9/10 (predictions)## ! Fold04: model 9/10 (predictions): There are new levels in a factor: NA## i Fold04: model 10/10## ✓ Fold04: model 10/10## i Fold04: model 10/10 (predictions)## ! Fold04: model 10/10 (predictions): There are new levels in a factor: NA## i Fold05: recipe## ✓ Fold05: recipe## i Fold05: model 1/10## ✓ Fold05: model 1/10## i Fold05: model 1/10 (predictions)## ! Fold05: model 1/10 (predictions): There are new levels in a factor: NA## i Fold05: model 2/10## ✓ Fold05: model 2/10## i Fold05: model 2/10 (predictions)## ! Fold05: model 2/10 (predictions): There are new levels in a factor: NA## i Fold05: model 3/10## ✓ Fold05: model 3/10## i Fold05: model 3/10 (predictions)## ! Fold05: model 3/10 (predictions): There are new levels in a factor: NA## i Fold05: model 4/10## ✓ Fold05: model 4/10## i Fold05: model 4/10 (predictions)## ! Fold05: model 4/10 (predictions): There are new levels in a factor: NA## i Fold05: model 5/10## ✓ Fold05: model 5/10## i Fold05: model 5/10 (predictions)## ! Fold05: model 5/10 (predictions): There are new levels in a factor: NA## i Fold05: model 6/10## ✓ Fold05: model 6/10## i Fold05: model 6/10 (predictions)## ! Fold05: model 6/10 (predictions): There are new levels in a factor: NA## i Fold05: model 7/10## ✓ Fold05: model 7/10## i Fold05: model 7/10 (predictions)## ! Fold05: model 7/10 (predictions): There are new levels in a factor: NA## i Fold05: model 8/10## ✓ Fold05: model 8/10## i Fold05: model 8/10 (predictions)## ! Fold05: model 8/10 (predictions): There are new levels in a factor: NA## i Fold05: model 9/10## ✓ Fold05: model 9/10## i Fold05: model 9/10 (predictions)## ! Fold05: model 9/10 (predictions): There are new levels in a factor: NA## i Fold05: model 10/10## ✓ Fold05: model 10/10## i Fold05: model 10/10 (predictions)## ! Fold05: model 10/10 (predictions): There are new levels in a factor: NA## i Fold06: recipe## ✓ Fold06: recipe## i Fold06: model 1/10## ✓ Fold06: model 1/10## i Fold06: model 1/10 (predictions)## ! Fold06: model 1/10 (predictions): There are new levels in a factor: NA## i Fold06: model 2/10## ✓ Fold06: model 2/10## i Fold06: model 2/10 (predictions)## ! Fold06: model 2/10 (predictions): There are new levels in a factor: NA## i Fold06: model 3/10## ✓ Fold06: model 3/10## i Fold06: model 3/10 (predictions)## ! Fold06: model 3/10 (predictions): There are new levels in a factor: NA## i Fold06: model 4/10## ✓ Fold06: model 4/10## i Fold06: model 4/10 (predictions)## ! Fold06: model 4/10 (predictions): There are new levels in a factor: NA## i Fold06: model 5/10## ✓ Fold06: model 5/10## i Fold06: model 5/10 (predictions)## ! Fold06: model 5/10 (predictions): There are new levels in a factor: NA## i Fold06: model 6/10## ✓ Fold06: model 6/10## i Fold06: model 6/10 (predictions)## ! Fold06: model 6/10 (predictions): There are new levels in a factor: NA## i Fold06: model 7/10## ✓ Fold06: model 7/10## i Fold06: model 7/10 (predictions)## ! Fold06: model 7/10 (predictions): There are new levels in a factor: NA## i Fold06: model 8/10## ✓ Fold06: model 8/10## i Fold06: model 8/10 (predictions)## ! Fold06: model 8/10 (predictions): There are new levels in a factor: NA## i Fold06: model 9/10## ✓ Fold06: model 9/10## i Fold06: model 9/10 (predictions)## ! Fold06: model 9/10 (predictions): There are new levels in a factor: NA## i Fold06: model 10/10## ✓ Fold06: model 10/10## i Fold06: model 10/10 (predictions)## ! Fold06: model 10/10 (predictions): There are new levels in a factor: NA## i Fold07: recipe## ✓ Fold07: recipe## i Fold07: model 1/10## ✓ Fold07: model 1/10## i Fold07: model 1/10 (predictions)## ! Fold07: model 1/10 (predictions): There are new levels in a factor: NA## i Fold07: model 2/10## ✓ Fold07: model 2/10## i Fold07: model 2/10 (predictions)## ! Fold07: model 2/10 (predictions): There are new levels in a factor: NA## i Fold07: model 3/10## ✓ Fold07: model 3/10## i Fold07: model 3/10 (predictions)## ! Fold07: model 3/10 (predictions): There are new levels in a factor: NA## i Fold07: model 4/10## ✓ Fold07: model 4/10## i Fold07: model 4/10 (predictions)## ! Fold07: model 4/10 (predictions): There are new levels in a factor: NA## i Fold07: model 5/10## ✓ Fold07: model 5/10## i Fold07: model 5/10 (predictions)## ! Fold07: model 5/10 (predictions): There are new levels in a factor: NA## i Fold07: model 6/10## ✓ Fold07: model 6/10## i Fold07: model 6/10 (predictions)## ! Fold07: model 6/10 (predictions): There are new levels in a factor: NA## i Fold07: model 7/10## ✓ Fold07: model 7/10## i Fold07: model 7/10 (predictions)## ! Fold07: model 7/10 (predictions): There are new levels in a factor: NA## i Fold07: model 8/10## ✓ Fold07: model 8/10## i Fold07: model 8/10 (predictions)## ! Fold07: model 8/10 (predictions): There are new levels in a factor: NA## i Fold07: model 9/10## ✓ Fold07: model 9/10## i Fold07: model 9/10 (predictions)## ! Fold07: model 9/10 (predictions): There are new levels in a factor: NA## i Fold07: model 10/10## ✓ Fold07: model 10/10## i Fold07: model 10/10 (predictions)## ! Fold07: model 10/10 (predictions): There are new levels in a factor: NA## i Fold08: recipe## ✓ Fold08: recipe## i Fold08: model 1/10## ✓ Fold08: model 1/10## i Fold08: model 1/10 (predictions)## ! Fold08: model 1/10 (predictions): There are new levels in a factor: NA## i Fold08: model 2/10## ✓ Fold08: model 2/10## i Fold08: model 2/10 (predictions)## ! Fold08: model 2/10 (predictions): There are new levels in a factor: NA## i Fold08: model 3/10## ✓ Fold08: model 3/10## i Fold08: model 3/10 (predictions)## ! Fold08: model 3/10 (predictions): There are new levels in a factor: NA## i Fold08: model 4/10## ✓ Fold08: model 4/10## i Fold08: model 4/10 (predictions)## ! Fold08: model 4/10 (predictions): There are new levels in a factor: NA## i Fold08: model 5/10## ✓ Fold08: model 5/10## i Fold08: model 5/10 (predictions)## ! Fold08: model 5/10 (predictions): There are new levels in a factor: NA## i Fold08: model 6/10## ✓ Fold08: model 6/10## i Fold08: model 6/10 (predictions)## ! Fold08: model 6/10 (predictions): There are new levels in a factor: NA## i Fold08: model 7/10## ✓ Fold08: model 7/10## i Fold08: model 7/10 (predictions)## ! Fold08: model 7/10 (predictions): There are new levels in a factor: NA## i Fold08: model 8/10## ✓ Fold08: model 8/10## i Fold08: model 8/10 (predictions)## ! Fold08: model 8/10 (predictions): There are new levels in a factor: NA## i Fold08: model 9/10## ✓ Fold08: model 9/10## i Fold08: model 9/10 (predictions)## ! Fold08: model 9/10 (predictions): There are new levels in a factor: NA## i Fold08: model 10/10## ✓ Fold08: model 10/10## i Fold08: model 10/10 (predictions)## ! Fold08: model 10/10 (predictions): There are new levels in a factor: NA## i Fold09: recipe## ✓ Fold09: recipe## i Fold09: model 1/10## ✓ Fold09: model 1/10## i Fold09: model 1/10 (predictions)## ! Fold09: model 1/10 (predictions): There are new levels in a factor: NA## i Fold09: model 2/10## ✓ Fold09: model 2/10## i Fold09: model 2/10 (predictions)## ! Fold09: model 2/10 (predictions): There are new levels in a factor: NA## i Fold09: model 3/10## ✓ Fold09: model 3/10## i Fold09: model 3/10 (predictions)## ! Fold09: model 3/10 (predictions): There are new levels in a factor: NA## i Fold09: model 4/10## ✓ Fold09: model 4/10## i Fold09: model 4/10 (predictions)## ! Fold09: model 4/10 (predictions): There are new levels in a factor: NA## i Fold09: model 5/10## ✓ Fold09: model 5/10## i Fold09: model 5/10 (predictions)## ! Fold09: model 5/10 (predictions): There are new levels in a factor: NA## i Fold09: model 6/10## ✓ Fold09: model 6/10## i Fold09: model 6/10 (predictions)## ! Fold09: model 6/10 (predictions): There are new levels in a factor: NA## i Fold09: model 7/10## ✓ Fold09: model 7/10## i Fold09: model 7/10 (predictions)## ! Fold09: model 7/10 (predictions): There are new levels in a factor: NA## i Fold09: model 8/10## ✓ Fold09: model 8/10## i Fold09: model 8/10 (predictions)## ! Fold09: model 8/10 (predictions): There are new levels in a factor: NA## i Fold09: model 9/10## ✓ Fold09: model 9/10## i Fold09: model 9/10 (predictions)## ! Fold09: model 9/10 (predictions): There are new levels in a factor: NA## i Fold09: model 10/10## ✓ Fold09: model 10/10## i Fold09: model 10/10 (predictions)## ! Fold09: model 10/10 (predictions): There are new levels in a factor: NA## i Fold10: recipe## ✓ Fold10: recipe## i Fold10: model 1/10## ✓ Fold10: model 1/10## i Fold10: model 1/10 (predictions)## ! Fold10: model 1/10 (predictions): There are new levels in a factor: NA## i Fold10: model 2/10## ✓ Fold10: model 2/10## i Fold10: model 2/10 (predictions)## ! Fold10: model 2/10 (predictions): There are new levels in a factor: NA## i Fold10: model 3/10## ✓ Fold10: model 3/10## i Fold10: model 3/10 (predictions)## ! Fold10: model 3/10 (predictions): There are new levels in a factor: NA## i Fold10: model 4/10## ✓ Fold10: model 4/10## i Fold10: model 4/10 (predictions)## ! Fold10: model 4/10 (predictions): There are new levels in a factor: NA## i Fold10: model 5/10## ✓ Fold10: model 5/10## i Fold10: model 5/10 (predictions)## ! Fold10: model 5/10 (predictions): There are new levels in a factor: NA## i Fold10: model 6/10## ✓ Fold10: model 6/10## i Fold10: model 6/10 (predictions)## ! Fold10: model 6/10 (predictions): There are new levels in a factor: NA## i Fold10: model 7/10## ✓ Fold10: model 7/10## i Fold10: model 7/10 (predictions)## ! Fold10: model 7/10 (predictions): There are new levels in a factor: NA## i Fold10: model 8/10## ✓ Fold10: model 8/10## i Fold10: model 8/10 (predictions)## ! Fold10: model 8/10 (predictions): There are new levels in a factor: NA## i Fold10: model 9/10## ✓ Fold10: model 9/10## i Fold10: model 9/10 (predictions)## ! Fold10: model 9/10 (predictions): There are new levels in a factor: NA## i Fold10: model 10/10## ✓ Fold10: model 10/10## i Fold10: model 10/10 (predictions)## ! Fold10: model 10/10 (predictions): There are new levels in a factor: NAenet7fit <- enet7_fit_1 %>%
mutate(model = "enet7")
model_fits <- rbind(model_fits, enet7fit)
show_best(model_fits, metric = "rmse", n = 10)## # A tibble: 10 x 9
## penalty mixture model .metric .estimator mean n std_err .config
## <dbl> <dbl> <chr> <chr> <chr> <dbl> <int> <dbl> <chr>
## 1 1 0.333 enet7 rmse standard 92.0 10 0.373 Model040
## 2 1 0.222 enet7 rmse standard 92.0 10 0.367 Model030
## 3 1 0.444 enet7 rmse standard 92.1 10 0.378 Model050
## 4 1 0.111 enet7 rmse standard 92.1 10 0.361 Model020
## 5 0.0774 NA lasso7 rmse standard 92.1 10 0.352 Model09
## 6 0.0774 1 enet7 rmse standard 92.1 10 0.352 Model099
## 7 0.0774 0.889 enet7 rmse standard 92.2 10 0.351 Model089
## 8 0.0774 0.778 enet7 rmse standard 92.2 10 0.350 Model079
## 9 1 0.556 enet7 rmse standard 92.2 10 0.385 Model060
## 10 0.0774 0.667 enet7 rmse standard 92.2 10 0.349 Model069
