Question 2

  1. False. Lasso is less flexible due to how it penalizes coefficients.

  2. False. Again, lasso is less flexible due to how it penalizes coefficients.

  3. True. Lasso does lead to greater prediction power as it reduces variance more than it increases bias.

  4. False. Lasso does not increase variance, it decreases it.

  1. False. Ridge is also less flexible than OLS.

  2. False. Again, ridge is less flexible than OLS.

  3. True. Ridge does decrease variance while slightly increasing bias.

  4. False. Ridge does not increase variance.

  1. True. Non-linear methods are more flexible and lower bias while increasing variance.

  2. True, I think. Non-linear methods are more flexible and I think this is just reversing the previous statement about the bias/variance trade-off.

  3. False. Non-linear methods are more flexible, not less.

  4. False. Non-linear methods are more flexible, not less.

Question 9

(a) Split the data into training and test sets

library(ISLR2)
library(tidymodels)
## ── Attaching packages ────────────────────────────────────── tidymodels 1.5.0 ──
## ✔ broom        1.0.13     ✔ recipes      1.3.3 
## ✔ dials        1.4.4      ✔ rsample      1.3.2 
## ✔ dplyr        1.2.1      ✔ tailor       0.1.0 
## ✔ ggplot2      4.0.3      ✔ tidyr        1.3.2 
## ✔ infer        1.1.0      ✔ tune         2.1.0 
## ✔ modeldata    1.5.1      ✔ workflows    1.3.0 
## ✔ parsnip      1.6.0      ✔ workflowsets 1.1.1 
## ✔ purrr        1.2.2      ✔ yardstick    1.4.0
## ── Conflicts ───────────────────────────────────────── tidymodels_conflicts() ──
## ✖ purrr::discard() masks scales::discard()
## ✖ dplyr::filter()  masks stats::filter()
## ✖ dplyr::lag()     masks stats::lag()
## ✖ recipes::step()  masks stats::step()
data("College")     # <-- capital C
college <- College  # <-- capital C

college$Private <- ifelse(college$Private == "Yes", 1, 0)

set.seed(1)
college_split <- initial_split(college, prop = 0.7)
college_train <- training(college_split)
college_test <- testing(college_split)

(b) Fit a linear model and report the test error

The root mean square error is shown below.

## [1] 1123.223

(c) Fit a ridge regression model with λ chosen by cross‑validation

The RMSE for this model is shown below.

## [1] 1058.789

(d) Fit a lasso model with λ chosen by cross‑validation

The RMSE for this model is shown below, along with the number of non‑zero coefficients.

## [1] 1108.005
## [1] 0

(e) Fit a PCR model with M chosen by cross‑validation

The RMSE for this model is shown below, along with the value of M selected by cross‑validation.

## [1] 1123.223
## [1] 17

(f) Fit a PLS model with M chosen by cross‑validation

The RMSE for this model is shown below, along with the value of M selected by cross‑validation.

## 
## Attaching package: 'pls'
## The following object is masked from 'package:stats':
## 
##     loadings
## 17 comps 
##       17
## [1] 1123.223

(g) Comment on the results

Given the similar values for RMSE obtained across models, it appears difficult to accurately predict the number of applications received. The mean number of applications is around 3,002, so only being able to predict within roughly 900–1100 applications is not very helpful. This suggests that “Apps” may be a poor dependent variable for the predictors available in this dataset. Better predictors might include variables related to cost, prestige, or other institutional characteristics. Overall, there is not much difference among the test errors resulting from these five approaches.

Question 11

  1. Across all methods, the RMSE values were very similar, suggesting that no single regression approach clearly outperformed the others. The ridge model’s RMSE is 7.6, meaning the model can predict the per capita crime rate accurately for each town within about 7.6 units. Lasso produced virtually the same result. After conducting best subset selection, I removed “age” from the model and performed a regular linear regression, which still performed about the same as the lasso and ridge models. The PCR and PLS models also produced RMSE values in the same range, with slightly lower errors but not enough to indicate a clearly superior method.
## [1] 7.664965
## → A | warning: A correlation computation is required, but `estimate` is constant and has 0
##                standard deviation, resulting in a divide by 0 error. `NA` will be returned.
## There were issues with some computations   A: x22There were issues with some computations   A: x43There were issues with some computations   A: x64There were issues with some computations   A: x85There were issues with some computations   A: x106There were issues with some computations   A: x127There were issues with some computations   A: x149There were issues with some computations   A: x171There were issues with some computations   A: x193There were issues with some computations   A: x214There were issues with some computations   A: x214
## [1] 7.666177

## [1] 7.590427
## [1] 7.590291
## Selected M = 10
## [1] 7.590268

(b)

Based on the cross‑validation and test set performance results, the linear model built using the 11 variables selected via Best Subset Selection, as well as the PCR and PLS models, all perform exceptionally well and produce highly competitive test RMSE values (≈7.59). If forced to choose a single optimal model, the Best Subset Selection linear model or the PLS model is preferred because they achieve the lowest test error while remaining relatively simple and interpretable.

(c)

No, the chosen Best Subset Selection model does not involve all of the features in the dataset. During cross‑validation, the variable age was identified as unnecessary and removed because it did not contribute meaningful predictive power once other variables (such as tax rate, distance to employment centers, and pollution levels) were included. Excluding it reduces variance without increasing bias, resulting in a more efficient and parsimonious model.