knitr::opts_chunk$set(echo = TRUE, message=FALSE,warning = FALSE)2. For parts (a) through (c), indicate which of i. through iv. is correct. Justify your answer.
i. More flexible and hence will give improved prediction accuracy when its increase in bias is less than its decrease in variance. ii. More flexible and hence will give improved prediction accuracy when its increase in variance is less than its decrease in bias. iii. Less flexible and hence will give improved prediction accuracy when its increase in bias is less than its decrease in variance. iv. Less flexible and hence will give improved prediction accuracy when its increase in variance is less than its decrease in bias.
(a) The lasso, relative to least squares, is:
(b) Repeat (a) for ridge regression relative to least squares.
(c) Repeat (a) for non-linear methods relative to least squares.
library("ISLR2")
library("glmnet")
library("pls")
library("leaps")
data(College)
data(Boston)9. In this exercise, we will predict the number of applications received using the other variables in the College data set.
(a) Split the data set into a training set and a test set.
set.seed(70)
College=na.omit(College)
trainCol=sample(1:nrow(College), nrow(College)/2)
testCol=setdiff(1:nrow(College), trainCol)(b) Fit a linear model using least squares on the training set, and report the test error obtained.
set.seed(70)
lmCol.fit = lm(Apps~.,data=College, subset = trainCol)
#summary(lmCol.fit)
lm.predCollege = predict(lmCol.fit, College[testCol,])
mean((lm.predCollege - College$Apps[testCol])^2)## [1] 1502545(c) Fit a ridge regression model on the training set, with λ chosen by cross-validation. Report the test error obtained.
x.ridge = model.matrix(Apps~.,data=College)[,-1]
y.ridge = College$Apps
set.seed(70)
ridge.cvCollege=cv.glmnet(x.ridge[trainCol, ],y.ridge[trainCol], alpha = 0)
ridge.predCollege=predict(ridge.cvCollege,s='lambda.min',newx=x.ridge[testCol,])
mean((ridge.predCollege-y.ridge[testCol])^2)## [1] 2478117(d) Fit a lasso model on the training set, with λ chosen by cross validation. Report the test error obtained, along with the number of non-zero coefficient estimates.
x.lasso= model.matrix(Apps~.,data=College)[,-1]
y.lasso = College$Apps
set.seed(70)
lasso.cvCollege=cv.glmnet(x.lasso[trainCol, ],y.lasso[trainCol], alpha = 1)
lasso.predCollege=predict(lasso.cvCollege,s='lambda.min',newx=x.lasso[testCol,])
mean((lasso.predCollege-y.lasso[testCol])^2)## [1] 1593951lasso.coefCollege = coef(lasso.cvCollege, s = "lambda.min")
sum(lasso.coefCollege[-1] != 0)## [1] 13(e) Fit a PCR model on the training set, with M chosen by cross validation. Report the test error obtained, along with the value of M selected by cross-validation.
set.seed(70)
pcr.fit3=pcr(Apps~.,data=College[trainCol,],scale=TRUE,validation='CV')
#validationplot(pcr.fit3,val.type = "MSEP")
cv.pcr1 = RMSEP(pcr.fit3)
ChosenM1 = which.min(cv.pcr1$val[1,1,-1])
ChosenM1## 17 comps
## 17pcr.pred2=predict(pcr.fit3,College[testCol,],ncomp = ChosenM1)
mean((pcr.pred2-College$Apps[testCol])^2)## [1] 1502545(f) Fit a PLS model on the training set, with M chosen by cross validation. Report the test error obtained, along with the value of M selected by cross-validation.
set.seed(70)
plsr.fit4=plsr(Apps~.,data=College[trainCol,],scale=TRUE,validation='CV')
#validationplot(plsr.fit4,val.type = "MSEP")
cv.pls2 = RMSEP(plsr.fit4)
ChosenM2 = which.min(cv.pls2$val[1,1,-1])
ChosenM2## 9 comps
## 9plsr.pred3=predict(plsr.fit4,College[testCol,],ncomp = ChosenM2)
mean((plsr.pred3-College$Apps[testCol])^2)## [1] 1588278(g) Comment on the results obtained. How accurately can we predict the number of college applications received? Is there much difference among the test errors resulting from these five approaches?
The test error obtained by fitting a linear model using least squares was 1,502,545 , The test error obtained by fitting a ridge regression model was 2,478,117
The test error obtained by fitting a lasso model was 1,593,951
The test error obtained by fitting a PCR model was 1,502,545
The test error obtained by fitting a PLS model was 1,588,278
Among all the determined test MSEs, the smallest ones were produced using the PCR model and linear model using least squares. They’re the exact same number.
The biggest observation test MSE by far came from using the ridge regression model.
The ridge MSE is larger by 884,166 to the second nearest large MSE.
Going off the test MSEs, we can make the assumption that all models fitted on the data set accurately predict the amount of applications received minus the ridge regression model due to the other 4 models MSEs being in close proximity to one another.
11. We will now try to predict per capita crime rate in the Boston dataset.
(a) Try out some of the regression methods explored in this chapter, such as best subset selection, the lasso, ridge regression, and PCR. Present and discuss results for the approaches that you consider.
Split
set.seed(70)
Boston=na.omit(Boston)
trainBost=sample(1:nrow(Boston), nrow(Boston)/2)
testBost=setdiff(1:nrow(Boston), trainBost)Linear Regression
set.seed(70)
lmCol.fitBost = lm(crim~.,data=Boston, subset = trainBost)
#summary(lmCol.fit)
lm.predBoston = predict(lmCol.fitBost, Boston[testBost,])
mean((lm.predBoston - Boston$crim[testBost])^2)## [1] 49.36843Ridge Regression
x.ridgeBost = model.matrix(crim~.,data=Boston)[,-1]
y.ridgeBost = Boston$crim
set.seed(70)
ridge.cvBoston=cv.glmnet(x.ridgeBost[trainBost, ],
y.ridgeBost[trainBost], alpha = 0)
ridge.predBoston=predict(ridge.cvBoston,s='lambda.min',
newx=x.ridgeBost[testBost,])
mean((ridge.predBoston-y.ridgeBost[testBost])^2)## [1] 49.72757Lasso Model
x.lassoBost= model.matrix(crim~.,data=Boston)[,-1]
y.lassoBost = Boston$crim
set.seed(70)
lasso.cvBoston=cv.glmnet(x.lassoBost[trainBost, ],
y.lassoBost[trainBost], alpha = 1)
lasso.predBoston=predict(lasso.cvBoston,s='lambda.min',
newx=x.lassoBost[testBost,])
mean((lasso.predBoston-y.lassoBost[testBost])^2)## [1] 49.75547lasso.coefBoston = coef(lasso.cvBoston, s = "lambda.min")
#sum(lasso.coefBoston[-1] != 0)PCR Model
set.seed(70)
pcr.fit3Bost=pcr(crim~.,data=Boston[trainBost,],scale=TRUE,validation='CV')
#validationplot(pcr.fit3,val.type = "MSEP")
cv.pcr1Bost = RMSEP(pcr.fit3Bost)
ChosenM1Bost = which.min(cv.pcr1Bost$val[1,1,-1])
#ChosenM1Bost
pcr.pred2Bost=predict(pcr.fit3Bost,Boston[testBost,],ncomp = ChosenM1Bost)
mean((pcr.pred2Bost-Boston$crim[testBost])^2)## [1] 49.36843Best Subset Selection
predict.regsubsets = function(object,newdata,id){
form = as.formula(object$call[[2]])
mat = model.matrix(form,newdata)
coefi = coef(object,id=id)
xvars = names(coefi)
mat[,xvars] %*% coefi
}
set.seed(70)
regfit.fullBoston = regsubsets(crim~.,Boston[trainBost,],nvmax = 12)
val.errorsBost=rep(NA,12)
for(i in 1:12){
pred.regBost=predict.regsubsets(regfit.fullBoston,
Boston[testBost, ],id = i)
val.errorsBost[i]=mean((Boston$crim[testBost]-pred.regBost)^2)
}
#val.errorsBost
which.min(val.errorsBost)## [1] 12val.errorsBost[12]## [1] 49.36843The test error obtained by fitting a linear model using least squares was 49.36843 , The test error obtained by fitting a ridge regression model was 49.72757
The test error obtained by fitting a lasso model was 49.75547
The test error obtained by fitting a PCR model was 49.36843
The test error obtained by fitting a Best Subset Selection model was 49.36843
It appears that all 5 models fitted produce Test MSEs that are small relative to the overall error, which inidcates that they’re all fairly accurate compared to each other.
(b) Propose a model (or set of models) that seem to perform well on this data set, and justify your answer. Make sure that you are evaluating model performance using validation set error, cross validation, or some other reasonable alternative, as opposed to using training error.
Best Subset Selection model, linear model using least squares, and the PCR model all yield the same value for the test MSE. This value of 49.36843 is the lowest Test MSE.
These 3 models perform the best because of this.
Personally, I would choose the least squares linear model due to its readability and simplicity.
(c) Does your chosen model involve all of the features in the data set? Why or why not?
Yes, the linear model examines all the predictors in the data set. It also examines coefficient for each predictor, which allows each predictor to add its own input to the prediction of ‘crim’.