Homework 5

Nicole Janny Salomons

Homework 5

Get the libraries and import the data:

library(glmnet)
train <- read.csv("digits_train.csv",as.is=TRUE)
valid <- read.csv("digits_valid.csv", as.is=TRUE)

train_x = train[,-1]
train_y = train[257]
valid_x = valid[,-1]
valid_y = valid[257]

Part 1

** Apply the lasso penalty to train a multinomial logistic regression classifier with type.multinomial=“grouped”. **

ty<- data.frame(train_y)
ty = factor(ty$digit)

m.lasso.grouped <- cv.glmnet(x = as.matrix(train_x) ,y = ty, family = c("multinomial"), type.multinomial = "grouped", alpha = 1)

Show a plot that displays the misclassification error on the y-axis for values of log(lambda) on the x-axis.

plot(m.lasso.grouped)

** 2) Report min and 1se**

lmin_lasso_grouped <- m.lasso.grouped$lambda.min
lmin_lasso_grouped

[1] 0.004828283

lse_lasso_grouped <- m.lasso.grouped$lambda.1se
lse_lasso_grouped

[1] 0.007005011

The minimum lambda was 0.004828283. And with one standard deviation its 0.007005011.

** Next, use both min and 1se to generate predictions on the validation set. **

pmin <- predict(m.lasso.grouped,newx = as.matrix(valid_x), s=lmin_lasso_grouped, type="class")
psd <- predict(m.lasso.grouped, newx = as.matrix(valid_x), s=lse_lasso_grouped, type="class")

** Report the resulting misclassification error rates.**

mean(pmin != valid_y)

[1] 0.03773585

mean(psd != valid_y)

[1] 0.03773585

The error rates for the minimum lamda were 3.77%, and with 1 standard deviation it was 3.77%.

** (3) For 1 standard deviation lambda, how many predictors are selected by glmnet()? How many non-zero coefficients are estimated in total for this value of lambda? **

#m.lasso.grouped$lambda.min
m.lasso.grouped$nzero[which(m.lasso.grouped$lambda == m.lasso.grouped$lambda.1se)]

The number of non-zero coeficients for lambda with 1 standard deviation is 128. And the number of predictors is also 128.

** 4) Repeat with ungrouped **

m.lasso.ungrouped <- cv.glmnet(x = as.matrix(train_x) ,y = ty, family = c("multinomial"), type.multinomial = "ungrouped", alpha = 1)

plot(m.lasso.ungrouped)

m.lasso.ungrouped <- cv.glmnet(x = as.matrix(train_x) ,y = ty, family = c("multinomial"), type.multinomial = "ungrouped", alpha = 1)
plot(m.lasso.ungrouped)

lmin_lasso_ungrouped <- m.lasso.ungrouped$lambda.min
lmin_lasso_ungrouped

[1] 0.0002767712

lse_lasso_ungrouped <- m.lasso.ungrouped$lambda.1se
lse_lasso_ungrouped

[1] 0.001779105

pmin_ung <- predict(m.lasso.ungrouped,newx = as.matrix(valid_x), s=lmin_lasso_ungrouped, type="class")
psd_ung <- predict(m.lasso.ungrouped, newx = as.matrix(valid_x), s=lse_lasso_ungrouped, type="class")
mean(pmin_ung != valid_y)

[1] 0.05974843

mean(psd_ung != valid_y)

[1] 0.07232704

m.lasso.ungrouped$nzero[which(m.lasso.ungrouped$lambda == m.lasso.ungrouped$lambda.1se)]

s50 
 36 

For ungrouped the minimum lambda was: 0.0002767712. And lambda with 1 standard deviation is: 0.001779105.

The misclassification for minimum lambda is: 5.97%.

The misclassification for 1 standard deviation is: 7.23%.

With one standard deviation the number of non-zero coefficients is 36.

With one standard deviation the number of predictors is also 36.

Part 2

** 1)Apply the ridge penalty to train a multinomial logistic regression classifier. **

m.ridge <- cv.glmnet(x = as.matrix(train_fold) ,y = train_fold_y, family = c("multinomial"), alpha = 0)

** Show a plot that displays the misclassification error on the y-axis for values of log(lambda) on the x-axis. **

plot(m.ridge)

** (2) Report min and 1se and their associated misclassification errors for the validation set. **

lmin_ridge <- m.ridge$lambda.min
lmin_ridge

[1] 0.01937868

lse_ridge <- m.ridge$lambda.1se
lse_ridge

[1] 0.02811515

pmin_ridge <- predict(m.ridge,newx = as.matrix(valid_x), s=lmin_ridge, type="class")
psd_ridge <- predict(m.ridge, newx = as.matrix(valid_x), s=lse_ridge, type="class")
mean(pmin_ridge != valid_y)

[1] 0.08176101

mean(psd_ridge != valid_y)

[1] 0.08805031

For ungrouped the minimum lambda was: 0.01937868. And lambda with 1 standard deviation is: 0.02811515.

The misclassification for minimum lambda is: 8.18%.

The misclassification for 1 standard deviation is: 8.81%.

Part 3

** In your own words, how do these 3 approaches differ? **

The grouped lasso approach will create the most simple model in my opinion, using only a subset of the predictors, and using the same predictors for all the classification. On the other hand, the ungrouped lasso approach will use different subsets of predictors creating a more detailed model. Both lasso approaches are similar that they do not use all of the predictors. And Ridge uses all of the predictors, but has some of the weights be very close to 0.

** Compare the performance of ridge, lasso, and your best approach from Homework 3 in prediction accuracy on the validation set. **

I will be using the best performance in prediction accuracy for each:

• Performance for lasso (ungrouped): 94.03%

• Performance for lasso (grouped): 96.23%

• Performance for ridge: 91.82%

• The best approach from homework (KNN): 87.42%

The best performance seems to come from lasso grouped.

** If you had to pick a favorite final model, what would you choose? Justify your answer. **

My favourite final model in this case is lasso grouped. It has the highest accuracy, and although it drops some of the predictors making it harder to interpret, at least it uses the same predictors accross groups.