Predictive Modeling Homework 3

Author

Cheryl Chiu (wky301)

Published

March 2, 2025

Exercise 13

This question should be answered using the Weekly data set, which is part of the ISLR2 package. This data is similar in nature to the Smarket data from this chapter’s lab, except that it contains 1,089 weekly returns for 21 years, from the beginning of 1990 to the end of 2010.

library(ISLR2)
library(MASS)
library(class) #KNN
library(e1071) #Bayes
library(ggplot2)
library(dplyr)
library(patchwork)
library(car)

(a) Produce some numerical and graphical summaries of the Weekly data. Do there appear to be any patterns?

data(Weekly)
summary(Weekly)

      Year           Lag1               Lag2               Lag3         
 Min.   :1990   Min.   :-18.1950   Min.   :-18.1950   Min.   :-18.1950  
 1st Qu.:1995   1st Qu.: -1.1540   1st Qu.: -1.1540   1st Qu.: -1.1580  
 Median :2000   Median :  0.2410   Median :  0.2410   Median :  0.2410  
 Mean   :2000   Mean   :  0.1506   Mean   :  0.1511   Mean   :  0.1472  
 3rd Qu.:2005   3rd Qu.:  1.4050   3rd Qu.:  1.4090   3rd Qu.:  1.4090  
 Max.   :2010   Max.   : 12.0260   Max.   : 12.0260   Max.   : 12.0260  
      Lag4               Lag5              Volume            Today         
 Min.   :-18.1950   Min.   :-18.1950   Min.   :0.08747   Min.   :-18.1950  
 1st Qu.: -1.1580   1st Qu.: -1.1660   1st Qu.:0.33202   1st Qu.: -1.1540  
 Median :  0.2380   Median :  0.2340   Median :1.00268   Median :  0.2410  
 Mean   :  0.1458   Mean   :  0.1399   Mean   :1.57462   Mean   :  0.1499  
 3rd Qu.:  1.4090   3rd Qu.:  1.4050   3rd Qu.:2.05373   3rd Qu.:  1.4050  
 Max.   : 12.0260   Max.   : 12.0260   Max.   :9.32821   Max.   : 12.0260  
 Direction 
 Down:484  
 Up  :605

pairs(Weekly)

🔴 Answer: The scatterplots of Lag1 to Lag5 do not have strong correlations. However, Volume is shown to increase when year increases, it seems to be strongly correlated.

(b) Use the full data set to perform a logistic regression with Direction as the response and the five lag variables plus Volume as predictors. Use the summary function to print the results. Do any of the predictors appear to be statistically significant? If so, which ones?

glm.fit <- glm(Direction ~ Lag1 + Lag2 + Lag3 + Lag4 + Lag5 
               + Volume, data = Weekly, family = binomial)
summary(glm.fit)


Call:
glm(formula = Direction ~ Lag1 + Lag2 + Lag3 + Lag4 + Lag5 + 
    Volume, family = binomial, data = Weekly)

Coefficients:
            Estimate Std. Error z value Pr(>|z|)   
(Intercept)  0.26686    0.08593   3.106   0.0019 **
Lag1        -0.04127    0.02641  -1.563   0.1181   
Lag2         0.05844    0.02686   2.175   0.0296 * 
Lag3        -0.01606    0.02666  -0.602   0.5469   
Lag4        -0.02779    0.02646  -1.050   0.2937   
Lag5        -0.01447    0.02638  -0.549   0.5833   
Volume      -0.02274    0.03690  -0.616   0.5377   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 1496.2  on 1088  degrees of freedom
Residual deviance: 1486.4  on 1082  degrees of freedom
AIC: 1500.4

Number of Fisher Scoring iterations: 4

🔴 Answer: Lag2 is the only lag variable that is statistically significant (p-value = 0.0296). Lag2 might the most useful predictor of the market’s direction.

(c) Compute the confusion matrix and overall fraction of correct predictions. Explain what the confusion matrix is telling you about the types of mistakes made by logistic regression

glm.probs <- predict(glm.fit, type = "response")
glm.pred <- ifelse(glm.probs > 0.5, "Up", "Down")

confusion_full <- table(glm.pred, Weekly$Direction)
confusion_full

        
glm.pred Down  Up
    Down   54  48
    Up    430 557

accuracy_full <- mean(glm.pred == Weekly$Direction)
accuracy_full

[1] 0.5610652

🔴 Answer: The confusion matrix shows that the model predicts Up more often (it incorrectly predicts Up for 430 Down days.). As a result, the model only has an accuracy of ~56%.

(d) Now fit the logistic regression model using a training data period from 1990 to 2008, with Lag2 as the only predictor. Compute the confusion matrix and the overall fraction of correct predictions for the held out data (that is, the data from 2009 and 2010).

train <- (Weekly$Year < 2009)

glm.fit2 <- glm(Direction ~ Lag2, data = Weekly, family = binomial, subset = train)

glm.probs2 <- predict(glm.fit2, Weekly[!train, ], type = "response")
glm.pred2 <- ifelse(glm.probs2 > 0.5, "Up", "Down")

confusion_test_glm <- table(glm.pred2, Weekly[!train, "Direction"])
confusion_test_glm

         
glm.pred2 Down Up
     Down    9  5
     Up     34 56

accuracy_test_glm <- mean(glm.pred2 == Weekly[!train, "Direction"])
accuracy_test_glm

[1] 0.625

🔴 Answer: For the logistic regression model model with 104 observations, it misclassifies 5 Up days as Down and 34 Down days as Up. This results in an overall accuracy of 62.5%, the model still tends to predict Up more.

(e) Repeat (d) using LDA.

lda.fit <- lda(Direction ~ Lag2, data = Weekly, subset = train)
lda.pred <- predict(lda.fit, Weekly[!train, ])
confusion_test_lda <- table(lda.pred$class, Weekly[!train, "Direction"])
confusion_test_lda

      
       Down Up
  Down    9  5
  Up     34 56

accuracy_test_lda <- mean(lda.pred$class == Weekly[!train, "Direction"])
accuracy_test_lda

[1] 0.625

🔴 Answer: For the LDA model model with 104 observations, it misclassifies 5 Up days as Down and 34 Down days as Up. This results in an overall accuracy of 62.5%, the model still tends to predict Up more. The result is the same as logistic regression model.

(f) Repeat (d) using QDA.

qda.fit <- qda(Direction ~ Lag2, data = Weekly, subset = train)
qda.pred <- predict(qda.fit, Weekly[!train, ])
confusion_test_qda <- table(qda.pred$class, Weekly[!train, "Direction"])
confusion_test_qda

      
       Down Up
  Down    0  0
  Up     43 61

accuracy_test_qda <- mean(qda.pred$class == Weekly[!train, "Direction"])
accuracy_test_qda

[1] 0.5865385

🔴 Answer: For the QDA model model with 104 observations, it predicted every observation as Up. It correctly identified 61 Up days, and misclassified all 43 Down days as Up, leading to an accuracy of ~59%.

(g) Repeat (d) using KNN with K = 1.

train.X <- as.matrix(Weekly[train, "Lag2"])
test.X <- as.matrix(Weekly[!train, "Lag2"])
train.Direction <- Weekly[train, "Direction"]

set.seed(123)  
knn.pred <- knn(train.X, test.X, train.Direction, k = 1)

confusion_test_knn <- table(knn.pred, Weekly[!train, "Direction"])
confusion_test_knn

        
knn.pred Down Up
    Down   21 29
    Up     22 32

accuracy_test_knn <- mean(knn.pred == Weekly[!train, "Direction"])
accuracy_test_knn

[1] 0.5096154

🔴 Answer: For the KNN model model with 104 observations, it predicted 21 Down days and 32 Up days, but misclassified 29 Up days as Down and 22 Down days as Up, leading to an accuracy of ~51%.

Repeat (d) using naive Bayes.

nb.fit <- naiveBayes(Direction ~ Lag2, data = Weekly, subset = train)
nb.pred <- predict(nb.fit, Weekly[!train, ])
confusion_test_nb <- table(nb.pred, Weekly[!train, "Direction"])
confusion_test_nb

       
nb.pred Down Up
   Down    0  0
   Up     43 61

accuracy_test_nb <- mean(nb.pred == Weekly[!train, "Direction"])
accuracy_test_nb

[1] 0.5865385

🔴 Answer: For the naive Bayes model model with 104 observations, it predicted every observation as Up. It correctly identified 61 Up days, and misclassified all 43 Down days as Up, leading to an accuracy of ~59%.

(i) Which of these methods appears to provide the best results on this data?

🔴 Answer: After running the above models on the test set (2009–2010), logistic regression and LDA with only Lag2 show the best predictive performance on this data set (both are 62.5%).

(j) Experiment with different combinations of predictors, including possible transformations and interactions, for each of the methods. Report the variables, method, and associated confu- sion matrix that appears to provide the best results on the held out data. Note that you should also experiment with values for K in the KNN classifier.

train <- Weekly$Year < 2009
WeeklyTrain <- Weekly[train, ]
WeeklyTest  <- Weekly[!train, ]

Logistic Regression with Lag2 and Lag3

glm.fit.alt <- glm(Direction ~ Lag2 + Lag3, 
                   data = WeeklyTrain, 
                   family = binomial)
glm.probs.alt <- predict(glm.fit.alt, WeeklyTest, type = "response")
glm.pred.alt <- ifelse(glm.probs.alt > 0.5, "Up", "Down")
acc.glm.alt <- mean(glm.pred.alt == WeeklyTest$Direction)

LDA with Lag2 and Lag3

lda.fit <- lda(Direction ~ Lag2 + Lag3, data = WeeklyTrain)
lda.pred <- predict(lda.fit, WeeklyTest)
acc.lda <- mean(lda.pred$class == WeeklyTest$Direction)

QDA with Lag2 and Lag3

qda.fit <- qda(Direction ~ Lag2 + Lag3, data = WeeklyTrain)
qda.pred <- predict(qda.fit, WeeklyTest)
acc.qda <- mean(qda.pred$class == WeeklyTest$Direction)

KNN with Lag2 and Lag3, and k=2

train.X <- as.matrix(WeeklyTrain[, c("Lag2", "Lag3")])
test.X  <- as.matrix(WeeklyTest[, c("Lag2", "Lag3")])
train.Direction <- WeeklyTrain$Direction

knn.pred <- knn(train.X, test.X, train.Direction, k = 2)
acc.knn <- mean(knn.pred == WeeklyTest$Direction)

Naive Bayes with Lag2 and Lag3

nb.fit <- naiveBayes(Direction ~ Lag2 + Lag3, data = WeeklyTrain)
nb.pred <- predict(nb.fit, WeeklyTest)
acc.nb <- mean(nb.pred == WeeklyTest$Direction)

All result

cat("Accuracy of new logistic regression model:", acc.glm.alt, "\n\n")

Accuracy of new logistic regression model: 0.625

cat("Accuracy of new LDA model:", acc.lda, "\n\n")

Accuracy of new LDA model: 0.625

cat("Accuracy of new QDA model:", acc.qda, "\n")

Accuracy of new QDA model: 0.6057692

cat("Accuracy of new KNN model:", acc.knn, "\n")

Accuracy of new KNN model: 0.4903846

cat("Accuracy of new naive Bayes model:", acc.nb, "\n")

Accuracy of new naive Bayes model: 0.5865385

🔴 Answer: After adding another variable Lag3 to the original model with only Lag2, both logistic regression and LDA models smaintained an accuracy of 62.5%, while the accuracy of QDA model increase from 58% to 61%, accuracy of new KNN (with K=2) model increase from 51% to 53%, and accuracy of naive Bayes remain at 58%, 52.9%, and 58.7%, respectively. This confirms that logistic regression and LDA continue to perform the best on this dataset. And adding other non-significant variable does not not enhance model performance.

Exercise 14

In this problem, you will develop a model to predict whether a given car gets high or low gas mileage based on the Auto data set.

attach(Auto)

The following object is masked from package:ggplot2:

    mpg

str(Auto)

'data.frame':   392 obs. of  9 variables:
 $ mpg         : num  18 15 18 16 17 15 14 14 14 15 ...
 $ cylinders   : int  8 8 8 8 8 8 8 8 8 8 ...
 $ displacement: num  307 350 318 304 302 429 454 440 455 390 ...
 $ horsepower  : int  130 165 150 150 140 198 220 215 225 190 ...
 $ weight      : int  3504 3693 3436 3433 3449 4341 4354 4312 4425 3850 ...
 $ acceleration: num  12 11.5 11 12 10.5 10 9 8.5 10 8.5 ...
 $ year        : int  70 70 70 70 70 70 70 70 70 70 ...
 $ origin      : int  1 1 1 1 1 1 1 1 1 1 ...
 $ name        : Factor w/ 304 levels "amc ambassador brougham",..: 49 36 231 14 161 141 54 223 241 2 ...
 - attr(*, "na.action")= 'omit' Named int [1:5] 33 127 331 337 355
  ..- attr(*, "names")= chr [1:5] "33" "127" "331" "337" ...

(a) Create a binary variable, mpg01, that contains a 1 if mpg contains a value above its median, and a 0 if mpg contains a value below its median. You can compute the median using the median() function. Note you may find it helpful to use the data.frame() function to create a single data set containing both mpg01 and the other Auto variables.

Auto$mpg01 <- ifelse(Auto$mpg > median(Auto$mpg), 1, 0)
Auto$mpg01 <- factor(Auto$mpg01)

(b) Explore the data graphically in order to investigate the associ- ation between mpg01 and the other features. Which of the other features seem most likely to be useful in predicting mpg01? Scatterplots and boxplots may be useful tools to answer this question. Describe your findings.

pairs(Auto)

create_boxplot <- function(data, yvar, title, convertYToFactor = FALSE) {
  if(convertYToFactor) {
    data <- data %>% mutate(across(all_of(yvar), as.factor))}
  ggplot(data, aes(x = mpg01, y = .data[[yvar]], fill = mpg01)) +
    geom_boxplot() + scale_fill_manual(values = c("0" = "lightpink", "1" = "lightblue")) + theme_classic() +labs(title = title, x = "mpg01", y = yvar)
}

p1 <- create_boxplot(Auto, "horsepower", "Horsepower vs. mpg01")
p2 <- create_boxplot(Auto, "weight", "Weight vs. mpg01")
p3 <- create_boxplot(Auto, "displacement", "Displacement vs. mpg01")
p4 <- ggplot(Auto, aes(x = mpg01, y = cylinders, fill = mpg01)) +
  geom_boxplot() +
  scale_fill_manual(values = c("0" = "lightpink", "1" = "lightblue")) +
  scale_y_continuous(breaks = c(3, 4, 5, 6, 8)) +
  theme_classic() +
  labs(title = "Cylinders vs mpg01",
       x = "mpg01 (0 = Low, 1 = High)",
       y = "Cylinders")
(p1 + p2) / (p3 + p4)

🔴 Answer: From the pairs plot, horsepower, weight, displacement, and cylinders are shown to be strongly related to mpg. The boxplots also show us that higher values of horsepower, weight, displacement, and cylinders are associated with mpg01 = 0 (lower mpg). These four features would be the most useful in predicting whether a car’s mpg is above or below the median.

(c) Split the data into a training set and a test set.

set.seed(1)
train_index <- sample(1:nrow(Auto), 0.7 * nrow(Auto))
AutoTrain <- Auto[train_index, ]
AutoTest  <- Auto[-train_index, ]
vars <- c("horsepower", "weight", "displacement", "cylinders")

(d) Perform LDA on the training data in order to predict mpg01 using the variables that seemed most associated with mpg01 in (b). What is the test error of the model obtained?

lda.fit <- lda(mpg01 ~ horsepower + weight + displacement + cylinders, 
               data = AutoTrain)
lda.pred <- predict(lda.fit, AutoTest)
conf.lda <- table(lda.pred$class, AutoTest$mpg01)
conf.lda

err.lda <- mean(lda.pred$class != AutoTest$mpg01)
err.lda

[1] 0.1186441

🔴 Answer: The LDA model has a test error rate of ~12%.

(e) Perform QDA on the training data in order to predict mpg01 using the variables that seemed most associated with mpg01 in (b). What is the test error of the model obtained?

qda.fit <- qda(mpg01 ~ horsepower + weight + displacement + cylinders, 
               data = AutoTrain)
qda.pred <- predict(qda.fit, AutoTest)
conf.qda <- table(qda.pred$class, AutoTest$mpg01)
conf.qda

err.qda <- mean(qda.pred$class != AutoTest$mpg01)
err.qda

[1] 0.1186441

🔴 Answer: The QDA model also has a test error rate of ~12%.

(f) Perform logistic regression on the training data in order to pre- dict mpg01 using the variables that seemed most associated with mpg01 in (b). What is the test error of the model obtained?

glm.fit <- glm(mpg01 ~ horsepower + weight + displacement + cylinders, 
               data = AutoTrain, family = binomial)
glm.probs <- predict(glm.fit, AutoTest, type = "response")
glm.pred <- ifelse(glm.probs > 0.5, 1, 0)
conf.glm <- table(glm.pred, AutoTest$mpg01)
conf.glm

        
glm.pred  0  1
       0 53  3
       1  8 54

err.glm <- mean(glm.pred != AutoTest$mpg01)
err.glm

[1] 0.09322034

🔴 Answer: The logistic regression model also has a test error rate of ~9%.

(g) Perform naive Bayes on the training data in order to predict mpg01 using the variables that seemed most associated with mpg01 in (b). What is the test error of the model obtained?

nb.fit <- naiveBayes(mpg01 ~ horsepower + weight + displacement + cylinders, 
                     data = AutoTrain)
nb.pred <- predict(nb.fit, AutoTest)
conf.nb <- table(nb.pred, AutoTest$mpg01)
conf.nb

       
nb.pred  0  1
      0 52  4
      1  9 53

err.nb <- mean(nb.pred != AutoTest$mpg01)
err.nb

[1] 0.1101695

🔴 Answer: The naive Bayes model also has a test error rate of ~11%.

(h) Perform KNN on the training data, with several values of K, in order to predict mpg01. Use only the variables that seemed most associated with mpg01 in (b). What test errors do you obtain? Which value of K seems to perform the best on this data set?

train.X <- as.matrix(AutoTrain[, vars])
test.X  <- as.matrix(AutoTest[, vars])
train.mpg01 <- AutoTrain$mpg01
k.values <- c(1, 3, 5, 7, 9)
for (k in k.values) {
  knn.pred <- knn(train.X, test.X, train.mpg01, k = k)
  conf.knn <- table(knn.pred, AutoTest$mpg01)
  err.knn <- mean(knn.pred != AutoTest$mpg01)
  
  cat("\nK =", k, ":\n")
  print(conf.knn)
  cat("Test Error =", err.knn, "\n")
}


K = 1 :
        
knn.pred  0  1
       0 51  6
       1 10 51
Test Error = 0.1355932 

K = 3 :
        
knn.pred  0  1
       0 52  4
       1  9 53
Test Error = 0.1101695 

K = 5 :
        
knn.pred  0  1
       0 51  5
       1 10 52
Test Error = 0.1271186 

K = 7 :
        
knn.pred  0  1
       0 50  4
       1 11 53
Test Error = 0.1271186 

K = 9 :
        
knn.pred  0  1
       0 49  5
       1 12 52
Test Error = 0.1440678

🔴 Answer: In KNN models, among each tested values of K, the KNN classifier achieves the lowest test error of about ~11% when K = 3.

Exercise 16

Using the Boston data set, fit classification models in order to predict whether a given census tract has a crime rate above or below the me- dian. Explore logistic regression, LDA, naive Bayes, and KNN models using various subsets of the predictors. Describe your findings. Hint: You will have to create the response variable yourself, using the variables that are contained in the Boston data set.

set.seed(1)

Load the Boston data

data(Boston)
Boston$crime01 <- factor(ifelse(Boston$crim > median(Boston$crim), 1, 0))
str(Boston)

'data.frame':   506 obs. of  15 variables:
 $ crim   : num  0.00632 0.02731 0.02729 0.03237 0.06905 ...
 $ zn     : num  18 0 0 0 0 0 12.5 12.5 12.5 12.5 ...
 $ indus  : num  2.31 7.07 7.07 2.18 2.18 2.18 7.87 7.87 7.87 7.87 ...
 $ chas   : int  0 0 0 0 0 0 0 0 0 0 ...
 $ nox    : num  0.538 0.469 0.469 0.458 0.458 0.458 0.524 0.524 0.524 0.524 ...
 $ rm     : num  6.58 6.42 7.18 7 7.15 ...
 $ age    : num  65.2 78.9 61.1 45.8 54.2 58.7 66.6 96.1 100 85.9 ...
 $ dis    : num  4.09 4.97 4.97 6.06 6.06 ...
 $ rad    : int  1 2 2 3 3 3 5 5 5 5 ...
 $ tax    : num  296 242 242 222 222 222 311 311 311 311 ...
 $ ptratio: num  15.3 17.8 17.8 18.7 18.7 18.7 15.2 15.2 15.2 15.2 ...
 $ black  : num  397 397 393 395 397 ...
 $ lstat  : num  4.98 9.14 4.03 2.94 5.33 ...
 $ medv   : num  24 21.6 34.7 33.4 36.2 28.7 22.9 27.1 16.5 18.9 ...
 $ crime01: Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...

Check correlation plot

pairs(Boston)

Boston$chas <- as.numeric(Boston$chas)
Boston$rad  <- as.numeric(Boston$rad)
other_vars <- setdiff(names(Boston), "crime01")
corr_scores <- cor(Boston$crim, Boston[, other_vars])
corr_scores

     crim         zn     indus        chas       nox         rm       age
[1,]    1 -0.2004692 0.4065834 -0.05589158 0.4209717 -0.2192467 0.3527343
            dis       rad       tax   ptratio      black     lstat       medv
[1,] -0.3796701 0.6255051 0.5827643 0.2899456 -0.3850639 0.4556215 -0.3883046

🔴 Answer: Higher crime rates are highly positively-correlated with higher indus, nox, age, rad, tax, and lstat. Furthermore, rad (0.63) and tax (0.58) shows the strongest correlations.

Split the data

train_index <- sample(1:nrow(Boston), round(0.7 * nrow(Boston)))
BostonTrain <- Boston[train_index, ]
BostonTest  <- Boston[-train_index, ]

Define the predictors

model_formula <- crime01 ~ rad + tax + nox + indus + lstat + age

Logistic Regression model

glm.fit <- glm(model_formula, data = BostonTrain, family = binomial)
glm.probs <- predict(glm.fit, BostonTest, type = "response")
glm.pred <- ifelse(glm.probs > 0.5, 1, 0)
glm.pred <- factor(glm.pred)
conf_glm <- table(glm.pred, BostonTest$crime01)
conf_glm

        
glm.pred  0  1
       0 61 12
       1 12 67

err_glm <- mean(glm.pred != BostonTest$crime01)
err_glm

[1] 0.1578947

vif(glm.fit)

     rad      tax      nox    indus    lstat      age 
1.507526 1.626011 3.635751 3.133313 1.339053 1.672006

🔴 Answer: The logistic regression model has a test error rate of ~11%. There is also no multicollinearity in the logistic regression model.

LDA model

lda.fit <- lda(model_formula, data = BostonTrain)
lda.pred <- predict(lda.fit, BostonTest)$class
conf_lda <- table(lda.pred, BostonTest$crime01)
conf_lda

        
lda.pred  0  1
       0 71 20
       1  2 59

err_lda <- mean(lda.pred != BostonTest$crime01)
err_lda

[1] 0.1447368

🔴 Answer: The LDA model has a test error rate of ~14%.

QDA model

qda.fit <- qda(model_formula, data = BostonTrain)
qda.pred <- predict(qda.fit, BostonTest)$class
conf_qda <- table(qda.pred, BostonTest$crime01)
conf_qda

        
qda.pred  0  1
       0 70 19
       1  3 60

err_qda <- mean(qda.pred != BostonTest$crime01)
err_qda

[1] 0.1447368

🔴 Answer: The QDA model has a test error rate of ~13%.

Naive Bayes model

nb.fit <- naiveBayes(model_formula, data = BostonTrain)
nb.pred <- predict(nb.fit, BostonTest)
conf_nb <- table(nb.pred, BostonTest$crime01)
conf_nb

       
nb.pred  0  1
      0 68 24
      1  5 55

err_nb <- mean(nb.pred != BostonTest$crime01)
err_nb

[1] 0.1907895

🔴 Answer: The naive Bayes model has a test error rate of ~16%.

KNN model with k=3

predictors <- c("rad", "tax", "nox", "indus", "lstat", "age")
train.X <- as.matrix(BostonTrain[, predictors])
test.X  <- as.matrix(BostonTest[, predictors])
train.Y <- BostonTrain$crime01
train.X <- scale(train.X)
test.X <- scale(test.X, center = attr(train.X, "scaled:center"), scale = attr(train.X, "scaled:scale"))
knn.pred <- knn(train.X, test.X, train.Y, k = 3)
conf_knn <- table(knn.pred, BostonTest$crime01)
conf_knn

        
knn.pred  0  1
       0 63  2
       1 10 77

err_knn <- mean(knn.pred != BostonTest$crime01)
err_knn

[1] 0.07894737

🔴 Answer: The KNN model with k=3 model has a test error rate of 6%.

🔴 Answer: Overall, KNN model with k=3 model provided the best performance on this data set using the selected predictors.