Assignment 3
Sunanda Das
2022-09-21
Chapter 4 Exercise 13
(a). Produce some numerical and graphical summaries of the Weekly data. Do there appear to be any patterns?
library(ISLR)
summary(Weekly)## Year Lag1 Lag2 Lag3
## Min. :1990 Min. :-18.1950 Min. :-18.1950 Min. :-18.1950
## 1st Qu.:1995 1st Qu.: -1.1540 1st Qu.: -1.1540 1st Qu.: -1.1580
## Median :2000 Median : 0.2410 Median : 0.2410 Median : 0.2410
## Mean :2000 Mean : 0.1506 Mean : 0.1511 Mean : 0.1472
## 3rd Qu.:2005 3rd Qu.: 1.4050 3rd Qu.: 1.4090 3rd Qu.: 1.4090
## Max. :2010 Max. : 12.0260 Max. : 12.0260 Max. : 12.0260
## Lag4 Lag5 Volume Today
## Min. :-18.1950 Min. :-18.1950 Min. :0.08747 Min. :-18.1950
## 1st Qu.: -1.1580 1st Qu.: -1.1660 1st Qu.:0.33202 1st Qu.: -1.1540
## Median : 0.2380 Median : 0.2340 Median :1.00268 Median : 0.2410
## Mean : 0.1458 Mean : 0.1399 Mean :1.57462 Mean : 0.1499
## 3rd Qu.: 1.4090 3rd Qu.: 1.4050 3rd Qu.:2.05373 3rd Qu.: 1.4050
## Max. : 12.0260 Max. : 12.0260 Max. :9.32821 Max. : 12.0260
## Direction
## Down:484
## Up :605
##
##
##
## library(corrplot)## corrplot 0.92 loadedcorrplot(cor(Weekly[,-9]), method="square")
- The only variable that appear to have any significant linear relation are Year and Volume. The correlational plot doesn’t illustrate that any other variables are linearly related.
(b). Use the full data set to perform a logistic regression with Direction as the response and the five lag variables plus Volume as predictors. Use the summary function to print the results. Do any of the predictors appear to be statistically significant? If so, which ones?
Weekly.fit<-glm(Direction~Lag1+Lag2+Lag3+Lag4+Lag5+Volume, data=Weekly,family='binomial')
summary(Weekly.fit)##
## Call:
## glm(formula = Direction ~ Lag1 + Lag2 + Lag3 + Lag4 + Lag5 +
## Volume, family = "binomial", data = Weekly)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -1.6949 -1.2565 0.9913 1.0849 1.4579
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 0.26686 0.08593 3.106 0.0019 **
## Lag1 -0.04127 0.02641 -1.563 0.1181
## Lag2 0.05844 0.02686 2.175 0.0296 *
## Lag3 -0.01606 0.02666 -0.602 0.5469
## Lag4 -0.02779 0.02646 -1.050 0.2937
## Lag5 -0.01447 0.02638 -0.549 0.5833
## Volume -0.02274 0.03690 -0.616 0.5377
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 1496.2 on 1088 degrees of freedom
## Residual deviance: 1486.4 on 1082 degrees of freedom
## AIC: 1500.4
##
## Number of Fisher Scoring iterations: 4- Out of the 6 predictors, at the level of significance α =0.05, only Lag2 (That is, the values from 2 weeks before the current value) seems to be statistically significant to predict the Direction. The estimate coefficient is of 0.058, which would mean that an increase of 1 in Lag2 represents an increase of e^0.058 = 1,06 in the odds of direction going up.
(c). Compute the confusion matrix and overall fraction of correct predictions. Explain what the confusion matrix is telling you about the types of mistakes made by logistic regression.
glm_probs_wk = predict(Weekly.fit, type = "response")
glm_pred_wk = rep("Down", length(glm_probs_wk))
glm_pred_wk[glm_probs_wk > 0.5] <- "Up"
table(glm_pred_wk, Weekly$Direction)##
## glm_pred_wk Down Up
## Down 54 48
## Up 430 557mean(glm_pred_wk == Weekly$Direction)## [1] 0.5610652- It appears as though 56.1% of the responses are predicted correctly.
- While the model correctly predicted the Up weekly trends 557/(48+557)=0.9207; 92.07% correct. In contrast Down weekly trends were predicted at a lower rate, 54/(430+54)=0.1115; or only 11.15% correctly predicted.
(d). Now fit the logistic regression model using a training data period from 1990 to 2008, with Lag2 as the only predictor. Compute the confusion matrix and the overall fraction of correct predictions for the held out data (that is, the data from 2009 and 2010).
training.data = Weekly[Weekly$Year<2009,]
test.data = Weekly[Weekly$Year>2008,]
simpglm = glm(Direction~Lag2, data= training.data, family = "binomial")
summary(simpglm)##
## Call:
## glm(formula = Direction ~ Lag2, family = "binomial", data = training.data)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -1.536 -1.264 1.021 1.091 1.368
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 0.20326 0.06428 3.162 0.00157 **
## Lag2 0.05810 0.02870 2.024 0.04298 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 1354.7 on 984 degrees of freedom
## Residual deviance: 1350.5 on 983 degrees of freedom
## AIC: 1354.5
##
## Number of Fisher Scoring iterations: 4logistic_probs <- predict(simpglm, test.data)
logistic_pred = rep("Down", length(test.data$Direction))
logistic_pred[logistic_probs > 0.5] <- "Up"
table(logistic_pred, test.data$Direction)##
## logistic_pred Down Up
## Down 41 56
## Up 2 5mean(logistic_pred == test.data$Direction)## [1] 0.4423077- When spliting up the whole Weekly dataset into a training and test dataset, the model correctly predicted weekly trends at rate of 62.5%, which is a moderate improvement from the model that utilized the whole dataset.
- Also this model such as the previous one did better at predicting upward trends(91.80%) compared to downward trends(20.93%); although this model was able to improve significantly on correctly predicting downward trends.
(e). Repeat (d) using LDA
library(MASS)
lda_wkly <- lda(Direction ~ Lag2, data = training.data)
lda_probs <- predict(lda_wkly, test.data)
table(lda_probs$class, test.data$Direction)##
## Down Up
## Down 9 5
## Up 34 56mean(lda_probs$class == test.data$Direction)## [1] 0.625- Using Linear Discriminant Analysis to develop a classifying model yielded similar results as the logistic regression model created in part D.
(f). Repeat (d) using QDA.
qda_wkly <- qda(Direction ~ Lag2, data = training.data)
qda_pred <- predict(qda_wkly, test.data)
table(qda_pred$class, test.data$Direction)##
## Down Up
## Down 0 0
## Up 43 61mean(qda_pred$class == test.data$Direction)## [1] 0.5865385- Quadratic Linear Analysis created a model with an accuracy of 58.65%, which is lower than the previous methods. Also this model only considered predicting the correctness of weekly upward trends disregrading the downward weekly trends.
(g). Repeat (d) using KNN with K = 1.
library(class)
train_10 <- (Weekly$Year< 2008)
Weekly.train <- Weekly[train_10,]
Weekly.test <- Weekly[!train_10,]
train.x_10 <- matrix(Weekly$Lag2[train_10])
test.x_10 <- matrix(Weekly$Lag2[!train_10])
train.direction_10 <- Weekly$Direction[train_10]
set.seed(1)
knn_pred_10 <- knn(train.x_10, test.x_10, train.direction_10, k=1)
table(knn_pred_10, Weekly.test$Direction)##
## knn_pred_10 Down Up
## Down 32 38
## Up 40 46mean(knn_pred_10== Weekly.test$Direction)## [1] 0.5- The K-Nearest neighbors resulted in a classifying model with an accuracy rate of 50% which is equal to random chance.
(h). Repeat (d) using naive Bayes.
library(naivebayes)## Warning: package 'naivebayes' was built under R version 4.2.1## naivebayes 0.9.7 loadednaive_wkly <- naive_bayes(Direction ~ Lag2, data = Weekly)
naive.pred2 = predict(naive_wkly, newdata=test.data)## Warning: predict.naive_bayes(): more features in the newdata are provided as
## there are probability tables in the object. Calculation is performed based on
## features to be found in the tables.mean(naive.pred2 == test.data$Direction)## [1] 0.5865385- using the naive bayes model, our predictions were correct 58.65% of the time; which is the same as the qda model.
(i). Which of these methods appears to provide the best results on this data? * The methods that have the highest accuracy rates are the Logistic Regression and Linear Discriminant Analysis; both having rates of 62.5%.
(j). Experiment with different combinations of predictors, including possible transformations and interactions, for each of the methods. Report the variables, method, and associated confusion matrix that appears to provide the best results on the held out data. Note that you should also experiment with values for K in the KNN classifier.
qda.fit2 = qda(Direction~Lag1 + Lag2 + Lag4, data= training.data)
qda.pred2 = predict(qda.fit2, newdata=test.data, type="response")
qda.class2 = qda.pred2$class
table(qda.class2, test.data$Direction)##
## qda.class2 Down Up
## Down 9 20
## Up 34 41mean(qda.class2 == test.data$Direction)## [1] 0.4807692- When using the QDA model to compare the relationship between Lag 1, Lag 2 and Lag 4, we see that the QDA model correctly predicted the market would go up on 41 days and down on 9 days. This means that our predictions were correct 48.08% of the time.
lda.fit2 = lda(Direction~Lag1 + Lag2 + Lag4, data= training.data)
lda.pred2 = predict(lda.fit2, newdata=test.data, type="response")
lda.class2 = lda.pred2$class
table(lda.class2, test.data$Direction)##
## lda.class2 Down Up
## Down 9 7
## Up 34 54mean(lda.class2 == test.data$Direction)## [1] 0.6057692- When using the LDA model to compare the relationship between Lag 1, Lag 2 and Lag 4, we see that the LDA model correctly predicted the market would go up on 54 days and down on 9 days. This means that our predictions were correct 60.6% of the time.
Exercise 14
(a). Create a binary variable, mpg01, that contains a 1 if mpg contains a value above its median, and a 0 if mpg contains a value below its median. You can compute the median using the median() function. Note you may find it helpful to use the data.frame() function to create a single data set containing both mpg01 and the other Auto variables
library(dplyr)##
## Attaching package: 'dplyr'## The following object is masked from 'package:MASS':
##
## select## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionAuto <- as_tibble(ISLR::Auto)
Auto_mpg01 <- Auto %>%
mutate(mpg01 = ifelse(mpg > median(mpg), 1, 0))(b). Explore the data graphically in order to investigate the association between mpg01 and the other features. Which of the other features seem most likely to be useful in predicting mpg01? Scatterplots and boxplots may be useful tools to answer this question. Describe your findings.
library(tidyverse)## Warning: package 'tidyverse' was built under R version 4.2.1## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
## ✔ ggplot2 3.3.6 ✔ purrr 0.3.4
## ✔ tibble 3.1.6 ✔ stringr 1.4.0
## ✔ tidyr 1.2.0 ✔ forcats 0.5.2
## ✔ readr 2.1.2## Warning: package 'readr' was built under R version 4.2.1## Warning: package 'forcats' was built under R version 4.2.1## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ✖ dplyr::select() masks MASS::select()Auto_mpg01 %>%
ggplot(aes(cut_number(mpg01, 2), displacement)) +
geom_boxplot()
Auto_mpg01 %>%
ggplot(aes(cut_number(mpg01, 2), horsepower)) +
geom_boxplot()
Auto_mpg01 %>%
ggplot(aes(cut_number(mpg01, 2), weight)) +
geom_boxplot()
Auto_mpg01 %>%
ggplot(aes(cut_number(mpg01, 2), acceleration)) +
geom_boxplot()
ggplot(Auto_mpg01) +
geom_count(aes(cut_number(mpg01, 2), cylinders))
ggplot(Auto_mpg01) +
geom_boxplot(aes(cut_number(mpg01, 2), cylinders))
Auto_mpg01 %>%
ggplot(aes(cut_number(mpg01, 2), mpg)) +
geom_boxplot()
* To predict mpg01, can use mpg obviously. there seems to be neg corr
with cylnders and displacement, horsepower, and weight but positive
correlation with acceleration.
(c). Split the data into a training set and a test set.
train <- (Auto_mpg01$year %% 2 == 0)
train_auto <- Auto_mpg01[train,]
test_auto <- Auto_mpg01[!train,](d). Perform LDA on the training data in order to predict mpg01 using the variables that seemed most associated with mpg01 in (b). What is the test error of the model obtained?
# LDA Models
lda_auto <- lda(mpg01 ~ cylinders + displacement + horsepower + weight,
data = Auto_mpg01,
subset = train)
lda_auto_pred <- predict(lda_auto, test_auto)
table(lda_auto_pred$class, test_auto$mpg01)##
## 0 1
## 0 86 9
## 1 14 73# test error rate is
1 - mean(lda_auto_pred$class == test_auto$mpg01)## [1] 0.1263736- Using LDA method to create a classifying model resulted in a test error rate of 12.6%.
(e). Perform QDA on the training data in order to predict mpg01 using the variables that seemed most associated with mpg01 in (b). What is the test error of the model obtained?
qda_auto <- qda(mpg01 ~ cylinders + displacement + horsepower + weight,
data = Auto_mpg01,
subset = train)
qda_auto_pred <- predict(qda_auto, test_auto)
table(qda_auto_pred$class, test_auto$mpg01)##
## 0 1
## 0 89 13
## 1 11 69# test error rate is
1 - mean(qda_auto_pred$class == test_auto$mpg01)## [1] 0.1318681- Using QDA method to create a classifying model resulted in a test error rate of 13.2%.
(f). Perform logistic regression on the training data in order to predict mpg01 using the variables that seemed most associated with mpg01 in (b). What is the test error of the model obtained?
logistic_auto <- glm(mpg01 ~ cylinders + displacement + horsepower + weight,
data = Auto_mpg01,
subset = train,
family = binomial)
logistic_probs_auto <- predict(logistic_auto, test_auto, type = "response")
logistic_pred_auto <- rep(0,length(test_auto$mpg01))
logistic_pred_auto[logistic_probs_auto > 0.5] <- 1
table(logistic_pred_auto, test_auto$mpg01)##
## logistic_pred_auto 0 1
## 0 89 11
## 1 11 71# test error rate is
1 - mean(logistic_pred_auto == test_auto$mpg01)## [1] 0.1208791- Using Logistic Regression method to create a classifying model resulted in a test error rate of 12.09%.
(g). Perform naive Bayes on the training data in order to predict mpg01 using the variables that seemed most associated with mpg01 in (b). What is the test error of the model obtained?
naive_auto <- naive_bayes(as.factor(mpg01) ~ cylinders + displacement + horsepower + weight, data = Auto_mpg01, subset = train)
naive_auto_pred <- predict(naive_auto, test_auto)## Warning: predict.naive_bayes(): more features in the newdata are provided as
## there are probability tables in the object. Calculation is performed based on
## features to be found in the tables.table(naive_auto_pred, test_auto$mpg01)##
## naive_auto_pred 0 1
## 0 88 11
## 1 12 71# test error rate is
1 - mean(naive_auto_pred == test_auto$mpg01)## [1] 0.1263736(h). Perform KNN on the training data, with several values of K, in order to predict mpg01. Use only the variables that seemed most associated with mpg01 in (b). What test errors do you obtain? Which value of K seems to perform the best on this data set?
train_auto_X <- cbind(
Auto_mpg01$cylinders,
Auto_mpg01$displacement,
Auto_mpg01$horsepower,
Auto_mpg01$weight
)[train,]
test_auto_X <- cbind(
Auto_mpg01$cylinders,
Auto_mpg01$displacement,
Auto_mpg01$horsepower,
Auto_mpg01$weight
)[!train,]
set.seed(1)
knn_auto1 <- knn(train_auto_X, test_auto_X, train_auto$mpg01, k = 1)
table(knn_auto1, test_auto$mpg01)##
## knn_auto1 0 1
## 0 83 11
## 1 17 71#test error
1 - mean(knn_auto1 == test_auto$mpg01)## [1] 0.1538462knn_auto10 <- knn(train_auto_X, test_auto_X, train_auto$mpg01, k = 10)
table(knn_auto10, test_auto$mpg01)##
## knn_auto10 0 1
## 0 77 7
## 1 23 75#test error
1 - mean(knn_auto10 == test_auto$mpg01)## [1] 0.1648352knn_auto100 <- knn(train_auto_X, test_auto_X, train_auto$mpg01, k = 100)
table(knn_auto100, test_auto$mpg01)##
## knn_auto100 0 1
## 0 81 7
## 1 19 75#test error
1 - mean(knn_auto100 == test_auto$mpg01)## [1] 0.1428571knn_auto5 <- knn(train_auto_X, test_auto_X, train_auto$mpg01, k = 5)
table(knn_auto5, test_auto$mpg01)##
## knn_auto5 0 1
## 0 82 9
## 1 18 73#test error
1 - mean(knn_auto5 == test_auto$mpg01)## [1] 0.1483516- K=5 performed better.
Exercise 16
predicting whether a given census tract has a crime rate above or below the median.
attach(Boston)
median_crime = median(crim)
#We will create crim_lvl variable that takes on two values: "0" or "1".
#"0" if crime rate below median and "1" if above median.
crim_lvl <- rep(0, 506)
crim_lvl[crim > median_crime] = 1
crim_lvl <- as.factor(crim_lvl)
Boston_2 <- data.frame(Boston, crim_lvl)Set up Training and Test Sets
set.seed(1)
train_13 <- rbinom(506, 1, 0.7)
Boston_2 <- cbind(Boston_2, train_13)
Boston.train <- Boston_2[train_13 == 1,]
Boston.test <- Boston_2[train_13 == 0,]Logistic Regression
log_13_fits <- glm(crim_lvl~nox + age + dis + medv, data = Boston.train, family = binomial)
summary(log_13_fits)##
## Call:
## glm(formula = crim_lvl ~ nox + age + dis + medv, family = binomial,
## data = Boston.train)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -2.17404 -0.35742 0.00278 0.26418 2.53635
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -24.900404 4.019591 -6.195 5.84e-10 ***
## nox 38.426015 5.859800 6.558 5.47e-11 ***
## age 0.016253 0.009966 1.631 0.1029
## dis 0.309361 0.164026 1.886 0.0593 .
## medv 0.087237 0.028230 3.090 0.0020 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 499.02 on 359 degrees of freedom
## Residual deviance: 210.55 on 355 degrees of freedom
## AIC: 220.55
##
## Number of Fisher Scoring iterations: 7- Looking at the fitted model, age is not a significant predictor, so we will remove it from the model.
log_13_fits <- glm(crim_lvl~nox + dis + medv, data = Boston.train, family = binomial)
summary(log_13_fits)##
## Call:
## glm(formula = crim_lvl ~ nox + dis + medv, family = binomial,
## data = Boston.train)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -2.19958 -0.39388 0.00252 0.26563 2.48475
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -23.94414 3.89175 -6.153 7.63e-10 ***
## nox 39.78032 5.77733 6.886 5.75e-12 ***
## dis 0.23393 0.15697 1.490 0.13615
## medv 0.07713 0.02678 2.880 0.00398 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 499.02 on 359 degrees of freedom
## Residual deviance: 213.25 on 356 degrees of freedom
## AIC: 221.25
##
## Number of Fisher Scoring iterations: 7- dis has now become another un-significant predictor after the removal of age. So we will remove it until only significant predictors are left.
log_13_fits <- glm(crim_lvl~nox + medv, data = Boston.train, family = binomial)
summary(log_13_fits)##
## Call:
## glm(formula = crim_lvl ~ nox + medv, family = binomial, data = Boston.train)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -2.17657 -0.38729 0.00523 0.30375 2.65695
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -19.74864 2.42833 -8.133 4.2e-16 ***
## nox 33.97633 3.88025 8.756 < 2e-16 ***
## medv 0.06605 0.02524 2.617 0.00887 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 499.02 on 359 degrees of freedom
## Residual deviance: 215.41 on 357 degrees of freedom
## AIC: 221.41
##
## Number of Fisher Scoring iterations: 6- Now we have our final set of predictors.
log_13_prob <- predict(log_13_fits, Boston.test, type = 'response')
log_13_preds <- rep(0, 146)
log_13_preds[log_13_prob > 0.5] = 1
dat <- matrix(data=table(log_13_preds, Boston.test$crim_lvl), nrow=2, ncol=2,
dimnames=list(c("Below median", "Above median"), c("Below", "Above")))
names(dimnames(dat)) <- c("predicted", "observed")
print(dat)## observed
## predicted Below Above
## Below median 63 16
## Above median 12 55#Error rate:
(16+12)/146## [1] 0.1917808- The logistic model selected considered nox and medv as the predictors. The other predictors were taken out as they were not statistically significant predictors. The model states that as the nox variable increases (nitrogen oxides concentration), our predicted probability of above-median-crime-rate for that neighborhood increases. The model also states that as the median value of owner-occupied homes increases then the predicted probability for a high crime-rate also increases, however the effect of this variable on the probability is significant less than that of the nox variable. The final error rate of this model with a threshold of 50% was 19.2%
LDA
lda_13_fits <- lda(crim_lvl~nox + age+ dis+medv, data = Boston_2, subset= (train_13==1))
lda_13_preds <- predict(lda_13_fits, Boston.test)
lda_13_class <- lda_13_preds$class
dat <- matrix(data=table(lda_13_class, Boston.test$crim_lvl), nrow=2, ncol=2,
dimnames=list(c("Below median", "Above median"), c("Below", "Above")))
names(dimnames(dat)) <- c("predicted", "observed")
print(dat)## observed
## predicted Below Above
## Below median 62 14
## Above median 13 57#Error rate:
(14 + 13)/146## [1] 0.1849315- The LDA model states that the nox, age, and medv variables have a positive correlation with the probability of a high crime rate (above median). The dis variable has a negative correlation with the predicted probablity. Using the LDA model, the error rate is slightly lower than that of the logistic model with the rate at 18.5%.
KNN For k=3
train.x_13 <- cbind(Boston.train$nox, Boston.train$tax, Boston.train$pratio)
test.x_13 <- cbind(Boston.test$nox, Boston.test$tax, Boston.test$pratio)
set.seed(1)
knn_pred_13 <- knn(train.x_13, test.x_13, Boston.train$crim_lvl, k=3)
table(knn_pred_13, Boston.test$crim_lvl)##
## knn_pred_13 0 1
## 0 71 5
## 1 4 66#Error rate:
(4 + 5)/146## [1] 0.06164384For k=5
knn_pred_13_2 <- knn(train.x_13, test.x_13, Boston.train$crim_lvl, k=5)
table(knn_pred_13_2, Boston.test$crim_lvl)##
## knn_pred_13_2 0 1
## 0 69 5
## 1 6 66#Error rate:
(5 + 6)/146## [1] 0.07534247- We ran to KNN models, one with a k of 3 and one with a k of 5. The error rates were 6.2% and 7.5% respectively.This model outperforms the Logistic Regression and LDA models developed in the examples above. The best model is the KNN model with a k of 3. However, the difficulty with the KNN approach is that it doesn’t tell us which predictors are important and how they can affect the probability of our outcome. What we can take away from this model is that if we want to lower the crime rate of a community, we should look at what low-crime communities are doing. Based on our variables selected, we should look at that low-crime communities are doing in terms of air pollution, tax rates/tax structures, and school funding.