ISLR Assignment 2
Chapter 3 (page 120) Problems 2, 9, 10, 12
Misty Stultz
Due 2/18/2022
Problem 2
Carefully explain the differences between the kNN classifier and kNN regression methods.
kNN classification is attempting to predict a the category that a response variable will fall into. kNN regression is attempting to predcit the value of the response variable.
Problem 9
This question involves the use of multiple linear regression on the Auto data set.
(a) Produce a scatterplot matrix which includes all of the variables in the data set.
#View(Auto)
plot(Auto)
(b) Compute the matrix of correlations between the variables using the function cor(). You will need to exclude the name variable, which is qualitative.
matcor <- cor(Auto[,1:8]) %>%
round(digits = 2)
print(matcor)## mpg cylinders displacement horsepower weight acceleration year
## mpg 1.00 -0.78 -0.81 -0.78 -0.83 0.42 0.58
## cylinders -0.78 1.00 0.95 0.84 0.90 -0.50 -0.35
## displacement -0.81 0.95 1.00 0.90 0.93 -0.54 -0.37
## horsepower -0.78 0.84 0.90 1.00 0.86 -0.69 -0.42
## weight -0.83 0.90 0.93 0.86 1.00 -0.42 -0.31
## acceleration 0.42 -0.50 -0.54 -0.69 -0.42 1.00 0.29
## year 0.58 -0.35 -0.37 -0.42 -0.31 0.29 1.00
## origin 0.57 -0.57 -0.61 -0.46 -0.59 0.21 0.18
## origin
## mpg 0.57
## cylinders -0.57
## displacement -0.61
## horsepower -0.46
## weight -0.59
## acceleration 0.21
## year 0.18
## origin 1.00(c) Use the lm() function to perform a multiple linear regression with mpg as the response and all other variables except name as the predictors. Use the summary() function to print the results. Comment on the output. For instance:
(i) Is there a relationship between the predictors and the response?
(ii) Which predictors appear to have a statistically significant relationship to the response?
(iii) What does the coefficient for the year variable suggest?
lm.mpg.all <- lm(mpg ~ ., data = Auto[,1:8])
summary(lm.mpg.all)##
## Call:
## lm(formula = mpg ~ ., data = Auto[, 1:8])
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.5903 -2.1565 -0.1169 1.8690 13.0604
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -17.218435 4.644294 -3.707 0.00024 ***
## cylinders -0.493376 0.323282 -1.526 0.12780
## displacement 0.019896 0.007515 2.647 0.00844 **
## horsepower -0.016951 0.013787 -1.230 0.21963
## weight -0.006474 0.000652 -9.929 < 2e-16 ***
## acceleration 0.080576 0.098845 0.815 0.41548
## year 0.750773 0.050973 14.729 < 2e-16 ***
## origin 1.426141 0.278136 5.127 4.67e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.328 on 384 degrees of freedom
## Multiple R-squared: 0.8215, Adjusted R-squared: 0.8182
## F-statistic: 252.4 on 7 and 384 DF, p-value: < 2.2e-16vif(lm.mpg.all)## cylinders displacement horsepower weight acceleration year
## 10.737535 21.836792 9.943693 10.831260 2.625806 1.244952
## origin
## 1.772386Is there a relationship between the predictors and the response?
Testing the hypotheses:
H0: βj = 0
Ha: βj ≠ 0
Looking at the F-Statistic (252.4) and corresponding p-value (2.2e-166), we can determine that there is a statistically significant relationship between the predictor and at least one of the response variables. The R2 of 0.8215 suggests that our current model explains approximately 82% of variation in the response variable.
Which predictors appear to have a statistically significant relationship to the response?
Based on the p-values associated with each variable, origin (4.67e-07), year (< 2e-16), weight (< 2e-16), and displacement (0.00844) are statistically significant.
What does the coefficient for the year variable suggest?
There is a positive relationship between year and gas mileage. For each year newer the car is there is an expected ~0.75 mpg increase in gas mileage.
(d) Use the plot() function to produce diagnostic plots of the linear regression fit. Comment on any problems you see with the fit. Do the residual plots suggest any unusually large outliers? Does the leverage plot identify any observations with unusually high leverage?
par(mfrow = c(2,2))
plot(lm.mpg.all) 
A linear model does not appear to bea perfect fit for this data. The plot of the residuals vs fitted shows a slight curve in the data inconsistent with a linear model. ased on the plot of the residuals vs leverage there may be at least one significant outlier (point 14). This can also be seen in the Normal Q-Q plot where the residuals start deviating from the line of normality.
(e) Use the * and : symbols to fit linear regression models with interaction effects. Do any interactions appear to be statistically significant?
lm.inter1 <- lm(mpg ~ weight * year, data = Auto[ ,1:8])
summary(lm.inter1)##
## Call:
## lm(formula = mpg ~ weight * year, data = Auto[, 1:8])
##
## Residuals:
## Min 1Q Median 3Q Max
## -8.0397 -1.9956 -0.0983 1.6525 12.9896
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -1.105e+02 1.295e+01 -8.531 3.30e-16 ***
## weight 2.755e-02 4.413e-03 6.242 1.14e-09 ***
## year 2.040e+00 1.718e-01 11.876 < 2e-16 ***
## weight:year -4.579e-04 5.907e-05 -7.752 8.02e-14 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.193 on 388 degrees of freedom
## Multiple R-squared: 0.8339, Adjusted R-squared: 0.8326
## F-statistic: 649.3 on 3 and 388 DF, p-value: < 2.2e-16lm.inter2 <- lm(mpg ~ acceleration * horsepower, data = Auto[ ,1:8])
summary(lm.inter2)##
## Call:
## lm(formula = mpg ~ acceleration * horsepower, data = Auto[, 1:8])
##
## Residuals:
## Min 1Q Median 3Q Max
## -13.3442 -2.7324 -0.4049 2.4210 15.8840
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 33.512440 3.420187 9.798 < 2e-16 ***
## acceleration 0.800296 0.211899 3.777 0.000184 ***
## horsepower 0.017590 0.027425 0.641 0.521664
## acceleration:horsepower -0.015698 0.002003 -7.838 4.45e-14 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.426 on 388 degrees of freedom
## Multiple R-squared: 0.6809, Adjusted R-squared: 0.6784
## F-statistic: 275.9 on 3 and 388 DF, p-value: < 2.2e-16The interaction between weight and year (the two most significant variables in the original model) appears to be significant. But when interactiong weight with other variables it appears that there may be some multi-collinearities.
I also looked at how the interaction between acceleration and horsepower would look since they were two variables that were not statistically significant by themselves and discovered that horsepower was still insignificant on its own, but the interaction with acceleration gave it a strong significance.
(f) Try a few different transformations of the variables, such as log(X), √X, X2. Comment on your findings.
Auto2 <- Auto
Auto2 <- transform(Auto2, mpg.log = log(mpg))
Auto2 <- transform(Auto2, mpg.sq = I(mpg^2))
Auto2 <- transform(Auto2, mpg.sqrt = sqrt(mpg))
par(mfrow = c(2,2))
plot(Auto2$mpg.log, main = "Log")
plot(Auto2$mpg.sqrt, main = "Square Root")
plot(Auto2$mpg.sq, main = "Squared")
lm.tran1 <- lm(mpg.sqrt ~ weight + log(weight), data = Auto2[ ,-c(9:11)])
#summary(lm.tran1)
#lm.fit1 <- lm(mpg.sq ~ weight, Auto2[ ,-c(9:11)])
#anova(lm.fit1,lm.tran1)
lm.tran2 <- lm(mpg.sqrt ~ horsepower + sqrt(horsepower), data = Auto2[ ,-c(9:11)])
#summary(lm.tran2)
#lm.fit2 <- lm(mpg.sq ~ horsepower, Auto2[ ,-c(9:11)])
#anova(lm.fit2, lm.tran2)
lm.tran3 <- lm(mpg.sqrt ~ year + log(year), data = Auto2[ ,-c(9:11)])
#summary(lm.tran3)
#lm.fit3 <- lm(mpg.sq ~ year, Auto2[ ,-c(9:11)])
#anova(lm.fit3, lm.tran3)
lm.tran4 <- lm(mpg.sqrt ~ I(weight^2) + sqrt(horsepower) + log(year) + . - mpg - year - horsepower - weight, data = Auto2[ ,-c(9:11)])
summary(lm.tran4)##
## Call:
## lm(formula = mpg.sqrt ~ I(weight^2) + sqrt(horsepower) + log(year) +
## . - mpg - year - horsepower - weight, data = Auto2[, -c(9:11)])
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.03371 -0.20364 -0.01189 0.18783 1.03549
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -1.427e+01 1.702e+00 -8.383 9.86e-16 ***
## I(weight^2) -4.929e-08 9.567e-09 -5.152 4.14e-07 ***
## sqrt(horsepower) -1.816e-01 2.925e-02 -6.207 1.40e-09 ***
## log(year) 5.046e+00 3.703e-01 13.626 < 2e-16 ***
## cylinders -7.811e-02 3.099e-02 -2.521 0.01212 *
## displacement 1.093e-03 7.137e-04 1.532 0.12640
## acceleration -3.374e-02 9.787e-03 -3.447 0.00063 ***
## origin 1.519e-01 2.648e-02 5.737 1.95e-08 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3218 on 384 degrees of freedom
## Multiple R-squared: 0.8423, Adjusted R-squared: 0.8394
## F-statistic: 293 on 7 and 384 DF, p-value: < 2.2e-16
By looking at the scatterplot matrix my initial thoughts are that the variables mpg, horsepower, weight, and year might need some form of transformation.
After transforming and graphing mpg, the sqrt transformation appears to be the most linear.
After transforming and comparing weight the sqauared transformation appears to yield the best results.
After transforming and comparing horsepower the sqrt transformation appears to yield the best results.
After transforming and comparing year the log transformation appears to yield the best results.
When combining all of the transformations with the rest of the variables we end up with an R2 of 0.8423. This indicates that the model explains more of the variation than in the data than the original model. Additional testing is needed to see if the model can be improved, but this is a good start.
Problem 10
This question should be answered using the Carseats data set.
(a) Fit a multiple regression model to predict Sales using Price, Urban, and US.
attach(Carseats)
#View(Carseats)
#summary(Carseats)
carseat.sales1 <- lm(Sales ~ Price + Urban + US)
summary(carseat.sales1)##
## Call:
## lm(formula = Sales ~ Price + Urban + US)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.9206 -1.6220 -0.0564 1.5786 7.0581
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.043469 0.651012 20.036 < 2e-16 ***
## Price -0.054459 0.005242 -10.389 < 2e-16 ***
## UrbanYes -0.021916 0.271650 -0.081 0.936
## USYes 1.200573 0.259042 4.635 4.86e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.472 on 396 degrees of freedom
## Multiple R-squared: 0.2393, Adjusted R-squared: 0.2335
## F-statistic: 41.52 on 3 and 396 DF, p-value: < 2.2e-16(b) Provide an interpretation of each coefficient in the model. Be careful—some of the variables in the model are qualitative!
Sales units are not specified, so there is no way to know if the figure is US dollars or some other unit. For that reason I will only refer to sales in “units” and not a specific currency.
Price: There is a negative relationship between the price and sales of carseats. For every 1 unit in price increased, the sales decrease by approximately 0.05 (thousand) units.
Urban: There is a negative relationship between the sales and the store being located in an urban location. If the store is located in an urban location there will be a decreasin in sales by approximately 0.02 (thousand) units.
US: There is a positive relationship betwen sales and the the store being located in the United States. When the store is located in the US the sales are higher by approximately 1.2 (thousand) units.
(c) Write out the model in equation form, being careful to handle the qualitative variables properly.
Y = 13.043469 - 0.054459X1 - 0.021916X2 + 1.200573X3 + \({\epsilon}\)
Where:
Y = Sales
X1 = Price
X2 = Urban (Store is located in an urban location), Yes = 1, No = 0
X3 = US (Store is located in the United States), Yes = 1, No = 0
(d) For which of the predictors can you reject the null hypothesis H0:βj=0?
With p-values near zero, we can reject the null hypothesis for the US and Price variables.
(e) On the basis of your response to the previous question, fit a smaller model that only uses the predictors for which there is evidence of association with the outcome.
carseat.sales2 <- lm(Sales ~ Price + US)
summary(carseat.sales2)##
## Call:
## lm(formula = Sales ~ Price + US)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.9269 -1.6286 -0.0574 1.5766 7.0515
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.03079 0.63098 20.652 < 2e-16 ***
## Price -0.05448 0.00523 -10.416 < 2e-16 ***
## USYes 1.19964 0.25846 4.641 4.71e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.469 on 397 degrees of freedom
## Multiple R-squared: 0.2393, Adjusted R-squared: 0.2354
## F-statistic: 62.43 on 2 and 397 DF, p-value: < 2.2e-16(f) How well do the models in (a) and (e) fit the data?
Neither of the models fit the data well. They dropping Urban from the model does not make much of a change to the R2, and both models only fit about 24% of the variability in the data.
(g) Using the model from (e), obtain 95% confidence intervals for the coefficient(s).
confint(carseat.sales2)## 2.5 % 97.5 %
## (Intercept) 11.79032020 14.27126531
## Price -0.06475984 -0.04419543
## USYes 0.69151957 1.70776632(h) Is there evidence of outliers or high leverage observations in the model from (e)?
par(mfrow = c(2,2))
plot(carseat.sales2)
There looks like there may be a few outliers in the data set based on the diagnostic plots, but more investigation would need to be done to ensure that they are true outliers. It is more likely that the model needs additional variables than there being a presence of outliers.
Problem 12
This problem involves simple linear regression without an intercept.
(a) Recall that the coefficient estimate \(\hat{\beta}\) for the linear regression of Y onto X without an intercept is given by (3.38). Under what circumstance is the coefficient estimate for the regression of X onto Y the same as the coefficient estimate for the regression of Y onto X?
The coefficient estimate for Y onto X is:
\(\hat{\beta} = \frac{\sum_{i} x_iy_i}{\sum_{i} {x^2}_j}\)
The coefficient estimate for X onto Y is:
\(\hat{\beta}' = \frac{\sum_{i} x_iy_i}{\sum_{i} {y^2}_j}\)
Therefore they can only be the same if:
\(\sum_{i} {{x^2}_j} = \sum_{i} {{y^2}_j}\)
(b) Generate an example in R with n = 100 observations in which the coefficient estimate for the regression of X onto Y is different from the coefficient estimate for the regression of Y onto X.
x <- rnorm(100)
sum(x^2)## [1] 101.9209y <- 2*rnorm(100)
sum(y^2)## [1] 369.5895diff1 <- lm(y ~ x)
diff2 <- lm(x ~ y)
coef(diff1)## (Intercept) x
## -0.3157175 -0.1961199coef(diff2)## (Intercept) y
## -0.06391473 -0.05537423(c) Generate an example in R with n = 100 observations in which the coefficient estimate for the regression of X onto Y is the same as the coefficient estimate for the regression of Y onto X.
x <- rnorm(100)
y <- x^1
same1 <- lm(x ~ y)
same2 <- lm(y ~ x)
coef(same1)## (Intercept) y
## 3.330669e-17 1.000000e+00coef(same2)## (Intercept) x
## 3.330669e-17 1.000000e+00detach(Carseats)