Assignment 2

library(tidyverse)
library(openintro)
library(ISLR)

Question 2

2. Carefully explain the differences between the KNN classifier and KNN regression methods.

The biggest difference is that the KNN classifier method is used to solve qualitative questions whereas KNN regression method is used to solve quantitative questions. In addition to that, the KNN classifier method estimates the conditional probability compared to the KNN regression method which estimates the zero of the function.

Question 9

(a) Produce a scatterplot matrix which includes all of the variables in the data set.

pairs(Auto)

(b) Compute the matrix of correlations between the variables using the function cor(). You will need to exclude the name variable, cor() which is qualitative.

names(Auto)

## [1] "mpg"          "cylinders"    "displacement" "horsepower"   "weight"      
## [6] "acceleration" "year"         "origin"       "name"

cor(Auto[1:8])

##                     mpg  cylinders displacement horsepower     weight
## mpg           1.0000000 -0.7776175   -0.8051269 -0.7784268 -0.8322442
## cylinders    -0.7776175  1.0000000    0.9508233  0.8429834  0.8975273
## displacement -0.8051269  0.9508233    1.0000000  0.8972570  0.9329944
## horsepower   -0.7784268  0.8429834    0.8972570  1.0000000  0.8645377
## weight       -0.8322442  0.8975273    0.9329944  0.8645377  1.0000000
## acceleration  0.4233285 -0.5046834   -0.5438005 -0.6891955 -0.4168392
## year          0.5805410 -0.3456474   -0.3698552 -0.4163615 -0.3091199
## origin        0.5652088 -0.5689316   -0.6145351 -0.4551715 -0.5850054
##              acceleration       year     origin
## mpg             0.4233285  0.5805410  0.5652088
## cylinders      -0.5046834 -0.3456474 -0.5689316
## displacement   -0.5438005 -0.3698552 -0.6145351
## horsepower     -0.6891955 -0.4163615 -0.4551715
## weight         -0.4168392 -0.3091199 -0.5850054
## acceleration    1.0000000  0.2903161  0.2127458
## year            0.2903161  1.0000000  0.1815277
## origin          0.2127458  0.1815277  1.0000000

(c) Use the lm() function to perform a multiple linear regression with mpg as the response and all other variables except name as the predictors. Use the summary() function to print the results. Comment on the output

model= lm(mpg~. -name, data = Auto)
summary(model)

## 
## Call:
## lm(formula = mpg ~ . - name, data = Auto)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -9.5903 -2.1565 -0.1169  1.8690 13.0604 
## 
## Coefficients:
##                Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  -17.218435   4.644294  -3.707  0.00024 ***
## cylinders     -0.493376   0.323282  -1.526  0.12780    
## displacement   0.019896   0.007515   2.647  0.00844 ** 
## horsepower    -0.016951   0.013787  -1.230  0.21963    
## weight        -0.006474   0.000652  -9.929  < 2e-16 ***
## acceleration   0.080576   0.098845   0.815  0.41548    
## year           0.750773   0.050973  14.729  < 2e-16 ***
## origin         1.426141   0.278136   5.127 4.67e-07 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.328 on 384 degrees of freedom
## Multiple R-squared:  0.8215, Adjusted R-squared:  0.8182 
## F-statistic: 252.4 on 7 and 384 DF,  p-value: < 2.2e-16

i. Is there a relationship between the predictors and the response?

We predicted that the overall p-value is <2.2e-16 which shows our model is significant and idnicates there is a relationship present between X and Y, or our predictor and response.

ii. Which predictors appear to have a statistically significant relationship to the response?

Displacement, weight, year, and origin appear to be statistically significant because of their low p-value with values less than .05.

iii. What does the coefficient for the year variable suggest?

The coefficient for the year variable is 0.750773, which indicates that mpg increases about 0.75 per year.

(d) Use the plot() function to produce diagnostic plots of the linear regression fit. Comment on any problems you see with the fit. Do the residual plots suggest any unusually large outliers? Does the leverage plot identify any observations with unusually high leverage?

par(mfrow = c(2,2))
plot(model)

Based off the first graph there appears to be non-linearity in our model. In the last graph, we see that there don’t appear to be any large outliers and based off the Cook’s D plotting we don’t see any values with unusually high leverage.

(e) Use the * and : symbols to fit linear regression models with interaction effects. Do any interactions appear to be statistically significant?

model2 = lm(mpg ~ cylinders * displacement + displacement * weight + acceleration * year + horsepower * acceleration + horsepower * weight, data = Auto[, 1:8])
summary(model2)

## 
## Call:
## lm(formula = mpg ~ cylinders * displacement + displacement * 
##     weight + acceleration * year + horsepower * acceleration + 
##     horsepower * weight, data = Auto[, 1:8])
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -9.2694 -1.5465  0.0983  1.3882 11.6711 
## 
## Coefficients:
##                           Estimate Std. Error t value Pr(>|t|)    
## (Intercept)              1.018e+02  2.204e+01   4.621 5.24e-06 ***
## cylinders                7.422e-01  6.390e-01   1.162  0.24612    
## displacement            -5.020e-02  1.761e-02  -2.850  0.00461 ** 
## weight                  -1.045e-02  1.090e-03  -9.588  < 2e-16 ***
## acceleration            -6.508e+00  1.344e+00  -4.841 1.88e-06 ***
## year                    -5.974e-01  2.656e-01  -2.249  0.02507 *  
## horsepower              -1.034e-01  6.166e-02  -1.677  0.09443 .  
## cylinders:displacement  -1.311e-03  2.830e-03  -0.463  0.64355    
## displacement:weight      1.353e-05  6.804e-06   1.988  0.04755 *  
## acceleration:year        8.710e-02  1.658e-02   5.254 2.48e-07 ***
## acceleration:horsepower -2.564e-03  1.965e-03  -1.305  0.19271    
## weight:horsepower        2.746e-05  1.318e-05   2.083  0.03789 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.808 on 380 degrees of freedom
## Multiple R-squared:  0.8742, Adjusted R-squared:  0.8706 
## F-statistic:   240 on 11 and 380 DF,  p-value: < 2.2e-16

After testing a few possible interactions, we see that there is no known relationship between cylinders and displacement or between acceleration and horsepower. However, there does appear to be a relationship between displacement and weight, acceleration and year, and weight and horsepower because their corresponding p-values are significant.

(f) Try a few different transformations of the variables, such as log(X), √ X, X2. Comment on your findings.

par(mfrow = c(2, 2))
plot(log(Auto$weight), Auto$acceleration)
plot(sqrt(Auto$weight), Auto$acceleration)
plot((Auto$weight)^2, Auto$acceleration)

par(mfrow = c(2, 2))

plot(log(Auto$mpg), Auto$acceleration)
plot(sqrt(Auto$mpg), Auto$acceleration)
plot((Auto$mpg)^2, Auto$acceleration)

par(mfrow = c(2, 2))

plot(log(Auto$cylinders), Auto$acceleration)
plot(sqrt(Auto$cylinders), Auto$acceleration)
plot((Auto$cylinders)^2, Auto$acceleration)

I took a deeper look at the acceleration variable and some of the variables that originally did not appear to have a relationship with acceleration (those being weight, mpg, and cylinders). After applying differing transformations to each secondary variable, there still does not appear to be any relationships between the variables and acceleration.

Question 10

(a) Fit a multiple regression model to predict Sales using Price, Urban, and US.

library(ISLR)
attach(Carseats)
fit1<-lm(Sales~Price+Urban+US)
summary(fit1)

## 
## Call:
## lm(formula = Sales ~ Price + Urban + US)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -6.9206 -1.6220 -0.0564  1.5786  7.0581 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 13.043469   0.651012  20.036  < 2e-16 ***
## Price       -0.054459   0.005242 -10.389  < 2e-16 ***
## UrbanYes    -0.021916   0.271650  -0.081    0.936    
## USYes        1.200573   0.259042   4.635 4.86e-06 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.472 on 396 degrees of freedom
## Multiple R-squared:  0.2393, Adjusted R-squared:  0.2335 
## F-statistic: 41.52 on 3 and 396 DF,  p-value: < 2.2e-16

(b) Provide an interpretation of each coefficient in the model. Be careful—some of the variables in the model are qualitative!

Both Urban and US are qualitative variables, in this regression model we are using the values of both Urban and US being “yes”. Knowing that, we continue to look at the regression model. The coefficient for the price variable tells us that as price increases by 1, then sales will decrease by-.054459. The coefficient for the variable Urban is not taken into consideration here because it does not appear to have a relationship with sales based off the large p value. Lastly, the coefficient for US tells us that sales in the US are 1.200573 more than sales not in the US.

(c) Write out the model in equation form, being careful to handle the qualitative variables properly.

Sales= 13.04 - 0.0545∗price - 0.0219∗UrbanYes + 1.2006∗USYes

(d) For which of the predictors can you reject the null hypothesis H0 : βj = 0?

We can reject the bull hypothesis for the predictor variable UrbanYes, because the p-value of 0.936 is not significant in this model.

(e) On the basis of your response to the previous question, fit a smaller model that only uses the predictors for which there is evidence of association with the outcome.

fit2<-lm(Sales~Price+US, data= Carseats)
summary(fit2)

## 
## Call:
## lm(formula = Sales ~ Price + US, data = Carseats)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -6.9269 -1.6286 -0.0574  1.5766  7.0515 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 13.03079    0.63098  20.652  < 2e-16 ***
## Price       -0.05448    0.00523 -10.416  < 2e-16 ***
## USYes        1.19964    0.25846   4.641 4.71e-06 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.469 on 397 degrees of freedom
## Multiple R-squared:  0.2393, Adjusted R-squared:  0.2354 
## F-statistic: 62.43 on 2 and 397 DF,  p-value: < 2.2e-16

(f) How well do the models in (a) and (e) fit the data?

When looking at their R-squared values of 0.2393, we can say that the models poorly fit the data. The R-squared suggests only 23.93% of the variability in the model can be explained.

(g) Using the model from (e), obtain 95 % confidence intervals for the coefficient(s).

confint(fit2)

##                   2.5 %      97.5 %
## (Intercept) 11.79032020 14.27126531
## Price       -0.06475984 -0.04419543
## USYes        0.69151957  1.70776632

(h) Is there evidence of outliers or high leverage observations in the model from (e)?

par(mfrow=c(2,2))
plot(fit2)

There do not appear to be any outliers based on the Normal Q-Q plot. As well as there does not appear to be any high leverage observations based on the Residuals vs. Leverage plot because nothing extends beyond the Cook’s D limits.

Question 12

(a) Recall that the coefficient estimate βˆ for the linear regression of Y onto X without an intercept is given by (3.38). Under what circumstance is the coefficient estimate for the regression of X onto Y the same as the coefficient estimate for the regression of Y onto X?

The coefficient estimate is the same when the sum of squared y values equals the sum of squared x values.

(b) Generate an example in R with n = 100 observations in which the coefficient estimate for the regression of X onto Y is different from the coefficient estimate for the regression of Y onto X.

x <- rnorm (100)
y <- x^2 
coefficients(lm(x ~ y))

## (Intercept)           y 
##  0.11757422  0.02160038

coefficients(lm(y ~ x))

## (Intercept)           x 
##  0.94121427  0.04116177

(c) Generate an example in R with n = 100 observations in which the coefficient estimate for the regression of X onto Y is the same as the coefficient estimate for the regression of Y onto X.

x <- rnorm(100)
y <- x
coefficients(lm(x ~ y))

## (Intercept)           y 
##           0           1

coefficients(lm(y ~ x))

## (Intercept)           x 
##           0           1

…