Assignment 02
Aranzazu Chaparro Molina - wvg029
2/18/2022
Exercise 2
Carefully explain the differences between the KNN classifier and KNN regression methods.
The main difference between them is that the KNN classifier method is a method used for discrete (categorical) response variables, and the KNN regression method is used more for a continuous response variable.
Also the main goal of the KNN regression method is to predict the value of the output variable by using a local average, while KNN classification tries to predict the class to which the response variable belongs by calculating the probability for class j to happen.
Exercise 9 - Auto
a) Produce a scatterplot matrix which includes all of the variables in the data set.
Auto <- read_csv("Auto.csv")## Rows: 397 Columns: 9## -- Column specification --------------------------------------------------------
## Delimiter: ","
## chr (2): horsepower, name
## dbl (7): mpg, cylinders, displacement, weight, acceleration, year, origin##
## i Use `spec()` to retrieve the full column specification for this data.
## i Specify the column types or set `show_col_types = FALSE` to quiet this message.str(Auto)## spec_tbl_df [397 x 9] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
## $ mpg : num [1:397] 18 15 18 16 17 15 14 14 14 15 ...
## $ cylinders : num [1:397] 8 8 8 8 8 8 8 8 8 8 ...
## $ displacement: num [1:397] 307 350 318 304 302 429 454 440 455 390 ...
## $ horsepower : chr [1:397] "130" "165" "150" "150" ...
## $ weight : num [1:397] 3504 3693 3436 3433 3449 ...
## $ acceleration: num [1:397] 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ...
## $ year : num [1:397] 70 70 70 70 70 70 70 70 70 70 ...
## $ origin : num [1:397] 1 1 1 1 1 1 1 1 1 1 ...
## $ name : chr [1:397] "chevrolet chevelle malibu" "buick skylark 320" "plymouth satellite" "amc rebel sst" ...
## - attr(*, "spec")=
## .. cols(
## .. mpg = col_double(),
## .. cylinders = col_double(),
## .. displacement = col_double(),
## .. horsepower = col_character(),
## .. weight = col_double(),
## .. acceleration = col_double(),
## .. year = col_double(),
## .. origin = col_double(),
## .. name = col_character()
## .. )
## - attr(*, "problems")=<externalptr>Auto$horsepower <- as.numeric(Auto$horsepower)## Warning: NAs introduced by coercionsum(is.na(Auto$horsepower))## [1] 5Auto <- na.omit(Auto)pairs(Auto[,1:8], pch = 20)
b) Compute the matrix of correlations between the variables using the function cor(). You will need to exclude the name variable, which is qualitative.
cor(Auto[,1:8])## mpg cylinders displacement horsepower weight
## mpg 1.0000000 -0.7776175 -0.8051269 -0.7784268 -0.8322442
## cylinders -0.7776175 1.0000000 0.9508233 0.8429834 0.8975273
## displacement -0.8051269 0.9508233 1.0000000 0.8972570 0.9329944
## horsepower -0.7784268 0.8429834 0.8972570 1.0000000 0.8645377
## weight -0.8322442 0.8975273 0.9329944 0.8645377 1.0000000
## acceleration 0.4233285 -0.5046834 -0.5438005 -0.6891955 -0.4168392
## year 0.5805410 -0.3456474 -0.3698552 -0.4163615 -0.3091199
## origin 0.5652088 -0.5689316 -0.6145351 -0.4551715 -0.5850054
## acceleration year origin
## mpg 0.4233285 0.5805410 0.5652088
## cylinders -0.5046834 -0.3456474 -0.5689316
## displacement -0.5438005 -0.3698552 -0.6145351
## horsepower -0.6891955 -0.4163615 -0.4551715
## weight -0.4168392 -0.3091199 -0.5850054
## acceleration 1.0000000 0.2903161 0.2127458
## year 0.2903161 1.0000000 0.1815277
## origin 0.2127458 0.1815277 1.0000000c) Use the lm() function to perform a multiple linear regression with mpg as the response and all other variables except name as the predictors. Use the summary() function to print the results.
auto_lm = lm(mpg ~ .-name, data = Auto)summary(auto_lm)##
## Call:
## lm(formula = mpg ~ . - name, data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.5903 -2.1565 -0.1169 1.8690 13.0604
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -17.218435 4.644294 -3.707 0.00024 ***
## cylinders -0.493376 0.323282 -1.526 0.12780
## displacement 0.019896 0.007515 2.647 0.00844 **
## horsepower -0.016951 0.013787 -1.230 0.21963
## weight -0.006474 0.000652 -9.929 < 2e-16 ***
## acceleration 0.080576 0.098845 0.815 0.41548
## year 0.750773 0.050973 14.729 < 2e-16 ***
## origin 1.426141 0.278136 5.127 4.67e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.328 on 384 degrees of freedom
## Multiple R-squared: 0.8215, Adjusted R-squared: 0.8182
## F-statistic: 252.4 on 7 and 384 DF, p-value: < 2.2e-16d) Use the plot() function to produce diagnostic plots of the linear regression fit. Comment on any problems you see with the fit. Do the residual plots suggest any unusually large outliers? Does the leverage plot identify any observations with unusually high leverage?
par(mfrow=c(2,2))
plot(auto_lm)
par(mfrow=c(1,1))At first glance we can see that the distribution of the data is normal (Normal Q-Q), with some outliers. However, with the Residuals vs Fitted we can see that the Homoscedasticity assumption might not be met. For the Cooks distance (Leverage plot), it will really depend on the threshold that we want to assume, but again, the same outliers found in the Normal Q-Q can be observed outside the -2 , 2 range of the plot. Point 14 seems to have a high leverage.
e) Use the * and : symbols to fit linear regression models with interaction effects. Do any interactions appear to be statistically significant?
We will include the Interaction effect of: displacement, weight, year and origin
auto_lm2 = lm(mpg ~ cylinders * displacement + displacement * weight, data = Auto)summary(auto_lm2)##
## Call:
## lm(formula = mpg ~ cylinders * displacement + displacement *
## weight, data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -13.2934 -2.5184 -0.3476 1.8399 17.7723
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 5.262e+01 2.237e+00 23.519 < 2e-16 ***
## cylinders 7.606e-01 7.669e-01 0.992 0.322
## displacement -7.351e-02 1.669e-02 -4.403 1.38e-05 ***
## weight -9.888e-03 1.329e-03 -7.438 6.69e-13 ***
## cylinders:displacement -2.986e-03 3.426e-03 -0.872 0.384
## displacement:weight 2.128e-05 5.002e-06 4.254 2.64e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.103 on 386 degrees of freedom
## Multiple R-squared: 0.7272, Adjusted R-squared: 0.7237
## F-statistic: 205.8 on 5 and 386 DF, p-value: < 2.2e-16We can see that the interaction that is significant is displacement and weight, while cylinders and displacement is not.
f) Try a few different transformations of the variables, such as log(X), √X, X2. Comment on your findings.
auto_lm3 = lm(mpg~sqrt(weight)+log(horsepower)+acceleration+I(acceleration^2), data = Auto)summary(auto_lm3)##
## Call:
## lm(formula = mpg ~ sqrt(weight) + log(horsepower) + acceleration +
## I(acceleration^2), data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -10.7338 -2.4273 -0.1604 2.1804 15.5506
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 122.51698 9.29220 13.185 < 2e-16 ***
## sqrt(weight) -0.44050 0.06744 -6.531 2.05e-10 ***
## log(horsepower) -12.42799 1.92756 -6.448 3.39e-10 ***
## acceleration -1.97369 0.60125 -3.283 0.00112 **
## I(acceleration^2) 0.04986 0.01788 2.788 0.00556 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.97 on 387 degrees of freedom
## Multiple R-squared: 0.7439, Adjusted R-squared: 0.7412
## F-statistic: 281 on 4 and 387 DF, p-value: < 2.2e-16Exercise 10 - Carseats
a) Fit a multiple regression model to predict Sales using Price, Urban, and US.
data(Carseats)
head(Carseats)## Sales CompPrice Income Advertising Population Price ShelveLoc Age Education
## 1 9.50 138 73 11 276 120 Bad 42 17
## 2 11.22 111 48 16 260 83 Good 65 10
## 3 10.06 113 35 10 269 80 Medium 59 12
## 4 7.40 117 100 4 466 97 Medium 55 14
## 5 4.15 141 64 3 340 128 Bad 38 13
## 6 10.81 124 113 13 501 72 Bad 78 16
## Urban US
## 1 Yes Yes
## 2 Yes Yes
## 3 Yes Yes
## 4 Yes Yes
## 5 Yes No
## 6 No Yesstr(Carseats)## 'data.frame': 400 obs. of 11 variables:
## $ Sales : num 9.5 11.22 10.06 7.4 4.15 ...
## $ CompPrice : num 138 111 113 117 141 124 115 136 132 132 ...
## $ Income : num 73 48 35 100 64 113 105 81 110 113 ...
## $ Advertising: num 11 16 10 4 3 13 0 15 0 0 ...
## $ Population : num 276 260 269 466 340 501 45 425 108 131 ...
## $ Price : num 120 83 80 97 128 72 108 120 124 124 ...
## $ ShelveLoc : Factor w/ 3 levels "Bad","Good","Medium": 1 2 3 3 1 1 3 2 3 3 ...
## $ Age : num 42 65 59 55 38 78 71 67 76 76 ...
## $ Education : num 17 10 12 14 13 16 15 10 10 17 ...
## $ Urban : Factor w/ 2 levels "No","Yes": 2 2 2 2 2 1 2 2 1 1 ...
## $ US : Factor w/ 2 levels "No","Yes": 2 2 2 2 1 2 1 2 1 2 ...car_lm = lm(Sales ~ Price+Urban+US, data= Carseats)summary(car_lm)##
## Call:
## lm(formula = Sales ~ Price + Urban + US, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.9206 -1.6220 -0.0564 1.5786 7.0581
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.043469 0.651012 20.036 < 2e-16 ***
## Price -0.054459 0.005242 -10.389 < 2e-16 ***
## UrbanYes -0.021916 0.271650 -0.081 0.936
## USYes 1.200573 0.259042 4.635 4.86e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.472 on 396 degrees of freedom
## Multiple R-squared: 0.2393, Adjusted R-squared: 0.2335
## F-statistic: 41.52 on 3 and 396 DF, p-value: < 2.2e-16b) Provide an interpretation of each coefficient in the model. Be careful—some of the variables in the model are qualitative!
We can see that when price increases by 1 dollar and other predictors stay constant, sales decrease by 0.0544 units sales. That means that when price increases by 1 dollar, the number of carseats sold decrease by 5.44%.
When it comes to whether the store is in a Urban area or not, it doesn’t affect, but it does affect that it is in the US, so a store in the US will sell 120% more than other stores not in the US.
c) Write out the model in equation form, being careful to handle the qualitative variables properly.
\(Sales = 13.043469 -0.054459Price-0.021916Urban_{Yes}+1.200573XUS_{Yes}\)
d) For which of the predictors can you reject the null hypothesis H0 : βj = 0?
For the predictor Urban, since its p-value is > 0.05 (0.936)
e) On the basis of your response to the previous question, fit a smaller model that only uses the predictors for which there is evidence of association with the outcome.
car_lm2 = lm(Sales ~ Price+US, data= Carseats)summary(car_lm2)##
## Call:
## lm(formula = Sales ~ Price + US, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.9269 -1.6286 -0.0574 1.5766 7.0515
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.03079 0.63098 20.652 < 2e-16 ***
## Price -0.05448 0.00523 -10.416 < 2e-16 ***
## USYes 1.19964 0.25846 4.641 4.71e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.469 on 397 degrees of freedom
## Multiple R-squared: 0.2393, Adjusted R-squared: 0.2354
## F-statistic: 62.43 on 2 and 397 DF, p-value: < 2.2e-16f) How well do the models in (a) and (e) fit the data?
Bad. To compare how they perform we will use the Adjusted R-squared, which for the model on (a) is: 0.2335 , and for the model in (e) is: 0.2354, which will suggest a 0.24% improve in the prediction. In conclusion, taking off 1 of the predictors doesn’t improve the prediction, and both of them perform badly.
g) Using the model from (e), obtain 95% confidence intervals for the coefficient(s).
confint(car_lm2)## 2.5 % 97.5 %
## (Intercept) 11.79032020 14.27126531
## Price -0.06475984 -0.04419543
## USYes 0.69151957 1.70776632h) Is there evidence of outliers or high leverage observations in the model from (e)?
To calculate the cutoff level or upper limit for the influence points, we will do the following formula: \(\frac{(p+1)}{n}\) which is \(\frac{(2+1)}{400} = 0.0075\).
par(mfrow=c(2,2))
plot(car_lm2)
summary(influence.measures(car_lm2))## Potentially influential observations of
## lm(formula = Sales ~ Price + US, data = Carseats) :
##
## dfb.1_ dfb.Pric dfb.USYs dffit cov.r cook.d hat
## 26 0.24 -0.18 -0.17 0.28_* 0.97_* 0.03 0.01
## 29 -0.10 0.10 -0.10 -0.18 0.97_* 0.01 0.01
## 43 -0.11 0.10 0.03 -0.11 1.05_* 0.00 0.04_*
## 50 -0.10 0.17 -0.17 0.26_* 0.98 0.02 0.01
## 51 -0.05 0.05 -0.11 -0.18 0.95_* 0.01 0.00
## 58 -0.05 -0.02 0.16 -0.20 0.97_* 0.01 0.01
## 69 -0.09 0.10 0.09 0.19 0.96_* 0.01 0.01
## 126 -0.07 0.06 0.03 -0.07 1.03_* 0.00 0.03_*
## 160 0.00 0.00 0.00 0.01 1.02_* 0.00 0.02
## 166 0.21 -0.23 -0.04 -0.24 1.02 0.02 0.03_*
## 172 0.06 -0.07 0.02 0.08 1.03_* 0.00 0.02
## 175 0.14 -0.19 0.09 -0.21 1.03_* 0.02 0.03_*
## 210 -0.14 0.15 -0.10 -0.22 0.97_* 0.02 0.01
## 270 -0.03 0.05 -0.03 0.06 1.03_* 0.00 0.02
## 298 -0.06 0.06 -0.09 -0.15 0.97_* 0.01 0.00
## 314 -0.05 0.04 0.02 -0.05 1.03_* 0.00 0.02_*
## 353 -0.02 0.03 0.09 0.15 0.97_* 0.01 0.00
## 357 0.02 -0.02 0.02 -0.03 1.03_* 0.00 0.02
## 368 0.26 -0.23 -0.11 0.27_* 1.01 0.02 0.02_*
## 377 0.14 -0.15 0.12 0.24 0.95_* 0.02 0.01
## 384 0.00 0.00 0.00 0.00 1.02_* 0.00 0.02
## 387 -0.03 0.04 -0.03 0.05 1.02_* 0.00 0.02
## 396 -0.05 0.05 0.08 0.14 0.98_* 0.01 0.00R shows some observations that violate various rules for each influence measure..
outyling.obs = c(26,29,43,50,51,58,69,126,160,166,172,175,210,270,298,314,353,357,368,377,384,387,396)
Carseats_new = Carseats[-outyling.obs,]
car_lm3 = lm(Sales~Price+US,data=Carseats_new)summary(car_lm3)##
## Call:
## lm(formula = Sales ~ Price + US, data = Carseats_new)
##
## Residuals:
## Min 1Q Median 3Q Max
## -5.263 -1.605 -0.039 1.590 5.428
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 12.925232 0.665259 19.429 < 2e-16 ***
## Price -0.053973 0.005511 -9.794 < 2e-16 ***
## USYes 1.255018 0.248856 5.043 7.15e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.29 on 374 degrees of freedom
## Multiple R-squared: 0.2387, Adjusted R-squared: 0.2347
## F-statistic: 58.64 on 2 and 374 DF, p-value: < 2.2e-16Even when the influential observations are removed, there’s not much change on the model nor the prediction (Adjusted R-squared).
Exercise 12 - Carseats
This problem involves simple linear regression without an intercept.
a) Recall that the coefficient estimate ˆ β for the linear regression of Y onto X without an intercept is given by (3.38). Under what circumstance is the coefficient estimate for the regression of X onto Y the same as the coefficient estimate for the regression of Y onto X?
When the variance(x)equals variance(y), if the variances are equal (maybe because you standardized the variables first), then so are the standard deviations. If that is the case, then they would have and equal Pearson’s r.
b) Generate an example in R with n = 100 observations in which the coefficient estimate for the regression of X onto Y is different from the coefficient estimate for the regression of Y onto X.
x = rnorm(100)
y = x^2
coefficients(lm(x ~ y))## (Intercept) y
## 0.009943457 -0.048667585coefficients(lm(y ~ x))## (Intercept) x
## 1.0875434 -0.1026285c) Generate an example in R with n = 100 observations in which the coefficient estimate for the regression of X onto Y is the same as the coefficient estimate for the regression of Y onto X.
x2 = rnorm(100)
y2 = x2
coefficients(lm(x2 ~ y2))## (Intercept) y2
## 0 1coefficients(lm(y2 ~ x2))## (Intercept) x2
## 0 1
Comment on the output. For instance:
i. Is there a relationship between the predictors and the response?
The test p-value is close to zero (< 2.2e-16) which suggests there is evidence to reject the null Hypothesis and assume that there is a relationship between the predictors and the response.
ii. Which predictors appear to have a statistically significant relationship to the response?
There is a significant relationship between the predictors: displacement, weight, year and origin, with the response variable mpg. However, cylinders can be converted to a factor variable and repeat the Multiple Regression Level to see if any of those levels would be significant, but for this report, we won’t do it.
iii. What does the coefficient for the year variable suggest?
The coefficient for year is: 0.750773, which suggests that there is almost a 1 to 1 effect on Y with 1 unit of increase OF X(year), in other words, the older the car (more years), the higher the mpg in a proportion of 0.750773.