Assignment #2
Yuan Torres
2023-02-04
Problem 2
Carefully explain the differences between the KNN classifier
and KNN regression methods.
KNN regression is used in quantitative situations.
KNN classifier is used in qualitative situations.
Problem 9
This question involves the use of multiple linear regression on the Auto data set.
library(tidyverse)
library (MASS)
library (ISLR2)
library (ISLR)
auto <- read.csv("Auto.csv",header = T, na.strings = "?")
view(auto)(a) Produce a scatterplot matrix which includes all of the variables in the data set.
head(auto)## mpg cylinders displacement horsepower weight acceleration year origin
## 1 18 8 307 130 3504 12.0 70 1
## 2 15 8 350 165 3693 11.5 70 1
## 3 18 8 318 150 3436 11.0 70 1
## 4 16 8 304 150 3433 12.0 70 1
## 5 17 8 302 140 3449 10.5 70 1
## 6 15 8 429 198 4341 10.0 70 1
## name
## 1 chevrolet chevelle malibu
## 2 buick skylark 320
## 3 plymouth satellite
## 4 amc rebel sst
## 5 ford torino
## 6 ford galaxie 500plot(auto)
(b) Compute the matrix of correlations between the variables using the function cor(). You will need to exclude the name variable, cor() which is qualitative.
auto1 <- auto
auto1$name=NULL
cor(auto1)## mpg cylinders displacement horsepower weight
## mpg 1.0000000 -0.7762599 -0.8044430 NA -0.8317389
## cylinders -0.7762599 1.0000000 0.9509199 NA 0.8970169
## displacement -0.8044430 0.9509199 1.0000000 NA 0.9331044
## horsepower NA NA NA 1 NA
## weight -0.8317389 0.8970169 0.9331044 NA 1.0000000
## acceleration 0.4222974 -0.5040606 -0.5441618 NA -0.4195023
## year 0.5814695 -0.3467172 -0.3698041 NA -0.3079004
## origin 0.5636979 -0.5649716 -0.6106643 NA -0.5812652
## acceleration year origin
## mpg 0.4222974 0.5814695 0.5636979
## cylinders -0.5040606 -0.3467172 -0.5649716
## displacement -0.5441618 -0.3698041 -0.6106643
## horsepower NA NA NA
## weight -0.4195023 -0.3079004 -0.5812652
## acceleration 1.0000000 0.2829009 0.2100836
## year 0.2829009 1.0000000 0.1843141
## origin 0.2100836 0.1843141 1.0000000(c) Use the lm() function to perform a multiple linear
regression with mpg as the response and all other variables except name
as the predictors. Use the summary() function to print the results.
Comment on the output. For instance:
1. Is there a relationship between the predictors and the response? Yes,
there is. There are multiple predictors that have a relationship with
the response because their associated p-value is significant. The
R-squared value implies that around 82% of the changes in the response
can be explained by the predictors.
2. Which predictors appear to have a statistically significant
relationship to the response?
Displacement, weight, year, and origin have a statistically significant
relationship with the response by showing significant p-values.
3. What does the coefficient for the year variable suggest?
The coefficient of “year” is significant and positive, which suggests
that if all other variables are constant then on average the response
variable” mpg” increases by 0.75 every year.
lm.auto <- lm(mpg~.-name, data=auto)
#or# lm.auto1 <- lm(mpg~.,data=auto1)
summary(lm.auto)##
## Call:
## lm(formula = mpg ~ . - name, data = auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.5903 -2.1565 -0.1169 1.8690 13.0604
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -17.218435 4.644294 -3.707 0.00024 ***
## cylinders -0.493376 0.323282 -1.526 0.12780
## displacement 0.019896 0.007515 2.647 0.00844 **
## horsepower -0.016951 0.013787 -1.230 0.21963
## weight -0.006474 0.000652 -9.929 < 2e-16 ***
## acceleration 0.080576 0.098845 0.815 0.41548
## year 0.750773 0.050973 14.729 < 2e-16 ***
## origin 1.426141 0.278136 5.127 4.67e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.328 on 384 degrees of freedom
## (5 observations deleted due to missingness)
## Multiple R-squared: 0.8215, Adjusted R-squared: 0.8182
## F-statistic: 252.4 on 7 and 384 DF, p-value: < 2.2e-16(d) Use the plot() function to produce diagnostic plots of
the linear regression fit. Comment on any problems you see with the fit.
Do the residual plots suggest any unusually large outliers? Does the
leverage plot identify any observations with unusually high leverage?
From the residual plots and leverage plot, we don’t see any large
outliers or high leverage. The Scale-Location plot displays if there are
outliers in the data. The data will be an outlier if standardized
residual is outside the range of [-3, 3]. Based on this graph, there
don’t seem to be any outliers because all values are within the range of
[0,2]. The Residuals vs Leverage plot shows observations that have high
leverage points. The Cook’s distance is shown with the dashed red line.
Points that are above the Cook’s distance are high leverage points.
Based on the Residuals vs. Leverage graph, there are no observations
that provide a high leverage.
par(mfrow = c(2,2))
plot(lm.auto)
(e) Use the * and : symbols to fit linear regression models
with interaction effects. Do any interactions appear to be statistically
significant?
Statistically significant interactions:
Displacement: year, acceleration: year, acceleration : origin
inter.auto1 <- lm(mpg~.*.,data=auto1)
summary(inter.auto1)##
## Call:
## lm(formula = mpg ~ . * ., data = auto1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.6303 -1.4481 0.0596 1.2739 11.1386
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.548e+01 5.314e+01 0.668 0.50475
## cylinders 6.989e+00 8.248e+00 0.847 0.39738
## displacement -4.785e-01 1.894e-01 -2.527 0.01192 *
## horsepower 5.034e-01 3.470e-01 1.451 0.14769
## weight 4.133e-03 1.759e-02 0.235 0.81442
## acceleration -5.859e+00 2.174e+00 -2.696 0.00735 **
## year 6.974e-01 6.097e-01 1.144 0.25340
## origin -2.090e+01 7.097e+00 -2.944 0.00345 **
## cylinders:displacement -3.383e-03 6.455e-03 -0.524 0.60051
## cylinders:horsepower 1.161e-02 2.420e-02 0.480 0.63157
## cylinders:weight 3.575e-04 8.955e-04 0.399 0.69000
## cylinders:acceleration 2.779e-01 1.664e-01 1.670 0.09584 .
## cylinders:year -1.741e-01 9.714e-02 -1.793 0.07389 .
## cylinders:origin 4.022e-01 4.926e-01 0.816 0.41482
## displacement:horsepower -8.491e-05 2.885e-04 -0.294 0.76867
## displacement:weight 2.472e-05 1.470e-05 1.682 0.09342 .
## displacement:acceleration -3.479e-03 3.342e-03 -1.041 0.29853
## displacement:year 5.934e-03 2.391e-03 2.482 0.01352 *
## displacement:origin 2.398e-02 1.947e-02 1.232 0.21875
## horsepower:weight -1.968e-05 2.924e-05 -0.673 0.50124
## horsepower:acceleration -7.213e-03 3.719e-03 -1.939 0.05325 .
## horsepower:year -5.838e-03 3.938e-03 -1.482 0.13916
## horsepower:origin 2.233e-03 2.930e-02 0.076 0.93931
## weight:acceleration 2.346e-04 2.289e-04 1.025 0.30596
## weight:year -2.245e-04 2.127e-04 -1.056 0.29182
## weight:origin -5.789e-04 1.591e-03 -0.364 0.71623
## acceleration:year 5.562e-02 2.558e-02 2.174 0.03033 *
## acceleration:origin 4.583e-01 1.567e-01 2.926 0.00365 **
## year:origin 1.393e-01 7.399e-02 1.882 0.06062 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.695 on 363 degrees of freedom
## (5 observations deleted due to missingness)
## Multiple R-squared: 0.8893, Adjusted R-squared: 0.8808
## F-statistic: 104.2 on 28 and 363 DF, p-value: < 2.2e-16(f) Try a few different transformations of the variables,
such as log(X), √ X, and X2. Comment on your findings.
log(horsepower) is more significant than horsepower
Squaring horsepower doesn’t change the significance
Squaring the weights doesn’t change the significance
Square root origin is more significant than origin.
summary(lm(mpg ~ . + log(horsepower), data=auto1))##
## Call:
## lm(formula = mpg ~ . + log(horsepower), data = auto1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -8.5777 -1.6623 -0.1213 1.4913 12.0230
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 8.674e+01 1.106e+01 7.839 4.54e-14 ***
## cylinders -5.530e-02 2.907e-01 -0.190 0.849230
## displacement -4.607e-03 7.108e-03 -0.648 0.517291
## horsepower 1.764e-01 2.269e-02 7.775 7.05e-14 ***
## weight -3.366e-03 6.561e-04 -5.130 4.62e-07 ***
## acceleration -3.277e-01 9.670e-02 -3.388 0.000776 ***
## year 7.421e-01 4.534e-02 16.368 < 2e-16 ***
## origin 8.976e-01 2.528e-01 3.551 0.000432 ***
## log(horsepower) -2.685e+01 2.652e+00 -10.127 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.959 on 383 degrees of freedom
## (5 observations deleted due to missingness)
## Multiple R-squared: 0.8592, Adjusted R-squared: 0.8562
## F-statistic: 292.1 on 8 and 383 DF, p-value: < 2.2e-16summary(lm(mpg ~ . + log(cylinders), data=auto1))##
## Call:
## lm(formula = mpg ~ . + log(cylinders), data = auto1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -11.762 -2.093 -0.180 1.730 12.942
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -1.333e+00 6.639e+00 -0.201 0.84096
## cylinders 3.673e+00 1.299e+00 2.827 0.00494 **
## displacement 2.008e-02 7.420e-03 2.707 0.00710 **
## horsepower -2.750e-02 1.398e-02 -1.967 0.04986 *
## weight -6.393e-03 6.442e-04 -9.924 < 2e-16 ***
## acceleration 1.059e-01 9.789e-02 1.082 0.27981
## year 7.482e-01 5.033e-02 14.865 < 2e-16 ***
## origin 1.268e+00 2.787e-01 4.548 7.29e-06 ***
## log(cylinders) -2.287e+01 6.912e+00 -3.308 0.00103 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.285 on 383 degrees of freedom
## (5 observations deleted due to missingness)
## Multiple R-squared: 0.8264, Adjusted R-squared: 0.8228
## F-statistic: 228 on 8 and 383 DF, p-value: < 2.2e-16summary(lm(mpg ~ . + log(displacement), data=auto1))##
## Call:
## lm(formula = mpg ~ . + log(displacement), data = auto1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -12.1562 -1.8388 -0.0423 1.6999 11.7871
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.529e+01 8.485e+00 5.337 1.62e-07 ***
## cylinders 3.391e-03 3.025e-01 0.011 0.991060
## displacement 7.744e-02 9.655e-03 8.021 1.29e-14 ***
## horsepower -4.380e-02 1.304e-02 -3.358 0.000864 ***
## weight -4.536e-03 6.404e-04 -7.083 6.80e-12 ***
## acceleration -1.352e-02 9.142e-02 -0.148 0.882479
## year 7.827e-01 4.695e-02 16.671 < 2e-16 ***
## origin 4.485e-01 2.799e-01 1.602 0.109926
## log(displacement) -1.537e+01 1.804e+00 -8.520 3.70e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.055 on 383 degrees of freedom
## (5 observations deleted due to missingness)
## Multiple R-squared: 0.8499, Adjusted R-squared: 0.8468
## F-statistic: 271.1 on 8 and 383 DF, p-value: < 2.2e-16summary(lm(mpg ~ . + I(horsepower^2), data=auto1))##
## Call:
## lm(formula = mpg ~ . + I(horsepower^2), data = auto1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -8.5497 -1.7311 -0.2236 1.5877 11.9955
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.3236564 4.6247696 0.286 0.774872
## cylinders 0.3489063 0.3048310 1.145 0.253094
## displacement -0.0075649 0.0073733 -1.026 0.305550
## horsepower -0.3194633 0.0343447 -9.302 < 2e-16 ***
## weight -0.0032712 0.0006787 -4.820 2.07e-06 ***
## acceleration -0.3305981 0.0991849 -3.333 0.000942 ***
## year 0.7353414 0.0459918 15.989 < 2e-16 ***
## origin 1.0144130 0.2545545 3.985 8.08e-05 ***
## I(horsepower^2) 0.0010060 0.0001065 9.449 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.001 on 383 degrees of freedom
## (5 observations deleted due to missingness)
## Multiple R-squared: 0.8552, Adjusted R-squared: 0.8522
## F-statistic: 282.8 on 8 and 383 DF, p-value: < 2.2e-16summary(lm(mpg ~ . + I(weight^2), data=auto1))##
## Call:
## lm(formula = mpg ~ . + I(weight^2), data = auto1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.4706 -1.6701 -0.1488 1.6383 12.5429
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.479e+00 4.614e+00 0.321 0.74867
## cylinders -2.840e-01 2.917e-01 -0.974 0.33083
## displacement 1.371e-02 6.793e-03 2.019 0.04418 *
## horsepower -2.435e-02 1.243e-02 -1.959 0.05083 .
## weight -2.049e-02 1.580e-03 -12.970 < 2e-16 ***
## acceleration 6.571e-02 8.895e-02 0.739 0.46055
## year 7.999e-01 4.615e-02 17.331 < 2e-16 ***
## origin 7.418e-01 2.603e-01 2.850 0.00461 **
## I(weight^2) 2.237e-06 2.341e-07 9.556 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.994 on 383 degrees of freedom
## (5 observations deleted due to missingness)
## Multiple R-squared: 0.8558, Adjusted R-squared: 0.8528
## F-statistic: 284.2 on 8 and 383 DF, p-value: < 2.2e-16summary(lm(mpg~.+ sqrt(weight), data = auto1))##
## Call:
## lm(formula = mpg ~ . + sqrt(weight), data = auto1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.6137 -1.6346 -0.2036 1.5941 12.6691
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 64.139861 9.614122 6.671 8.89e-11 ***
## cylinders -0.445801 0.291816 -1.528 0.12742
## displacement 0.013567 0.006816 1.991 0.04725 *
## horsepower -0.022813 0.012459 -1.831 0.06786 .
## weight 0.021590 0.003042 7.097 6.21e-12 ***
## acceleration 0.050527 0.089268 0.566 0.57171
## year 0.798541 0.046284 17.253 < 2e-16 ***
## origin 0.720882 0.261992 2.752 0.00621 **
## sqrt(weight) -3.061177 0.325549 -9.403 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.003 on 383 degrees of freedom
## (5 observations deleted due to missingness)
## Multiple R-squared: 0.855, Adjusted R-squared: 0.8519
## F-statistic: 282.2 on 8 and 383 DF, p-value: < 2.2e-16summary(lm(mpg~.+ sqrt(origin), data = auto1))##
## Call:
## lm(formula = mpg ~ . + sqrt(origin), data = auto1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.0095 -2.0785 -0.0982 1.9856 13.3608
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.521e+01 8.713e+00 -4.042 6.42e-05 ***
## cylinders -4.897e-01 3.212e-01 -1.524 0.12821
## displacement 2.398e-02 7.653e-03 3.133 0.00186 **
## horsepower -1.818e-02 1.371e-02 -1.326 0.18549
## weight -6.710e-03 6.551e-04 -10.243 < 2e-16 ***
## acceleration 7.910e-02 9.822e-02 0.805 0.42110
## year 7.770e-01 5.178e-02 15.005 < 2e-16 ***
## origin -7.714e+00 3.764e+00 -2.049 0.04110 *
## sqrt(origin) 2.497e+01 1.026e+01 2.435 0.01535 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.307 on 383 degrees of freedom
## (5 observations deleted due to missingness)
## Multiple R-squared: 0.8242, Adjusted R-squared: 0.8205
## F-statistic: 224.5 on 8 and 383 DF, p-value: < 2.2e-16Problem 10
This question should be answered using the Carseats data set.
library(ISLR)
attach(Carseats)(a) Fit a multiple regression model to predict Sales using Price, Urban, and US.
lmsales <- lm(Sales~Price + Urban + US, data =Carseats)
summary (lmsales)##
## Call:
## lm(formula = Sales ~ Price + Urban + US, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.9206 -1.6220 -0.0564 1.5786 7.0581
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.043469 0.651012 20.036 < 2e-16 ***
## Price -0.054459 0.005242 -10.389 < 2e-16 ***
## UrbanYes -0.021916 0.271650 -0.081 0.936
## USYes 1.200573 0.259042 4.635 4.86e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.472 on 396 degrees of freedom
## Multiple R-squared: 0.2393, Adjusted R-squared: 0.2335
## F-statistic: 41.52 on 3 and 396 DF, p-value: < 2.2e-16(b) Provide an interpretation of each coefficient in the
model. Be careful—some of the variables in the model are
qualitative!
Price(estimate) = - 0.054
Interpretation = the effect of a 1 unit ($1） increase in Price (for
fixed values of Urban & US) equates to a decrease of $54.459 in
‘sales’
Urban(p-value)=0.936
Interpretation =urban is a categorical variable. The coefficient of the
‘Urban’ variable shows that there is no relationship between ‘Sales’ and
‘Urban’ since the p-value is large(greater than 0.05).
US = 1.200
Sales inside of US increase $1,200 higher than sales outside of the US
when price increase $1.
(c) Write out the model in equation form, being careful to
handle the qualitative variables properly.
Sales=β0+β1∗Price+β2∗UrbanYes+β3∗USYes
\(Sales=13.04−0.05∗Price−0.02∗UrbanYes+1.2∗USYes\)
(d) For which of the predictors can you reject the null
hypothesis H0 : βj = 0?
‘Price’ and ‘US’ due to the significance of the p-value(less than
0.05).
(e) On the basis of your response to the previous question, fit a smaller model that only uses the predictors for which there is evidence of association with the outcome.
lmsales1<-lm(Sales~Price+US,data=Carseats)
summary(lmsales1)##
## Call:
## lm(formula = Sales ~ Price + US, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.9269 -1.6286 -0.0574 1.5766 7.0515
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.03079 0.63098 20.652 < 2e-16 ***
## Price -0.05448 0.00523 -10.416 < 2e-16 ***
## USYes 1.19964 0.25846 4.641 4.71e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.469 on 397 degrees of freedom
## Multiple R-squared: 0.2393, Adjusted R-squared: 0.2354
## F-statistic: 62.43 on 2 and 397 DF, p-value: < 2.2e-16(f) How well do the models in (a) and (e) fit the data?
The R-squared for both models shows that about 23% of the variability
can be explained by the model. these two models are mediocre.
(g) Using the model from (e), obtain 95 % confidence
intervals for the coefficient(s).
That there is a 95% probability that the true parameter for Price falls
within the interval (-0.0648, -0.0442), and a 5% probability that it
doesn’t.
confint(lmsales1)## 2.5 % 97.5 %
## (Intercept) 11.79032020 14.27126531
## Price -0.06475984 -0.04419543
## USYes 0.69151957 1.70776632(h) Is there evidence of outliers or high leverage
observations in the model from (e)?
the average leverage \(\frac{(p+1)}{n}\) which for us is \(\frac{(2+1)}{400} = 0.0075\).
As shown in the “studentized residual” figure above, All studentized
residuals appear to be bounded by -3 to 3, so no potential outliers are
suggested from the linear regression.
As shown in the “residuals vs leverage” figure above, all the leverage
fall inside the “cook`s distance” which means there is no potential high
leverage in the data.
par(mfrow = c(2, 2))
plot(lmsales1)
plot(predict(lmsales1),rstudent(lmsales1))
summary(influence.measures(lmsales1))## Potentially influential observations of
## lm(formula = Sales ~ Price + US, data = Carseats) :
##
## dfb.1_ dfb.Pric dfb.USYs dffit cov.r cook.d hat
## 26 0.24 -0.18 -0.17 0.28_* 0.97_* 0.03 0.01
## 29 -0.10 0.10 -0.10 -0.18 0.97_* 0.01 0.01
## 43 -0.11 0.10 0.03 -0.11 1.05_* 0.00 0.04_*
## 50 -0.10 0.17 -0.17 0.26_* 0.98 0.02 0.01
## 51 -0.05 0.05 -0.11 -0.18 0.95_* 0.01 0.00
## 58 -0.05 -0.02 0.16 -0.20 0.97_* 0.01 0.01
## 69 -0.09 0.10 0.09 0.19 0.96_* 0.01 0.01
## 126 -0.07 0.06 0.03 -0.07 1.03_* 0.00 0.03_*
## 160 0.00 0.00 0.00 0.01 1.02_* 0.00 0.02
## 166 0.21 -0.23 -0.04 -0.24 1.02 0.02 0.03_*
## 172 0.06 -0.07 0.02 0.08 1.03_* 0.00 0.02
## 175 0.14 -0.19 0.09 -0.21 1.03_* 0.02 0.03_*
## 210 -0.14 0.15 -0.10 -0.22 0.97_* 0.02 0.01
## 270 -0.03 0.05 -0.03 0.06 1.03_* 0.00 0.02
## 298 -0.06 0.06 -0.09 -0.15 0.97_* 0.01 0.00
## 314 -0.05 0.04 0.02 -0.05 1.03_* 0.00 0.02_*
## 353 -0.02 0.03 0.09 0.15 0.97_* 0.01 0.00
## 357 0.02 -0.02 0.02 -0.03 1.03_* 0.00 0.02
## 368 0.26 -0.23 -0.11 0.27_* 1.01 0.02 0.02_*
## 377 0.14 -0.15 0.12 0.24 0.95_* 0.02 0.01
## 384 0.00 0.00 0.00 0.00 1.02_* 0.00 0.02
## 387 -0.03 0.04 -0.03 0.05 1.02_* 0.00 0.02
## 396 -0.05 0.05 0.08 0.14 0.98_* 0.01 0.00
Problem 12
This problem involves simple linear regression without an intercept.
(a) Recall that the coefficient estimate βˆ for the linear
regression of Y onto X without an intercept is given by (3.38). Under
what circumstance is the coefficient estimate for the regression of X
onto Y the same as the coefficient estimate for the regression of Y onto
X?
sum of squares of observed y values = sum of squares of observed x
values.
(b) Generate an example in R with n = 100 observations in which the coefficient estimate for the regression of X onto Y is different from the coefficient estimate for the regression of Y onto X
x <- rnorm(100)
y <- x^2
coefficients(lm(x ~ y))## (Intercept) y
## -0.04464288 0.06615758coefficients(lm(y ~ x))## (Intercept) x
## 0.82301463 0.07747444(c) Generate an example in R with n = 100 observations in which the coefficient estimate for the regression of X onto Y is the same as the coefficient estimate for the regression of Y onto X.
x <- 1:100
y <- 100:1
fit.X <- lm(x ~ y +0)
summary(fit.X)##
## Call:
## lm(formula = x ~ y + 0)
##
## Residuals:
## Min 1Q Median 3Q Max
## -49.75 -12.44 24.87 62.18 99.49
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## y 0.5075 0.0866 5.86 6.09e-08 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 50.37 on 99 degrees of freedom
## Multiple R-squared: 0.2575, Adjusted R-squared: 0.25
## F-statistic: 34.34 on 1 and 99 DF, p-value: 6.094e-08fit.Y <- lm(y ~ x +0)
summary(fit.Y)##
## Call:
## lm(formula = y ~ x + 0)
##
## Residuals:
## Min 1Q Median 3Q Max
## -49.75 -12.44 24.87 62.18 99.49
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## x 0.5075 0.0866 5.86 6.09e-08 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 50.37 on 99 degrees of freedom
## Multiple R-squared: 0.2575, Adjusted R-squared: 0.25
## F-statistic: 34.34 on 1 and 99 DF, p-value: 6.094e-08The coefficients are the same: 0.5075