Statistical Data Analysis
2024-04-03
Boston
This problem involves the Boston data set, which we saw in the lab for this chapter. We will now try to predict per capita crime rate using the other variables in this data set. In other words, per capita crime rate is the response, and the other variables are the predictors.
library(ISLR2)## Warning: 패키지 'ISLR2'는 R 버전 4.4.0에서 작성되었습니다head(Boston)## crim zn indus chas nox rm age dis rad tax ptratio lstat medv
## 1 0.00632 18 2.31 0 0.538 6.575 65.2 4.0900 1 296 15.3 4.98 24.0
## 2 0.02731 0 7.07 0 0.469 6.421 78.9 4.9671 2 242 17.8 9.14 21.6
## 3 0.02729 0 7.07 0 0.469 7.185 61.1 4.9671 2 242 17.8 4.03 34.7
## 4 0.03237 0 2.18 0 0.458 6.998 45.8 6.0622 3 222 18.7 2.94 33.4
## 5 0.06905 0 2.18 0 0.458 7.147 54.2 6.0622 3 222 18.7 5.33 36.2
## 6 0.02985 0 2.18 0 0.458 6.430 58.7 6.0622 3 222 18.7 5.21 28.7변수 설명
crim: 마을별 1인당 범죄율
zn: 25,000 평방피트이상인 주거용 토지지의 비율
indus: 마을 당 비소매상업부지의의 비율
chas: 찰스강에 대한 더미변수(강 경계에 위치한 경우 1, 아니면 0)
nox: 10ppm 당 일산화질소 농도
rm: 거주지당 평균 방의 개수
age:1940년 이전에 건축된 주택의 자가거주 비율
dis: 5개의 보스턴 직업센터까지의 접근성 지수
rad: 방사형 도로까지의 접근성 지수
tax: 10,000 달러 당 재산세율
ptratio: 마을별 학생/교사 비율
lstat: 하위계층의 비율
medv: 자가주택의 가격의 중앙값
- For each predictor, fit a simple linear regression model to predict the response. Describe your results. In which of the models is there a statistically significant association between the predictor and the response? Create some plots to back up your assertions.
m_zn = lm(crim ~ zn, data = Boston)
summary(m_zn)##
## Call:
## lm(formula = crim ~ zn, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -4.429 -4.222 -2.620 1.250 84.523
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.45369 0.41722 10.675 < 2e-16 ***
## zn -0.07393 0.01609 -4.594 5.51e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.435 on 504 degrees of freedom
## Multiple R-squared: 0.04019, Adjusted R-squared: 0.03828
## F-statistic: 21.1 on 1 and 504 DF, p-value: 5.506e-06m_indus = lm(crim ~ indus, data = Boston)
summary(m_indus)##
## Call:
## lm(formula = crim ~ indus, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -11.972 -2.698 -0.736 0.712 81.813
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -2.06374 0.66723 -3.093 0.00209 **
## indus 0.50978 0.05102 9.991 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.866 on 504 degrees of freedom
## Multiple R-squared: 0.1653, Adjusted R-squared: 0.1637
## F-statistic: 99.82 on 1 and 504 DF, p-value: < 2.2e-16m_chas = lm(crim ~ chas, data = Boston)
summary(m_chas)##
## Call:
## lm(formula = crim ~ chas, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.738 -3.661 -3.435 0.018 85.232
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.7444 0.3961 9.453 <2e-16 ***
## chas -1.8928 1.5061 -1.257 0.209
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.597 on 504 degrees of freedom
## Multiple R-squared: 0.003124, Adjusted R-squared: 0.001146
## F-statistic: 1.579 on 1 and 504 DF, p-value: 0.2094m_nox = lm(crim ~ nox, data = Boston)
summary(m_nox)##
## Call:
## lm(formula = crim ~ nox, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -12.371 -2.738 -0.974 0.559 81.728
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -13.720 1.699 -8.073 5.08e-15 ***
## nox 31.249 2.999 10.419 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.81 on 504 degrees of freedom
## Multiple R-squared: 0.1772, Adjusted R-squared: 0.1756
## F-statistic: 108.6 on 1 and 504 DF, p-value: < 2.2e-16m_rm = lm(crim ~ rm, data = Boston)
summary(m_rm)##
## Call:
## lm(formula = crim ~ rm, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.604 -3.952 -2.654 0.989 87.197
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 20.482 3.365 6.088 2.27e-09 ***
## rm -2.684 0.532 -5.045 6.35e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.401 on 504 degrees of freedom
## Multiple R-squared: 0.04807, Adjusted R-squared: 0.04618
## F-statistic: 25.45 on 1 and 504 DF, p-value: 6.347e-07m_age = lm(crim ~ age, data = Boston)
summary(m_age)##
## Call:
## lm(formula = crim ~ age, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.789 -4.257 -1.230 1.527 82.849
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.77791 0.94398 -4.002 7.22e-05 ***
## age 0.10779 0.01274 8.463 2.85e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.057 on 504 degrees of freedom
## Multiple R-squared: 0.1244, Adjusted R-squared: 0.1227
## F-statistic: 71.62 on 1 and 504 DF, p-value: 2.855e-16m_dis = lm(crim ~ dis, data = Boston)
summary(m_dis)##
## Call:
## lm(formula = crim ~ dis, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.708 -4.134 -1.527 1.516 81.674
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 9.4993 0.7304 13.006 <2e-16 ***
## dis -1.5509 0.1683 -9.213 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.965 on 504 degrees of freedom
## Multiple R-squared: 0.1441, Adjusted R-squared: 0.1425
## F-statistic: 84.89 on 1 and 504 DF, p-value: < 2.2e-16m_rad = lm(crim ~ rad, data = Boston)
summary(m_rad)##
## Call:
## lm(formula = crim ~ rad, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -10.164 -1.381 -0.141 0.660 76.433
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -2.28716 0.44348 -5.157 3.61e-07 ***
## rad 0.61791 0.03433 17.998 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 6.718 on 504 degrees of freedom
## Multiple R-squared: 0.3913, Adjusted R-squared: 0.39
## F-statistic: 323.9 on 1 and 504 DF, p-value: < 2.2e-16m_tax = lm(crim ~ tax, data = Boston)
summary(m_tax)##
## Call:
## lm(formula = crim ~ tax, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -12.513 -2.738 -0.194 1.065 77.696
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -8.528369 0.815809 -10.45 <2e-16 ***
## tax 0.029742 0.001847 16.10 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 6.997 on 504 degrees of freedom
## Multiple R-squared: 0.3396, Adjusted R-squared: 0.3383
## F-statistic: 259.2 on 1 and 504 DF, p-value: < 2.2e-16m_ptratio = lm(crim ~ ptratio, data = Boston)
summary(m_ptratio)##
## Call:
## lm(formula = crim ~ ptratio, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.654 -3.985 -1.912 1.825 83.353
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -17.6469 3.1473 -5.607 3.40e-08 ***
## ptratio 1.1520 0.1694 6.801 2.94e-11 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.24 on 504 degrees of freedom
## Multiple R-squared: 0.08407, Adjusted R-squared: 0.08225
## F-statistic: 46.26 on 1 and 504 DF, p-value: 2.943e-11m_lstat = lm(crim ~ lstat, data = Boston)
summary(m_lstat)##
## Call:
## lm(formula = crim ~ lstat, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -13.925 -2.822 -0.664 1.079 82.862
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.33054 0.69376 -4.801 2.09e-06 ***
## lstat 0.54880 0.04776 11.491 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.664 on 504 degrees of freedom
## Multiple R-squared: 0.2076, Adjusted R-squared: 0.206
## F-statistic: 132 on 1 and 504 DF, p-value: < 2.2e-16m_medv = lm(crim ~ medv, data = Boston)
summary(m_medv)##
## Call:
## lm(formula = crim ~ medv, data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.071 -4.022 -2.343 1.298 80.957
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 11.79654 0.93419 12.63 <2e-16 ***
## medv -0.36316 0.03839 -9.46 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.934 on 504 degrees of freedom
## Multiple R-squared: 0.1508, Adjusted R-squared: 0.1491
## F-statistic: 89.49 on 1 and 504 DF, p-value: < 2.2e-16chars 변수를 제외한 다른 변수들의 계수들 모두 통계적으로 유의하다는 결과가 나왔다. 그러나 모두 결정계수(R-squared) 값은 높지 않다. 가장 높은 값이 0.3913으로 crim과 rad의 상관관계를 나타내는 모델의 값이다.
Scatter plot
plot(Boston$zn,Boston$crim)
abline(m_zn,col="red")
plot(Boston$indus,Boston$crim)
abline(m_indus,col="red")
plot(Boston$chas,Boston$crim)
abline(m_chas,col="red")
plot(Boston$nox,Boston$crim)
abline(m_nox,col="red")
plot(Boston$rm,Boston$crim)
abline(m_rm,col="red")
plot(Boston$age,Boston$crim)
abline(m_age,col="red")
plot(Boston$dis,Boston$crim)
abline(m_dis,col="red")
plot(Boston$rad,Boston$crim)
abline(m_rad,col="red")
plot(Boston$tax,Boston$crim)
abline(m_tax,col="red")
plot(Boston$ptratio,Boston$crim)
abline(m_ptratio,col="red")
plot(Boston$lstat,Boston$crim)
abline(m_lstat,col="red")
plot(Boston$medv,Boston$crim)
abline(m_medv,col="red")
Diagnostic Plots
1)잔차가 선형성인지, 선 주위로 무작위로 분포했는지: 대체적으로
선형성을 가지지만, 잔차들이 한쪽에 몰려있는 모습이 나타난다.
2)잔차가 정규분포를 따르는지(직선에 근접했는지): 정규분포를 따르지 않는
것으로 보인다.
3)잔차가 동일분산을 따르는지(값들이 패턴없이 일정하게 분포되어있는지):
잔차들이 뭉쳐있는 모습이 많이 보인다.
4)이상값이 얼마나 영향을 주는지(점선으로 된 곡선 영역 밖에 있는지) :
이상치들은 있지만, 큰 영향을 주지는 않는다.
par(mfrow=c(2,2))
plot(m_zn)
par(mfrow=c(2,2))
plot(m_indus)
par(mfrow=c(2,2))
plot(m_indus)
par(mfrow=c(2,2))
plot(m_chas)
par(mfrow=c(2,2))
plot(m_nox)
par(mfrow=c(2,2))
plot(m_rm)
par(mfrow=c(2,2))
plot(m_age)
par(mfrow=c(2,2))
plot(m_dis)
par(mfrow=c(2,2))
plot(m_rad)
par(mfrow=c(2,2))
plot(m_tax)
par(mfrow=c(2,2))
plot(m_ptratio)
par(mfrow=c(2,2))
plot(m_lstat)
par(mfrow=c(2,2))
plot(m_medv)
- Fit a multiple regression model to predict the response using all of the predictors. Describe your results. For which predictors can we reject the null hypothesis \(H_0: \beta_j=0\)?
m_all = lm(crim ~ ., data=Boston)
summary(m_all)##
## Call:
## lm(formula = crim ~ ., data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -8.534 -2.248 -0.348 1.087 73.923
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.7783938 7.0818258 1.946 0.052271 .
## zn 0.0457100 0.0187903 2.433 0.015344 *
## indus -0.0583501 0.0836351 -0.698 0.485709
## chas -0.8253776 1.1833963 -0.697 0.485841
## nox -9.9575865 5.2898242 -1.882 0.060370 .
## rm 0.6289107 0.6070924 1.036 0.300738
## age -0.0008483 0.0179482 -0.047 0.962323
## dis -1.0122467 0.2824676 -3.584 0.000373 ***
## rad 0.6124653 0.0875358 6.997 8.59e-12 ***
## tax -0.0037756 0.0051723 -0.730 0.465757
## ptratio -0.3040728 0.1863598 -1.632 0.103393
## lstat 0.1388006 0.0757213 1.833 0.067398 .
## medv -0.2200564 0.0598240 -3.678 0.000261 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 6.46 on 493 degrees of freedom
## Multiple R-squared: 0.4493, Adjusted R-squared: 0.4359
## F-statistic: 33.52 on 12 and 493 DF, p-value: < 2.2e-16각 변수들을 개별적으로 회귀식을 만들었을 때보다 결정계수의 값이 더 크다. 통계적으로 유의하다는 결과가 나온 zn, nox, dis, rad, lstat, medv가 귀무가설을 기각 할 수 있다.
- How do your results from (a) compare to your results from (b)? Create a plot displaying the univariate regression coefficients from (a) on the x-axis, and the multiple regression coefficients from (b) on the y-axis. That is, each predictor is displayed as a single point in the plot. Its coefficient in a simple linear regression model is shown on the x-axis, and its coefficient estimate in the multiple linear regression model is shown on the y-axis.
x = c(coefficients(m_zn)[2],
coefficients(m_indus)[2],
coefficients(m_chas)[2],
coefficients(m_nox)[2],
coefficients(m_rm)[2],
coefficients(m_age)[2],
coefficients(m_dis)[2],
coefficients(m_rad)[2],
coefficients(m_tax)[2],
coefficients(m_ptratio)[2],
coefficients(m_lstat)[2],
coefficients(m_medv)[2])
y = coefficients(m_all)[2:13]
plot(x, y)
다중회귀의 계수들은 0에 가깝거나 음의 값인 경우가 많고, 단순회귀분석의 경우 양의 값을 가지는 경우가 많다. 다중회귀일때의 계수와 단순회귀분석일때의 계수가 많이 다르다는 것을 알 수 있다.
- Is there evidence of non-linear association between any of the predictors and the response? To answer this question, for each predictor X, fit a model of the form \[ Y=\beta_0+\beta_1X+\beta_2X^2+\beta_3X^3+\epsilon \] indus, nox, age, dis, ptratio, medev 변수들에서는 2차,3차 변수들의 계수들이 유의함을 보이고 있다. 비선형관계가 있음을 보여주고 있다.
unlin_zn <- lm(crim~ zn + I(zn^2) +I(zn^3),data=Boston)
summary(unlin_zn)##
## Call:
## lm(formula = crim ~ zn + I(zn^2) + I(zn^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -4.821 -4.614 -1.294 0.473 84.130
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.846e+00 4.330e-01 11.192 < 2e-16 ***
## zn -3.322e-01 1.098e-01 -3.025 0.00261 **
## I(zn^2) 6.483e-03 3.861e-03 1.679 0.09375 .
## I(zn^3) -3.776e-05 3.139e-05 -1.203 0.22954
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.372 on 502 degrees of freedom
## Multiple R-squared: 0.05824, Adjusted R-squared: 0.05261
## F-statistic: 10.35 on 3 and 502 DF, p-value: 1.281e-06unlin_indus <- lm(crim~ indus + I(indus^2) +I(indus^3),data=Boston)
summary(unlin_indus)##
## Call:
## lm(formula = crim ~ indus + I(indus^2) + I(indus^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -8.278 -2.514 0.054 0.764 79.713
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.6625683 1.5739833 2.327 0.0204 *
## indus -1.9652129 0.4819901 -4.077 5.30e-05 ***
## I(indus^2) 0.2519373 0.0393221 6.407 3.42e-10 ***
## I(indus^3) -0.0069760 0.0009567 -7.292 1.20e-12 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.423 on 502 degrees of freedom
## Multiple R-squared: 0.2597, Adjusted R-squared: 0.2552
## F-statistic: 58.69 on 3 and 502 DF, p-value: < 2.2e-16unlin_chas <- lm(crim~ chas + I(chas^2) +I(chas^3),data=Boston)
summary(unlin_chas)##
## Call:
## lm(formula = crim ~ chas + I(chas^2) + I(chas^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.738 -3.661 -3.435 0.018 85.232
##
## Coefficients: (2 not defined because of singularities)
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.7444 0.3961 9.453 <2e-16 ***
## chas -1.8928 1.5061 -1.257 0.209
## I(chas^2) NA NA NA NA
## I(chas^3) NA NA NA NA
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.597 on 504 degrees of freedom
## Multiple R-squared: 0.003124, Adjusted R-squared: 0.001146
## F-statistic: 1.579 on 1 and 504 DF, p-value: 0.2094unlin_nox <- lm(crim~ nox + I(nox^2) +I(nox^3),data=Boston)
summary(unlin_nox)##
## Call:
## lm(formula = crim ~ nox + I(nox^2) + I(nox^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.110 -2.068 -0.255 0.739 78.302
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 233.09 33.64 6.928 1.31e-11 ***
## nox -1279.37 170.40 -7.508 2.76e-13 ***
## I(nox^2) 2248.54 279.90 8.033 6.81e-15 ***
## I(nox^3) -1245.70 149.28 -8.345 6.96e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.234 on 502 degrees of freedom
## Multiple R-squared: 0.297, Adjusted R-squared: 0.2928
## F-statistic: 70.69 on 3 and 502 DF, p-value: < 2.2e-16unlin_rm <- lm(crim~ rm + I(rm^2) +I(rm^3),data=Boston)
summary(unlin_rm)##
## Call:
## lm(formula = crim ~ rm + I(rm^2) + I(rm^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -18.485 -3.468 -2.221 -0.015 87.219
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 112.6246 64.5172 1.746 0.0815 .
## rm -39.1501 31.3115 -1.250 0.2118
## I(rm^2) 4.5509 5.0099 0.908 0.3641
## I(rm^3) -0.1745 0.2637 -0.662 0.5086
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.33 on 502 degrees of freedom
## Multiple R-squared: 0.06779, Adjusted R-squared: 0.06222
## F-statistic: 12.17 on 3 and 502 DF, p-value: 1.067e-07unlin_age <- lm(crim~ age + I(age^2) +I(age^3),data=Boston)
summary(unlin_age)##
## Call:
## lm(formula = crim ~ age + I(age^2) + I(age^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.762 -2.673 -0.516 0.019 82.842
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -2.549e+00 2.769e+00 -0.920 0.35780
## age 2.737e-01 1.864e-01 1.468 0.14266
## I(age^2) -7.230e-03 3.637e-03 -1.988 0.04738 *
## I(age^3) 5.745e-05 2.109e-05 2.724 0.00668 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.84 on 502 degrees of freedom
## Multiple R-squared: 0.1742, Adjusted R-squared: 0.1693
## F-statistic: 35.31 on 3 and 502 DF, p-value: < 2.2e-16unlin_dis <- lm(crim~ dis + I(dis^2) +I(dis^3),data=Boston)
summary(unlin_dis)##
## Call:
## lm(formula = crim ~ dis + I(dis^2) + I(dis^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -10.757 -2.588 0.031 1.267 76.378
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 30.0476 2.4459 12.285 < 2e-16 ***
## dis -15.5543 1.7360 -8.960 < 2e-16 ***
## I(dis^2) 2.4521 0.3464 7.078 4.94e-12 ***
## I(dis^3) -0.1186 0.0204 -5.814 1.09e-08 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.331 on 502 degrees of freedom
## Multiple R-squared: 0.2778, Adjusted R-squared: 0.2735
## F-statistic: 64.37 on 3 and 502 DF, p-value: < 2.2e-16unlin_rad <- lm(crim~ rad + I(rad^2) +I(rad^3),data=Boston)
summary(unlin_rad)##
## Call:
## lm(formula = crim ~ rad + I(rad^2) + I(rad^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -10.381 -0.412 -0.269 0.179 76.217
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.605545 2.050108 -0.295 0.768
## rad 0.512736 1.043597 0.491 0.623
## I(rad^2) -0.075177 0.148543 -0.506 0.613
## I(rad^3) 0.003209 0.004564 0.703 0.482
##
## Residual standard error: 6.682 on 502 degrees of freedom
## Multiple R-squared: 0.4, Adjusted R-squared: 0.3965
## F-statistic: 111.6 on 3 and 502 DF, p-value: < 2.2e-16unlin_tax <- lm(crim~ tax + I(tax^2) +I(tax^3),data=Boston)
summary(unlin_tax)##
## Call:
## lm(formula = crim ~ tax + I(tax^2) + I(tax^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -13.273 -1.389 0.046 0.536 76.950
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.918e+01 1.180e+01 1.626 0.105
## tax -1.533e-01 9.568e-02 -1.602 0.110
## I(tax^2) 3.608e-04 2.425e-04 1.488 0.137
## I(tax^3) -2.204e-07 1.889e-07 -1.167 0.244
##
## Residual standard error: 6.854 on 502 degrees of freedom
## Multiple R-squared: 0.3689, Adjusted R-squared: 0.3651
## F-statistic: 97.8 on 3 and 502 DF, p-value: < 2.2e-16unlin_ptratio <- lm(crim~ ptratio + I(ptratio^2) +I(ptratio^3),data=Boston)
summary(unlin_ptratio)##
## Call:
## lm(formula = crim ~ ptratio + I(ptratio^2) + I(ptratio^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.833 -4.146 -1.655 1.408 82.697
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 477.18405 156.79498 3.043 0.00246 **
## ptratio -82.36054 27.64394 -2.979 0.00303 **
## I(ptratio^2) 4.63535 1.60832 2.882 0.00412 **
## I(ptratio^3) -0.08476 0.03090 -2.743 0.00630 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 8.122 on 502 degrees of freedom
## Multiple R-squared: 0.1138, Adjusted R-squared: 0.1085
## F-statistic: 21.48 on 3 and 502 DF, p-value: 4.171e-13unlin_lstat <- lm(crim~ lstat + I(lstat^2) +I(lstat^3),data=Boston)
summary(unlin_lstat)##
## Call:
## lm(formula = crim ~ lstat + I(lstat^2) + I(lstat^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -15.234 -2.151 -0.486 0.066 83.353
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.2009656 2.0286452 0.592 0.5541
## lstat -0.4490656 0.4648911 -0.966 0.3345
## I(lstat^2) 0.0557794 0.0301156 1.852 0.0646 .
## I(lstat^3) -0.0008574 0.0005652 -1.517 0.1299
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.629 on 502 degrees of freedom
## Multiple R-squared: 0.2179, Adjusted R-squared: 0.2133
## F-statistic: 46.63 on 3 and 502 DF, p-value: < 2.2e-16unlin_medv <- lm(crim~ medv + I(medv^2) +I(medv^3),data=Boston)
summary(unlin_medv)##
## Call:
## lm(formula = crim ~ medv + I(medv^2) + I(medv^3), data = Boston)
##
## Residuals:
## Min 1Q Median 3Q Max
## -24.427 -1.976 -0.437 0.439 73.655
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 53.1655381 3.3563105 15.840 < 2e-16 ***
## medv -5.0948305 0.4338321 -11.744 < 2e-16 ***
## I(medv^2) 0.1554965 0.0171904 9.046 < 2e-16 ***
## I(medv^3) -0.0014901 0.0002038 -7.312 1.05e-12 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 6.569 on 502 degrees of freedom
## Multiple R-squared: 0.4202, Adjusted R-squared: 0.4167
## F-statistic: 121.3 on 3 and 502 DF, p-value: < 2.2e-16Auto
library(ISLR2)
head(Auto)## mpg cylinders displacement horsepower weight acceleration year origin
## 1 18 8 307 130 3504 12.0 70 1
## 2 15 8 350 165 3693 11.5 70 1
## 3 18 8 318 150 3436 11.0 70 1
## 4 16 8 304 150 3433 12.0 70 1
## 5 17 8 302 140 3449 10.5 70 1
## 6 15 8 429 198 4341 10.0 70 1
## name
## 1 chevrolet chevelle malibu
## 2 buick skylark 320
## 3 plymouth satellite
## 4 amc rebel sst
## 5 ford torino
## 6 ford galaxie 500변수 설명
mpg: 연비
cylinders: 엔진 실린더 개수
displacement: 배기량
horsepower: 마력(hp)
weight: 자동차 무게
acceleration: 초당 가속력
year: 출시 년도
origin: 제조국(1: 미국, 2: 유럽, 3: 일본)
car name: 자동차 이름
- Simple linear regression model (name 변수는 제외함.)
m_cylin = lm(mpg ~ cylinders, data = Auto)
summary(m_cylin)##
## Call:
## lm(formula = mpg ~ cylinders, data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -14.2413 -3.1832 -0.6332 2.5491 17.9168
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 42.9155 0.8349 51.40 <2e-16 ***
## cylinders -3.5581 0.1457 -24.43 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.914 on 390 degrees of freedom
## Multiple R-squared: 0.6047, Adjusted R-squared: 0.6037
## F-statistic: 596.6 on 1 and 390 DF, p-value: < 2.2e-16m_disp = lm(mpg ~ displacement, data = Auto)
summary(m_disp)##
## Call:
## lm(formula = mpg ~ displacement, data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -12.9170 -3.0243 -0.5021 2.3512 18.6128
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 35.12064 0.49443 71.03 <2e-16 ***
## displacement -0.06005 0.00224 -26.81 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.635 on 390 degrees of freedom
## Multiple R-squared: 0.6482, Adjusted R-squared: 0.6473
## F-statistic: 718.7 on 1 and 390 DF, p-value: < 2.2e-16m_horse = lm(mpg ~ horsepower, data = Auto)
summary(m_horse)##
## Call:
## lm(formula = mpg ~ horsepower, data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -13.5710 -3.2592 -0.3435 2.7630 16.9240
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 39.935861 0.717499 55.66 <2e-16 ***
## horsepower -0.157845 0.006446 -24.49 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.906 on 390 degrees of freedom
## Multiple R-squared: 0.6059, Adjusted R-squared: 0.6049
## F-statistic: 599.7 on 1 and 390 DF, p-value: < 2.2e-16m_weight = lm(mpg ~ weight, data = Auto)
summary(m_weight)##
## Call:
## lm(formula = mpg ~ weight, data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -11.9736 -2.7556 -0.3358 2.1379 16.5194
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 46.216524 0.798673 57.87 <2e-16 ***
## weight -0.007647 0.000258 -29.64 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.333 on 390 degrees of freedom
## Multiple R-squared: 0.6926, Adjusted R-squared: 0.6918
## F-statistic: 878.8 on 1 and 390 DF, p-value: < 2.2e-16m_accel = lm(mpg ~ acceleration, data = Auto)
summary(m_accel)##
## Call:
## lm(formula = mpg ~ acceleration, data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -17.989 -5.616 -1.199 4.801 23.239
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.8332 2.0485 2.359 0.0188 *
## acceleration 1.1976 0.1298 9.228 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7.08 on 390 degrees of freedom
## Multiple R-squared: 0.1792, Adjusted R-squared: 0.1771
## F-statistic: 85.15 on 1 and 390 DF, p-value: < 2.2e-16m_year = lm(mpg ~ year, data = Auto)
summary(m_year)##
## Call:
## lm(formula = mpg ~ year, data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -12.0212 -5.4411 -0.4412 4.9739 18.2088
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -70.01167 6.64516 -10.54 <2e-16 ***
## year 1.23004 0.08736 14.08 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 6.363 on 390 degrees of freedom
## Multiple R-squared: 0.337, Adjusted R-squared: 0.3353
## F-statistic: 198.3 on 1 and 390 DF, p-value: < 2.2e-16m_origin = lm(mpg ~ origin, data = Auto)
summary(m_origin)##
## Call:
## lm(formula = mpg ~ origin, data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -13.2416 -5.2533 -0.7651 3.8967 18.7115
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 14.8120 0.7164 20.68 <2e-16 ***
## origin 5.4765 0.4048 13.53 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 6.447 on 390 degrees of freedom
## Multiple R-squared: 0.3195, Adjusted R-squared: 0.3177
## F-statistic: 183.1 on 1 and 390 DF, p-value: < 2.2e-16모두 pvalue값이 작아서 통계적으로 유의한 값임을 알 수 있다. acceleration과 year, origin을 제외한 다른 변수들의 R-squard 값들이 모두 0.6이상으로 매우 크다.
Scatter plot
plot(Auto$cylinders,Auto$mpg)
abline(m_cylin,col="red")
plot(Auto$displacement,Auto$mpg)
abline(m_disp,col="red")
plot(Auto$horsepower,Auto$mpg)
abline(m_horse,col="red")
plot(Auto$weight,Auto$mpg)
abline(m_weight,col="red")
plot(Auto$acceleration,Auto$mpg)
abline(m_accel,col="red")
plot(Auto$year,Auto$mpg)
abline(m_year,col="red")
plot(Auto$origin,Auto$mpg)
abline(m_origin,col="red")
Diagnostic Plots
1)잔차가 선형성인지, 선 주위로 무작위로 분포했는지: 대부분 선형의
형태를 가지고 있지만, 잔차가 뭉쳐있는 모습이 보인다. 2)잔차가 정규분포를
따르는지(직선에 근접했는지): 정규분포에 거의 근사하다.
3)잔차가 동일분산을 따르는지(값들이 패턴없이 일정하게 분포되어있는지):
한 곳에 모여 있는 잔차들이 있다. 4)이상값이 얼마나 영향을
주는지(점선으로 된 곡선 영역 밖에 있는지) : 이상값들이 많지 않고 다른
값들과의 차이도 크지 않다.
par(mfrow=c(2,2))
plot(m_cylin)
par(mfrow=c(2,2))
plot(m_disp)
par(mfrow=c(2,2))
plot(m_horse)
par(mfrow=c(2,2))
plot(m_weight)
par(mfrow=c(2,2))
plot(m_accel)
par(mfrow=c(2,2))
plot(m_year)
par(mfrow=c(2,2))
plot(m_origin)
- Multiple regression model
For which predictors can we reject the null hypothesis \(H_0: \beta_j=0\)?
다중회귀분석모델의 결정계수가 0.8로 더 상승했음을 알 수 있다. displacement, weight, year, origin의 pvalue 값이 작으므로 귀무가설을 기각할 수 있다.
m_auto = lm(mpg ~ .-name, data=Auto)
summary(m_auto)##
## Call:
## lm(formula = mpg ~ . - name, data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.5903 -2.1565 -0.1169 1.8690 13.0604
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -17.218435 4.644294 -3.707 0.00024 ***
## cylinders -0.493376 0.323282 -1.526 0.12780
## displacement 0.019896 0.007515 2.647 0.00844 **
## horsepower -0.016951 0.013787 -1.230 0.21963
## weight -0.006474 0.000652 -9.929 < 2e-16 ***
## acceleration 0.080576 0.098845 0.815 0.41548
## year 0.750773 0.050973 14.729 < 2e-16 ***
## origin 1.426141 0.278136 5.127 4.67e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.328 on 384 degrees of freedom
## Multiple R-squared: 0.8215, Adjusted R-squared: 0.8182
## F-statistic: 252.4 on 7 and 384 DF, p-value: < 2.2e-16- How do your results from (a) compare to your results from (b)? Create a plot displaying the univariate regression coefficients from (a) on the x-axis, and the multiple regression coefficients from (b) on the y-axis. That is, each predictor is displayed as a single point in the plot. Its coefficient in a simple linear regression model is shown on the x-axis, and its coefficient estimate in the multiple linear regression model is shown on the y-axis.
다중회귀와 단순회귀의 계수들의 부호는 displacement 제외 같다. 단순회귀 분석의 값들이 더 크다.
xa = c(coefficients(m_cylin)[2],
coefficients(m_disp)[2],
coefficients(m_horse)[2],
coefficients(m_weight)[2],
coefficients(m_accel)[2],
coefficients(m_year)[2],
coefficients(m_origin)[2])
ya = coefficients(m_auto)[2:8]
plot(xa, ya)
- Is there evidence of non-linear association between any of the predictors and the response? To answer this question, for each predictor X, fit a model of the form \[ Y=\beta_0+\beta_1X+\beta_2X^2+\beta_3X^3+\epsilon \] Cylinder, origin에서는 세제곱의 계수까지 유의하하고, displacement,horsepower는 제곱의 계수까지 유의하다. 나머지 변수들은 비선형관계가 없다.
unlin_cylin <- lm(mpg~ cylinders + I(cylinders^2) +I(cylinders^3),data=Auto)
summary(unlin_cylin)##
## Call:
## lm(formula = mpg ~ cylinders + I(cylinders^2) + I(cylinders^3),
## data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -11.2869 -2.9058 -0.9627 2.3403 18.0218
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -131.7352 30.4664 -4.324 1.95e-05 ***
## cylinders 94.2603 16.7231 5.637 3.34e-08 ***
## I(cylinders^2) -17.5120 2.9386 -5.959 5.69e-09 ***
## I(cylinders^3) 1.0027 0.1658 6.047 3.46e-09 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.693 on 388 degrees of freedom
## Multiple R-squared: 0.6413, Adjusted R-squared: 0.6385
## F-statistic: 231.2 on 3 and 388 DF, p-value: < 2.2e-16unlin_disp <- lm(mpg~ displacement + I(displacement^2)+I(displacement^3),data=Auto)
summary(unlin_disp)##
## Call:
## lm(formula = mpg ~ displacement + I(displacement^2) + I(displacement^3),
## data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -15.6791 -2.3900 -0.2987 2.1156 20.4528
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.445e+01 2.591e+00 17.157 < 2e-16 ***
## displacement -1.769e-01 3.975e-02 -4.451 1.12e-05 ***
## I(displacement^2) 3.452e-04 1.743e-04 1.980 0.0484 *
## I(displacement^3) -2.363e-07 2.310e-07 -1.023 0.3069
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.365 on 388 degrees of freedom
## Multiple R-squared: 0.6896, Adjusted R-squared: 0.6872
## F-statistic: 287.4 on 3 and 388 DF, p-value: < 2.2e-16unlin_horse <- lm(mpg~ horsepower + I(horsepower^2) +I(horsepower^3),data=Auto)
summary(unlin_horse)##
## Call:
## lm(formula = mpg ~ horsepower + I(horsepower^2) + I(horsepower^3),
## data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -14.7039 -2.4491 -0.1519 2.2035 15.8159
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.068e+01 4.563e+00 13.298 < 2e-16 ***
## horsepower -5.689e-01 1.179e-01 -4.824 2.03e-06 ***
## I(horsepower^2) 2.079e-03 9.479e-04 2.193 0.0289 *
## I(horsepower^3) -2.147e-06 2.378e-06 -0.903 0.3673
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.375 on 388 degrees of freedom
## Multiple R-squared: 0.6882, Adjusted R-squared: 0.6858
## F-statistic: 285.5 on 3 and 388 DF, p-value: < 2.2e-16unlin_weight <- lm(mpg~ weight + I(weight^2) +I(weight^3),data=Auto)
summary(unlin_weight)##
## Call:
## lm(formula = mpg ~ weight + I(weight^2) + I(weight^3), data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -12.6259 -2.7080 -0.3552 1.8385 16.0816
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.170e+01 1.104e+01 5.587 4.36e-08 ***
## weight -1.793e-02 1.091e-02 -1.643 0.101
## I(weight^2) 1.515e-06 3.450e-06 0.439 0.661
## I(weight^3) 1.846e-11 3.503e-10 0.053 0.958
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.182 on 388 degrees of freedom
## Multiple R-squared: 0.7151, Adjusted R-squared: 0.7129
## F-statistic: 324.7 on 3 and 388 DF, p-value: < 2.2e-16unlin_accel <- lm(mpg~ acceleration + I(acceleration^2) +I(acceleration^3),data=Auto)
summary(unlin_accel)##
## Call:
## lm(formula = mpg ~ acceleration + I(acceleration^2) + I(acceleration^3),
## data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -17.7828 -5.5661 -0.8348 4.7540 22.7574
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -36.684955 26.024431 -1.410 0.159
## acceleration 8.043898 5.017719 1.603 0.110
## I(acceleration^2) -0.352493 0.315457 -1.117 0.265
## I(acceleration^3) 0.005575 0.006460 0.863 0.389
##
## Residual standard error: 7.028 on 388 degrees of freedom
## Multiple R-squared: 0.1955, Adjusted R-squared: 0.1893
## F-statistic: 31.43 on 3 and 388 DF, p-value: < 2.2e-16unlin_year <- lm(mpg~ year + I(year^2) +I(year^3),data=Auto)
summary(unlin_year)##
## Call:
## lm(formula = mpg ~ year + I(year^2) + I(year^3), data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -13.2690 -5.0507 -0.8684 4.8333 17.9470
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.499e+03 3.435e+03 1.019 0.309
## year -1.316e+02 1.359e+02 -0.969 0.333
## I(year^2) 1.640e+00 1.789e+00 0.916 0.360
## I(year^3) -6.706e-03 7.846e-03 -0.855 0.393
##
## Residual standard error: 6.225 on 388 degrees of freedom
## Multiple R-squared: 0.3687, Adjusted R-squared: 0.3638
## F-statistic: 75.54 on 3 and 388 DF, p-value: < 2.2e-16unlin_origin <- lm(mpg~ origin + I(origin^2) +I(origin^3),data=Auto)
summary(unlin_origin)##
## Call:
## lm(formula = mpg ~ origin + I(origin^2) + I(origin^3), data = Auto)
##
## Residuals:
## Min 1Q Median 3Q Max
## -12.451 -5.034 -1.034 3.649 18.966
##
## Coefficients: (1 not defined because of singularities)
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 7.7422 2.7268 2.839 0.00476 **
## origin 14.6521 3.4402 4.259 2.58e-05 ***
## I(origin^2) -2.3609 0.8791 -2.686 0.00755 **
## I(origin^3) NA NA NA NA
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 6.396 on 389 degrees of freedom
## Multiple R-squared: 0.3318, Adjusted R-squared: 0.3284
## F-statistic: 96.6 on 2 and 389 DF, p-value: < 2.2e-16Carseats
library(ISLR2)
head(Carseats)## Sales CompPrice Income Advertising Population Price ShelveLoc Age Education
## 1 9.50 138 73 11 276 120 Bad 42 17
## 2 11.22 111 48 16 260 83 Good 65 10
## 3 10.06 113 35 10 269 80 Medium 59 12
## 4 7.40 117 100 4 466 97 Medium 55 14
## 5 4.15 141 64 3 340 128 Bad 38 13
## 6 10.81 124 113 13 501 72 Bad 78 16
## Urban US
## 1 Yes Yes
## 2 Yes Yes
## 3 Yes Yes
## 4 Yes Yes
## 5 Yes No
## 6 No Yes변수 설명
Sales: 지역별 매출
CompPrice: 지역별 경쟁업체가 부과한 가격
Income: 지역 소득수준
Advertising: 지역별 광고 예산
Population: 지역 인구
Price: 자동차 좌석 가격
ShelveLoc: 자동차 좌석 보관 위치의 품질
Age: 지역 인구 연령
Education: 지역별 교육수준
Urban: 도시인지.시골인지
US: 판매매 위치가 미국인지, 그외 지역인지
- Simple linear regression model
compPrice, Population, Education을 제외한 변수들의 계수는 통계적으로 유의하다. 결정계수 값이 모두 낮다. 가장 높은 값이 0.198으로 Sales와 Price 회귀모델의 결정계수 값이다.
m_comp = lm(Sales ~ CompPrice, data = Carseats)
summary(m_comp)##
## Call:
## lm(formula = Sales ~ CompPrice, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.6618 -2.0422 -0.0108 1.7440 8.5846
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.021469 1.159945 5.191 3.34e-07 ***
## CompPrice 0.011801 0.009212 1.281 0.201
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.822 on 398 degrees of freedom
## Multiple R-squared: 0.004106, Adjusted R-squared: 0.001604
## F-statistic: 1.641 on 1 and 398 DF, p-value: 0.2009m_income = lm(Sales ~ Income, data = Carseats)
summary(m_income)##
## Call:
## lm(formula = Sales ~ Income, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.2629 -1.9447 -0.1772 1.7654 8.9064
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.44356 0.37061 17.386 < 2e-16 ***
## Income 0.01533 0.00500 3.067 0.00231 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.795 on 398 degrees of freedom
## Multiple R-squared: 0.02309, Adjusted R-squared: 0.02063
## F-statistic: 9.407 on 1 and 398 DF, p-value: 0.00231m_adver = lm(Sales ~ Advertising, data = Carseats)
summary(m_adver)##
## Call:
## lm(formula = Sales ~ Advertising, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.3770 -1.9634 -0.1037 1.7222 8.3208
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.7370 0.1925 35.007 < 2e-16 ***
## Advertising 0.1144 0.0205 5.583 4.38e-08 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.723 on 398 degrees of freedom
## Multiple R-squared: 0.07263, Adjusted R-squared: 0.0703
## F-statistic: 31.17 on 1 and 398 DF, p-value: 4.378e-08m_popul = lm(Sales ~ Population, data = Carseats)
summary(m_popul)##
## Call:
## lm(formula = Sales ~ Population, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.5864 -2.0176 -0.0597 1.6892 8.7213
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 7.2401837 0.2906658 24.909 <2e-16 ***
## Population 0.0009672 0.0009593 1.008 0.314
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.824 on 398 degrees of freedom
## Multiple R-squared: 0.002547, Adjusted R-squared: 4.116e-05
## F-statistic: 1.016 on 1 and 398 DF, p-value: 0.314m_price= lm(Sales ~ Price, data = Carseats)
summary(m_price)##
## Call:
## lm(formula = Sales ~ Price, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.5224 -1.8442 -0.1459 1.6503 7.5108
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.641915 0.632812 21.558 <2e-16 ***
## Price -0.053073 0.005354 -9.912 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.532 on 398 degrees of freedom
## Multiple R-squared: 0.198, Adjusted R-squared: 0.196
## F-statistic: 98.25 on 1 and 398 DF, p-value: < 2.2e-16m_Age = lm(Sales ~ Age, data = Carseats)
summary(m_Age)##
## Call:
## lm(formula = Sales ~ Age, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -8.1900 -1.8648 -0.1261 1.7449 8.3969
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 9.65115 0.47365 20.376 < 2e-16 ***
## Age -0.04041 0.00850 -4.754 2.79e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.751 on 398 degrees of freedom
## Multiple R-squared: 0.05374, Adjusted R-squared: 0.05136
## F-statistic: 22.6 on 1 and 398 DF, p-value: 2.789e-06m_Edu = lm(Sales ~ Education, data = Carseats)
summary(m_Edu)##
## Call:
## lm(formula = Sales ~ Education, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.4347 -2.0322 -0.0357 1.7788 8.6113
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 8.27461 0.76304 10.844 <2e-16 ***
## Education -0.05599 0.05395 -1.038 0.3
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.824 on 398 degrees of freedom
## Multiple R-squared: 0.002699, Adjusted R-squared: 0.0001936
## F-statistic: 1.077 on 1 and 398 DF, p-value: 0.2999Scatter plot
plot(Carseats$CompPrice,Carseats$Sales)
abline(m_comp,col="red")
plot(Carseats$Income,Carseats$Sales)
abline(m_income,col="red")
plot(Carseats$Advertising,Carseats$Sales)
abline(m_adver,col="red")
plot(Carseats$Population,Carseats$Sales)
abline(m_popul,col="red")
plot(Carseats$Price,Carseats$Sales)
abline(m_price,col="red")
plot(Carseats$Age,Carseats$Sales)
abline(m_Age,col="red")
plot(Carseats$Education,Carseats$Sales)
abline(m_Edu,col="red")
Diagnostic Plots
1)잔차가 선형성인지, 선 주위로 무작위로 분포했는지: 모두 수평에
가까운 직선의 형태를 가지고 있다. 2)잔차가 정규분포를 따르는지(직선에
근접했는지): 모두 정규분포를 따른다고 볼 수 있다.
3)잔차가 동일분산을 따르는지(값들이 패턴없이 일정하게 분포되어있는지):
CompPrice와 Price의 잔차는 뭉쳐있는 모습이 있고, income과 Population은
거의 균등하게 분포되어 있다.
4)이상값이 얼마나 영향을 주는지(점선으로 된 곡선 영역 밖에 있는지) :
이상값들이 거의 없으며 큰 영향을 주는 이상값들도 없다.
par(mfrow=c(2,2))
plot(m_comp)
par(mfrow=c(2,2))
plot(m_income)
par(mfrow=c(2,2))
plot(m_adver)
par(mfrow=c(2,2))
plot(m_popul)
par(mfrow=c(2,2))
plot(m_price)
par(mfrow=c(2,2))
plot(m_Age)
par(mfrow=c(2,2))
plot(m_Edu)
- Multiple regression model
For which predictors can we reject the null hypothesis \(H_0: \beta_j=0\)?
Population과 Education을 제외한 모든 변수들의 계수가 통계적으로 유의하므로 귀무가설을 기각할 수 있다. 결정계수 값이 0.5이상이 나옴.
m_car = lm(Sales ~ .-ShelveLoc-Urban-US, data=Carseats)
summary(m_car)##
## Call:
## lm(formula = Sales ~ . - ShelveLoc - Urban - US, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -5.0598 -1.3515 -0.1739 1.1331 4.8304
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 7.7076934 1.1176260 6.896 2.15e-11 ***
## CompPrice 0.0939149 0.0078395 11.980 < 2e-16 ***
## Income 0.0128717 0.0034757 3.703 0.000243 ***
## Advertising 0.1308637 0.0151219 8.654 < 2e-16 ***
## Population -0.0001239 0.0006877 -0.180 0.857092
## Price -0.0925226 0.0050521 -18.314 < 2e-16 ***
## Age -0.0449743 0.0060083 -7.485 4.75e-13 ***
## Education -0.0399844 0.0371257 -1.077 0.282142
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.929 on 392 degrees of freedom
## Multiple R-squared: 0.5417, Adjusted R-squared: 0.5335
## F-statistic: 66.18 on 7 and 392 DF, p-value: < 2.2e-16- How do your results from (a) compare to your results from (b)? Create a plot displaying the univariate regression coefficients from (a) on the x-axis, and the multiple regression coefficients from (b) on the y-axis. That is, each predictor is displayed as a single point in the plot. Its coefficient in a simple linear regression model is shown on the x-axis, and its coefficient estimate in the multiple linear regression model is shown on the y-axis.
단순회귀분석과 다중회귀분석의 값들들이 서로 유사한 것들이 보인다.
xc = c(coefficients(m_comp)[2],
coefficients(m_income)[2],
coefficients(m_adver)[2],
coefficients(m_popul)[2],
coefficients(m_price)[2],
coefficients(m_Age)[2],
coefficients(m_Edu)[2])
yc = coefficients(m_car)[2:8]
plot(xc, yc)
- Is there evidence of non-linear association between any of the predictors and the response? To answer this question, for each predictor X, fit a model of the form
Carseats 데이터의 경우 모두 비선형관계가 없음. \[ Y=\beta_0+\beta_1X+\beta_2X^2+\beta_3X^3+\epsilon \]
unlin_comp <- lm(Sales~ CompPrice + I(CompPrice^2) +I(CompPrice^3),data=Carseats)
summary(unlin_comp)##
## Call:
## lm(formula = Sales ~ CompPrice + I(CompPrice^2) + I(CompPrice^3),
## data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.7428 -2.0330 -0.0746 1.6843 8.4670
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.074e+01 3.096e+01 1.316 0.189
## CompPrice -7.729e-01 7.579e-01 -1.020 0.308
## I(CompPrice^2) 5.780e-03 6.121e-03 0.944 0.346
## I(CompPrice^3) -1.387e-05 1.631e-05 -0.850 0.396
##
## Residual standard error: 2.82 on 396 degrees of freedom
## Multiple R-squared: 0.01075, Adjusted R-squared: 0.003252
## F-statistic: 1.434 on 3 and 396 DF, p-value: 0.2324unlin_income <- lm(Sales~ Income + I(Income^2)+I(Income^3),data=Carseats)
summary(unlin_income)##
## Call:
## lm(formula = Sales ~ Income + I(Income^2) + I(Income^3), data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.3307 -1.9700 -0.1845 1.7124 8.8275
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.083e+00 2.073e+00 2.935 0.00353 **
## Income 2.465e-02 1.070e-01 0.230 0.81791
## I(Income^2) 4.036e-06 1.641e-03 0.002 0.99804
## I(Income^3) -6.203e-07 7.689e-06 -0.081 0.93574
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.8 on 396 degrees of freedom
## Multiple R-squared: 0.02424, Adjusted R-squared: 0.01685
## F-statistic: 3.279 on 3 and 396 DF, p-value: 0.02101unlin_adver <- lm(Sales~Advertising + I(Advertising^2) +I(Advertising^3),data=Carseats)
summary(unlin_adver)##
## Call:
## lm(formula = Sales ~ Advertising + I(Advertising^2) + I(Advertising^3),
## data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.585 -1.956 -0.121 1.676 8.531
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.8100240 0.2186284 31.149 <2e-16 ***
## Advertising -0.0056316 0.1211034 -0.047 0.963
## I(Advertising^2) 0.0148466 0.0139508 1.064 0.288
## I(Advertising^3) -0.0004311 0.0004053 -1.064 0.288
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.726 on 396 degrees of freedom
## Multiple R-squared: 0.07531, Adjusted R-squared: 0.0683
## F-statistic: 10.75 on 3 and 396 DF, p-value: 8.322e-07unlin_popul <- lm(Sales~ Population + I(Population^2) +I(Population^3),data=Carseats)
summary(unlin_popul)##
## Call:
## lm(formula = Sales ~ Population + I(Population^2) + I(Population^3),
## data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.7285 -2.0013 -0.0518 1.7449 8.5792
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 7.284e+00 6.325e-01 11.518 <2e-16 ***
## Population -2.110e-03 1.043e-02 -0.202 0.840
## I(Population^2) 2.083e-05 4.598e-05 0.453 0.651
## I(Population^3) -3.204e-08 5.769e-08 -0.555 0.579
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.829 on 396 degrees of freedom
## Multiple R-squared: 0.004224, Adjusted R-squared: -0.00332
## F-statistic: 0.56 on 3 and 396 DF, p-value: 0.6417unlin_price <- lm(Sales~ Price + I(Price^2) +I(Price^3),data=Carseats)
summary(unlin_price)##
## Call:
## lm(formula = Sales ~ Price + I(Price^2) + I(Price^3), data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.508 -1.866 -0.115 1.621 7.572
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.634e+01 4.033e+00 4.050 6.15e-05 ***
## Price -1.272e-01 1.125e-01 -1.131 0.259
## I(Price^2) 6.424e-04 1.025e-03 0.626 0.531
## I(Price^3) -1.771e-06 3.035e-06 -0.583 0.560
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.537 on 396 degrees of freedom
## Multiple R-squared: 0.1989, Adjusted R-squared: 0.1928
## F-statistic: 32.78 on 3 and 396 DF, p-value: < 2.2e-16unlin_Age <- lm(Sales~ Age + I(Age^2) +I(Age^3),data=Carseats)
summary(unlin_Age)##
## Call:
## lm(formula = Sales ~ Age + I(Age^2) + I(Age^3), data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.6426 -2.0234 0.0684 1.7014 8.1503
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.724e+00 5.037e+00 1.335 0.183
## Age 6.639e-02 3.177e-01 0.209 0.835
## I(Age^2) -5.382e-04 6.294e-03 -0.086 0.932
## I(Age^3) -5.678e-06 3.954e-05 -0.144 0.886
##
## Residual standard error: 2.735 on 396 degrees of freedom
## Multiple R-squared: 0.06909, Adjusted R-squared: 0.06204
## F-statistic: 9.797 on 3 and 396 DF, p-value: 3.011e-06unlin_Edu <- lm(Sales~ Education + I(Education^2) +I(Education^3),data=Carseats)
summary(unlin_Edu)##
## Call:
## lm(formula = Sales ~ Education + I(Education^2) + I(Education^3),
## data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.4571 -2.0296 -0.0405 1.7992 8.6928
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 54.49341 28.86171 1.888 0.0597 .
## Education -10.01933 6.41892 -1.561 0.1193
## I(Education^2) 0.69968 0.46731 1.497 0.1351
## I(Education^3) -0.01605 0.01115 -1.440 0.1507
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.819 on 396 degrees of freedom
## Multiple R-squared: 0.01105, Adjusted R-squared: 0.003563
## F-statistic: 1.476 on 3 and 396 DF, p-value: 0.2207