Assigment #2
2024-02-21
library(tidyverse)## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.2 ✔ readr 2.1.4
## ✔ forcats 1.0.0 ✔ stringr 1.5.0
## ✔ ggplot2 3.4.4 ✔ tibble 3.2.1
## ✔ lubridate 1.9.2 ✔ tidyr 1.3.0
## ✔ purrr 1.0.1
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errorslibrary(MASS)##
## Attaching package: 'MASS'
##
## The following object is masked from 'package:dplyr':
##
## selectlibrary(modelr)
library(car)## Loading required package: carData
##
## Attaching package: 'car'
##
## The following object is masked from 'package:dplyr':
##
## recode
##
## The following object is masked from 'package:purrr':
##
## some- Carefully explain the differences between the KNN classifier and KNN regression methods.
The primary distinction lies in how they handle prediction outcomes. KNN regression computes the average value of nearby neighbors’ responses, ideal for continuous variables. Conversely, KNN classification determines the predominant class among nearby neighbors, effectively delineating the space into regions based on decision boundaries.
getwd()## [1] "C:/Users/omarc/OneDrive/Documentos/CVS files/ALL CSV FILES - 2nd Edition"setwd("C:/Users/omarc/OneDrive/Documentos/CVS files/ALL CSV FILES - 2nd Edition")# Read the CSV file
auto_data <- read.csv("Auto.csv", header = TRUE)
# Check the structure of the data frame
str(auto_data)## 'data.frame': 397 obs. of 9 variables:
## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ...
## $ cylinders : int 8 8 8 8 8 8 8 8 8 8 ...
## $ displacement: num 307 350 318 304 302 429 454 440 455 390 ...
## $ horsepower : chr "130" "165" "150" "150" ...
## $ weight : int 3504 3693 3436 3433 3449 4341 4354 4312 4425 3850 ...
## $ acceleration: num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ...
## $ year : int 70 70 70 70 70 70 70 70 70 70 ...
## $ origin : int 1 1 1 1 1 1 1 1 1 1 ...
## $ name : chr "chevrolet chevelle malibu" "buick skylark 320" "plymouth satellite" "amc rebel sst" ...# View the first few rows of the data frame
head(auto_data)## mpg cylinders displacement horsepower weight acceleration year origin
## 1 18 8 307 130 3504 12.0 70 1
## 2 15 8 350 165 3693 11.5 70 1
## 3 18 8 318 150 3436 11.0 70 1
## 4 16 8 304 150 3433 12.0 70 1
## 5 17 8 302 140 3449 10.5 70 1
## 6 15 8 429 198 4341 10.0 70 1
## name
## 1 chevrolet chevelle malibu
## 2 buick skylark 320
## 3 plymouth satellite
## 4 amc rebel sst
## 5 ford torino
## 6 ford galaxie 500# Convert "horsepower" to numeric
auto_data$horsepower <- as.numeric(auto_data$horsepower)## Warning: NAs introduced by coercion# Verify the conversion
class(auto_data$horsepower)## [1] "numeric"#Clean data
auto_data <- na.omit(auto_data)plot(auto_data)
auto_data %>%
dplyr::select(-name) %>%
cor()## mpg cylinders displacement horsepower weight
## mpg 1.0000000 -0.7776175 -0.8051269 -0.7784268 -0.8322442
## cylinders -0.7776175 1.0000000 0.9508233 0.8429834 0.8975273
## displacement -0.8051269 0.9508233 1.0000000 0.8972570 0.9329944
## horsepower -0.7784268 0.8429834 0.8972570 1.0000000 0.8645377
## weight -0.8322442 0.8975273 0.9329944 0.8645377 1.0000000
## acceleration 0.4233285 -0.5046834 -0.5438005 -0.6891955 -0.4168392
## year 0.5805410 -0.3456474 -0.3698552 -0.4163615 -0.3091199
## origin 0.5652088 -0.5689316 -0.6145351 -0.4551715 -0.5850054
## acceleration year origin
## mpg 0.4233285 0.5805410 0.5652088
## cylinders -0.5046834 -0.3456474 -0.5689316
## displacement -0.5438005 -0.3698552 -0.6145351
## horsepower -0.6891955 -0.4163615 -0.4551715
## weight -0.4168392 -0.3091199 -0.5850054
## acceleration 1.0000000 0.2903161 0.2127458
## year 0.2903161 1.0000000 0.1815277
## origin 0.2127458 0.1815277 1.0000000lm_mpg_allothers <- lm(mpg ~ .-name, data = auto_data)
summary(lm_mpg_allothers)##
## Call:
## lm(formula = mpg ~ . - name, data = auto_data)
##
## Residuals:
## Min 1Q Median 3Q Max
## -9.5903 -2.1565 -0.1169 1.8690 13.0604
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -17.218435 4.644294 -3.707 0.00024 ***
## cylinders -0.493376 0.323282 -1.526 0.12780
## displacement 0.019896 0.007515 2.647 0.00844 **
## horsepower -0.016951 0.013787 -1.230 0.21963
## weight -0.006474 0.000652 -9.929 < 2e-16 ***
## acceleration 0.080576 0.098845 0.815 0.41548
## year 0.750773 0.050973 14.729 < 2e-16 ***
## origin 1.426141 0.278136 5.127 4.67e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.328 on 384 degrees of freedom
## Multiple R-squared: 0.8215, Adjusted R-squared: 0.8182
## F-statistic: 252.4 on 7 and 384 DF, p-value: < 2.2e-16Is there a relationship between the predictors and the response? We can conclude that there is a relationship between at least one of the predictors and the response variable, since the p-value for the F-statistic is almost zero.
Which predictors appear to have a statistically significant relationship to the response? The predictors which shown a statiscally significant relationship are displacement, weight, year and origin.
What does the coeficient for the year variable suggest?
Also, the coefficent for the year variable suggest that, holding constant all the other predictors, each year the cars increase their efficiency by 0.75 miles per gallon, on average.
- Use the plot() function to produce diagnostic plots of the linear regression fit. Comment on any problems you see with the fit. Do the residual plots suggest any unusually large outliers? Does the leverage plot identify any observations with unusually high leverage?
par(mfrow = c(2, 2))
plot(lm_mpg_allothers)
There are no particularly high residuals / outliers in the model.
However, the observation n°14 has a relatively high leverage (compared
to the other observations).
- Use the * and : symbols to fit linear regression models with interaction effects. Do any interactions appear to be statistically significant?
lm(mpg ~ year*weight*horsepower, data = auto_data) %>%
summary()##
## Call:
## lm(formula = mpg ~ year * weight * horsepower, data = auto_data)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.8796 -1.6800 -0.0639 1.2556 11.5534
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -1.577e+02 3.694e+01 -4.270 2.47e-05 ***
## year 2.886e+00 4.906e-01 5.884 8.75e-09 ***
## weight 1.889e-02 1.286e-02 1.469 0.14273
## horsepower 1.696e+00 4.208e-01 4.029 6.74e-05 ***
## year:weight -3.943e-04 1.712e-04 -2.303 0.02180 *
## year:horsepower -2.540e-02 5.652e-03 -4.494 9.27e-06 ***
## weight:horsepower -3.218e-04 1.112e-04 -2.893 0.00403 **
## year:weight:horsepower 4.972e-06 1.500e-06 3.314 0.00101 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.826 on 384 degrees of freedom
## Multiple R-squared: 0.8712, Adjusted R-squared: 0.8689
## F-statistic: 371.2 on 7 and 384 DF, p-value: < 2.2e-16In this case all the interaction coefficients are statistically significant.
- Try a few different transformations of the variables, such as (log(X)), ((X)), (X^2). Comment on your findings.
lm(mpg ~ horsepower + I(horsepower^2), data = auto_data) %>% plot()
lm(mpg ~ poly(horsepower, 3), data = auto_data) %>% summary()##
## Call:
## lm(formula = mpg ~ poly(horsepower, 3), data = auto_data)
##
## Residuals:
## Min 1Q Median 3Q Max
## -14.7039 -2.4491 -0.1519 2.2035 15.8159
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 23.446 0.221 106.105 <2e-16 ***
## poly(horsepower, 3)1 -120.138 4.375 -27.460 <2e-16 ***
## poly(horsepower, 3)2 44.090 4.375 10.078 <2e-16 ***
## poly(horsepower, 3)3 -3.949 4.375 -0.903 0.367
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.375 on 388 degrees of freedom
## Multiple R-squared: 0.6882, Adjusted R-squared: 0.6858
## F-statistic: 285.5 on 3 and 388 DF, p-value: < 2.2e-16In previous diagnostic plots we seen that the relationship between horsepower and mpg was probably non-linear (because the residuals showed a pattern across the horsepower variable). Now we can see that adding a cuadratic term for horsepower fixes this, and captures the non-linear relationship present in the data. However, adding a cubic term doesn’t increses the explanatory power of the model.
- This question should be answered using the Carseats data set
- Fit a multiple regression model to predict Sales using Price, Urban, and Urban.
# Read the CSV file
Carseats <- read.csv("Carseats.csv", header = TRUE)
# Check the structure of the data frame
str(Carseats)## 'data.frame': 400 obs. of 11 variables:
## $ Sales : num 9.5 11.22 10.06 7.4 4.15 ...
## $ CompPrice : int 138 111 113 117 141 124 115 136 132 132 ...
## $ Income : int 73 48 35 100 64 113 105 81 110 113 ...
## $ Advertising: int 11 16 10 4 3 13 0 15 0 0 ...
## $ Population : int 276 260 269 466 340 501 45 425 108 131 ...
## $ Price : int 120 83 80 97 128 72 108 120 124 124 ...
## $ ShelveLoc : chr "Bad" "Good" "Medium" "Medium" ...
## $ Age : int 42 65 59 55 38 78 71 67 76 76 ...
## $ Education : int 17 10 12 14 13 16 15 10 10 17 ...
## $ Urban : chr "Yes" "Yes" "Yes" "Yes" ...
## $ US : chr "Yes" "Yes" "Yes" "Yes" ...# View the first few rows of the data frame
head(Carseats)## Sales CompPrice Income Advertising Population Price ShelveLoc Age Education
## 1 9.50 138 73 11 276 120 Bad 42 17
## 2 11.22 111 48 16 260 83 Good 65 10
## 3 10.06 113 35 10 269 80 Medium 59 12
## 4 7.40 117 100 4 466 97 Medium 55 14
## 5 4.15 141 64 3 340 128 Bad 38 13
## 6 10.81 124 113 13 501 72 Bad 78 16
## Urban US
## 1 Yes Yes
## 2 Yes Yes
## 3 Yes Yes
## 4 Yes Yes
## 5 Yes No
## 6 No Yes# Load the Carseats dataset
data("Carseats")## Warning in data("Carseats"): data set 'Carseats' not found# Convert "ShelveLoc" to factor with specified levels
Carseats$ShelveLoc <- factor(Carseats$ShelveLoc, levels = c("Bad", "Medium", "Good"))
# Convert "Urban" to numeric
Carseats$Urban <- ifelse(Carseats$Urban == "No", 0, 1)
# Convert "US" to numeric
Carseats$US <- ifelse(Carseats$US == "No", 0, 1)
# Verify the conversion
str(Carseats)## 'data.frame': 400 obs. of 11 variables:
## $ Sales : num 9.5 11.22 10.06 7.4 4.15 ...
## $ CompPrice : int 138 111 113 117 141 124 115 136 132 132 ...
## $ Income : int 73 48 35 100 64 113 105 81 110 113 ...
## $ Advertising: int 11 16 10 4 3 13 0 15 0 0 ...
## $ Population : int 276 260 269 466 340 501 45 425 108 131 ...
## $ Price : int 120 83 80 97 128 72 108 120 124 124 ...
## $ ShelveLoc : Factor w/ 3 levels "Bad","Medium",..: 1 3 2 2 1 1 2 3 2 2 ...
## $ Age : int 42 65 59 55 38 78 71 67 76 76 ...
## $ Education : int 17 10 12 14 13 16 15 10 10 17 ...
## $ Urban : num 1 1 1 1 1 0 1 1 0 0 ...
## $ US : num 1 1 1 1 0 1 0 1 0 1 ...lm_carseats <- lm(Sales ~ Price + Urban + US, data = Carseats)
summary(lm_carseats)##
## Call:
## lm(formula = Sales ~ Price + Urban + US, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.9206 -1.6220 -0.0564 1.5786 7.0581
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.043469 0.651012 20.036 < 2e-16 ***
## Price -0.054459 0.005242 -10.389 < 2e-16 ***
## Urban -0.021916 0.271650 -0.081 0.936
## US 1.200573 0.259042 4.635 4.86e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.472 on 396 degrees of freedom
## Multiple R-squared: 0.2393, Adjusted R-squared: 0.2335
## F-statistic: 41.52 on 3 and 396 DF, p-value: < 2.2e-16- Provide an interpretation of each coefficient in the model. Be careful—some of the variables in the model are qualitative!
ANSWER:
The Intercept estimate tell us that when all the other variables are zero (Price is equal to zero, the store is in a rural location, and is not located in the US) we can expect to see sales of 13,043 (asumming that the effects are linear and additive).
The Price coefficient indicates that an increase of 1 in the final price brings sales down by 54 units. The UrbanYes coefficient is not statistically significant, so we can’t conclude that the sales level is different in urban areas vs. rural areas (holding constant all the other variables). Finally, the USYes coefficient is significant, and its value implies that a store in the US sales 1,200 more units, on average, than a non-US store.
lm_carseats <- lm(Sales ~ Price + Urban + US, data = Carseats)
summary(lm_carseats)##
## Call:
## lm(formula = Sales ~ Price + Urban + US, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.9206 -1.6220 -0.0564 1.5786 7.0581
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.043469 0.651012 20.036 < 2e-16 ***
## Price -0.054459 0.005242 -10.389 < 2e-16 ***
## Urban -0.021916 0.271650 -0.081 0.936
## US 1.200573 0.259042 4.635 4.86e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.472 on 396 degrees of freedom
## Multiple R-squared: 0.2393, Adjusted R-squared: 0.2335
## F-statistic: 41.52 on 3 and 396 DF, p-value: < 2.2e-16Write out the model in equation form, being careful to handle the qualitative variables properly. Salesi = β0 + β1 * Pricei +((β2 if Urban ((β3&if US 0&if not Urban)) + 0if not US))
For which of the predictors can you reject the null hypothesis (H_0 : _j = 0)?
ANSWER: The Intercept, Price, Urban, and US.
- On the basis of your response to the previous question, fit a smaller model that only uses the predictors for which there is evidence of association with the outcome.
lm_carseats_simp <- lm(Sales ~ Price + US, data = Carseats)
summary(lm_carseats_simp)##
## Call:
## lm(formula = Sales ~ Price + US, data = Carseats)
##
## Residuals:
## Min 1Q Median 3Q Max
## -6.9269 -1.6286 -0.0574 1.5766 7.0515
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.03079 0.63098 20.652 < 2e-16 ***
## Price -0.05448 0.00523 -10.416 < 2e-16 ***
## US 1.19964 0.25846 4.641 4.71e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.469 on 397 degrees of freedom
## Multiple R-squared: 0.2393, Adjusted R-squared: 0.2354
## F-statistic: 62.43 on 2 and 397 DF, p-value: < 2.2e-16- How well do the models in (a) and (e) fit the data?
ANSWER: The fit of both models is very similar, but the second model has a better adjusted R-squared, because we have removed a variable with no explanatory power (adjusted R-squared penalizes for each variable we add to the model, so including variables with little correlation with the response don’t bring the number upwards, as it happens with the simple R-squared).
- Using the model from (e), obtain 95% confidence intervals for the coefficient(s).
confint(lm_carseats_simp)## 2.5 % 97.5 %
## (Intercept) 11.79032020 14.27126531
## Price -0.06475984 -0.04419543
## US 0.69151957 1.70776632- Is there evidence of outliers or high leverage observations in the model from (e)?
par(mfrow = c(2,2))
plot(lm_carseats_simp)
ANSWER: In the diagnostic plots we see little evidence of outlier
observations. There seems to be an observation with relatively high
leverage, but it’s not problematic since it has a very low residual.
- This problem involves simple linear regression without an intercept.
- Recall that the coefficient estimate () for the linear regression of (Y) onto (X) without an intercept is given by (3.38). Under what circumstance is the coefficient estimate for the regression of (X) onto (Y) the same as the coefficient estimate for the regression of (Y) onto (X)?
The numerator in the () is always the same weather the regression is y ~ x or x ~ y. However, the denominator is usually different. If we want the whole coefficient to be the same in both cases, then this equality must be true:
( = )
- Generate an example in R with (n = 100) observations in which the coefficient estimate for the regression of (X) onto (Y) is different from the coefficient estimate for the regression of (Y) onto (X).
set.seed(1989)
x_1 <- rnorm(100)
y_1 <- rnorm(100) + 2*x_1
lm(x_1 ~ y_1 + 0) %>% summary()##
## Call:
## lm(formula = x_1 ~ y_1 + 0)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.94313 -0.25160 0.00501 0.26080 1.28222
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## y_1 0.41411 0.01899 21.81 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.4325 on 99 degrees of freedom
## Multiple R-squared: 0.8277, Adjusted R-squared: 0.826
## F-statistic: 475.7 on 1 and 99 DF, p-value: < 2.2e-16lm(y_1 ~ x_1 + 0) %>% summary()##
## Call:
## lm(formula = y_1 ~ x_1 + 0)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.69131 -0.63116 -0.05623 0.52089 2.72321
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## x_1 1.99887 0.09164 21.81 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.9503 on 99 degrees of freedom
## Multiple R-squared: 0.8277, Adjusted R-squared: 0.826
## F-statistic: 475.7 on 1 and 99 DF, p-value: < 2.2e-16- Generate an example in R with (n = 100) observations in which the coefficient estimate for the regression of (X) onto (Y) is the same as the coefficient estimate for the regression of (Y) onto (X).
x_2 <- rnorm(100)
y_2 <- sample(x_2, 100, replace = FALSE)
lm(x_2 ~ y_2 + 0) %>% summary()##
## Call:
## lm(formula = x_2 ~ y_2 + 0)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.15853 -0.68461 -0.07436 0.63679 2.64401
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## y_2 0.17061 0.09903 1.723 0.088 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.965 on 99 degrees of freedom
## Multiple R-squared: 0.02911, Adjusted R-squared: 0.0193
## F-statistic: 2.968 on 1 and 99 DF, p-value: 0.08804lm(y_2 ~ x_2 + 0) %>% summary()##
## Call:
## lm(formula = y_2 ~ x_2 + 0)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.12868 -0.65179 -0.07524 0.54794 2.83044
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## x_2 0.17061 0.09903 1.723 0.088 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.965 on 99 degrees of freedom
## Multiple R-squared: 0.02911, Adjusted R-squared: 0.0193
## F-statistic: 2.968 on 1 and 99 DF, p-value: 0.08804