Regression Model
Jackson Pham
2024-04-23
Executive Summary
This analysis uses the mtcars dataset to investigate the impact of
transmission type on fuel efficiency, measured in miles per gallon
(MPG). The results confirm that manual transmissions generally achieve
higher MPG than automatics, with a coding of 0 for automatic
transmission and 1 for manual transmission. Supported by statistical
modeling and diagnostics, these findings underscore the fuel efficiency
advantages of manual transmissions, providing valuable insights for
consumers and the automotive industry. Conclusively, our analysis
suggests that cars with manual transmission add an average of
14.079 + (-4.141) * wt MPG compared to automatic
transmission cars.
# Load libraries
library(ggplot2)Exploratory Data Analysis
# Load the mtcars dataset
cars <- mtcars
# Convert the values in "am" column, 1 for automatic, and 0 for manual
cars$am <- factor(ifelse(cars$am == 0, "Automatic", "Manual"))
# View the dataset
head(cars)## mpg cyl disp hp drat wt qsec vs am gear carb
## Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0 Manual 4 4
## Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0 Manual 4 4
## Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1 Manual 4 1
## Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1 Automatic 3 1
## Hornet Sportabout 18.7 8 360 175 3.15 3.440 17.02 0 Automatic 3 2
## Valiant 18.1 6 225 105 2.76 3.460 20.22 1 Automatic 3 1Inference
# Simple t.test on the mtcars data
cars_t_test <- t.test(mpg ~ am, data = cars)
cars_t_test$p.value## [1] 0.001373638With a p-value of 0.001373638 from the t-test, we reject the null hypothesis, indicating a significant difference in miles per gallon (mpg) between manual and automatic transmission.
# Perform some basic EDA. Plot the distribution of MPG for cars with manual and automatic transmissions to see if there's a visible difference.
cars_plot <- ggplot(cars, aes(x = factor(am), y = mpg, fill = factor(am))) +
geom_boxplot() +
labs(x = "Transmission (0 = Automatic, 1 = Manual)", y = "Miles per Gallon", fill = "Transmission") +
theme_minimal()
# Print the plot
cars_plot
From the plot, it is evident that there is a significant difference in miles per gallon (MPG) between cars with automatic transmissions and those with manual transmissions.
Statistical Modeling
# Full regression model to help finding the step model in the next step
full_fit <- lm(mpg ~ . , data = cars)
# Using step model to find the best fit regression
stepModel <- step(full_fit, k=log(nrow(cars)))## Start: AIC=87.02
## mpg ~ cyl + disp + hp + drat + wt + qsec + vs + am + gear + carb
##
## Df Sum of Sq RSS AIC
## - cyl 1 0.0799 147.57 83.572
## - vs 1 0.1601 147.66 83.590
## - carb 1 0.4067 147.90 83.643
## - gear 1 1.3531 148.85 83.847
## - drat 1 1.6270 149.12 83.906
## - disp 1 3.9167 151.41 84.394
## - hp 1 6.8399 154.33 85.006
## - qsec 1 8.8641 156.36 85.423
## - am 1 10.5467 158.04 85.765
## <none> 147.49 87.021
## - wt 1 27.0144 174.51 88.937
##
## Step: AIC=83.57
## mpg ~ disp + hp + drat + wt + qsec + vs + am + gear + carb
##
## Df Sum of Sq RSS AIC
## - vs 1 0.2685 147.84 80.165
## - carb 1 0.5201 148.09 80.219
## - gear 1 1.8211 149.40 80.499
## - drat 1 1.9826 149.56 80.534
## - disp 1 3.9009 151.47 80.942
## - hp 1 7.3632 154.94 81.665
## - qsec 1 10.0933 157.67 82.224
## - am 1 11.8359 159.41 82.575
## <none> 147.57 83.572
## - wt 1 27.0280 174.60 85.488
##
## Step: AIC=80.16
## mpg ~ disp + hp + drat + wt + qsec + am + gear + carb
##
## Df Sum of Sq RSS AIC
## - carb 1 0.6855 148.53 76.847
## - gear 1 2.1437 149.99 77.160
## - drat 1 2.2139 150.06 77.175
## - disp 1 3.6467 151.49 77.479
## - hp 1 7.1060 154.95 78.201
## - am 1 11.5694 159.41 79.110
## - qsec 1 15.6830 163.53 79.925
## <none> 147.84 80.165
## - wt 1 27.3799 175.22 82.136
##
## Step: AIC=76.85
## mpg ~ disp + hp + drat + wt + qsec + am + gear
##
## Df Sum of Sq RSS AIC
## - gear 1 1.565 150.09 73.717
## - drat 1 1.932 150.46 73.795
## - disp 1 10.110 158.64 75.489
## - am 1 12.323 160.85 75.932
## - hp 1 14.826 163.35 76.426
## <none> 148.53 76.847
## - qsec 1 26.408 174.94 78.618
## - wt 1 69.127 217.66 85.610
##
## Step: AIC=73.72
## mpg ~ disp + hp + drat + wt + qsec + am
##
## Df Sum of Sq RSS AIC
## - drat 1 3.345 153.44 70.956
## - disp 1 8.545 158.64 72.023
## - hp 1 13.285 163.38 72.965
## <none> 150.09 73.717
## - am 1 20.036 170.13 74.261
## - qsec 1 25.574 175.67 75.286
## - wt 1 67.572 217.66 82.146
##
## Step: AIC=70.96
## mpg ~ disp + hp + wt + qsec + am
##
## Df Sum of Sq RSS AIC
## - disp 1 6.629 160.07 68.844
## - hp 1 12.572 166.01 70.011
## <none> 153.44 70.956
## - qsec 1 26.470 179.91 72.583
## - am 1 32.198 185.63 73.586
## - wt 1 69.043 222.48 79.380
##
## Step: AIC=68.84
## mpg ~ hp + wt + qsec + am
##
## Df Sum of Sq RSS AIC
## - hp 1 9.219 169.29 67.170
## <none> 160.07 68.844
## - qsec 1 20.225 180.29 69.186
## - am 1 25.993 186.06 70.193
## - wt 1 78.494 238.56 78.147
##
## Step: AIC=67.17
## mpg ~ wt + qsec + am
##
## Df Sum of Sq RSS AIC
## <none> 169.29 67.170
## - am 1 26.178 195.46 68.306
## - qsec 1 109.034 278.32 79.614
## - wt 1 183.347 352.63 87.187# Extract coefficients
summary(stepModel)##
## Call:
## lm(formula = mpg ~ wt + qsec + am, data = cars)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.4811 -1.5555 -0.7257 1.4110 4.6610
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 9.6178 6.9596 1.382 0.177915
## wt -3.9165 0.7112 -5.507 6.95e-06 ***
## qsec 1.2259 0.2887 4.247 0.000216 ***
## amManual 2.9358 1.4109 2.081 0.046716 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.459 on 28 degrees of freedom
## Multiple R-squared: 0.8497, Adjusted R-squared: 0.8336
## F-statistic: 52.75 on 3 and 28 DF, p-value: 1.21e-11Summary of the
lm(formula = mpg ~ wt + qsec + am, data = cars) shows that
it have 3 significant variable of wt, qsec, and am that the Adjusted
R-squared value is 0.8336, which means that the model can explain about
83% of the variance of the MPG variable.
# Further analysis of the regression model between wt and am
full_fit_adjusted <- lm(mpg ~ wt + qsec + am + wt:am, data = cars)
# Extract coefficients
summary(full_fit_adjusted)##
## Call:
## lm(formula = mpg ~ wt + qsec + am + wt:am, data = cars)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.5076 -1.3801 -0.5588 1.0630 4.3684
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 9.723 5.899 1.648 0.110893
## wt -2.937 0.666 -4.409 0.000149 ***
## qsec 1.017 0.252 4.035 0.000403 ***
## amManual 14.079 3.435 4.099 0.000341 ***
## wt:amManual -4.141 1.197 -3.460 0.001809 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.084 on 27 degrees of freedom
## Multiple R-squared: 0.8959, Adjusted R-squared: 0.8804
## F-statistic: 58.06 on 4 and 27 DF, p-value: 7.168e-13Including the interaction term wt:am in the model
lm(formula = mpg ~ wt + qsec + am, data = cars) is
essential to determine if the effect of a car’s weight on fuel
efficiency varies with the type of transmission. The significance of
this interaction indicates that the impact of weight on MPG changes
depending on whether the car has a manual or automatic transmission.
The best_fit model yield R-adjusted value of 88.04% which means that the model can explain about 88% of the variance in MPG variable.
# Using anova to find the best fit model
anova(full_fit_adjusted, full_fit)## Analysis of Variance Table
##
## Model 1: mpg ~ wt + qsec + am + wt:am
## Model 2: mpg ~ cyl + disp + hp + drat + wt + qsec + vs + am + gear + carb
## Res.Df RSS Df Sum of Sq F Pr(>F)
## 1 27 117.28
## 2 21 147.49 6 -30.218# Pick the best model mpg ~ wt + qsec + am + wt:am with the highest R-adjusted value
summary(full_fit_adjusted)$coef## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 9.723053 5.8990407 1.648243 0.1108925394
## wt -2.936531 0.6660253 -4.409038 0.0001488947
## qsec 1.016974 0.2520152 4.035366 0.0004030165
## amManual 14.079428 3.4352512 4.098515 0.0003408693
## wt:amManual -4.141376 1.1968119 -3.460340 0.0018085763Based on the result coefficient of the best_fit model, we can
conclude that car with manual transmission add
14.079 + (-4.141) * wt MPG on average compare to automatic
transmission car.
Residual Analysis and Diagnostics
# As for the Dfbetas, the measure of how much an observation has effected the estimate of a regression coefficient, this is the following result:
sum((abs(dfbetas(full_fit_adjusted)))>1)## [1] 0The sum((abs(dfbetas(best_fit)))>1) is qual to 0.
Therefore, the above analyses meet all basic assumptions of linear
regression and well answer the questions.
# Plot statistical value
par(mfrow = c(2, 2))
plot(full_fit_adjusted)