The eternal debate

As readers of this magazine know, the debate between what kind of transmission is better (automatic or manual) has caused heated debates in the automobile community. Tradition, pride, progress and many other values have littered this debate. And in all of this, there is no single objective metric that would give credit to either camp. This article aims to repair that by exploring the mtcars data and defining, once and for all, how transmission impacts car performance (specifically, miles per hour).

#The data

Let’s start by looking at the data.

##                    mpg cyl disp  hp drat    wt  qsec vs am gear carb
## Mazda RX4         21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
## Mazda RX4 Wag     21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
## Datsun 710        22.8   4  108  93 3.85 2.320 18.61  1  1    4    1
## Hornet 4 Drive    21.4   6  258 110 3.08 3.215 19.44  1  0    3    1
## Hornet Sportabout 18.7   8  360 175 3.15 3.440 17.02  0  0    3    2
## Valiant           18.1   6  225 105 2.76 3.460 20.22  1  0    3    1

There sure are a lot of variables here! From some that appear continuous like miles per gallon (mpg) to others that are binary like transmission mode (am) or the shape of the engine (vs). Here dear reader we must remember the difference between independent and dependent variables.

Simple variable analysis

For the next calculation, mpg will be our dependent variable and am our independent variable.

...
## 
## Call:
## lm(formula = mpg ~ as.factor(am), data = mtcars)
## Coefficients:
##                Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      17.147      1.125  15.247 1.13e-15 ***
## as.factor(am)1    7.245      1.764   4.106 0.000285 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 4.902 on 30 degrees of freedom
## Multiple R-squared:  0.3598, Adjusted R-squared:  0.3385 
## F-statistic: 16.86 on 1 and 30 DF,  p-value: 0.000285
...

With a simple linear model we can already see that the mode of transmission has a significant effect. With a p-value of less than 0.001, this model indicates that having a manual transmission is expected to be correlated with more than 7 miles per gallon! Is this the end of the discussion?

As much as we’d like to have this case closed, I’m afraid no good analysis can be done without looking at the residuals, that is to say, by evaluating how much of the variance stays unexplained by the model. The plot “Boxplot of Residuals by Transmission Mode” can help us visualize this unexplained variation. And what do we see? A much bigger spread of residuals for those in the manual group (am = 1)!

Multi-variable analysis

This surely seems like the initial model doesn’t account very well for the differences in performance. Let’s try again, now controlling for other relevant variables.

...
## 
## Call:
## lm(formula = mpg ~ as.factor(am) + wt + as.factor(vs), data = mtcars)
## Coefficients:
##                Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     30.0787     3.7480   8.025 9.71e-09 ***
## as.factor(am)1   1.4913     1.4863   1.003 0.324262    
## wt              -3.7845     0.8981  -4.214 0.000236 ***
## as.factor(vs)1   3.6150     1.2761   2.833 0.008454 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.779 on 28 degrees of freedom
## Multiple R-squared:  0.8079, Adjusted R-squared:  0.7873 
## F-statistic: 39.25 on 3 and 28 DF,  p-value: 3.659e-10
...

By controlling for a few more variables, the initial impressive result ceases to be significant at all. In fact, just controlling for the weight of the vehicle and the shape of the engine is enough to give a p-value of 0.32, way too high to determine there’s a real difference between manual and automatic! Instead, weight and shape of the motor seem to be much more relevant measurements.

It wouldn’t be fair to jump to conclusions here, so let’s now see if this model can stand the pressure of analysing the residuals.

At least visibly, in the Boxplot of Residuals by Transmission Mode (controlled) it seems the answer is yes! The residuals indicate a much more similar dispersion between the two modes of transmission, which goes to show just how much more variance can be explained by the other variables!

Conclusion

So, what is the conclusion? The numbers don’t lie and the numbers say… there’s no way to say! Even if a single variable analysis seems to favor manual with and advantage of more than 7 miles per galon, a more complete analysis indicates there is no statistically significant difference between manual and automatic transmission. If anything, readers will have to turn to their car’s weight as the reason their next gaz bill is so high.