Simple linear regression studies how one variable (the response \(Y\)) changes with another variable (the predictor \(x\)).

• We fit a straight line through the data
• We can use the line to describe the relationship
• We can use the line to predict new values

Our question: do heavier cars get fewer miles per gallon?

• \(Y\) = mpg (miles per gallon)
• \(x\) = wt (weight in 1000 lbs)

We use the built-in mtcars data set (32 cars).

##                    mpg    wt  hp
## Mazda RX4         21.0 2.620 110
## Mazda RX4 Wag     21.0 2.875 110
## Datsun 710        22.8 2.320  93
## Hornet 4 Drive    21.4 3.215 110
## Hornet Sportabout 18.7 3.440 175
## Valiant           18.1 3.460 105

As weight goes up, mpg goes down.

\[Y = \beta_0 + \beta_1 x + \varepsilon\]

• \(\beta_0\) is the intercept
• \(\beta_1\) is the slope
• \(\varepsilon\) is the random error

The error is assumed to be normal:

\[\varepsilon \sim N(0, \sigma^2)\]

We pick the line that makes the sum of squared errors as small as possible:

\[SSE = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2\]

The line that does this is

\[\hat{y} = \hat{\beta}_0 + \hat{\beta}_1 x\]

where

\[\hat{\beta}_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2} \qquad \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x}\]

This is the code that fits the line and makes the plot on the next slide.

fit <- lm(mpg ~ wt, data = mtcars)

ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point() +
  geom_smooth(method = "lm")

## 
## Call:
## lm(formula = mpg ~ wt, data = mtcars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
## wt           -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528, Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

From the output above:

• The fitted line is \(\hat{y} = 37.29 - 5.34x\)
• Slope: each extra 1000 lbs lowers mpg by about 5.3
• R-squared is about 0.75, so weight explains about 75% of the variation in mpg
• The p-value for the slope is tiny, so weight is clearly related to mpg

We test whether the slope is zero:

\[H_0: \beta_1 = 0 \qquad \text{vs.} \qquad H_a: \beta_1 \neq 0\]

The test statistic is

\[t = \frac{\hat{\beta}_1}{SE(\hat{\beta}_1)}\]

Here the p-value is far below 0.05, so we reject \(H_0\).

A residual is the error of the line for one car: \(e_i = y_i - \hat{y}_i\).

Here is the same data with horsepower added. You can rotate the plot.

• Simple linear regression fits a line: \(\hat{y} = \hat{\beta}_0 + \hat{\beta}_1 x\)
• lm() does the work in R
• In our example, heavier cars have lower gas mileage
• The relationship is strong and statistically significant