• A way to describe the relationship between two variables
• One variable \(x\) is used to predict another variable \(y\)
• We fit a straight line through the data
• Example: does a car’s speed help predict its stopping distance?

The simple linear regression model is:

\[Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i, \quad i = 1, \dots, n\]

• \(\beta_0\) is the intercept
• \(\beta_1\) is the slope
• \(\varepsilon_i\) is the random error, with \(\varepsilon_i \sim N(0, \sigma^2)\)

We choose the line that minimizes the sum of squared errors:

\[SSE = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2\]

The estimates are:

\[\hat{\beta}_1 = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2}, \qquad \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x}\]

The built-in cars data set has the speed (mph) and stopping distance (ft) of 50 cars.

The code below fits the model and creates the plot on the previous slide.

library(ggplot2)

fit <- lm(dist ~ speed, data = cars)
coef(fit)

ggplot(cars, aes(x = speed, y = dist)) +
  geom_point() +
  geom_smooth(method = "lm", se = FALSE) +
  labs(x = "Speed (mph)", y = "Stopping Distance (ft)")

## (Intercept)       speed 
##  -17.579095    3.932409

The fitted line is:

\[\hat{y} = -17.58 + 3.93 x\]

• Each extra 1 mph of speed adds about 3.93 ft to the stopping distance

Residuals are \(e_i = y_i - \hat{y}_i\). They should look randomly scattered around 0.

With two predictors, we fit a plane instead of a line. Here: mpg vs. wt and hp from mtcars.

• Linear regression fits a straight line to predict \(y\) from \(x\)
• The line is found by least squares
• In R, use lm(y ~ x) to fit the model
• Check the residuals to see if the model is reasonable