• It is a method to study the relationship between two variables.
• \(x\) is the predictor (input) and \(y\) is the response (output).
• We fit a straight line through the data.
• We can use the line to predict \(y\) for a new value of \(x\).

Example: can we predict the stopping distance of a car from its speed?

The simple linear regression model is

\[y_i = \beta_0 + \beta_1 x_i + \varepsilon_i, \quad i = 1, \dots, n\]

• \(\beta_0\) is the intercept
• \(\beta_1\) is the slope
• \(\varepsilon_i\) is the random error, with \(\varepsilon_i \sim N(0, \sigma^2)\)

The fitted line is \(\hat{y} = \hat{\beta}_0 + \hat{\beta}_1 x\).

We choose \(\hat{\beta}_0\) and \(\hat{\beta}_1\) to make the sum of squared errors as small as possible:

\[SSE = \sum_{i=1}^{n} (y_i - \hat{\beta}_0 - \hat{\beta}_1 x_i)^2\]

The solution is

\[\hat{\beta}_1 = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2}, \qquad \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x}\]

The cars data set in R has 50 cars with their speed (mph) and stopping distance (ft).

model = lm(dist ~ speed, data = cars)
coef(model)

## (Intercept)       speed 
##  -17.579095    3.932409

summary(model)$r.squared

## [1] 0.6510794

So the fitted line is \(\hat{y} = -17.58 + 3.93x\) and \(R^2 = 0.65\).

library(ggplot2)

ggplot(cars, aes(x = speed, y = dist)) +
  geom_point(color = "blue", size = 2) +
  geom_smooth(method = "lm", color = "red", se = FALSE) +
  labs(title = "Fitted Regression Line",
       x = "Speed (mph)", y = "Stopping distance (ft)")

• Slope = 3.93: when speed goes up by 1 mph, the stopping distance goes up by about 3.93 ft.
• \(R^2 = 0.65\): speed explains about 65% of the variation in stopping distance.
• Prediction: for a car going 20 mph,

\[\hat{y} = -17.58 + 3.93(20) \approx 61.07 \text{ ft}\]

With two predictors the line becomes a plane. Here is the trees data: Girth, Height and Volume of 31 trees.

• Simple linear regression fits a straight line \(\hat{y} = \hat{\beta}_0 + \hat{\beta}_1 x\).
• The estimates come from least squares.
• In R we use lm() to fit the model.
• For the cars data, faster cars need longer distance to stop.