Simple linear regression models the linear relationship between a response variable \(Y\) and a single predictor variable \(X\).

• Widely used in prediction and inference
• Foundation for more complex regression models
• Assumes a straight-line relationship between \(X\) and \(Y\)

Example dataset: We use R’s built-in cars dataset, which records the speed (mph) and stopping distance (ft) of cars from the 1920s.

The simple linear regression model is:

\[Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i\]

where:

• \(Y_i\) is the response for observation \(i\)
• \(X_i\) is the predictor for observation \(i\)
• \(\beta_0\) is the intercept (value of \(Y\) when \(X = 0\))
• \(\beta_1\) is the slope (change in \(Y\) per unit increase in \(X\))
• \(\varepsilon_i \sim \mathcal{N}(0, \sigma^2)\) is random error

The coefficients \(\beta_0\) and \(\beta_1\) are estimated by Ordinary Least Squares (OLS), minimizing:

\[RSS = \sum_{i=1}^{n} \left( Y_i - \hat{Y}_i \right)^2 = \sum_{i=1}^{n} \left( Y_i - \hat{\beta}_0 - \hat{\beta}_1 X_i \right)^2\]

The closed-form solutions are:

\[\hat{\beta}_1 = \frac{\sum_{i=1}^{n}(X_i - \bar{X})(Y_i - \bar{Y})}{\sum_{i=1}^{n}(X_i - \bar{X})^2}, \qquad \hat{\beta}_0 = \bar{Y} - \hat{\beta}_1 \bar{X}\]

A random scatter around zero suggests the linearity and constant-variance assumptions hold reasonably well.

Points following the diagonal line indicate the normality assumption is approximately satisfied.

The coefficient of determination measures the proportion of variance in \(Y\) explained by \(X\):

\[R^2 = 1 - \frac{RSS}{TSS} = 1 - \frac{\sum(Y_i - \hat{Y}_i)^2}{\sum(Y_i - \bar{Y})^2}\]

• \(R^2 \in [0, 1]\); values closer to 1 indicate a better fit
• For the cars model: \(R^2 \approx 0.65\), meaning speed explains about 65% of the variation in stopping distance

## R-squared: 0.6511

Key takeaway: Simple linear regression is an interpretable, powerful tool for modeling the relationship between two continuous variables — and a critical foundation for all of regression analysis.