Simple Linear Regression models the linear relationship between: - A response variable \(Y\) (dependent) - A single predictor variable \(X\) (independent)

It is one of the fundamental tools in statistics primarily used to: 1. Understand how \(Y\) changes as \(X\) changes 2. Predict the future values of \(Y\) given a new \(X\) 3. Quantify the strength and the direction of a relationship

The population regression model is:

\[Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i\] This is used as: - \(\beta_0\) = intercept — the expected value of \(Y\) when \(X = 0\) - \(\beta_1\) = slope — the expected change in \(Y\) for a one-unit increase in \(X\) - \(\varepsilon_i \sim \mathcal{N}(0, \sigma^2)\) — random error term, assumed i.i.d.

The fitted (estimated) regression line is: \[\hat{Y}_i = \hat{\beta}_0 + \hat{\beta}_1 X_i\] ## Estimating the Coefficients We use Ordinary Least Squares to minimize the sum of squared residuals:

\[\text{SSE} = \sum_{i=1}^{n}(Y_i - \hat{Y}_i)^2\]

The OLS estimators are:

\[\hat{\beta}_1 = \frac{\sum_{i=1}^{n}(X_i - \bar{X})(Y_i - \bar{Y})}{\sum_{i=1}^{n}(X_i - \bar{X})^2}\]

\[\hat{\beta}_0 = \bar{Y} - \hat{\beta}_1 \bar{X}\]

We use R’s built-in cars dataset to provide an example: - speed: speed of the car (mph) — predictor \(X\) - dist: stopping distance (ft) — response \(Y\)

summary(cars)

##      speed           dist       
##  Min.   : 4.0   Min.   :  2.00  
##  1st Qu.:12.0   1st Qu.: 26.00  
##  Median :15.0   Median : 36.00  
##  Mean   :15.4   Mean   : 42.98  
##  3rd Qu.:19.0   3rd Qu.: 56.00  
##  Max.   :25.0   Max.   :120.00

model <- lm(dist ~ speed, data = cars)
summary(model)

## 
## Call:
## lm(formula = dist ~ speed, data = cars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -29.069  -9.525  -2.272   9.215  43.201 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) -17.5791     6.7584  -2.601   0.0123 *  
## speed         3.9324     0.4155   9.464 1.49e-12 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 15.38 on 48 degrees of freedom
## Multiple R-squared:  0.6511, Adjusted R-squared:  0.6438 
## F-statistic: 89.57 on 1 and 48 DF,  p-value: 1.49e-12

Coefficient of Determination \(R^2\):

\[R^2 = 1 - \frac{\text{SSE}}{\text{SST}} = 1 - \frac{\sum(Y_i - \hat{Y}_i)^2}{\sum(Y_i - \bar{Y})^2}\]

Using our fitted model:

\[\hat{\text{dist}} = -17.58 + 3.93 \times \text{speed}\]

Example: Predict stopping distance for a car traveling at 40 mph:

\[\hat{\text{dist}} = -17.58 + 3.93 \times 40 = 139.72 \text{ ft}\]

new_data <- data.frame(speed = 40)
predict(model, newdata = new_data, interval = "prediction", level = 0.95)

##        fit      lwr      upr
## 1 139.7173 102.3311 177.1034

Conclusion: There is a strong positive linear relationship between car speed and stopping distance. For every 1 mph increase in speed, stopping distance increases by approximately 3.93 feet.