Simple Linear Regression is a statistical method that allows us to:

• Model the relationship between two continuous variables
• Make predictions based on one variable (predictor) to estimate another (response)
• Understand the strength and direction of the relationship

Key Question: Can we predict Y based on X?

The mathematical model for simple linear regression is:

\[Y = \beta_0 + \beta_1 X + \epsilon\]

Where:

• \(Y\) = Response variable (dependent variable)
• \(X\) = Predictor variable (independent variable)
  
• \(\beta_0\) = Y-intercept (value of Y when X = 0)
• \(\beta_1\) = Slope (change in Y for one unit change in X)
• \(\epsilon\) = Random error term

We estimate the coefficients using the Least Squares Method, which minimizes the sum of squared residuals:

\[\hat{\beta_1} = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2}\]

\[\hat{\beta_0} = \bar{y} - \hat{\beta_1}\bar{x}\]

Where \(\bar{x}\) and \(\bar{y}\) are the means of X and Y, and \(n\) is the number of observations.

Let’s use a real example: predicting car stopping distance based on speed.

# Using built-in 'cars' dataset
data(cars)
head(cars, 5)

##   speed dist
## 1     4    2
## 2     4   10
## 3     7    4
## 4     7   22
## 5     8   16

• speed: Speed of car (mph)
• dist: Stopping distance (feet)

# Fit the linear regression model
model <- lm(dist ~ speed, data = cars)

# Display the model summary
summary(model)

## 
## Call:
## lm(formula = dist ~ speed, data = cars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -29.069  -9.525  -2.272   9.215  43.201 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) -17.5791     6.7584  -2.601   0.0123 *  
## speed         3.9324     0.4155   9.464 1.49e-12 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 15.38 on 48 degrees of freedom
## Multiple R-squared:  0.6511, Adjusted R-squared:  0.6438 
## F-statistic: 89.57 on 1 and 48 DF,  p-value: 1.49e-12

From our model output:

\[\text{Distance} = -17.58 + 3.93 \times \text{Speed}\]

Interpretation:

• Intercept (\(\beta_0 = -17.58\)): Theoretical stopping distance at 0 mph (not meaningful in practice)
• Slope (\(\beta_1 = 3.93\)): For every 1 mph increase in speed, stopping distance increases by approximately 3.93 feet

R-squared (\(R^2\)): 0.6511

\[R^2 = 1 - \frac{\sum(y_i - \hat{y_i})^2}{\sum(y_i - \bar{y})^2}\]

• About 65% of the variation in stopping distance is explained by speed
• This indicates a moderately strong positive relationship

Root Mean Squared Error (RMSE): 15.38 feet

For linear regression to be valid, we assume:

1. Linearity: Relationship between X and Y is linear
2. Independence: Observations are independent
3. Homoscedasticity: Constant variance of residuals
4. Normality: Residuals are normally distributed

These should be checked using diagnostic plots!

Simple Linear Regression is a powerful tool for:

• Understanding relationships between variables
• Making predictions
• Quantifying the strength of associations

In our example: Speed is a significant predictor of stopping distance, which has important safety implications!

Next Steps: Multiple regression, polynomial regression, or other advanced techniques.