Linear regression models the relationship between a dependent variable (Y) and an independent variable (X), assuming a linear relationship.

Linear Regression Equation:

\[Y = \beta_0 + \beta_1 X + \varepsilon\]

Where: \(Y\) = outcome, \(X\) = predictor, \(\beta_0\) = intercept, \(\beta_1\) = slope, \(\varepsilon\) = error term

Example: Predicting fuel efficiency (mpg) based on car weight using the mtcars dataset

We use Ordinary Least Squares (OLS) to estimate \(\beta_0\) and \(\beta_1\) by minimizing the sum of squared residuals.

The OLS estimators are:

\[\hat{\beta}_1 = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2}\]

\[\hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x}\]

These formulas give us the “best fit” line through the data.

# Load the mtcars dataset
data(mtcars)

# Fit linear regression model with dependent and independent variables
model <- lm(mpg ~ wt, data = mtcars)

# Create scatter plot with regression line (lm)
ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point(size = 3, alpha = 0.6) +
  geom_smooth(method = "lm", se = TRUE) +
  labs(x = "Weight (1000 lbs)", y = "MPG") +
  theme_minimal()

R² (Coefficient of Determination): \[R^2 = 1 - \frac{\sum(y_i - \hat{y}_i)^2}{\sum(y_i - \bar{y})^2}\] Our model: \(R^2 =\) 0.753 (75.3% of variance explained!)

Key Assumptions (GeeksForGeeks):

1. Linearity: Relationship between weight and MPG is linear
2. Independence: Each car observation is independent
3. Homoscedasticity: Constant variance of errors
4. Normality: Errors follow normal distribution

Key Findings from mtcars Analysis:

• Strong negative relationship between car weight and fuel efficiency
• Model explains 75.3% of variance in MPG (High correlation)
• Each additional 1000 lbs reduces MPG by ~5.34 miles per gallon