2025-04-13

Introduction

What is Linear Regression in Math?

  • a statistical method used to model the relationship between a dependent variable and one or more independent variables

What is the purpose?

  • Linear regression aims to find the best-fitting straight line

The Regression Equation

The linear regression model is:

\[ y = \beta_0 + \beta_1 x + \varepsilon \]

Where:

  • \(y\) is the response variable
  • \(x\) is the predictor variable
  • \(\beta_0\) is the y-intercept
  • \(\beta_1\) is the slope
  • \(\varepsilon\) is the error term

Assumptions of Linear Regression

  1. Linearity
  2. Independence of observations
  3. Constant variance (homoscedasticity)
  4. Normally distributed residuals

These assumptions help ensure reliable model results.

Example Dataset: Study Time vs Test Score

Random Dataset for Regression graph:

set.seed(123)
study_time <- seq(1, 10, by = 1)
test_score <- 50 + 5 * study_time + rnorm(10, mean = 0, sd = 5)
data <- data.frame(study_time, test_score)
invisible(head(data))
model <- lm(test_score ~ study_time, data = data)
invisible(summary(model))
invisible(library(ggplot2))

ggplot(data, aes(x = study_time, y = test_score)) +
  geom_point(color = "#8C1D40", size = 3) +
  invisible(geom_smooth(method = "lm", se = FALSE, color = "black")) +
  theme_minimal() +
  invisible(labs(title = "Study Time vs Test Score", x = "Hours Studied", y = "Test Score"))
## `geom_smooth()` using formula = 'y ~ x'

Regression Graph

Residual Plot

residuals <- resid(model)
fitted <- fitted(model)
res_data <- data.frame(fitted, residuals)

ggplot(res_data, aes(x = fitted, y = residuals)) +
  geom_point(color = "#003366", size = 3) +
  geom_hline(yintercept = 0, linetype = "dashed") +
  theme_minimal() +
  labs(title = "Residual Plot", x = "Fitted Values", y = "Residuals")

Plotly

Understanding Plotly Graph

This interactive plot visualizes the relationship between:

  • \(x\): Hours Studied
  • \(y\): Test Score
  • Added variation (noise) in the data

Even though the data roughly follows a linear trend, each point is affected by random noise \(\varepsilon\), which represents:

\[ \varepsilon \sim \mathbb{N}(0, \sigma^2) \]

This means that actual scores vary due to unpredictable factors, even if more study time generally leads to higher scores.

Key Takeaway:

  • The pattern shows a positive linear relationship: as \(x\) increases, \(y\) tends to increase.
  • The spread around reflects residual variation in real-world observations.