2026-10-18

What is simple linear regression?

  • Measures the relationship between two variables
  • Should be used when comparing an independent variable with a dependent variable

Example: Hours Studied & Exam Scores

  • Independent variable: Hours studied
  • Dependent variable: Exam scores
  • Is there any correlation between the amount of time studied and students’ exam scores?

The Regression Equation

Simple linear regression uses the regression equation:

\[ \hat{y} = b_0 + b_1x \]

  • \(\hat{y}\) = predicted value of the dependent variable

  • \(b_0\) = y-intercept

  • \(b_1\) = slope of the regression line

  • \(x\) = value of the independent variable

Finding the Line of Best Fit

  • The difference between the observed value and the predicted value from the line of best fit is called the residual.

  • The least squares method finds the line that minimizes the sum of the squared residuals.

\[ \text{Residual} = y - \hat{y} \]

\[ \text{SSE} = \sum_{i=1}^{n}(y_i-\hat{y}_i)^2 \]

The Regression Line

Using the hours studied and exam scores data:

\[ \hat{y} = 43.24 + 5.04x \]

  • \(b_0 = 43.24\): The predicted exam score for a student who studies 0 hours is 43.24.

  • \(b_1 = 5.04\): For each additional hour studied, the predicted exam score increases by about 5.04 points.

Making a Prediction

Suppose a student studies for 5 hours:

\[ \hat{y} = 43.24 + 5.04(5) \]

\[ \hat{y} = 68.44 \]

  • For a student who studies for 5 hours, their predicted exam score is approximately 68.44 points.

Visualize the Regression Line

The regression line shows the predicted exam scores for each number of hours studied.

Interactive Regression Plot

This interactive plot allows us to explore the relationship between hours studied and exam scores.

Creating the Regression Plot in R

The regression plot can be created using ggplot2.

ggplot(study_data, aes(x = hours, y = scores)) + 
  geom_point() + 
  geom_smooth(method = "lm", se = FALSE) + 
  xlab("Hours Studied") + ylab("Exam Scores")

Key Takeaways

  • Simple linear regression models the relationship between an independent variable and a dependent variable.

  • The least squares method finds the regression line that minimizes the sum of the squared residuals.

  • The regression equation can be used to predict the dependent variable based on a given value of the independent variable.