What is Simple Linear Regression?

Simple Linear regression is a statistical method that models the relationship between two continuous variables by fitting a straight line to the data.

The straight line illustrates how one variable changes as the other variable changes.

Linear Regression Research Example

Linear Regression has many uses in research:

  • Imagine we want to understand if completing more LSAT practice exams is likely to improve someone’s LSAT score.

We could collect information from students regarding:

  • The # of practice exams they completed
  • There official LSAT scores

Simple Linear Regression can then be used to observe the relationship between the two variables.

Independent and Dependent Variables

With simple linear regression, one variable is used to predict another.

In my LSAT example we could say:

  • Independent variable (x) = # of LSAT exams complete (The variable used to make the prediction)

  • Dependent Variable (y) = Official LSAT Score (variable we are predicting)

The Linear Regression Equation

Linear regression used the below formula to calculate predicted values: \[ \hat{Y} = b_0 + b_1X \]

  • \(\hat{Y}\) = Predicted value of y (dependent variable)
  • \(b_0\) = y-intercept
  • \(b_1\) = Slope of regression line
  • \(X\) = independent variable

Predicting an LSAT Score Example

Supposed we have the below hypothetical linear regression equation: \[ \hat{Y} = 145 + 1.5X \]

Lets say a student completed 10 practice exams: \[ \hat{Y} = 145 + 1.5(10) = 160 \] The predicted official LSAT score for a student who completes 10 practice exams is 160!

Visualizing LSAT Scores Example

  • Each point represents the number of completed LSAT practice exams and the corresponding LSAT scores.

  • This scatter-plot demonstrates a positive relationship between completing practice exams and increasing LSAT scores.

Visualizing the Regression Line

The purple line is the regression line. It shows a positive relationship between the two variables.

Interactive LSAT Score Display

The below graph allows for you to explore the linear regression relationship shown

Calculating Linear Regression in R

Fun Fact: R is able to calculate the best fitting regression line with the lm() function

model = lm(lsatScores ~ practiceExams, data = lsatData)
model
## 
## Call:
## lm(formula = lsatScores ~ practiceExams, data = lsatData)
## 
## Coefficients:
##   (Intercept)  practiceExams  
##       141.714          1.929
  • Intercept = 141.714 (Predicted LSAT score for students who completed 0 practice exams)
  • Slope = 1.929 (The # of points the LSAT score is predicted to increase for each practice exam completed)