Simple linear regression is a statistical technique used to examine and estimate the relationship between two continuous or quantitative variables (Castro & Mordkoff, 2023). One variable serves as the predictor (or explanatory) variable and the other as the outcome (or response) variable (Castro & Mordkoff, 2023).

One of the main goals of performing a simple linear regression analysis is to create a model that is able to predict the value of the outcome variable (Y) based on the value of the predictor variable (X) (Castro & Mordkoff, 2023). To model the relationship between the two variables, a linear regression equation is utilized (Castro & Mordkoff, 2023).

The linear regression equation used to model the relationship between a predictor variable and a outcome variable is the following: \(\hat {Y} = \beta_0 + \beta_1X + \varepsilon\).

• \(\hat {Y} = \text{the predicted value of the outcome (Y) variable.}\)

• \(\beta_0 = \text {the intercept (value of the variable when the predictor variable (X) = 0).}\)

• \(\beta_1 = \text{the slope of the regression line.}\)

• \(X = \text{the value of the predictor variable.}\)

• \(\varepsilon = \text{error component.}\)

This equation graphically establishes a regression line, which contains the information that best explains the relationship between the predictor variable and the outcome variable (Gallo, 2015).

The error component \((\varepsilon)\) of our regression equation is crucial for the success of our model. It acknowledges the fact that not all of the data points will sit on the regression line (San, 2021). This accounts for errors in measurement, omitted variables, and random variation (San, 2021).

The line of best fit works to minimize any distance between the actual scores and the scores predicted by our model by utilizing \(\beta\) values that minimize the residual sum of squares (San, 2021). The residual sum of squares is the summation of the actual Y values minus the predicted Y values, squared:

\(\text {SSR} = \Sigma^n_{i=1} (y_i - \hat{y} )^2\)

However, we can obtain all of the information needed in our equation to create the regression line for our model in R using a few key functions.

To better understand the information we have covered, we will examine the relationship between an individuals SAT score and their respective college GPA. Many universities regard an applicants SAT score as a major deciding factor in any admission decision, as the score received is believed to be an indicator of a student’s potential to succeed academically within their college career. Therefore, in this scenario, SAT score will be our predictor variable and college GPA will be our outcome variable. To begin, we will start by creating a scatter plot to better visualize any trends there may be in the data. Then, we will add the regression line in a secondary plot to better understand the relationship between the two variables.

x = df$SAT.total.score
y = df$college.gpa
xax = list(title = "SAT Composite Score", titlefont = list(
  family = "Modern Computer Roman"))
yax = list(title = "College GPA", titlefont = list(
  family = "Modern Computer Roman"), range = c(2.00, 4.50))
fig = plot_ly(x = x, y = y, type = "scatter", mode = "markers", 
              name = "data", width = 800, height = 430) %>%
  layout(xaxis = xax, yaxis = yax) %>%
  layout(margin = list(l = 150, r = 50,b = 20,t = 60))

g = ggplot(df, aes(x = SAT.total.score, y = college.gpa)) + 
  geom_point()
g = g + geom_smooth(method = "lm") + 
  coord_cartesian(ylim = c(2.00, 4.50))

The scatter plot shows a moderate positive linear relationship between SAT score and college GPA. The trend depicted is consistent with the correlation coefficient obtained for the relationship between the two variables (0.53328). We can use the following code to access the information used in the regression equation to produce the regression line in the scatter plot:

x = df$SAT.total.score
y = df$college.gpa
model1 = lm(y~x)
summary(model1)

According to the information produced by this code, we can use the following regression equation to predict a student’s college GPA based on their SAT score: \(\hat{Y} = 1.6329317 + 0.0008718x_i + \varepsilon\)

We can also examine the relationship between a student’s ACT score and their college GPA:

Similarly to the scatter plot we created to visualize the relationship between SAT score and college GPA, the data shows a moderate positive linear relationship between ACT score and college GPA. This is consistent with the correlation coefficient obtained using the cor() function (0.5012183). We can use the following code to access the information used in the regression equation to produce the regression line in the scatter plot:

a = df$ACT.composite.score
b = df$college.gpa
model2 = lm(b~a)
summary(model2)

According to the information produced by this code, we can use the following regression equation to predict a student’s college GPA based on their ACT score: \(\hat{Y} = 2.150653 + 0.042851x_i + \varepsilon\)

Castro, L., & Mordkoff, J. T. (2023, January 16). Unit 9. Simple Linear Regression. Data Analysis in the Psychological Sciences: A Practice, Applied, Multimedia Approach. https://pressbooks.uiowa.edu/data-analysis-in-the-psychological-sciences/chapter/unit-9-simple-linear-regression/

Gallo, A. (2015, November 4). A Refresher on Regression Analysis. Harvard Business Review. http://hbr.org/2015/11/a-refresher-on-regression-analysis

San, C. (2021, May 20). Simple Linear Regression: A Layman’s Explanation. Medium. https://medium.com/nerd-for-tech/simple-linear-regression-a-laymans-explanation-d60f1f2fc41b