2025-10-26

What is Simple Linear Regression?

Linear regression is a statistical tool for graphing data, which generates a straight line which models the relationship between two variables.

It is a very useful tool as it is a simple way to show a trend in a linearly related data set.

Isaac Newton is credited with inventing the linear regression method in 1700 in his work on the ordinary least squares method, however Legendre and Gauss popularized it with their work on the prediction of planetary movement (Belenkiy).

How is a Linear Regression Defined?

The equation for a linear regression is defined as:

\[y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_p x_p + \epsilon\]

Where:

  • \(y\) is the dependent variable
  • \(\beta_0\) is the intercept
  • \(\beta_1, \beta_2, \ldots, \beta_p\) are the regression coefficients
  • \(x_1, x_2, \ldots, x_p\) are the independent variables
  • \(\epsilon\) is the random error term, assumed to be \(\epsilon \sim N(0, \sigma^2)\)

Linear Regression in slope form

A more palatable way of presenting the linear regression equation is in slope form:

\[Y_i = \beta_0 + \beta_1 X_i\]

  • \(Y_i\): Dependent Variable (outcome)
  • \(\beta_0\): Constant/Intercept (baseline)
  • \(\beta_1\): Slope/Coefficient (effect size)
  • \(X_i\): Independent Variable (predictor)

Simple Linear Regression Example 1

The fitted model:

\[\widehat{\text{mpg}} = 37.29 -5.34 \times \text{wt}\]

Interpretation: For every 1000 lb increase in weight, MPG decreases by 5.34 miles per gallon.

Simple Linear Regression Example 2

The fitted model:

\[\widehat{\text{pressure}} = -147.9 + 1.51 \times \text{temperature}\]

Interpretation: For every 1°C increase in temperature, pressure increases by 1.51 mm of Hg.

Simple Linear Regression Example 3

The fitted model:

\[\widehat{\text{Sepal Length}} = 4.31 + 0.41 \times \text{Petal Length}\]

Interpretation: For every 1 cm increase in petal length, sepal length increases by 0.41 cm.

References

  • Belenkiy, Ari, and Eduardo Vila Echagüe. “History of one defeat: Reform of the julian calendar as envisaged by Isaac Newton.” Notes and Records of the Royal Society, vol. 59, no. 3, 22 Sept. 2005, pp. 223–254, https://doi.org/10.1098/rsnr.2005.0096.

  • “Linear Regression.” Wikipedia, Wikimedia Foundation, 22 Sept. 2025, en.wikipedia.org/wiki/Linear_regression.