Introduction to Linear Regression

In statistics, simple linear regression is used to determine if a statistical relationship is present when a deterministic relationship has not be previously identified. In other words, it can be used to determine if there is a statistical relationship between two quantitative variables.

\(\hat{y_n} = b_0 + b_1 x_i\)

where \(\hat{y_n}\) the predicted response for experimental unit i, \(x_i\) is the predictor value for experimental unit i, and \(b+0\) is the intercept and \(b_1\) is the slope for the equation of the line that minimizes the sum of the squared prediction errors.

Penguins Example

The graph below is a 3D scatter plot showing the bill length, flipper length, and body mass of penguins that were studied in the Palmer Archipelago, Antarctica. As you may notice, it is difficult to derive a relationship between these variables with this graph. Graphing two variables at a time using a simple linear regression model can help us better determine if there is a relationship between any of these variables.

Penguin Code

The code below was used to create the graph seen on the previous slide.

data(penguins)
attach(penguins)
The following objects are masked from penguins (pos = 3):

    bill_dep, bill_len, body_mass, flipper_len, island, sex, species,
    year
penguins = penguins[!is.na(penguins$bill_len),]
fig <- plot_ly(penguins, x=bill_len, 
               y=body_mass, 
               z=flipper_len, 
               type="scatter3d", mode="markers", color=species) %>%
  hide_colorbar() %>%
  layout(scene = list(xaxis = list(title = 'Bill Length'),
                      yaxis = list(title = 'Body Mass'),
                      zaxis = list(title = 'Flipper Length')))
fig

Penguin Body Mass vs. Bill Length Pt 1

Here we can see a plot of penguin bill length vs. body mass. A general positive relationship begins to emerge, but adding a simple linear regression will give a better idea of these variables relationship, if any.

Penguin Body Mass vs. Bill Length Pt 2

Note the linear regression line and formula added to the graph: \(y = 26.9 + 0.00405x\). The grey band surrounding the black line represents the 95% confidence interval of the true value of y falling within that band.

Penguin Flipper Length vs. Body Mass

Below is a graph comparing the body mass of a penguin vs. flipper length.

Penguin Flipper Length vs. Bill Length

Conclusions

  • Penguin body mass vs flipper length showed the highest correlation with an \(R^2\) value of 0.72
  • Penguin Body Mass vs. Bill Length had the weakest correlation with an \(R^2\) value of 0.35
  • Linear regression allowed us to evaluate possible relationships between variables such a penguin mass, flipper length, and bill length.
  • Graphing each pair of variables using simple linear regression allows for better communication and visualization of findings.

References

Horst AM, Hill AP, Gorman KB (2020). palmerpenguins: Palmer Archipelago (Antarctica) penguin data. doi:10.5281/zenodo.3960218. R package version 0.1.0, https://allisonhorst.github.io/palmerpenguins/.

Pardoe, I. (n.d.). 2.1 - what is simple linear regression?. 2.1 - What is Simple Linear Regression? | STAT 462. https://online.stat.psu.edu/stat462/node/91/

R Core Team (2026). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. doi:10.32614/R.manuals https://doi.org/10.32614/R.manuals. https://www.R-project.org/.