2024-03-19

Simple Linear Regression Defined

  • In statistics, a simple linear regression (SLR) is a type of model used to estimate the linear relationship between a single response variable and a single explanatory variable.
  • An SLR model will take the general form: \(y = α + βx\)
    • \(y\) is the response variable
    • \(α\) is the intercept
    • \(β\) is the slope
    • \(x\) is the explanatory variable

An Example

Truth and Statistics

  • An SLR model is an estimate. It does not reveal the underlying relationship between the variables it models.
  • Given a set of data points we can describe the relationship between x and y as: \(y_i = α + βx_i + ε_i\)
    • \(ε_i\) is the difference between \(y_i\) and \(\hat{y}_i\)

ε Visualized

Multiple Linear Regression

  • Multiple linear regression extends the principles of single linear regression to situations with more than one explanatory variable.
  • In these situations the model is an n-dimensional plane with the formula: \(z = α + β_1x_1 + β_2x_2 + ... + β_nx_n\)

Multiple Linear Regression Code

  • The code for the plot on the next slide.
tree_lm <- lm(Volume ~ Height + Girth, data=trees)

ax_x <- seq(min(trees$Height), max(trees$Height), by=0.1)
ax_y <- seq(min(trees$Girth), max(trees$Girth), by=0.1)
lm_surface <- expand.grid(Height=ax_x, Girth=ax_y, KEEP.OUT.ATTRS=F)
lm_surface$Volume <- predict.lm(tree_lm, newdata=lm_surface)
lm_surface <- acast(lm_surface, Girth ~ Height, value.var="Volume")

plot_ly() %>%
  add_trace(data=trees, x=~Height, y=~Girth, z=~Volume) %>%
  add_trace(x=ax_x, y=ax_y, z=lm_surface, type="surface")

Multiple Linear Regression Plot