• What is linear regression?
• Mathematical model
• Example: Predicting height from age
• Visualizations
• Model evaluation
• R code
• Extension: 3D regression surface

Linear regression models the relationship between a dependent variable \(y\) and one or more independent variables \(x\).

• Simple linear regression: one predictor
  
• Goal: Fit a line that best predicts \(y\) from \(x\)

\[ y = \beta_0 + \beta_1 x + \epsilon \]

Where:
- \(\beta_0\) is the intercept
- \(\beta_1\) is the slope
- \(\epsilon \sim \mathcal{N}(0, \sigma^2)\) is the error term

\[ \hat{\beta}_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}, \quad \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x} \]

These estimates minimize the sum of squared residuals.

set.seed(123)
age <- runif(100, 5, 15)
height <- 50 + 5 * age + rnorm(100, 0, 5)
data <- data.frame(age, height)

ggplot(data, aes(x = age, y = height)) +
  geom_point() +
  geom_smooth(method = "lm", color = "blue") +
  labs(title = "Height vs Age with Linear Fit", x = "Age (years)", y = "Height (cm)")

## `geom_smooth()` using formula = 'y ~ x'

model <- lm(height ~ age, data = data)
residuals <- resid(model)

ggplot(data.frame(age, residuals), aes(x = age, y = residuals)) +
  geom_point() +
  geom_hline(yintercept = 0, color = "red") +
  labs(title = "Residual Plot", x = "Age (years)", y = "Residuals")

set.seed(123)
x1 <- runif(100, 5, 15)
x2 <- runif(100, 30, 60)
y <- 40 + 3 * x1 + 2 * x2 + rnorm(100, 0, 5)
df3d <- data.frame(x1, x2, y)
model3d <- lm(y ~ x1 + x2, data = df3d)

x_seq <- seq(min(x1), max(x1), length.out = 20)
y_seq <- seq(min(x2), max(x2), length.out = 20)
z_grid <- outer(x_seq, y_seq, Vectorize(function(a, b) {
  predict(model3d, newdata = data.frame(x1 = a, x2 = b))
}))

plot_ly(df3d, x = ~x1, y = ~x2, z = ~y, type = "scatter3d", mode = "markers") %>%
  add_surface(x = x_seq, y = y_seq, z = z_grid)

## Warning: 'surface' objects don't have these attributes: 'mode'
## Valid attributes include:
## '_deprecated', 'autocolorscale', 'cauto', 'cmax', 'cmid', 'cmin', 'coloraxis', 'colorbar', 'colorscale', 'connectgaps', 'contours', 'customdata', 'customdatasrc', 'hidesurface', 'hoverinfo', 'hoverinfosrc', 'hoverlabel', 'hovertemplate', 'hovertemplatesrc', 'hovertext', 'hovertextsrc', 'ids', 'idssrc', 'legendgroup', 'legendgrouptitle', 'legendrank', 'lighting', 'lightposition', 'meta', 'metasrc', 'name', 'opacity', 'opacityscale', 'reversescale', 'scene', 'showlegend', 'showscale', 'stream', 'surfacecolor', 'surfacecolorsrc', 'text', 'textsrc', 'type', 'uid', 'uirevision', 'visible', 'x', 'xcalendar', 'xhoverformat', 'xsrc', 'y', 'ycalendar', 'yhoverformat', 'ysrc', 'z', 'zcalendar', 'zhoverformat', 'zsrc', 'key', 'set', 'frame', 'transforms', '_isNestedKey', '_isSimpleKey', '_isGraticule', '_bbox'

• Simple linear regression estimates a line of best fit
• We assess performance using residual plots
• 3D regression is an extension to multiple predictors