2026-02-07

Definition

Simple linear regression models the relationship between:

  • One response variable (Y)
  • One explanatory variable (X)

It assumes a linear relationship between the two variables.

The Model

The simple linear regression model is:

\[ Y = \beta_0 + \beta_1 X + \varepsilon \]

Where: \[ ( \beta_0 ): intercept \] \[ ( \beta_1 ): slope \] \[ ( \varepsilon ): random error term \]

Assumptions

The classical assumptions are:

\[ E(\varepsilon) = 0, \quad Var(\varepsilon) = \sigma^2 \]

\[ \varepsilon_i \text{ are independent and normally distributed} \]

These assumptions allow us to perform inference.

Example: Housing Prices

We consider a simple dataset where:

  • (X): house size (square feet)
  • (Y): house price (in thousands of dollars)

Scatter Plot

Fitted Regression Line

R Code Example

model <- lm(price ~ size, data = housing)
summary(model)

The estimated slope tells us how much price increases per additional square foot.

3D Visualization (plotly)

We add a third variable: house age.

Interpretation

  • Larger houses tend to have higher prices
  • Linear regression provides a simple but powerful model
  • Visualization helps validate assumptions and patterns

This approach is widely used in economics, engineering, and data science.

Why Linear Regression Matters

  • Easy to interpret
  • Foundation for more complex models
  • Useful for prediction and inference

Linear regression is often the first statistical model used in practice.