2025-10-19

What is Simple Linear Regression?

Simple Linear Regression is a statistical method that models the relationship between two quantitative variables —
a dependent variable \(y\) and an independent variable \(x\), using a line drawn through those variables’ data points, known as a regression line. It helps us understand how changes in \(x\) affect \(y\), and is used for prediction or trend estimation.

The general form of the simple linear regression model is:

\[ y_i = \beta_0 + \beta_1 x_i + \epsilon_i \]

where:
- \(y_i\): the dependent (response) variable
- \(x_i\): the independent (predictor) variable
- \(\beta_0\): the intercept (value of \(y\) when \(x = 0\))
- \(\beta_1\): the slope (how much \(y\) changes when \(x\) increases by 1 unit)
- \(\epsilon_i\): the random error term

Dataset Overview: mtcars

The mtcars dataset is a built-in R dataset containing measurements on 32 automobile models from Motor Trend (1974).
It records various engine and design attributes such as:

  • mpg: miles per gallon (fuel efficiency)
  • wt: weight of the car (in 1000 lbs)
  • hp: horsepower
  • disp: engine displacement
  • cyl: number of cylinders

We are interested in exploring whether a car’s weight (wt) influences its fuel efficiency (mpg).

Structure of the Data

The dataset has 32 rows (each representing one car model) and 11 columns (each a measured variable).
Here we display the first few rows:

##                    mpg    wt  hp cyl
## Mazda RX4         21.0 2.620 110   6
## Mazda RX4 Wag     21.0 2.875 110   6
## Datsun 710        22.8 2.320  93   4
## Hornet 4 Drive    21.4 3.215 110   6
## Hornet Sportabout 18.7 3.440 175   8
## Valiant           18.1 3.460 105   6

MPG Distribution by Car (Bar Plot)

We can also visualize the miles per gallon (mpg) values for each car in the dataset to see how fuel efficiency varies across models.

Fitting the Regression Model

To fit a Simple Linear Regression (SLR) model in R,
we use the built-in lm() function, which stands for linear model.

We model the relationship between miles per gallon (mpg)
and car weight (wt) as:

\[ mpg_i = \beta_0 + \beta_1 wt_i + \epsilon_i \]

In R, this is written as:

# Fit a simple linear regression model
model <- lm(mpg ~ wt, data = mtcars)

# Display model summary
summary(model)
Call:
lm(formula = mpg ~ wt, data = mtcars)

Residuals:
    Min      1Q  Median      3Q     Max 
-4.5432 -2.3647 -0.1252  1.4096  6.8727 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
wt           -5.3445     0.5591  -9.559 1.29e-10 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 3.046 on 30 degrees of freedom
Multiple R-squared:  0.7528,    Adjusted R-squared:  0.7446 
F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

Visualizing the Regression Line

To visualize the relationship between car weight (wt) and fuel efficiency (mpg),
we use a scatter plot with a fitted regression line.
This helps us see how well a linear model fits the data.

\[ \hat{y} = \hat{\beta_0} + \hat{\beta_1}x \]

Residual Plot (Model Diagnostics)

To check how well the linear model fits, we inspect the residuals: \[ e_i \;=\; y_i - \hat{y}_i \] A good fit shows no clear pattern around the line \(e_i=0\) and roughly constant spread.

Interactive Visualization (Part 1 — Code)

We first create a ggplot object to visualize the relationship
between car weight (wt) and fuel efficiency (mpg).

library(ggplot2)

p <- ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point(size = 2, color = "steelblue", alpha = 0.8) +
  geom_smooth(method = "lm", se = TRUE, color = "#8C1D40", fill = "pink", linewidth = 0.9) +
  labs(
    title = "Simple Linear Regression: MPG vs. Weight",
    x = "Car Weight (1000 lbs)",
    y = "Miles per Gallon (MPG)"
  ) +
  theme_minimal(base_size = 12)

Interactive Visualization (Part 2 — Plot)

Here’s the scatter plot with regression line created by the code on the previous slide.

Interactive Visualization (Part 1 — Code)

We convert the ggplot from the previous slide into an
interactive Plotly chart using the function ggplotly().

library(ggplot2)
library(plotly)

p <- ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point(size = 2, color = "steelblue", alpha = 0.8) +
  geom_smooth(method = "lm", se = TRUE, color = "#8C1D40", fill = "pink", linewidth = 0.9) +
  labs(
    title = "Interactive Simple Linear Regression",
    x = "Car Weight (1000 lbs)",
    y = "Miles per Gallon (MPG)"
  ) +
  theme_minimal(base_size = 12)

ggplotly(p)

Interactive Visualization (Part 2 — Interactive Plot)

Here’s the interactive version of our regression plot,
created with ggplotly() from the Plotly library. Unlike the previous slide, this plot is interactive — hover, zoom, and pan to explore data points.

Model Interpretation & Final Visualization

We summarize the fitted simple linear regression model:

\[ \widehat{mpg} \;=\; \hat{\beta}_0 \;+\; \hat{\beta}_1 \, wt \]

  • \(\hat{\beta}_1\) (slope): change in expected MPG per +1000 lbs increase in weight.
  • \(R^2\): proportion of variation in MPG explained by weight.