Simple linear regression is a statistical method used to study the relationship between two quantitative variables.

• Predictor variable (x): the variable used to explain or predict
• Response variable (y): the variable we want to predict
• The goal is to find a straight line that best describes the relationship

In this example, we will study the relationship between car weight and fuel efficiency (MPG).

A simple linear regression model is written as

\[ Y = \beta_0 + \beta_1X + \epsilon \]

where:

• \(Y\) is the response variable
• \(X\) is the predictor variable
• \(\beta_0\) is the intercept
• \(\beta_1\) is the slope
• \(\epsilon\) represents random error

For prediction, we use the fitted equation

\[ \hat{Y} = b_0 + b_1X \]

We will use the built-in R dataset mtcars.

For this example:

• wt = vehicle weight in thousands of pounds
• mpg = miles per gallon
• Each point represents one car

##                      wt  mpg
## Mazda RX4         2.620 21.0
## Mazda RX4 Wag     2.875 21.0
## Datsun 710        2.320 22.8
## Hornet 4 Drive    3.215 21.4
## Hornet Sportabout 3.440 18.7
## Valiant           3.460 18.1

This lets us ask a simple question:

Do heavier cars tend to have lower fuel efficiency?

## `geom_smooth()` using formula = 'y ~ x'

The graph shows a clear negative relationship. As vehicle weight increases, MPG generally decreases.

The slope can be calculated using

\[ b_1 = \frac{\sum (x_i-\bar{x})(y_i-\bar{y})} {\sum (x_i-\bar{x})^2} \]

The intercept is

\[ b_0 = \bar{y} - b_1\bar{x} \]

For the mtcars data, R estimates:

## (Intercept)          wt 
##   37.285126   -5.344472

So the fitted line is approximately

\[ \widehat{MPG} = 37.29 - 5.34(\text{Weight}) \]

The fitted equation is:

\[ \widehat{MPG} = 37.29 - 5.34(\text{Weight}) \]

The slope is approximately -5.34.

This means that, according to this model, increasing vehicle weight by 1,000 pounds is associated with a decrease of about 5.34 MPG on average.

The negative slope matches what we saw in the scatter plot.

## `geom_smooth()` using formula = 'y ~ x'

Move the mouse over the points to see their values.

A residual is the difference between the observed value and the predicted value.

\[ e_i = y_i - \hat{y}_i \]

A residual plot helps us check whether a linear model is reasonable.

library(ggplot2)
library(plotly)

data(mtcars)

model <- lm(mpg ~ wt, data = mtcars)
summary(model)

ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point() +
  geom_smooth(method = "lm")

The lm() function fits the linear regression model, and ggplot() creates the visualization.

## 
## Call:
## lm(formula = mpg ~ wt, data = mtcars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
## wt           -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528, Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

Important values include:

• Slope: direction and size of the relationship
• R-squared: how much variation in MPG is explained by weight
• p-value: evidence about whether the relationship is statistically significant

For this dataset, vehicle weight explains a large amount of the variation in MPG.

Simple linear regression helps us describe and predict relationships between two quantitative variables.

From this example:

• Heavier cars generally have lower MPG
• The fitted slope is negative
• The regression line gives us a prediction equation
• Residuals help us check the quality of the model
• R makes it easy to calculate, visualize, and explore the model

Linear regression is widely used in science, engineering, business, and many other fields.