Simple linear regression is used to study the relationship between two quantitative variables. The independent variable is represented by \(X\) and the dependent variable is represented by \(Y\)

Example

Can the weight of a car predict its fuel efficiency?

A simple linear regression model can be expressed as: \[ Y=\beta_0+\beta_1X+\epsilon \] where \(Y\) is the dependent variable, \(X\) is the independent variable, \(\beta_0\) is the intercept, \(\beta_1\) is the slope, and \(\epsilon\) is random error.

Here, the fitted regression line is: \[ \hat{Y}=b_0+b_1X \]

We will use the built in ‘mtcars’ dataset for this exercise.

Let, ‘wt’: car weight in thousands of pounds and ‘mpg’ : miles per gallon

Our research question is: Do heavier cars tend to have lower fuel efficiency?

##                      wt  mpg
## Mazda RX4         2.620 21.0
## Mazda RX4 Wag     2.875 21.0
## Datsun 710        2.320 22.8
## Hornet 4 Drive    3.215 21.4
## Hornet Sportabout 3.440 18.7
## Valiant           3.460 18.1

## `geom_smooth()` using formula = 'y ~ x'

This graph shows a negative relationship between the weight of the car and the fuel efficiency. Thus, from this graph we can conclude that heavier cars generally have a lower fuel economy.

The difference between the actual value and the predicted value for an observation is expressed as: \[ e_1=y_1-\hat{y}_i \] The regression with the lowest sum of squared residuals is the chosen regression. The sum of squared residuals is expressed by: \[ SSE=\sum_{i=1}^{n}(y_i-\hat{y}_i)^2 \] This lets us calculate the slope of the regression. The slope tells us how much \(Y\) is expected to change when \(X\) increases by 1 unit. The slope is estimated as: \[ b_1=\frac{\sum(x_i-\bar{x})(y_i-\bar{y})}{\sum(x_i-\bar{x})^2} \]

The ‘lm()’ function fits a linear regression model.

model<-lm(mpg~wt,data=mtcars)
summary(model)

## 
## Call:
## lm(formula = mpg ~ wt, data = mtcars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
## wt           -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528, Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

Here the slope is -5.344 and the intercept is 37.285.

## `geom_smooth()` using formula = 'y ~ x'

This interactive graph allows us to explore individual observations.

A residual plot can help us understand the fitted regression model better. The residuals here show the difference between the observed and the predicted values.