2024-03-17

What is Simple Linear Regression?

Simple linear regression is a statistical model that is used to estimate the relationship between 2 quantatative variables.

Simple linear regression is useful if someone wants to know:

  1. How strong the relationship is between two different variables
  2. The value of the dependent variable at a certain value of the independent variable.

Linear regression models use a straight line.

Assumptions of Simple Linear Regression

  1. Homogeneity of variance: the size of the error of our prediction (line) does not vary by a lot across the values of the independent variable.
  2. Independence of observations: the observations in the data set were collected using valid sampling methods.
  3. Normality: the data follows a normal distribution.
  4. The relationship between the independent and dependent variable is linear: the line of best fit is a straight line (as opposed to a curve, etc.)

Linear Regression Formula

The formula to perform a linear regression is:

\(y = \beta_0 + \beta_1X + \epsilon\)

Where \(y\) is the predicted value of the dependent variable for a given value of the dependent variable (X).

\(\beta_0\) is the y-intercept.

\(\beta_1\) is the regression coefficient (the slope)

\(X\) is the independent variable

\(\epsilon\) is the error of the estimate

mtcars Dataset

For this example I will be using the mtcars dataset, a summary of the data and a basic scatter plot is provided below (scroll):

data("mtcars")
summary(mtcars)
      mpg             cyl             disp             hp       
 Min.   :10.40   Min.   :4.000   Min.   : 71.1   Min.   : 52.0  
 1st Qu.:15.43   1st Qu.:4.000   1st Qu.:120.8   1st Qu.: 96.5  
 Median :19.20   Median :6.000   Median :196.3   Median :123.0  
 Mean   :20.09   Mean   :6.188   Mean   :230.7   Mean   :146.7  
 3rd Qu.:22.80   3rd Qu.:8.000   3rd Qu.:326.0   3rd Qu.:180.0  
 Max.   :33.90   Max.   :8.000   Max.   :472.0   Max.   :335.0  
      drat             wt             qsec             vs        
 Min.   :2.760   Min.   :1.513   Min.   :14.50   Min.   :0.0000  
 1st Qu.:3.080   1st Qu.:2.581   1st Qu.:16.89   1st Qu.:0.0000  
 Median :3.695   Median :3.325   Median :17.71   Median :0.0000  
 Mean   :3.597   Mean   :3.217   Mean   :17.85   Mean   :0.4375  
 3rd Qu.:3.920   3rd Qu.:3.610   3rd Qu.:18.90   3rd Qu.:1.0000  
 Max.   :4.930   Max.   :5.424   Max.   :22.90   Max.   :1.0000  
       am              gear            carb      
 Min.   :0.0000   Min.   :3.000   Min.   :1.000  
 1st Qu.:0.0000   1st Qu.:3.000   1st Qu.:2.000  
 Median :0.0000   Median :4.000   Median :2.000  
 Mean   :0.4062   Mean   :3.688   Mean   :2.812  
 3rd Qu.:1.0000   3rd Qu.:4.000   3rd Qu.:4.000  
 Max.   :1.0000   Max.   :5.000   Max.   :8.000  
ggplot(mtcars, aes(x = wt , y = mpg)) +
  geom_point() +
  labs(x = "Car Weight (tons)", y = "Miles Per Gallon")

Applying the Linear Regression Model on mtcars

# Calculate the linear regression of the previous slide's plot
linearReg <- lm(mpg ~ wt, data = mtcars)
summary(linearReg) # Provides the results of the model
Call:
lm(formula = mpg ~ wt, data = mtcars)

Residuals:
    Min      1Q  Median      3Q     Max 
-4.5432 -2.3647 -0.1252  1.4096  6.8727 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
wt           -5.3445     0.5591  -9.559 1.29e-10 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 3.046 on 30 degrees of freedom
Multiple R-squared:  0.7528,    Adjusted R-squared:  0.7446 
F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

The Coefficients table will give us our linear regression equation.

The intercept row gives us the estimate for our \(\beta_0\) and the wt row gives us the estimate for our \(\beta_1\) and the error column gives us our \(\epsilon\).

Our linear regression model for the plot would then be:

\(y = 37.2851 - 5.3445X \pm 0.5591\)

Plot the Linear Regression Model

Now, let’s plot the linear regression line on the scatter plot.

Interactive Plot of Linear Regression Model

Now, we will plot the same model on a plotly plot so you can interact with the graph:

Sources