Simple Linear Regression (SLR) is a statistical tool that:
The model is expressed as: \[Y = \beta_0 + \beta_1X + \epsilon\]
Predictor (\(X\)): The independent variable used to predict the response variable.
Intercept (\(\beta_0\)): The expected value when the predictor is zero.
Slope (\(\beta_1\)): The change in the response for a one-unit change in the predictor.
Error term (\(\epsilon\)): Represents random variability not captured by the model.
Linear: The relationship between predictor and response is linear.
Independent of Errors: There is no relationship between the residuals (errors) and the Y variable
Homoscedastic: There is an equal variance of the residuals for all values of X
Normality of Errors: Residuals are normally distributed.
Simple linear regression uses least squares method to find the best fit line.
The least squares estimates are given by minimizing the residual sum of squares: \[\min_{\beta} \sum_{i=1}^{n} \left( y_i - \beta_0 - \beta_1 x_i \right)^2\]
We will use the built-in mtcars data set extracted from 1974 Motor Trends US magazine which contains fuel consumption and 10 aspects of automobile design and performance for 32 automobiles (1973–74 models):
head(mtcars[, c("mpg", "wt")], 3)
mpg wt Mazda RX4 21.0 2.620 Mazda RX4 Wag 21.0 2.875 Datsun 710 22.8 2.320
We can use this data to determine a relationship between fuel economy and weight.
Using the linear model function, you can find the best (\(\beta_0\)) and (\(\beta_1\)) along with the residuals.
model <- lm(mpg ~ wt, data = mtcars) summary(model)
Call:
lm(formula = mpg ~ wt, data = mtcars)
Residuals:
Min 1Q Median 3Q Max
-4.5432 -2.3647 -0.1252 1.4096 6.8727
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 37.2851 1.8776 19.858 < 2e-16 ***
wt -5.3445 0.5591 -9.559 1.29e-10 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 3.046 on 30 degrees of freedom
Multiple R-squared: 0.7528, Adjusted R-squared: 0.7446
F-statistic: 91.38 on 1 and 30 DF, p-value: 1.294e-10
You can see no linear correlation between the residuals and the expected value of MPG.