2024-06-09

Simple Linear Regression

Introduction to Simple Linear Regression

Simple linear regression is a statistical method that allows us to summarize and study relationships between two continuous (quantitative) variables.

The Simple Linear Regression Model

The simple linear regression model is represented by the equation:

\[ y = \beta_0 + \beta_1 x + \epsilon \]

where: \(y\) is the dependent variable

\(x\) is the independent variable

\(\beta_0\) is the y-intercept

\(\beta_1\) is the slope

\(\epsilon\) is the error term

Assumptions of Simple Linear Regression

  1. Linearity: The relationship between the independent and dependent variable is linear.
  2. Independence: Observations are independent of each other.
  3. Homoscedasticity: Constant variance of the errors.
  4. Normality: The errors of the model are normally distributed.

Estimating the Regression Line

The coefficients \(\beta_0\) and \(\beta_1\) are estimated using the least squares method, which minimizes the sum of squared residuals:

\[ \hat{\beta_1} = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2} \] \[ \hat{\beta_0} = \bar{y} - \hat{\beta_1}\bar{x} \]

Interpretation of the Regression Coefficients

  • The intercept \(\hat{\beta_0}\): the expected value of \(y\) when \(x\) is 0.
  • The slope \(\hat{\beta_1}\): the expected change in \(y\) for a one-unit change in \(x\).

Example with Data

## 
## Call:
## lm(formula = mpg ~ wt, data = mtcars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
## wt           -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528, Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

Let’s interpret the output of the linear regression model:

The model suggests a strong negative linear relationship between car weight and fuel efficiency (mpg). The weight of the car is a significant predictor of mpg. The model explains about 75.28% of the variability in mpg. The intercept and slope are both highly significant, as indicated by their p-values. This detailed interpretation helps in understanding the influence of car weight on fuel efficiency and the overall fit of the regression model.

Scatter Plot with Regression Line

## `geom_smooth()` using formula = 'y ~ x'

Residual Plot

Residual standard error: 3.046 on 30 degrees of freedom

The residual standard error (RSE) is a measure of the quality of the linear regression fit. It represents the average amount that the observed values deviate from the regression line. Here, the RSE is 3.046, suggesting that the actual mpg values typically deviate from the predicted values by about 3.046 mpg.

Conclusion and Further Reading

Simple linear regression is a fundamental tool in statistical analysis. For further reading, consider:

. “Introduction to Statistical Learning” by Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani.

. “Applied Linear Statistical Models” by John Neter, Michael H. Kutner, Christopher J. Nachtsheim, and William Wasserman.