2024-02-25

Simple Linear Regression

Simple Linear Regression is a statistical method for modeling the relationship between two quantitative variables.

  • Dependent Variable (Y): The outcome or the variable that we try to predict.
  • Independent Variable (X): The predictor or the variable that we use to make predictions about Y.

The Linear Regression Model

The model can be represented by the equation:

\[ Y = \beta_0 + \beta_1X + \epsilon \] - \(\beta_0\) is the intercept, - \(\beta_1\) is the slope of the line, - \(\epsilon\) is the error term.

Assumptions of Linear Regression

Linear regression analysis requires several assumptions about the data:

  1. Linear relationship: There exists a linear relationship between the independent variable, X, and the dependent variable, Y.
  2. Independence: The residuals (errors) of the data are independent.
  3. Homoscedasticity: The residuals exhibit constant variance at every level of the independent variable.
  4. Normality of residuals: The residuals of the model are normally distributed.

Example Dataset: mtcars

# Summary statistics for miles per gallon
summary(mtcars$mpg)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   10.40   15.43   19.20   20.09   22.80   33.90
# Summary statistics for weight (in 1000 lbs)
summary(mtcars$wt)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   1.513   2.581   3.325   3.217   3.610   5.424

Visualizing the Relationship between Weight and MPG

## `geom_smooth()` using formula = 'y ~ x'

Fitting a Linear Model

## 
## Call:
## lm(formula = mpg ~ wt, data = mtcars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
## wt           -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528, Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

Exploring Complex Relationships with 3D Plots

Checking Residuals for Homoscedasticity

Conclusion

Through this exercise, we’ve seen how simple linear regression can be a powerful tool for understanding and predicting relationships between variables. By carefully checking model assumptions and considering the broader context, we can apply these techniques to a wide range of problems in engineering, science, and beyond.