Introduction

  • Simple linear regression is a statistical method for modeling the relationship between a dependent variable \(Y\) and an independent variable \(X\).
  • The model equation is: \[ Y = \beta_0 + \beta_1 X + \epsilon \]

Dataset

  • We will use the mtcars dataset available in R.
  • The model will predict mpg (miles per gallon) using hp (horsepower).
library(ggplot2)
library(plotly)

data(mtcars)
summary(mtcars)
##       mpg             cyl             disp             hp       
##  Min.   :10.40   Min.   :4.000   Min.   : 71.1   Min.   : 52.0  
##  1st Qu.:15.43   1st Qu.:4.000   1st Qu.:120.8   1st Qu.: 96.5  
##  Median :19.20   Median :6.000   Median :196.3   Median :123.0  
##  Mean   :20.09   Mean   :6.188   Mean   :230.7   Mean   :146.7  
##  3rd Qu.:22.80   3rd Qu.:8.000   3rd Qu.:326.0   3rd Qu.:180.0  
##  Max.   :33.90   Max.   :8.000   Max.   :472.0   Max.   :335.0  
##       drat             wt             qsec             vs        
##  Min.   :2.760   Min.   :1.513   Min.   :14.50   Min.   :0.0000  
##  1st Qu.:3.080   1st Qu.:2.581   1st Qu.:16.89   1st Qu.:0.0000  
##  Median :3.695   Median :3.325   Median :17.71   Median :0.0000  
##  Mean   :3.597   Mean   :3.217   Mean   :17.85   Mean   :0.4375  
##  3rd Qu.:3.920   3rd Qu.:3.610   3rd Qu.:18.90   3rd Qu.:1.0000  
##  Max.   :4.930   Max.   :5.424   Max.   :22.90   Max.   :1.0000  
##        am              gear            carb      
##  Min.   :0.0000   Min.   :3.000   Min.   :1.000  
##  1st Qu.:0.0000   1st Qu.:3.000   1st Qu.:2.000  
##  Median :0.0000   Median :4.000   Median :2.000  
##  Mean   :0.4062   Mean   :3.688   Mean   :2.812  
##  3rd Qu.:1.0000   3rd Qu.:4.000   3rd Qu.:4.000  
##  Max.   :1.0000   Max.   :5.000   Max.   :8.000

Scatter Plot of Data

ggplot(mtcars, aes(x = hp, y = mpg)) + 
  geom_point() + 
  ggtitle("Scatter Plot of MPG vs Horsepower")

Regression Model

The simple linear regression model estimates the coefficients:

lm_model <- lm(mpg ~ hp, data = mtcars)
summary(lm_model)
## 
## Call:
## lm(formula = mpg ~ hp, data = mtcars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -5.7121 -2.1122 -0.8854  1.5819  8.2360 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 30.09886    1.63392  18.421  < 2e-16 ***
## hp          -0.06823    0.01012  -6.742 1.79e-07 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.863 on 30 degrees of freedom
## Multiple R-squared:  0.6024, Adjusted R-squared:  0.5892 
## F-statistic: 45.46 on 1 and 30 DF,  p-value: 1.788e-07

Regression Line on Scatter Plot

ggplot(mtcars, aes(x = hp, y = mpg)) + 
  geom_point() + 
  geom_smooth(method = "lm", se = FALSE, color = "blue") +
  ggtitle("Regression Line for MPG vs Horsepower")
## `geom_smooth()` using formula = 'y ~ x'

3D Visualization

library(plotly)
p <- plot_ly(mtcars, x = ~hp, y = ~mpg, z = ~wt, type = "scatter3d", mode = "markers")
p

Mathematical Formulation

  • The estimated regression equation: \[ \hat{Y} = \beta_0 + \beta_1 X \]
  • The coefficients are obtained by minimizing the sum of squared residuals: \[ \sum (Y_i - \hat{Y}_i)^2 \]

Conclusion

  • Simple linear regression is useful for understanding relationships between variables.
  • The model helps predict mpg from hp.
  • Further extensions include multiple regression and nonlinear modeling