A Simple Linear Regression (Hwk3)

Main Overview

• Understanding the meaning of ‘Simple Linear Regression (SLR)’
• Model, assumptions, and interpretation
• Example like (egineering): predicting the fuel efficiency (mpg) by using the car’s weight (wt)
• Visualizing (ggplot2) + interactive 3D (plotly)
• Inference: hypothesis test and confidence interval for slope
• Model diagnostics
• Summarry

---

what is SLR?

• We relate a numeric response \(Y\) to a single numeric predictor \(X\).
• The goal is to estimate a line \(\hat{Y} = \hat{\beta}_0 + \hat{\beta}_1 X\) that best fits data.
• Typically uses in engineering / CS:
  • Predicting the latency from payload size
  • Estimating the energy use from CPU load
  • Predicting the fuel efficiency from vehicle weight (our example)

---

Model

We assume for observation \(i\):

\[ Y_i = \beta_1 X_i + \varepsilon_i, \quad \varepsilon_i \sim \text{i.i.d.} N(0, \sigma^2). \]

The Least Square Estimate

\[ \hat{\beta}_1 = \frac{\sum_i (X_i - \bar{X})(Y_i - \bar{Y})}{\sum_i (X_i - \bar{X})^2}, \quad \hat{\beta}_0 = \bar{Y} - \hat{\beta}_1\,\bar{X}. \]

---

Assumptions

1. Linearity: \(E[Y\mid X] = \beta_0 + \beta_1 X\).
2. Independence: observations are independent.
3. Homoscedasticity: constant variance \(\operatorname{Var}(\varepsilon_i) = \sigma^2\).
4. Normality of errors: \(\varepsilon_i\) are normally distributed.

These are the diagnostic tools to check if these are reasonable.

---

Data examples (mtcars)

Modeling mpg from wt (car weight, 1000 lbs).

df <- mtcars %>% select(mpg, wt, hp)
head(df)

##                    mpg    wt  hp
## Mazda RX4         21.0 2.620 110
## Mazda RX4 Wag     21.0 2.875 110
## Datsun 710        22.8 2.320  93
## Hornet 4 Drive    21.4 3.215 110
## Hornet Sportabout 18.7 3.440 175
## Valiant           18.1 3.460 105

summary(df)

##       mpg              wt              hp       
##  Min.   :10.40   Min.   :1.513   Min.   : 52.0  
##  1st Qu.:15.43   1st Qu.:2.581   1st Qu.: 96.5  
##  Median :19.20   Median :3.325   Median :123.0  
##  Mean   :20.09   Mean   :3.217   Mean   :146.7  
##  3rd Qu.:22.80   3rd Qu.:3.610   3rd Qu.:180.0  
##  Max.   :33.90   Max.   :5.424   Max.   :335.0

---

ggplot #1 - Scatter + Fitted Line

p1 <- ggplot(df, aes(x = wt, y = mpg)) +
  geom_point(size = 2, alpha = 0.8) +
  geom_smooth(method = "lm", se = TRUE) +
  labs(title = "mpg vs wt (with OLS fit)",
       x = "Weight (1000 lbs)",
       y = "Miles per gallon")
p1

## `geom_smooth()` using formula = 'y ~ x'

The fuel efficiency decreases as the weight increases.

---

Fit the Model + Inference

fit <- lm(mpg ~ wt, data = df)
tidy_fit <- tidy(fit, conf.int = TRUE)
glance_fit <- glance(fit)
tidy_fit

## # A tibble: 2 × 7
##   term        estimate std.error statistic  p.value conf.low conf.high
##   <chr>          <dbl>     <dbl>     <dbl>    <dbl>    <dbl>     <dbl>
## 1 (Intercept)    37.3      1.88      19.9  8.24e-19    33.5      41.1 
## 2 wt             -5.34     0.559     -9.56 1.29e-10    -6.49     -4.20

glance_fit

## # A tibble: 1 × 12
##   r.squared adj.r.squared sigma statistic  p.value    df logLik   AIC   BIC
##       <dbl>         <dbl> <dbl>     <dbl>    <dbl> <dbl>  <dbl> <dbl> <dbl>
## 1     0.753         0.745  3.05      91.4 1.29e-10     1  -80.0  166.  170.
## # ℹ 3 more variables: deviance <dbl>, df.residual <int>, nobs <int>

Interpretation: The slope estimates expected change in mpg per 1 (1000 lb) increase in weight.

---

Hypothesis Test for Slope (Math)

We often test

\[ H_0: \beta_1 = 0 \quad \text{vs} \quad H_1: \beta_1 \ne 0. \]

Test statistic:

\[ t = \frac{\hat{\beta}_1 - 0}{\operatorname{SE}(\hat{\beta}_1)} \sim t_{n-2}\,, \]

with p-value from the \(t\)-distribution. Confidence interval for \(\beta_1\):

\[ \hat{\beta}_1 \pm t_{\alpha/2,\, n-2}\;\operatorname{SE}(\hat{\beta}_1). \]

---

ggplot #2 — Residuals vs Fitted

aug <- augment(fit)
p2 <- ggplot(aug, aes(x = .fitted, y = .resid)) +
  geom_hline(yintercept = 0, linetype = 2) +
  geom_point(alpha = 0.85) +
  labs(title = "Residuals vs Fitted",
       x = "Fitted values",
       y = "Residuals")
p2

Check linearity & equal variance.

---

ploty (3D) - mpg vs wt & hp

Even though the model can be simple (1 predictor), 3D helps explore another variable (hp).

plot_ly(df, x = ~wt, y = ~hp, z = ~mpg,
        type = "scatter3d", mode = "markers") %>%
  layout(title = "Interactive 3D: mpg vs wt & hp",
         scene = list(xaxis = list(title = "wt"),
                      yaxis = list(title = "hp"),
                      zaxis = list(title = "mpg")))

---

Code Slide - Reproduce ggplot #1

library(ggplot2)

ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point(size = 2, alpha = 0.8) +
  geom_smooth(method = "lm", se = TRUE) +
  labs(title = "mpg vs wt (with OLS fit)",
       x = "Weight (1000 lbs)",
       y = "Miles per gallon")

---

Diagnostics & The Next Steps

• Consider adding predictors. Example (hp, cycl) to move to multiple regression.
• Tranformation if the residuals shows a curvature (log-mpg).
• Always check the assumptions using plots and domain knowledge.
• Communicate eefect sizes and uncertainty (CIs).

Summary

• SLR is known as a foundational predictive and inferential tool.
• Slope \(\hat{\beta}_1\) quantifies can be expected changing in \(Y\) per unit of \(X\).
• Use ggplot2 for a clearer visuals and plotly for interactivity.
• Verify assumptions with diagnostics before trusting conclusions.

---