The goal of linear regression is to model the relationship between two continuous variables. There is an independent variable which is the predictor, and a dependent variable known as the response. It is used to test how changes in one variable influence the outcomes of another.

For Example

• How weight influences fuel efficiency

• How height influences weight

• How time spent studying influences exam scores

The regression model is \(y = \beta_0 + \beta_1 x + \varepsilon\).

• \(\beta_0\) is the intercept. It gives us the expected y value when x is zero. Assuming that x being equal to 0 makes sense, then the intercept is the baseline level of the outcome.

• \(\beta_1\) is the slope. It is the average change in y as x increments by one. A larger magnitude indicates a stronger relationship between x and y.

  • A positive slope means that as x increases, so does y.

  • A negative slope means that as x increases, y decreases.

• \(\varepsilon\) is the random error. It is the variation seen in y that cannot be predicted by x.

There are 4 core assumptions when in linear regression which are conditions that the data is expected to satisfy so that our research is valid.

• Linearity: The relationship between x and y is linear.
  • \(E(y \mid x) = \beta_0 + \beta_1 x\)
• Independence: Observations are independent of one another.
  • \(\text{Cov}(\varepsilon_i, \varepsilon_j) = 0 \quad \text{for } i \neq j\)

• Normality of Errors: random error (\(\varepsilon\)) is normally distributed with a mean of zero.
  • \(\varepsilon_i \sim N(0, \sigma^2)\)
• Homoscedasticity: Variance should be constant accross all x values.
  • \(\text{Var}(\varepsilon_i \mid x) = \sigma^2\)

The points on the graph are spread near the line and evenly distributed above and below it. It shows that the model has linearity, homoscedasticity, and a mean of errors equaling zero.

## 
## Call:
## lm(formula = Petal.Length ~ Petal.Width, data = iris)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.33542 -0.30347 -0.02955  0.25776  1.39453 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  1.08356    0.07297   14.85   <2e-16 ***
## Petal.Width  2.22994    0.05140   43.39   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.4782 on 148 degrees of freedom
## Multiple R-squared:  0.9271, Adjusted R-squared:  0.9266 
## F-statistic:  1882 on 1 and 148 DF,  p-value: < 2.2e-16

The summary shows us that Petal.Width is strong predictor of Petal.Length

• The residuals show us that the model fits the data well. This is because they are all closely centered around 0. They indicate how far our predictions are from the actual values.

• The coefficients tell us that the predicted Petal.Length is 1.084(\(\beta_0\)) and the slope is 2.230(\(\beta_1\)).

• The residual standard error tells us that on average, the predication is 0.48 cm off from the actual Petal.Length.

• The \(R^2\) values tells us that 92.7% of variation in Petal.Length is predictable by Petal.Width

ggplot(women, aes(x = height, y = weight)) +
  geom_point(color = "red", size = 2) + 
  geom_smooth(method = "lm", se = TRUE, color = "blue") + 
  labs(title = "Linear Regression: Height vs Weight", 
    x = "Height (in)", y = "Weight (lb)") + 
  theme_minimal(base_size = 12)

From this scatter plot, we can see that Ozone’s influence on Wind is weak because of how dispersed and uneven the points are. There is no line of best fit that could be drawn that would show a strong correlation.