In Statistics, Linear Regression is a model that estimates the relationship between a dependent variable and one or more independent variables also termed as ‘regressor’ (Linear Regression, Wikipedia).

It assumes that there is a linear relationship between the input and output, meaning the output changes at a constant rate as the input changes. This relationship is represented by a straight line (Linear Regression in Machine Learning, GeekforGeeks).

• Simple Linear Regression
• Multiple Linear Regression
• Logistic Regression etc.

The goal of simple linear regression is to model the relationship between a single independent variable, \(X\), and a dependent variable, \(Y\).

The model is described by the following equation:

\[Y = \beta_0 + \beta_1 X + \epsilon\]

Where: - \(Y\) is the dependent variable (in my example, MPG). - \(X\) is the independent variable (in my example, Weight). - \(\beta_0\) is the y-intercept of the line. - \(\beta_1\) is the slope of the line. - \(\epsilon\) represents the random error term.

To determine if a true linear relationship exists, we perform a hypothesis test on the slope coefficient, \(\beta_1\).

The null hypothesis (\(H_0\)) states that there is no linear relationship between the variables (the slope is zero). The alternative hypothesis (\(H_a\)) states that there is one.

\[H_0: \beta_1 = 0\] \[H_a: \beta_1 \neq 0\]

If we find enough evidence to reject the null hypothesis (\(H_0\)), we can conclude that the relationship between weight and MPG is statistically significant.

library(ggplot2)
ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point(
    color = "maroon", 
    size = 3, 
    alpha = 0.8) +
  geom_smooth(
    method = "lm", 
    se = TRUE, 
    color = "black"
  ) +
  labs(
    title = "Fuel Efficiency Decreases as Car Weight Increases",
    x = "Weight (1000 lbs)",
    y = "Miles Per Gallon (MPG)"
  ) + theme_minimal()

## `geom_smooth()` using formula = 'y ~ x'