2024-03-20

What is Simple Linear Regression?

Linear regression is a statistical method used to model the relationship between two variables, typically referred to as the independent variable (X) and the dependent variable (Y). The method assumes a linear relationship between X and Y, meaning that a change in X results in a proportional change in Y.

In simple linear regression, there is only one independent variable, while in multiple linear regression, there are multiple independent variables.

Formula

Simple linear regression is a statistical method used to model the relationship between two variables: an independent variable (X) and a dependent variable (Y). It assumes a linear relationship between X and Y, represented by the equation:

\[ Y = \beta_0 + \beta_1X + \varepsilon \]

  • \(Y\): Dependent variable
  • \(X\): Independent variable
  • \(\beta_0\): Intercept
  • \(\beta_1\): Slope
  • \(\varepsilon\): Error term

Estimation

The coefficients \(\beta_0\) and \(\beta_1\) are estimated using the least squares method to minimize the sum of squared differences between the observed and predicted values of Y.

\[ \hat{\beta}_1 = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n} (x_i - \bar{x})^2} \]

\[ \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x} \]

  • \(\hat{\beta}_0\): Estimated intercept
  • \(\hat{\beta}_1\): Estimated slope
  • \(\bar{x}\): Mean of X
  • \(\bar{y}\): Mean of Y

Slide with R Code for Example 1

set.seed(123)
n <- 1000
advertising <- rnorm(n, mean = 50, sd = 10)
sales <- 20 + 0.5 * advertising + rnorm(n, mean = 0, sd = 5)

# Create a dataframe
data <- data.frame(sales, advertising)

ggplot(data, aes(x = advertising, y = sales)) +
  geom_point(alpha = 0.5) +  # Add transparency to points for better visualization
  geom_smooth(method = "lm", se = FALSE, color = "red") +  # Add linear regression line
  labs(x = "Advertising Expenditure", y = "Sales", title = "Scatter Plot with Linear Regression")

Example 1 of Linear Regression

## `geom_smooth()` using formula = 'y ~ x'

Example 2 of Linear Regression

## A marker object has been specified, but markers is not in the mode
## Adding markers to the mode...

Example 3 of Linear Regression

Conclusion

Simple linear regression is a powerful tool for understanding the relationship between two variables and making predictions based on that relationship. By estimating the intercept and slope coefficients, we can create a linear model that can be used for prediction and inference.