2024-02-18

Introduction to Simple Linear Regression

  • Linear regression is a statistical method which attempts to model a linear relationship between two variables by fitting a linear equation to previously observed data.
  • It can be used to predict or approximate values based on the linear equation without actually observing the value.
  • Before attempting a linear regression, you should first determine if a relationship is present in the data. A quick way you can check is by graphing a scatter plot.

Scatter Plot Example

- This graph appears to depict a positive linear relationship, so this dataset could be good choice for linear regression.

Scatter Plot Example

- This graph clearly does not depict a linear relationship, so this dataset does not seem like a good choice for linear regression.

The Regression Equation

  • The simple linear regression model can be represented as:

\[ Y = m X + b \]

  • Where:
    • \(Y\) is the dependent variable,
    • \(X\) is the independent variable,
    • \(b\) is the intercept, and
    • \(m\) is the slope
  • \(m\) and \(b\) are values we must calculate to find the linear equation that best fits the data

Least-Squares Regression

  • The most common method for fitting a regression is the least-squares method. This method calculates the best-fitting line for the data by minimizing the sum of squares of the vertical deviations of each data point from the line (if the point lies exactly on the line, the vertical deviation is 0).

Linear Regression in R

  • Here is how to perform a linear regression on a randomly generated normally distributed dataset:
suppressMessages(library(plotly))
set.seed(123)
# generate random data
x <- rnorm(100)
y <- x + rnorm(100)
# fit linear regression model
lm_model <- lm(y ~ x)
# create scatter plot
scatter_plot <- plot_ly(x = ~x, y = ~y, type = 'scatter', mode = 'markers') %>%
  layout(title = "Scatter Plot with Line of Best Fit",
         xaxis = list(title = "X"),
         yaxis = list(title = "Y"))
# add line of best fit
abline <- data.frame(x = range(x), y = predict(lm_model, data.frame(x = range(x))))
scatter_plot <- scatter_plot %>% add_lines(x = ~x, y = ~y, data = abline)
scatter_plot

Linear Regression in R (cont.)

  • Here is what the code on the previous slide produces:

Calculating the Linear Equation of the Line of Best Fit in R

coefficients <- coef(lm_model)
intercept <- coefficients[1]
slope <- coefficients[2]
cat("Equation of the line of best fit: Y =", round(intercept, 2), "+", round(slope, 2), "X")
## Equation of the line of best fit: Y = -0.1 + 0.95 X

Equation of the line of best fit: \[ Y = -0.1 + 0.95 X \]