Simple Linear Regression - Introduction

  • Simple Linear Regression is a statistical method used to measure the relationship between two quantitative (numeric) variables.
  • There is an independent variable and a dependent variable. The dependent variable “depends” on the value of the independent variable.
  • The independent variable is used to predict the value of the dependent variable through a linear equation.

How to do this in R?

  • In R language, we can simulate Simple Linear Regression by doing the following:
  • Determine your independent and dependent variables. These can be generated (see the following example), or they can be taken from a dataset.
  • Use the lm() function to generate the simple linear regression.
  • I show an example of this method in the following slide using plotly plot.

Example 1 - with Plotly plot

set.seed(123)
x <- 1:50
y <- 3 * x + rnorm(50, mean = 0, sd = 10)
model <- lm(y ~ x)
plot_ly(x = ~x, y = ~y, mode = "markers", type = "scatter", name = "Data") %>%
  add_trace(x = ~x, y = ~predict(model), mode = "lines", name = "Regression Line") %>%
  layout(title = "Simple Linear Regression Example",
         xaxis = list(title = "X"),
         yaxis = list(title = "Y"))

Example 2: with ggplot

## `geom_smooth()` using formula = 'y ~ x'

Explanation of the previous plots

  • In the previous examples, we performed SLR by having 50 x values (1-50) and generating random values for the y values.
  • The previous two plots are identical: the first one is created with plotly, and the second with ggplot2.
  • Now, I will show another example using ggplot. This plot will use residuals. It will show the difference between the observed and predicted values (residuals) versus the independent variable.

Example 3: Using ggplot again

Recap - What Next?

  • So far, we have gone over a brief introduction to SLR and shown 3 different examples using plotly and ggplot.
  • Now, I will touch on two SLR topics that need Latex to write equations:
    1. an Equation of Simple Linear Regression, and
    1. Coefficient of Determination (R-squared).
  • I will not show the R code used to generate the Latex, as it will mess up the formatting. I will show only the generated Latex text.

Latex pt. 1: Equation of SLR

  • The equation for Simple Linear Regression is (written in Latex): \[ Y = \beta_0 + \beta_1 X + \varepsilon\] Where
  • Y represents the dependent variable
  • X represents the independent variable
  • \(\beta_0\) is the intercept (the value of Y when X = 0)
  • \(\beta_1\) is the slope
  • and \(\varepsilon\) represents the error term.

Latex pt. 2: Coefficient of Determination (R-squared)

  • The coefficient of determination (R-squared) measures the proportion of the variance in the dependent variable that is predictable from the independent variable.
  • We can represent it mathematically as:

\[ R^2 = 1 - RSS / TSS\] Where - \(R^2\) is the coefficient of determination - RSS is the sum of squares of residuals - and TSS is the total sum of squares.