2024-02-03

Simple Linear Regression

Simple Linear Regression (SLR) is a statistical method that allows us to describe and study the relationship between two continuous variables.

Linear regression can be used to:

  • Describe the relationship between two variables
  • Predict the value of one variable given the value of another
  • Test the significance of the relationship between two variables

Definition

The slope of the line of \(\displaystyle \widehat{y}\) (where \(\displaystyle \widehat{y}\) is the predicted value of \(y\)) is given by:

\(\displaystyle \widehat{y} = b_0 + b_1x\)

\(\displaystyle b_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}\)

\(\displaystyle b_0 = \bar{y} - b_1\bar{x}\)

\(\displaystyle \bar{x}\) and \(\displaystyle \bar{y}\) are the means of the \(x\) and \(y\) variables, respectively.

Example: Trees Data

Consider the trees dataset included in base R. This dataset contains the diameter (labeled girth), height, and volume of 31 black cherry trees. We will use the girth variable to predict the volume variable.

data(trees)
head(trees)
##   Girth Height Volume
## 1   8.3     70   10.3
## 2   8.6     65   10.3
## 3   8.8     63   10.2
## 4  10.5     72   16.4
## 5  10.7     81   18.8
## 6  10.8     83   19.7

Girth vs Volume vs Height

When considering all three dimensions of our data, we’re able to explore possible correlations in Plotly via:

plot_ly(
  trees, 
  x = ~Girth, 
  y = ~Volume, 
  z = ~Height, 
  type = "scatter3d", 
  mode = "markers"
)

3D Plot

Interpetation

In this example, we can identify possible correlations between the Girth, Volume, and Height of the trees in the trees data set. We’ll avoid plotting a three-dimensional linear regression plane, but the general formula for an N-dimensional regression is:

\(\displaystyle\theta=(X^TX)^{-1}X^Ty\)

Scatterplot of Girth vs. Volume

We can see that the data appears to have a positive linear relationship.

Girth vs. Volume with Regression

## `geom_smooth()` using formula = 'y ~ x'

Girth vs. Volume Code

In order to plot the line of linear regression, we leveraged ggplot:

ggplot(trees, aes(x = Girth, y = Volume)) +
  geom_point() +
  geom_smooth(method = "lm", se = FALSE)