Put simply, linear regression is a mathematical method for modeling the relationship between an independent variable (usually x) and a dependent variable (usually y) in the form of a line.

For example, when you see a scatter plot with a line representing the data, that is linear regression at work!

It’s important to remember when looking at a linear regression line that the relationship it represents can be stronger (points are closer to the regression line) or weaker (points are further from the regression line - more spread out).

This ggplot example shows a positive relationship, which means that as the independent variable (x) increases, the dependent variable (y) also increases.

Conversely, this ggplot shows a negative relationship, which means that as the independent variable (x) increases, the dependent variable (y) decreases. This relationship is also slightly weaker than the previous one because the data is more spread out.

Though different sources will use different variable names, linear regression really comes down to a simple equation for a line: \[y = mx + b\] The complicated part comes in calculating the values of \(m\) and \(b\). Here we use the least squares method: \[m = \frac{n\Sigma(xy) - \Sigma(x)\Sigma(y)}{n\Sigma(x^2)-(\Sigma(x))^2}\] \[b = \frac{\Sigma(y) - m\Sigma(x)}{n}\] Where \(n\) is the number of data points being analyzed.

Here we see what it would look like to solve the linear regression formula by hand using some example data (n=10).
x = [1, 1, 2, 3, 4, 5, 6, 6, 7, 9]
y = [2, 1, 2, 4, 5, 5, 7, 6, 8, 9]
\(\Sigma(x)\) = 1+1+2+3+4+5+6+6+7+9 = 44
\(\Sigma(y)\) = 2+1+2+4+5+5+7+6+8+9 = 49
\(xy\) = [2, 1, 4, 12, 20, 25, 42, 36, 56, 81]
\(\Sigma(xy)\) = 2+1+4+12+20+25+42+36+56+81 = 279
\(\Sigma(x^2)\) = 1+1+4+9+16+25+36+36+49+81 = 258
\(m = \frac{10(279) - (44)(49)}{10(258) - 44^2} = \frac{634}{644} \approx 0.984\\ b = \frac{49 - 0.984(44)}{10} \approx 0.5704\)
After all that work, we find that the equation of then linear regression line is: \(y = 0.984x + 0.5704\)

The code below uses R to calculate a linear regression line and create an interactive plotly plot. See the next slide for the plot that it produces.

flighttimes = head(flights, 500)
flighttimes = select(flighttimes, dep_delay, arr_delay)
flighttimes = flighttimes[!is.na(flighttimes$dep_delay) & 
                            !is.na(flighttimes$arr_delay) & 
                            flighttimes$dep_delay < 500,]
flights_mod = lm(arr_delay~dep_delay, data=flighttimes)
x = flighttimes$dep_delay; y = flighttimes$arr_delay
xax <- list(title = "Departure Delay")
yax <- list(title = "Arrival Delay")
flights_fig <- plot_ly(x=x, y=y, type="scatter", name="Flights", 
              mode="markers", width=800, height=400) %>%
  add_lines(x=x, y=fitted(flights_mod), name="Regression Line") %>%
  layout(xaxis = xax, yaxis = yax) %>%
  layout(margin=list(l=75, r=75, b=20, t=20))
config(flights_fig, displaylogo=FALSE)

This plotly plot shows linear regression using real data about flights! Here we see a positive relationship between departure delays and arrival delays. Some flights have been filtered out for ease of understanding.

Now it’s your turn to try out plotting and linear regression using R!
Best of luck in your future linear regression adventures!