Linear regression estimates a line of form \(y = m\cdot x + b\) to best fit given data where variables estimates are calculated according to the following equations.

\[m = \frac{S_{xy}}{S_{xx}} = \frac{\Sigma(x_i - \bar{x})(y_i-\bar{y})}{(x_i-\bar{x})^2}\] \[b = \bar{y} - m\cdot\bar{x}\] \[\bar{y} = \frac{\Sigma y_i}{n} \quad ; \quad \bar{x} = \frac{\Sigma x_i}{n}\]

Thus, \(\hat{y} = m\cdot x + b\), where \(\hat{y}\) is the estimated value of y using the model

Additionally, \(y_i = m\cdot x_i + b + \epsilon_i\), where \(\epsilon\) is the error of the model

We can use linear regression to find the extent of linear relationships between variables.

Using the sum of squares \(r^2\), we can see what variable the miles per gallon (MPG) of cars is most dependent on. Variables tested will be:

1. Number of Cylinders

2. Horsepower

3. Drat (rear axle ratio)

4. Weight

Here is the code for linear regression in R and plot using ggplot2

data(mtcars) #load data

model <- lm(mpg~cyl, data=mtcars) #store linear model of mpg vs cyl in "model"

# This code obtains the slope, intercept, and r^2 value for labeling on the plot
info <- substitute(y == b + m %.% x*","~~r^2~"="~rsquare, 
        list(b = format(unname(coef(model)[1]), digits = 2),
              m = format(unname(coef(model)[2]), digits = 2),
             rsquare = format(summary(model)$r.squared, digits = 3)));
eq <- as.character(as.expression(info)) #coerce into character for labeling

## ggplot can also create a linear regression model of mpg vs cyl
ggplot(mtcars,aes(cyl, mpg)) +                       # define data set, x, and y
  geom_point() +                                     #scatter plot
  geom_smooth(method='lm') +                         #linear model
  geom_text(x = 8, y = 30, label = eq, parse = TRUE) #add in label

## `geom_smooth()` using formula = 'y ~ x'

## `geom_smooth()` using formula = 'y ~ x'

## `geom_smooth()` using formula = 'y ~ x'

## `geom_smooth()` using formula = 'y ~ x'

## `geom_smooth()` using formula = 'y ~ x'

model <- lm(mpg~ cyl+wt, data=mtcars)
b <- format(unname(coef(model)[1]), digits = 2)
m1 <- format(unname(coef(model)[2]), digits = 2)
m2 <- format(unname(coef(model)[3]), digits = 2)
rsquare <- format(summary(model)$r.squared, digits = 4) 
#old technique doesn't work, as plotly cannot parse the equation like ggplot
#equation and r^2 is rewritten below to be put into the plot
eq = ("-1.5*Cylinders - 3.2*Weight + 40 ; r^2 = 0.830")

x = seq(4,8, by=0.2); y = seq(1,6, by=0.25) # Vectors of same length
Cylinders = matrix(rep(x,length(y)),nrow=length(x),byrow = T)
Wt = matrix(rep(y,length(x)),ncol=length(y),byrow = F) # Create mesh grids
MPG = Cylinders*coef(model)[2] + Wt*coef(model)[3] + coef(model)[1]
xax = list(title = "Cylinders");yax = list(title = "Weight (1000 lbs)")
zax = list(title = "MPG")
fig <- plot_ly(x=Cylinders, y=Wt, z=MPG, showscale = F) %>%
layout(scene = list(xaxis = xax, yaxis = yax, zaxis = zax))
fig <- fig %>% add_surface(z = MPG) #add plane
fig <- fig %>% add_trace(data = mtcars, x = mtcars$cyl, y = mtcars$wt, z = mtcars$mpg,
mode = "markers", type = "scatter3d", marker = list(size = 4, color = "red",
  symbol = 10)) %>% # add data points 
  layout(title = 'MPG vs Weight vs Cylinders')
fig <- fig %>% add_annotations(x = 1.15, y = .9, z = 15, text = eq, showarrow = F) #label
fig