In statistics and data analysis, simple linear regression is a method to produce a linear function to predict the dependent variable based on the independent variable within a set of data.

This model takes into account the errors of the data and finds a “best fit” linear line to match the correlations between the x and y data.

The model of simple linear regression simply is a linear formula \(y_i= \alpha + \beta x_i + \epsilon_i\). With \(\alpha\) being the y-intercept and \(\beta\) being the slope. \(\epsilon\) is the error that comes from making predictions in data analysis.

The goal of this approach is to find the expected values \(\hat{\alpha}\) and \(\hat{\beta}\). This is where the “best fit” is found. This will hopefully produce a formula that has the least sum of square residuals: \(\hat{\epsilon}\).

Which is given by \(\hat{\epsilon}_i = y_i - \alpha - \beta x_i\).

To calculate the expectation values \(\hat{\alpha}\) and \(\hat{\beta}\) one must first find the mean \(\bar{x}\) and \(\bar{y}\).

Then plug those values into this formula to find \(\hat{\alpha}\) and \(\hat{\beta}\):

\[ \hat{\alpha} = \bar{y} - (\hat{\beta} \bar{x}) \] \[ \hat{\beta} = \frac{\sum_{i=1}^{n} (x_i - \bar{x}) (y_i - \bar{y})}{\sum_{i=1}^{n} (x_i - \bar{x})^2} = \frac{\sum_{i=1}^{n} \Delta x_i \Delta y_i}{\sum_{i=1}^{n} \Delta x_i^2} \]

A great example of the simple linear regression is from a data set that comes with Rstudios, cars. This plot predictions the relationship between the speed of cars and their distance that they slow down.

Here is the simple code showing this plot.

library(ggplot2)
attach(cars)


c <- ggplot(data = cars, aes(x = speed, y = dist)) + geom_point() 

#plot with linear regression
c + geom_smooth(method="lm")

The first example showed a simple linear regression with a plot with data that looked to have a correlation between the independent and dependent data.

Here is a an example where there does not seem to be no correlation. The data looks sporadic although a formula can still be applied with a mean of the independent and dependent data. Yet it can be seen that most points are very far from the line so it is not a good prediction.

This compares the relationship of weight vs height for large sample of people at a work place.

One last plot of This data will be another example of a linear regression of data that does seem to have some correlation. This plot is interactive to compare the predicted formula with the real data. This is an example from the data set iris and compares the length of the flower petals vs the width of the flower petals.

library(plotly)
i <- ggplot(iris, aes(x = Petal.Length, y = Petal.Width)) + 
  geom_point(aes(color=Species))
s <- i + geom_smooth(method="lm" )  + 
  ggtitle("Petal Length vs Petal Width of Flower")

ggplotly(s)