Linear Regression demonstrates the relationship between two or more variables in a data set, in the same way that a graph shows the relationship between x and y. Linear Regression is a common algorithm for statistical modeling because the results are easily understood.

Linear Regression assumes a linear relationship between the variables. This can be demonstrated by the women dataset when plotting height vs weight.

g <- ggplot(women, aes(height, weight)) + geom_point()
g

Linear Regression modeling creates a ‘line of best fit’ of the form y = A + Bx. The coefficient A represents the y-intercept, and B represents the slope. This provides a predicted outcome for each input.

To test how good of a fit the model is, we can look at the residuals. Residuals are the distance from each real point to the predicted point.

To do this in R:

g = g + geom_smooth(method = "lm") 

Here is an example using the same women dataset.

## `geom_smooth()` using formula 'y ~ x'

To see whether a Linear Regression model is a good fit, there are a few things we can look at. First, we can look at the P-value. The smaller it is, the more accurate the data is.

Secondly, we can look how small the residuals are. The goal is to have these values be as close to zero as possible, which would mean the model is very close to the actual answer.

Lastly, we can look at the R-squared value. The closer this value is to 1, the better the model.

Here is an interactive plot showing the actual values, the predicted values, and the residuals for each point.

We can clearly see that Simple Linear Regression was a great statistical model for this data set. However, we know that not all variables have a linear relationship. There are many different ways to modify a linear regression model to better fit your data.

Source: https://www.datacamp.com/community/tutorials/linear-regression-R