Linear regression is a type of statistical analysis between a dependent variable and one or more independent variables. Linear regression will try to fit a line to the data that can be used to predict the independent variables.

Linear Regression is a reliable, and relatively simple predictive model. However It can be susceptible to things like overfitting and has a high sensitivity to outlier data points.

In this figure it can be sometimes difficult to see how data is trending, and even harder to predict what future trends may look like. This would be a prefect application of Linear Regression.

\(Y = \beta_0 + \beta_1X + \epsilon\)

\(\beta_0 = \bar{Y} - \beta_1\bar{X}\)

\(SSE = \sum_{i=1}^{n} (Y_i - \hat{Y}_i)^2\)

While in the previous graph, it was difficult to see a trend between crime and urban population, using this graph it becomes clear that there is a slight upward trend between murder rate and urban population.

One of the biggest issues with linear regression can be Overfitting. For example, in the model below, the regression line is intentionally over fitted:

ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point() + geom_smooth(method = "lm", formula = y ~ poly(x, 10), 
                             se = FALSE, color = "orange") + 
  labs(title = "Scatter Plot of MPG vs. Weight with Overfitted Regression", 
       x = "Weight (1000 lbs)", y = "Miles per Gallon (MPG)") + theme_minimal()

You can see that using this regression line would cause not result in very accurate predictions.

The code for the previous graph look like the following:

ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point() + geom_smooth(method = "lm", formula = y ~ poly(x, 10), 
                             se = FALSE, color = "orange") + 
  labs(title = "Scatter Plot of MPG vs. Weight with Overfitted Regression", 
       x = "Weight (1000 lbs)", y = "Miles per Gallon (MPG)") + 
       theme_minimal()

Linear Regression is an powerful and simple tool in your toolbox to be able to predict future values based on some input data.