Linear regression is a statistical method used when we want to model a relationship between a explanatory (dependent) variable and a response (independent) variable. A model will be generated that fits a line of best fit through data points that can be used to estimate and predict future behavior of the input-output relationship. This is useful to make predictions about the output with limited information about the input.

Regression is used in all areas of science and engineering work for its use in analyzing patterns and providing quick error estimations about relationships. Regression does not NEED to be linear, but a linear relationship is the simplest case of regression that will be presented here.

Now, we can generate a model using R code

model <- lm(iris$Petal.Width ~ iris$Petal.Length, data = iris)
model_coefficients <- coef(model)
intercept <- model_coefficients[1]
slope <- model_coefficients[2]

ggplot(iris, aes(x = Petal.Length, y = Petal.Width)) +
  geom_point(aes(color = Species), size = 1.0) +
  geom_smooth(method = "lm", formula = y ~ x, color="black", se = TRUE) +
  labs(title = "Bestfit: Length vs.Width",
       x = "Petal Length",
       y = "Petal Width") +
  theme_bw()

Which produces the model that defines a best fit line with the following equation: \[y = \text{ -0.36} + \text{0.42} \cdot x\]

Looking back at the earlier plot we can now make predictions about the type of iris species, and its width, just from taking observations of the petal lengths.

Note: In most cases the setosa species exists for widths < 1, the versicolor species for 1< widths < 1.5, and the virginica for widths > 1.5.

From the linear fit line we can see that the model could be beneficial for predicting the petal width, with some associated error.

Another useful piece of information is the distance of each observation from the expected output of the model, known as the residuals. This is a measure of how well your model would have predicted the output without knowledge of the real observations. This is easily computed in R with the residuals() command. The equation that determines the residuals are a simple difference:

\[Residualerror_i = observedValue_i - {predictedValue}_i\]

iris$residuals <- residuals(model) # save residuals to iris 

ggplot(iris, aes(x = Petal.Length, y = residuals)) +
  geom_point(aes(color = Species), size = 1.0) +
  geom_hline(yintercept = 0, linetype="dashed", color="black") + # center line
  labs(title = "Residuals of model in Length",
       x = "Petal Length",
       y = "Residuals") +
  theme_bw()

In this case for both Setosa and Versicolor the residual errors are bounded by +/- 0.3 units, while the virginica has a larger residual range. This is expected as the range of results from the observed widths was also wider.