Linear Regression

Linear regression is a statistical method used to model the relationship between a dependent variable (or response variable) and one or more independent variables (or predictor variables). The primary purpose of linear regression is to understand and quantify the relationship between these variables, and to make predictions based on this relationship.

Sum of Squares

\(Q(b_{0},b_{1})=\sum_{i=1}^{n}e_{i}^{2}=\sum_{i=1}^{n}(y_{i}-(b_{0}+b_{1}x_{i}))^{2}\)

In linear regression, the goal is to find the line of best fit that minimizes the differences between the observed values (actual data points) and the values predicted by the regression model. These differences are often referred to as residuals or errors. The sum of squared errors (SSE) is the sum of the squares of these residuals across all data points.

Minimize the Error

To minimize \(Q(b_{0},b_{1})\) with respect to \(b_{0}\) and \(b_{1}\), we take the partial derivatives of \(Q\) with respect to each coefficient and set them equal to zero.

\(\frac{\partial Q}{\partial b_{0}} = 0\)

\(\frac{\partial Q}{\partial b_{1}} = 0\)

Once we find the values of \(b_{0}\) and \(b_{1}\) that minimize \(Q\), we have the coefficients of the linear regression model that best fits the data.

World Solar Prices

Here the negative sloping line of best fit signifies a downward trend in solar PV prices between 1975 and 2022.

Stress Level and Quality of Sleep

The negative slope of the linear regression signifies a negative correlation between the variables, meaning that higher stress levels tend to lead to lower quality of sleep.

Daily Steps and Heart Rate

While it appears there should be a relatively strong negative correlation here, the presence of outliers reduces the accuracy of the line of best fit by increasing the error.

Temperature and Ozone

There appears to be a strong positive correlation between temperature and ozone.

R Code

g <- ggplot(airquality, aes(x=Temp, y=Ozone)) + geom_point() + geom_smooth(method="lm")
ggplotly(g)