2024-06-08

Overview of Linear Regression

Linear Regression is used to describe the linear relationship between an explanatory variable (x) and a response variable (y).

A line is fitted to the data, and using known values of the explanatory variable, we are able to predict values of the response variable.

source: (ncl.ac.uk)

Dataset iris

data(iris)
head(iris)
##   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## 1          5.1         3.5          1.4         0.2  setosa
## 2          4.9         3.0          1.4         0.2  setosa
## 3          4.7         3.2          1.3         0.2  setosa
## 4          4.6         3.1          1.5         0.2  setosa
## 5          5.0         3.6          1.4         0.2  setosa
## 6          5.4         3.9          1.7         0.4  setosa

Sepal Length vs. Petal Length

g = ggplot(data = iris, aes(x = Sepal.Length, y = Petal.Length)) +
    geom_point()
g

Correlation Coefficient, r

cor(iris$Petal.Length, iris$Sepal.Length)
## [1] 0.8717538

The correlation coefficient, r, is a measure of the strength of the linear relationship between two variables.

This r value of 0.8717538 suggests a strong positive linear relationship between sepal length and petal length. Therefore, it would be reasonable to fit a regression line through the data.

Fitted Line Through Data

## `geom_smooth()` using formula = 'y ~ x'

Plotly Graph of Data

## `geom_smooth()` using formula = 'y ~ x'

Linear Regression Equation

\(\begin{equation}\hat{Y}_i = \hat{\beta}_0 + \hat{\beta}_1 X_i + \hat{\epsilon}_i\end{equation}\)


\(\hat{Y}_i = Response\) \(Variable\)

\(\hat{\beta}_0 = Y-intercept\)

\(\hat{\beta}_1 = Slope\)

\(X_i = Explanatory\) \(Variable\)

\(\hat{\epsilon}_i = Error\)

Least Squares Regression

## 
## Call:
## lm(formula = iris$Petal.Length ~ iris$Sepal.Length)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -2.47747 -0.59072 -0.00668  0.60484  2.49512 
## 
## Coefficients:
##                   Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       -7.10144    0.50666  -14.02   <2e-16 ***
## iris$Sepal.Length  1.85843    0.08586   21.65   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.8678 on 148 degrees of freedom
## Multiple R-squared:   0.76,  Adjusted R-squared:  0.7583 
## F-statistic: 468.6 on 1 and 148 DF,  p-value: < 2.2e-16

Linear Regression Equation Interpretation

\(\begin{equation}\hat{Y}_i = -7.10144 + 1.85843X_i + \hat{\epsilon}_i\end{equation}\)

According to the regression equation, for every 1 unit increase in sepal length, we predict a 1.85843 unit increase in petal length.