2024-03-19

Simple Linear Regression

Definition: Linear regression is a data analysis technique that predicts the value of unknown data by using another related and known data value. It mathematically models the unknown or dependent variable and the known or independent variable as a linear equation. (aws.amazon.com)

Random Data Set to Show Simple Linear Regression Model

  • (1, 4, 4, 5, 6, 6, 6, 8, 12, 14, 14, 15, 18, 19, 22, 23, 25, 26)
  • (1, 2, 3, 3, 4, 4, 5, 6, 6, 6, 7, 8, 8, 9, 10, 11, 11, 12)
df <- data.frame(set1=c(1, 4, 4, 5, 6, 6, 6, 8, 12, 14, 14, 15, 18, 
                        19, 22, 23, 25, 26),
                 set2=c(1, 2, 3, 3, 4, 4, 5, 6, 6, 6, 7, 8, 8, 9, 10, 
                        11, 11, 12))

Viewing Model Summary

modelSum <- lm(set1~set2, data = df)
summary(modelSum)
## 
## Call:
## lm(formula = set1 ~ set2, data = df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.6024 -0.7538  0.1220  1.2327  2.3976 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  -2.7651     0.8826  -3.133  0.00642 ** 
## set2          2.3946     0.1227  19.523 1.39e-12 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.666 on 16 degrees of freedom
## Multiple R-squared:  0.9597, Adjusted R-squared:  0.9572 
## F-statistic: 381.2 on 1 and 16 DF,  p-value: 1.387e-12

ggplot Linear Regression Model

ggplot(df, aes(set1, set2)) + geom_point() + stat_smooth(method = "lm", col="blue")

ggplot Linear Regression Model Alternative x and y swapped

ggplot(df, aes(set2, set1)) + geom_point() + stat_smooth(method = "lm", col="blue")

Simple Formula for Simple Linear Regression

Linear Regression Line Equation (ncl.ac.uk):

\(\overset{ʌ}{Y} = a + bx\)

\(a = y - bx\)

\(b = \frac{\sum (x_i - x)(y_i - y)}{\sum (x_i - x)^2} = \frac{\sum (xy) - \frac{\sum x \sum y}{n}}{\sum(x^2) - \frac{(\sum x)^2}{n}}\)

\(x = \frac{\sum x}{n}\)

\(y = \frac{\sum y}{n}\)

Equation For Data Set

(Set 2 is X and Set 1 is Y)

Mean of Independent Variable

\(x = \frac{\sum x}{n}\)

\(=\frac{1 + 2 + 3 + 3 + 4 + 4 + 5 + 6 + 6 + 6 + 7 + 8 + 8 + 9 + 10 + 11 + 11 + 12}{18}\)

\(= \frac{116}{18}\)

\(= 6.44444444444\)

Equation For Data Set (Continued…)

Mean of Dependent Variable

\(y = \frac{\sum y}{n}\)

\(=\frac{1 + 4 + 4 + 5 + 6 + 6 + 6 + 8 + 12 + 14 + 14 + 15 + 18 + 19 + 22 + 23 + 25 + 26}{18}\)

\(= \frac{228}{18}\)

\(= 12.6666666667\)

Equation For Data Set (Continued…) Calculating b

\(\sum (x_i - x)(y_i - y) = 441.6666667\)

\(\sum (x_i - x)^2 = 184.4444444\)

\(\frac{441.6666667}{184.4444444}\)

\(= 2.394578314 (3.d.p)\)

Equation For Data Set (Continued…) Calculating a

\(a = y - bx\)

\(= 12.6666666667 - 2.394578314 x 6.44444444444\)

\(= -2.765060246 (3.d.p.)\)

Equation For Data Set FINAL EQUATION

\(\overset{ʌ}{Y} = a + bx -> -2.765060246 + 2.394578314x\)

\(= 2.395x - 2.765\)

Plotly Linear Regression Model

Code for plotly graph:

lmModel <- linear_reg() %>%
  set_engine('lm') %>%
  set_mode('regression') %>%
  fit(set1 ~ set2, data = df)

xR <- seq(min(df$set2), max(df$set2), length.out = 100)
xR <- matrix(xR, nrow=100, ncol=1)
xdf <- data.frame(xR)
colnames(xdf) <- c('set2')

ydf <- lmModel %>% predict(xdf)

colnames(ydf) <- c('set1')
newDF <- data.frame(xdf, ydf)

plotlyLR <- plot_ly(df, x = ~set2, y = ~set1, type = 'scatter', mode = 'markers', 
                    name = 'Data')
plotlyLR <- plotlyLR %>% add_trace(data = newDF, x = ~set2, y = ~set1, name = 
                                     'Regression Line', mode = 'lines', alpha = 1)

Plotly Linear Regression Model

plotlyLR