2024-03-18

Simple Linear Regression

  • Linear Regression is used to create a prediction model
  • Simple regression includes one independent variable (x) and one dependent variable (y)
  • The dependent variable is the variable you are trying to predict
  • Equation form: \(y = \beta_0 + \beta_1(x)\)
  • \(\beta_0\) represents the y-intercept

Using R to make a linear model

  • In this presentation, we are going to use the trees data set to explain simple linear regression
  • The trees dataset has three variables: Girth, Height, and Volume
  • Using the trees dataset:
data("trees")
head(trees,5)
  Girth Height Volume
1   8.3     70   10.3
2   8.6     65   10.3
3   8.8     63   10.2
4  10.5     72   16.4
5  10.7     81   18.8

Regression problem

  • In order to create a regression model, we first need to determine which variable we are trying to predict or which variable we want as the dependent variable (y)
  • For this example we want to use Volume as the dependent variable which means that girth and height are the independent variables.
  • Thus the equation will have the following form:
    • \(Volume = \beta_0 + \beta_1(Girth)\)
    • \(Volume = \beta_0 + \beta_1(Height)\)

Code used for linear regression

model <- lm(Volume ~ Girth, trees)
summary(model)
model2 <- lm(Volume ~ Height, trees)
summary(model2)

Linear Model 1 Summary Output (x = Girth)

Call:
lm(formula = Volume ~ Girth, data = trees)

Residuals:
   Min     1Q Median     3Q    Max 
-8.065 -3.107  0.152  3.495  9.587 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -36.9435     3.3651  -10.98 7.62e-12 ***
Girth         5.0659     0.2474   20.48  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 4.252 on 29 degrees of freedom
Multiple R-squared:  0.9353,    Adjusted R-squared:  0.9331 
F-statistic: 419.4 on 1 and 29 DF,  p-value: < 2.2e-16

Interpreting the Results

  • When looking at a simple linear regression model, we look at the R-squared value to help us determine the significance of the model
  • Referring to the previous slide, the R-squared value stands at 0.9353, indicating a strong fit of the model to the data. This value is considered highly desirable, as we typically seek an R-squared exceeding 0.5 for a linear model.
  • This means that 93.53% of the variation in Volume is accounted for by the Girth variable.
  • The correlation coefficient is the square root of r-squared meaning the value for r is approximately 0.967 which is near one. This means that there is a strong positive correlation between the variables Girth and Volume.
  • Given that the p-values for the t-tests (Pr(>|t|)) are below \(\alpha\) (typically set at 0.05), both the intercept and the girth variable exhibit significance in predicting volume.
  • Using the estimate values from the previous slide, the equation for the model is :
    • \(Volume = -36.9435 + 5.0659(Girth)\)

Code for Scatter plot

ggplotly(ggplot(trees, aes(x=Girth, y=Volume)) +
  geom_point(size=2) +
  ggtitle("Girth vs Volume Scatter Plot"))

Result

Code for Adding Best-Fit Line

mod = lm(Volume ~ Girth, data=trees)
xax <- list(
  title = "Girth",
  titlefont = list(family="Modern Computer Roman")
)
yax <- list(
  title = "Volume",
  titlefont = list(family="Modern Computer Roman"),
  range = c(0,80)
)
x = trees$Girth; y = trees$Volume
plot_ly(x=trees$Girth, y=trees$Volume, type="scatter", mode="markers", name="data",
               width=690, height=300) %>%

  # plotting the linear regression model 
  add_lines(x = x, y = fitted(mod), name="fitted") %>%
  layout(xaxis=xax, yaxis=yax)

Plot

Linear Model 2 Summary output (x=Height)

Call:
lm(formula = Volume ~ Height, data = trees)

Residuals:
    Min      1Q  Median      3Q     Max 
-21.274  -9.894  -2.894  12.068  29.852 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -87.1236    29.2731  -2.976 0.005835 ** 
Height        1.5433     0.3839   4.021 0.000378 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 13.4 on 29 degrees of freedom
Multiple R-squared:  0.3579,    Adjusted R-squared:  0.3358 
F-statistic: 16.16 on 1 and 29 DF,  p-value: 0.0003784

Interpreting the Results

  • Using the same strategy as before, the r-squared value for this model is 0.3579 which is NOT ideal for a regression model.
  • This means that 35.79% if the variation in volume is accounted for by the height variable in the model.
  • The correlation coefficient is \(\sqrt{0.3579} = 0.598\) which means there is a moderate positive relationship between the variable Volume and Height.
  • The p-values for the t-test (Pr(>|t|)) are significant for both the intercept and the independent variable since their values are less than alpha (\(\alpha=0.05\) by default).
  • The equation for this model is:
    • \(Volume = -87.1236 + 1.5433(Height)\)

Code for Scatter plot

ggplotly(ggplot(trees, aes(x=Height, y=Volume)) +
  geom_point(size=2) +
  ggtitle("Height vs Volume Scatter Plot"))

Result

Code for Adding Best-fit line

ggplotly(ggplot(trees, aes(x=Height, y=Volume)) +
  geom_point(size=2) +
  geom_smooth(method="lm") +
  ggtitle("Height vs Volume"))

Best-fit Plot

`geom_smooth()` using formula = 'y ~ x'

Conclusion

  • Comparing the two models produced, it is clear that the best model to predict Volume is: -\(Volume = -36.9435 + 5.0659(Girth)\)
  • This is mainly due to the fact that the R-squared value for model 1 is significantly greater than model 2 (model1 (x=Girth) = 0.93, model 2 (x=Height)= 0.36)
  • The correlation coefficient for model 1 is 0.967 which is close to 1 suggesting a strong positive correlation between the variables and the correlation coefficient for model 2 is 0.598 which suggests a positive correlation between the two variables that is neither strong nor weak.