Models the relationship between a dependent variable and one/more independent variables
Enables us to make predictions on how changes in the independent variable(s) affect the dependent variable
Simple Linear Regression Equation: \[ y = \beta_0 + \beta_1x + \varepsilon \]
- \(y\): Dependent Variable
- \(x\): Independent Variable
- \((\beta_0)\): Intercept (Constant)
- \((\beta_1)\): Coefficient (Slope)
- \((\varepsilon)\): Error Term (Implicit - Not used in prediction)
Multiple Linear Regression Equation: \[ y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \ldots + \beta_nx_n + \varepsilon \]
\(x_1, x_2, \ldots, x_n\): Independent Variables
\((\beta_0, \beta_1, \ldots, \beta_n)\): Coefficients
\((\varepsilon)\): Error Term (Implicit - Not used in prediction)
-It represents the variability/noise between the observed values and the values predicted by linear regression
Example 1
- Let’s use linear regression to predict the relationship between study hours and exam scores
- Using the model summary below, the linear regression equation is: \[Score = 37.0667 + 5.6970*(Hours)\]
hours <- c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10) exam_scores <- c(43, 49, 54, 60, 64, 72, 75, 85, 88, 94) dataset <- data.frame(hours, exam_scores) model <- lm(exam_scores ~ hours, data = dataset) summary(model)
## ## Call: ## lm(formula = exam_scores ~ hours, data = dataset) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.94545 -0.29394 0.05455 0.46364 2.35758 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 37.0667 0.8649 42.86 9.69e-11 *** ## hours 5.6970 0.1394 40.87 1.41e-10 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 1.266 on 8 degrees of freedom ## Multiple R-squared: 0.9952, Adjusted R-squared: 0.9946 ## F-statistic: 1670 on 1 and 8 DF, p-value: 1.414e-10
Example (Plotted)
ggplot(dataset, aes(x = hours, y = exam_scores)) +
geom_point() +
geom_smooth(method = "lm", se = FALSE) +
labs(title = "Linear Regression",
x = "Hours", y = "Exam Scores")
`geom_smooth()` using formula = 'y ~ x'
Example 2
- This examples also uses simple linear regression to predict a dependent variable (mpg) based on the independent variable of weight (wt)
- Using the model summary below, the linear regression equation is: \[MPG = 37.22727 - 3.87783*WT\]
data(mtcars) model <- lm(mpg ~ wt, data = mtcars) summary(model)
## ## Call: ## lm(formula = mpg ~ wt, data = mtcars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -4.5432 -2.3647 -0.1252 1.4096 6.8727 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 37.2851 1.8776 19.858 < 2e-16 *** ## wt -5.3445 0.5591 -9.559 1.29e-10 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 3.046 on 30 degrees of freedom ## Multiple R-squared: 0.7528, Adjusted R-squared: 0.7446 ## F-statistic: 91.38 on 1 and 30 DF, p-value: 1.294e-10
Example 2 (Plotted)
- The graph adds an extra dimension of horsepower (hp) to show how higher horsepower cars are associated with lower MPG and higher weight
## `geom_smooth()` using formula = 'y ~ x'
Example 3
- This example uses multiple linear regression to predict home prices based on the number of bedrooms and square footage (2 independent variables).
- Using the model summary below the multiple linear regression equation is: \[Price = 6666.7 + 24703.7*(bedrooms) + 95.56*(sqft)\]
houses <- data.frame( bedrooms = c(2, 3, 4, 2, 3, 4, 3, 4, 5), size_sqft = c(1500, 1800, 2000, 1200, 1600, 1900, 1700, 2000, 2300), price = c(200000, 250000, 300000, 180000, 220000, 280000, 240000, 310000, 350000) ) model <- lm(price ~ bedrooms + size_sqft, data = houses) summary(model)
## ## Call: ## lm(formula = price ~ bedrooms + size_sqft, data = houses) ## ## Residuals: ## Min 1Q Median 3Q Max ## -13667 -3222 37 3407 13407 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 6666.67 28244.96 0.236 0.8213 ## bedrooms 24703.70 11110.45 2.223 0.0679 . ## size_sqft 95.56 34.38 2.780 0.0320 * ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 9415 on 6 degrees of freedom ## Multiple R-squared: 0.9785, Adjusted R-squared: 0.9713 ## F-statistic: 136.3 on 2 and 6 DF, p-value: 9.997e-06
Example 3 (Plotted)
- We can plot the multiple linear regression equation using a 3d plot
## A line object has been specified, but lines is not in the mode ## Adding lines to the mode...
\(R^2\) in Linear Regression
- In all 3 examples, the summary also mentions a \(R^2\) value which quantifies the model fit
- It is the proportion of variance in the dependent variable that can be explained by the independent variable
- R-squared ranges from 0 to 1, where 0 indicates that the model does not explain any variability in the dependent variable, and 1 indicates that the model explains all of the variability.
- High \(R^2\) indicates that the regression model is a good fit
- Example 1: \(R^2\) -> 0.9952; The independent variable explains 99.52% of the variability in the dependent variable, showing strong predictive capability
Conclusion: Linear Regression
- Linear regression offers a powerful framework for modeling relationships between variables.
- Through metrics like \(R^2\), it provides insights into the goodness-of-fit and predictive performance of the model.
- It is widely applicable across diverse fields, linear regression serves as an important tool for data analysis and decision-making.