2024-3-12

Introduction to Simple Linear Regression

Simple Linear Regression is a method to model the relationship between a single independent variable and a dependent variable by fitting a linear equation to observed data.

Applications of Linear Regression

Linear regression can be applied in numerous fields such as economics for predicting GDP, in health for estimating blood pressure levels based on weight, or in engineering for stress analysis.

Assumptions of Linear Regression

The main assumptions behind linear regression include: 1. Linearity 2. Independence 3. Homoscedasticity 4. Normality of residuals

R Code for Linear Model

# Example dataset
data(mtcars)
# Fitting the linear model
model <- lm(mpg ~ wt, data = mtcars)
summary(model)
## 
## Call:
## lm(formula = mpg ~ wt, data = mtcars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
## wt           -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528, Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

Plotly Graph for Linear Regression

library(plotly)
## Loading required package: ggplot2
## 
## Attaching package: 'plotly'
## The following object is masked from 'package:ggplot2':
## 
##     last_plot
## The following object is masked from 'package:stats':
## 
##     filter
## The following object is masked from 'package:graphics':
## 
##     layout
p <- plot_ly(data = mtcars, x = ~wt, y = ~mpg, type = 'scatter', mode = 'markers') %>%
  add_lines(x = ~wt, y = fitted(model))
p

GGplot Graph for Linear Regression

library(ggplot2)
# Scatter plot with regression line
ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point() +
  geom_smooth(method = "lm", se = FALSE)
## `geom_smooth()` using formula = 'y ~ x'

# Residuals plot
ggplot(model, aes(x = .fitted, y = .resid)) +
  geom_point() +
  geom_hline(yintercept = 0, linetype = "dashed")

Mathematical Representation

The equation of a line is given by:

\(y = \beta_0 + \beta_1x + \epsilon\)

Where:

\(y\) is the dependent variable, \(x\) is the independent variable, \(\beta_0\) is the intercept, \(\beta_1\) is the slope of the line, \(\epsilon\) is the error term.

Estimating the Coefficients

The coefficients \(\beta_0\) and \(\beta_1\) are estimated using the method of least squares. The formulas for \(\beta_1\) and \(\beta_0\) are:

\[ \beta_1 = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2} \]

\[ \beta_0 = \bar{y} - \beta_1\bar{x} \]

where \(\bar{x}\) and \(\bar{y}\) are the mean values of \(x\) and \(y\) respectively, and \(n\) is the number of observations.

Assumptions of the Model

The simple linear regression model relies on several key assumptions:

  1. Linearity: The relationship between \(x\) and \(y\) is linear.
  2. Independence: The residuals (\(\epsilon_i\)) are independent.
  3. Homoscedasticity: The variance of the residuals (\(\epsilon_i\)) is constant across all levels of \(x\).
  4. Normality: The residuals (\(\epsilon_i\)) are normally distributed.

Prediction

Once the model is fitted, it can be used for prediction. For a given value of \(x\), \(x_p\), the predicted value of \(y\), \(y_p\), is given by:

\[ y_p = \beta_0 + \beta_1x_p \]