Simple Linear Regression

2025-10-18

Introduction

Simple Linear Regression is a model that shows a relationship between two variables.

Examples:

Predicting the house prices based on its area.
Estimating salary based on the years of experience.
Predicting the yield of crop based on the rainfall.

Variable’s Roles

Dependent Variable:

This is the variable whose value we want to predict or estimate.
Its value depends on some other variable.
It is also known as the response and denoted by Y.

Independent Variable:

This is the variable that helps to predict or estimates the other variable.
Its values is independent.
It is also known as the predictor and denoted by X.

Simple Linear Regression Equation:

\[Y = \beta_0 + \beta_1 X + \epsilon\] Where:

Y is the dependent variable.
X is the independent variable.
\(\beta_0\)is the constant or intercept.
\(\beta_1\) is the slope.
\(\epsilon\) is the error term.

Least Square Method

The least Square Method is used to estimate the parameters \(\beta_0\) and \(\beta_1\) by minimizing the sum of the squares of the vertical deviation. Estimated Parameters:

\[\hat{\beta}_1 = \frac{\sum_{i=1}^{n}(X_i - \bar{X})(Y_i - \bar{Y})}{\sum_{i=1}^{n}(X_i - \bar{X})^2}\]

\[\hat{\beta}_0 = \bar{Y} - \hat{\beta}_1\bar{X}\]

Simple Linear Regression Example:

We will be looking at the cars data for this. We will try to predict Distance using the Speed of car.

Predictor (X): Speed of car
Response (Y): Distance traveled

The following is the summary and first few rows of the data we are looking at:

     speed           dist       
 Min.   : 4.0   Min.   :  2.00  
 1st Qu.:12.0   1st Qu.: 26.00  
 Median :15.0   Median : 36.00  
 Mean   :15.4   Mean   : 42.98  
 3rd Qu.:19.0   3rd Qu.: 56.00  
 Max.   :25.0   Max.   :120.00

  speed dist
1     4    2
2     4   10
3     7    4
4     7   22
5     8   16

Scatterplot for the Data

Simple Linear Regression Equation for cars:

lm(dist ~ speed, data = cars)

Call:
lm(formula = dist ~ speed, data = cars)

Coefficients:
(Intercept)        speed  
    -17.579        3.932

From this we can see that the equation we get is:

\[ dist = -17.579 + (3.932)*speed \]

Plot for Simple Linear Regression:

This creates a scatterplot with a fitted linear regression line.

ggplot(cars, 
       aes(x=speed, y=dist)) + 
  geom_smooth(method = 'lm') + 
  geom_point()  +
  labs(title = "Simple Linear Regression of Dist & Speed",
       x = "Speed", y = "Distance")

Introduction

Examples:

Variable’s Roles

Dependent Variable:

Independent Variable:

Simple Linear Regression Equation:

Least Square Method

Simple Linear Regression Example:

Scatterplot for the Data

Simple Linear Regression Equation for cars:

Plot for Simple Linear Regression:

Plot for Simple Linear Regression:

Visualizing Residuals

3D view of the Simple Linear Regression:

Summary