• Simple linear regression is a statistical method used to model the linear relationship between a dependent variable (y) and a single independent variable (x).

• The relationship is expressed in the form of an equation:

  \[ y = \beta_0 + \beta_1x + \epsilon \]

  • \(\beta_0\) is the y intercept
  • \(\beta_1\) is the slope
  • \(\epsilon\) is the error term

• Linearity: the relationship between x and y is linear.

• Independence: The observations are independent of each other.

• Homoscedasticity: The variants of the residuals is constant.

• Normality: The residuals are normally distributed.

The objective is to find the best-fitting line. This is achieved by minimizing the sum of the squared residuals.

\[ e = y - (\beta_0 + \beta_1 x) = y - \hat{y} \]

The objective function \( Q \) to minimize is:

\[ Q = \sum e^2 = \sum (y - (\beta_0 + \beta_1 x))^2 \]

We will use the Boston housing dataset from the `MASS` package. We’ll predict the median value of owner-occupied homes (medv) based on the average number of rooms (rm).

## Loading required package: ggplot2

## 
## Attaching package: 'plotly'

## The following object is masked from 'package:ggplot2':
## 
##     last_plot

## The following object is masked from 'package:stats':
## 
##     filter

## The following object is masked from 'package:graphics':
## 
##     layout

## 
## Attaching package: 'MASS'

## The following object is masked from 'package:plotly':
## 
##     select

## 
## Call:
## lm(formula = medv ~ rm, data = Boston)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -23.346  -2.547   0.090   2.986  39.433 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  -34.671      2.650  -13.08   <2e-16 ***
## rm             9.102      0.419   21.72   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 6.616 on 504 degrees of freedom
## Multiple R-squared:  0.4835, Adjusted R-squared:  0.4825 
## F-statistic: 471.8 on 1 and 504 DF,  p-value: < 2.2e-16

p1 <- ggplot(Boston, aes(x=rm, y=medv)) +
  geom_point() +
  geom_smooth(method="lm", se=FALSE) +
  labs(title="Housing Prices vs. Number of Rooms",
       x="Average Number of Rooms",
       y="Median Value of Homes (in $1000s)") +
  annotate("text", x=6, y=40, label=paste("Y =",
      round(coef(model)[1], 2), "+", round(coef(model)[2], 2), "X"), size=4, color="blue")

## `geom_smooth()` using formula = 'y ~ x'

Comment: There’s a positive correlation between the number of rooms to the median value of homes, indicating that the number of rooms is a good predictor for the price of the home. The linear equations is:

\(y = -34.67 + 9.1x\)

Comment: The residuals are randomly scattered around 0, suggesting that the model is well fitted for the data

Simple Linear Regression is a powerful tool for predicting outcomes and identifying relationships between variables. In this example for housing prices in Boston, we saw how the number of rooms in a house can be used to predict its median value using linear regression.