Simple Linear Regression is a statistical technique that models the relationship between an independent and dependent variable (x and y).

We use it to predict values and understand how one variable affects another.

In simple linear regression, we assume a linear relationship between the two variables :

[y = _0 + _1 x + ]

where : - \(y\): dependent variable - \(x\): independent variable - \(\beta_0\): intercept - \(\beta_1\): slope - \(\epsilon\): random error

Estimating the Model

The best-fitting line minimizes the sum of squared residuals:

\[\text{Minimize| \sum_{i=1}^{n} (y_i - \hat{y_i})^2\]

The slope and intercept estimates are given by :

\[ \hat{\beta_1} = \frac{\text{Cov}(X,Y)}{\text{Var}(X)}, \quad \hat{\beta_0} = \bar{Y} - \hat{\beta_1}\bar{X} \]

## 
## Call:
## lm(formula = y ~ x, data = data)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -5.964 -1.808 -0.113  1.558  5.201 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   3.9303     1.3860   2.836    0.011 *  
## x             1.9519     0.1157  16.870 1.78e-12 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.984 on 18 degrees of freedom
## Multiple R-squared:  0.9405, Adjusted R-squared:  0.9372 
## F-statistic: 284.6 on 1 and 18 DF,  p-value: 1.778e-12

Simple Linear Regression provides a foundation for understanding relationships and making predictions. It’s the building block for multiple regression, logistic regression, and many machine learning models.