Simple linear regression is a statistical method to model the relation between two variables.

There is an inherent assumption that there is a linear relationship between the X value (independent) and the Y value (dependent).

The simple linear regression model can be represented as:

\[ Y = \beta_0 + \beta_1X + \varepsilon \]

where, \(Y\) is the dependent variable,

\(X\) is the independent variable,

\(\beta_0\) is the intercept,

\(\beta_1\) is the slope coefficient, and

\(\varepsilon\) is the residual.

1. The relationship between the independent and dependent variables is linear.
2. The residuals are independent of each other.
3. The variance of the residuals is constant across all levels of the independent variable.
4. The residuals are normally distributed.

The data set I will be using to demonstrate simple linear regression is ‘kid.weights’ from the ‘UsingR’ library. The dataset contains 4 columns, Age, Weight, Height and Gender. For this example, we will example in the relationship between age and weight in Males. Below is the data we will study.

##   age weight height
## 1  58     38     38
## 2 103     87     43
## 3  87     50     48
## 4 138     98     61
## 7  28     24     29
## 9 107    144     59

We now create a simple linear regression model using age as the independent variable, and height as the dependent variable. Given below is the summary of the model created.

## 
## Call:
## lm(formula = height ~ age, data = filteredData)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -14.8472  -2.3931   0.6208   3.1043   8.1060 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 24.400013   0.627414   38.89   <2e-16 ***
## age          0.249567   0.009593   26.02   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 4.366 on 119 degrees of freedom
## Multiple R-squared:  0.8505, Adjusted R-squared:  0.8492 
## F-statistic: 676.8 on 1 and 119 DF,  p-value: < 2.2e-16

As seen in the scatter plot, most residuals are closer the x axis suggesting that variances are constant. This suggests that the linear regression model is well fitted to the data.

residuals = resid(model)
filteredData$residuals = residuals
plot_ly(filteredData, x = ~height, y = ~residuals, type = 'scatter', 
    mode = "markers") %>% layout(title = 'Residuals vs Height', 
    xaxis = list(title = 'Height (in inches)'), yaxis = list(title = 'Residuals'))

The plot of the residuals through a histogram suggests a normal distribution. This also suggests that the model is well fitted to the data.

The model can be written as: \[ y_i = 24.400013 + 0.249567X \] Here is how to calculate a residual: \[ e_i = y_i - \hat{y_i} \] Let us take an example from the data set to see how accurate the model is: let X = 103, a data point that is in the data set, using the model we get \[ y_i = 24.400013 + 0.249567 * 103 = 24.400013 + 25.705401 = 50.1054525 \]`.
Residual: \[ e_i = 43 - 50.1054525 = -7.1054525 \]

testData = data.frame(age = c(100, 200, 300))
predictHeight = predict(model, newdata = testData)

Let us now predict values using the model, for x = 100, 200, 300. Using the model we get:

##        1        2        3 
## 49.35675 74.31349 99.27023