Simple Linear Regression

This practice will be focussing on simple linear regression model. The regression model is basically to answer four broad questions: What is the nature of association between x and y?; Is there an association between x and y?; What is the expected response of a new given predictor values? and what is the expected change of mean response variable for each unit increase or decrease (change) in predictor value?

In this practice, we will work with a datset called Animals from MASS package.

Simple linear regression model follows the mathametical equation:

\[ y_{i} = E(Y_{i}) +\epsilon_{i} = \beta_{0} + \beta_{1}{x_{i}} +\epsilon_{i} \ (1) \]

The equation (1) is formed as population regression line, but we often don’t know the whole population. Therefore, we have to reply on a sample of data from the population to estimate the population regression line

Now we will use a similar equation, but estimate sample regression line. The following equation supports that idea

\[ y_{i}^{'}= \alpha + \ b_{1}x_{1} +\epsilon_{1} \ (2) \] As we notice that we will use the sample intercept (2) to estimae the population intercept(1), and the slope (2) to estimate population slope (1)

Before fitting the simple linear regression, it is important to clarify that error terms are LINE conditions, and the model should satisfy all these conditions.If you want to understand more about LINE conditions, here is LINE condition

Plotting the relationship between brain and body

# Get to know the dataset
library(MASS)
head(Animals)

It is observed that there is week or no linear relationship between brain and body. But how about using log transformation for both variables

plot(brain~body,data=Animals,main="Brain vs Body",col=4,pch=16,log="xy")

Oh, It looks much better and there is a positively linear relationship between natural log of body and natural log of brain

Now fitting a simple linear regression

m1<-lm(log(brain)~log(body),data=Animals)
summary(m1)

Call:
lm(formula = log(brain) ~ log(body), data = Animals)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.2890 -0.6763  0.3316  0.8646  2.5835 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  2.55490    0.41314   6.184 1.53e-06 ***
log(body)    0.49599    0.07817   6.345 1.02e-06 ***
---
Signif. codes:  
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.532 on 26 degrees of freedom
Multiple R-squared:  0.6076,    Adjusted R-squared:  0.5925 
F-statistic: 40.26 on 1 and 26 DF,  p-value: 1.017e-06

We may be interested in calculating CI for model

confint(m1,level = 0.95)

                2.5 %    97.5 %
(Intercept) 1.7056829 3.4041133
log(body)   0.3353152 0.6566742

As the model coefficients reported p-value<0.05, there is a significant evidence that there is a linear association bt natural log of body and natural log of brain

The main objective of model is the prediction. Thus, we will make a prediction about brain when a given body value is 32.

new_point<-data.frame(body=c(32))
pre_value<-predict(m1,new_point)
# But this gave the value of natural log of brain, to get brain value we need to use formula as follow
2.7^pre_value

      1 
69.7588 

We can calculate CI of a given value

predict(m1,new_point,interval = "confidence")

       fit      lwr      upr
1 4.273885 3.676913 4.870857

Calculate a prediction interval of a given value

predict(m1,new_point,interval = "prediction")

       fit      lwr      upr
1 4.273885 1.069607 7.478162

• Conclusion:

The simple linear regression model is very useful in many cases.