2024-03-22

Simple Linear Regression

Linear regression is a statistical technique that attempts to analyze the potential relationship between two variables. This is accomplished by fitting a line that estimates the correlation between your variables and attempts to predict the value of the dependent variable based on the value of your independent variable. For more the two variables multiple linear regression could be used.

A typical SLR model equation is written as: \(y=\beta_0 + \beta_1 x +\varepsilon\) where the \(y\) is the dependent variable \(x\) is the independent variable, \(\beta_0\) is the y-intercept, \(\beta_1\) is the slope coeffcient, and \(\varepsilon\) is the random error.

\(\beta_1\) and \(\beta_0\) are determined by: \[\frac{\beta_1=\Sigma^{m}_{i=1}(X_i-\bar{X})(Y_i-\bar{Y})}{\Sigma^{n}_{i=1}(X_i-\bar{X})^2}\] \[\beta_0= \bar{Y}-\hat{\beta_1}*X\]

Heart Rate Versus Temperature Example

To further discover what SLR can show us, lets examine the normtemp dataset.

##   temperature gender hr
## 1        96.3   male 70
## 2        96.7   male 71
## 3        96.9   male 74
## 4        97.0   male 80
## 5        97.1   male 73
## 6        97.1   male 75

This dataset contains the body temperature and heart rate of 130 individuals. This data was initially complied to investigate the generally accepted normal temperature of 98.6 degrees.

Plotly Scatterplot

As seen on the scatterplot, there does appear to be some relationship between these variables in the dataset. While the data is clustered a bit, the correlation seems to be positive, although it is a bit weak.

ggplot separated

ggplot(normtemp, aes(temperature, hr)) +
  geom_jitter(width = 0.1, height = 0.1) + facet_wrap(~gender) + theme_bw() + theme(plot.title = element_text(hjust = 0.5)) + labs(title = "Scatterplot Separated by Gender", x= "Temperature", y= "Heart Rate") + geom_smooth(method="lm", level=0.99, se = FALSE)
## `geom_smooth()` using formula = 'y ~ x'

Separated Plot

With the data filtered out, it will be easier to see the differences between the female and male participants. While both groups have some positive correlation, females have a much more positive correlation than when compared to males. However, we may be able to gain more insight on this relationship if we look at the raw residual data.

Residual Plot

ggplot(mod, aes(x = .fitted, y = .resid)) + geom_point() + geom_hline(yintercept = 0) + labs(title = "Residual vs. Fitted Values GGPlot", x="Fitted Values", y="Residual Values")

Residual Data

Using the summary function, we can take a look at the SLR residual data. RD refers to the distance of the data to the fitted linear regression line.

summary(mod)
## 
## Call:
## lm(formula = hr ~ temperature, data = normtemp)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -16.6413  -4.6356   0.3247   4.8304  15.8474 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)   
## (Intercept) -166.2847    80.9123  -2.055  0.04190 * 
## temperature    2.4432     0.8235   2.967  0.00359 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 6.858 on 128 degrees of freedom
## Multiple R-squared:  0.06434,    Adjusted R-squared:  0.05703 
## F-statistic: 8.802 on 1 and 128 DF,  p-value: 0.003591

Residual Analysis

-The residuals are symmetrically distributed around the line.

-We have significant p-values and the standard error is rather high.

-From this discovery of the high error, we can determine that this sample of data may not closely represent the popular.

-However, coupled with the fact that the p-value is very low, the SLR data seems to indicate a rejection of the null hypothesis.

-Meaning that not only is there a relationship between these variables but it is also statistically significant.

Residual Analysis Continued

cor(normtemp$temperature, normtemp$hr)
## [1] 0.2536564

When examined further through the correlation function, we can see that there is some substantial correlation of 0.25 between temperature and heart rate. However, it is not an overwhelming high correlation, it is still statistically significant.

This coincides with the generally accepted belief that temperature does have some affect on one’s heart rate, although it can vary from person to person.

Conclusion

It is important to remember that while simple linear regression can provide great insight when investigating relationships, it is merely an estimate.

There could be many extraneous variables affecting the dependent variable at any given moment. Or perhaps that amount of effect an independent variable has can flucuate.

When used in tandem with other statistical techniques: \(y=\beta_0 + \beta_1 x +\varepsilon\),or how its otherwise written (where \(\alpha\) is a constant): \(y=\alpha + \beta x\), simple linear regression can provide invaluable insights.