Simple Linear Regression is the statistical method that allows us to study the relationship between two variables. One variable is labeled the independent or explanatory variable and the other is the labeled the dependent or response variable. The relationship between the variables is modeled and we can understand this relationship better through studying the line of best fit.

• Purpose: Predict the dependent variable from the independent variable.
• Use: Predict sales, market analysis, predict house prices, etc…

Formula: \[ y = \beta_0 + \beta_1x + \epsilon \]

Where:

• y is the dependent variable
• x is the independent variable
• \(\beta_0\) is the y-intercept of the regression line
• \(\beta_1\) is the slope of the regression line
• \(\epsilon\) is the error term, which is the difference between the observed and predicted values

In order to obtain valid results and perform linear regression properly, these assumptions must be checked.

• Linearity: The relationship between the variables must be linear.
• Homoscedasticity: The variance of residual is the same for all values of x.
• Normality: For all values of x, y is normally distributed.
• Independence: All observations are independent of each other.

I will be using the “mtcars” data set as an example of simple linear regression. The data set contains a data frame with 32 observations on 11 different variables. For this example, we will look at the relationship between the car’s gross horsepower and the car’s 1/4 mile time.

In this data set: - “hp”: Gross horsepower (Independent Variable) - “qsec”: 1/4 mile time (seconds) (Dependent Variable)

data(mtcars)
head(mtcars[, c("hp", "qsec")])

                   hp  qsec
Mazda RX4         110 16.46
Mazda RX4 Wag     110 17.02
Datsun 710         93 18.61
Hornet 4 Drive    110 19.44
Hornet Sportabout 175 17.02
Valiant           105 20.22

Below is a scatter plot of the data set. The x axis shows gross horsepower and the y axis shows 1/4 mile time. This plot was made by using ‘plotly’:

Below is the same plot, but it is interactive!

Below is the R code for performing a simple linear regression on the data set to measure the relationship between a car’s horsepower and its 1/4 mile time.

results = lm(qsec ~ hp, data=mtcars)

Call:
lm(formula = qsec ~ hp, data = mtcars)

Residuals:
    Min      1Q  Median      3Q     Max 
-2.1766 -0.6975  0.0348  0.6520  4.0972 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 20.556354   0.542424  37.897  < 2e-16 ***
hp          -0.018458   0.003359  -5.495 5.77e-06 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.282 on 30 degrees of freedom
Multiple R-squared:  0.5016,    Adjusted R-squared:  0.485 
F-statistic: 30.19 on 1 and 30 DF,  p-value: 5.766e-06

Now that we have performed the analysis and fit the regression model, we can interpret the results:

• The coefficient for hp tells us how much change in qsec is expected for each unit change in hp.
• The \(R^2\) value says what proportion of the variance in qsec is explained by hp.
• The p-value tests a hypothesis that claims that the coefficient is different from 0.

What we see:

• The coefficient is negative, which indicates that as horsepower increases, the 1/4 mile time decreases.
• The \(R^2\) value is .485, which indicates a medium correlation between the variables.
• The P-value is very small, which indicates that the relationship is statistically significant.

These plots show that the four key assumptions are confirmed:

par(mfrow=c(2,2), mar=c(2, 2, 2, 2))
plot(results)

• The relationship between a car’s gross horsepower and 1/4 mile time is a statistically significant, negative relationship.
• The \(R^2\) value is .485, which means that the horsepower explains about 48.5% of the variance in 1/4 mile time.
• The coefficient of gross horsepower shows that for each unit of horsepower, the 1/4 mile time decreases by .018 seconds.

These results show that a higher horsepower leads to faster 1/4 mile times.