Linear regression is a statistical method used to model the relationship between:

• a numeric outcome \(Y\), and
  
• one or more predictors \(X\)

Main uses:

• Explain the relationship between variables
  
• Predict \(Y\) from \(X\)

Does studying actually lead to higher exam scores?

We’ll focus on two variables:

• Hours Studied \(\rightarrow\) predictor \((X)\)
  
• Exam Score \(\rightarrow\) outcome \((Y)\)

A simple linear regression model shows the relationship between study time and exam score:

\[ Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i \]

• \(Y_i\): exam score \(i\)
  
• \(X_i\): hours studied \(i\)
  
• \(\beta_0\): predicted score when \(X=0\)
  
• \(\beta_1\): expected change in score for each +1 hour studied
  
• \(\varepsilon_i\): random variation / other factors

The fitted regression line predicts:

\[ \hat{y}_i = \hat{\beta}_0 + \hat{\beta}_1 x_i \]

A residual is the prediction error:

\[ e_i = y_i - \hat{y}_i \]

Least squares chooses \(\hat{\beta}_0\) and \(\hat{\beta}_1\) to minimize:

\[ \text{SSE}=\sum_{i=1}^n (y_i - \hat{y}_i)^2 \]

Fitting the regression line to our data:

model <- lm(score ~ hours, data = exam)
coef(model)

## (Intercept)       hours 
##   51.958052    1.885365

A good linear model should have random residuals around 0

• In the example, students who studied more generally scored higher
• The line on the graph shows the overall pattern
• The residual plot shows if the line is a good fit