We will use plots, a regression model, and a measure of prediction error to explore this question.
We simulate data for 120 students, recording three values for each student:
The data is simulated for this lesson. It does not prove that one study habit causes a higher score.
The plot shows the relationship between study time and exam score. When the interactive version is available, hover over a point to see that student’s practice-problem count.
A multiple linear regression uses both study hours and practice problems:
\[ \widehat{y}_i = \widehat{\beta}_0 + \widehat{\beta}_1 x_{i1} + \widehat{\beta}_2 x_{i2} \]
Here, \(\widehat{y}_i\) is the predicted exam score, \(x_{i1}\) is study hours, and \(x_{i2}\) is the number of practice problems.
The model chooses coefficients that minimize squared prediction errors:
\[ \min_{\beta_0,\beta_1,\beta_2} \sum_{i=1}^{n}(y_i - \beta_0 - \beta_1x_{i1} - \beta_2x_{i2})^2 \]
These are estimated relationships in our simulated data, not proof of cause and effect.
A residual is the difference between a student’s observed score and their predicted score:
\[e_i = y_i - \widehat{y}_i\]
The root mean squared error (RMSE) summarizes prediction errors in points:
\[ \mathrm{RMSE} = \sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_i-\widehat{y}_i)^2} \]
Training RMSE: 7.75 score points.
The error should also be tested on new data before treating the model as reliable.
This is the main R code used to fit the model and calculate its error:
model <- lm(exam_score ~ study_hours + practice_problems,
data = students)
predicted <- predict(model, newdata = students)
errors <- students$exam_score - predicted
rmse <- sqrt(mean(errors^2))
summary(model)
rmse
Statistics helps us understand both what a model predicts and where it may be wrong.