Today

  • From model comparison to model checks
  • Residuals: observed minus predicted
  • LINE-M examples: Linearity, Independence, Normality, Equality of variance
  • Multicollinearity and VIF
  • How to check and address LINE-M problems
  • Writing a model-checking paragraph
  • Formative report guide and submission checklist
  • Next week: binary logistic regression

From Choice to Checks

In Part 1, model comparison helped us choose between nested models.

Today we ask a different question:

Is the selected model trustworthy enough to interpret?

Model checks do not prove that a model is correct.

They help us spot problems that would make the model hard to trust.

What We Check

For a normal linear model, use LINE-M as a practical checklist:

  • Linearity: does the model capture the shape of the relationship?
  • Independence: are observations independent?
  • Normality: are residuals roughly normally distributed?
  • Equality of variance: is residual spread roughly stable?
  • Multicollinearity: do predictors overlap too strongly?

The first four are model assumptions.

The final one is a predictor-overlap check, usually assessed with VIF.

Residuals

\[ \text{residual} = \text{observed value} - \text{predicted value} \]

model_rt <- lm(rt ~ age + sex, data = blomkvist)

model_check_data <- mutate(blomkvist,
                           predicted = predict(model_rt),
                           residual = rt - predicted)
# A tibble: 265 × 7
      id   age sex    smoker    rt predicted residual
   <dbl> <dbl> <chr>  <chr>  <dbl>     <dbl>    <dbl>
 1     1    84 male   former  702.      762.    -60.8
 2     2    37 female no      471.      552.    -81.3
 3     3    62 female yes     639.      695.    -56.8
 4     4    85 female former  708       827.   -119. 
 5     5    73 male   former  607.      699.    -92.0
 6     6    65 male   no      542.      653.   -112. 
 7     7    30 female no      571.      512.     58.9
 8     8    49 female former  509.      621.   -112. 
 9     9    83 male   former  737.      757.    -20.0
10    10    58 male   yes     550.      613.    -63.5
# ℹ 255 more rows

Why Residuals Matter

A model makes predictions.

Residuals show where those predictions went wrong.

If the residuals look random, the errors are not systematically linked to the predictors or fitted values. That supports the model because the main structure in the data has been captured.

Well-behaved residuals are roughly centred around zero, have a similar spread across fitted values, and do not show strong skew, extreme outliers, or clear patterns.

If residuals show strong patterns, the model may be missing something important.

LINE-M Checks

For a normal linear model, remember LINE-M:

Letter Check What it means
L Linearity The model captures the shape of the relationship.
I Independence Observations and residuals are not dependent on each other.
N Normality Residuals are roughly normally distributed.
E Equality of variance Residual spread is roughly similar across fitted values and predictors.
M Multicollinearity Predictors do not overlap too strongly with each other.

These are not pass/fail boxes. The LINE checks are model assumptions; M is an additional predictor-overlap check. Together, they help decide whether the model is trustworthy enough to interpret.

L: Linearity

Linearity means the model captures the shape of the relationship.

For a continuous predictor, plot residuals against that predictor.

Interpretation

  • random cloud around zero: reassuring
  • curve or trend: the age effect may not be captured well

Possible response

Add a justified curved age term, interaction, transformation, or a different model.

I: Independence

Independence is mainly about study design: see sleepstudy data. Observations from the same participant are likely to be more similar to each other.

Ask:

  • does each row come from a different participant?
  • are there repeated measurements from same person?
  • are observations grouped by class, school, lab, item, or time point?

If observations are dependent, a simple lm() may be too simple.

Possible response

Use a model that represents the dependency, such as a multilevel, repeated-measures, or time-series model.

N: Normality

Normality means residuals are roughly normally distributed.

A histogram shows whether residuals are roughly centred on zero and whether the distribution is very asymmetric.

Interpretation

  • roughly symmetric around zero: reassuring
  • strong skew or isolated extreme values: possible Normality concern

Possible response

Check for data errors or outliers; consider a (log) transformation or a model better matched to the outcome.

N: Residual Skew

Skew describes asymmetry.

Values close to zero suggest little skew. Larger positive or negative values suggest stronger asymmetry.

moments::skewness(model_check_data$residual)
[1] 1.86

A skew value is not a pass/fail test.

It is one summary to use alongside the plot.

E: Equality of Variance

Residual spread should be roughly similar across fitted values and predictors.

Interpretation

  • similar spread across predicted values: reassuring
  • funnel shape or changing spread: possible Equality-of-variance problem
  • curve: possible Linearity problem that is also visible here

Possible response

Transform the outcome or model the variance directly.

M: Multicollinearity

Multicollinearity means predictors overlap strongly with each other. When a model has more than one predictor, those predictors can share information.

For example:

  • age may relate to reaction time
  • smoking status may relate to reaction time
  • age and smoking status might also be related to each other in the sample

If predictors overlap strongly, the model has a harder time estimating their separate effects.

How VIF Is Calculated

For each predictor, we ask: can the other predictors predict this predictor?

  1. Regress one predictor on the other predictors.
  2. Record the \(R^2\) from that model.
  3. Calculate:

\[ \text{VIF} = \frac{1}{1 - R^2} \]

If \(R^2 = 0\), then \(\text{VIF} = 1\): no overlap with the other predictors.

If \(R^2\) is high, VIF becomes large: strong overlap with the other predictors.

If VIF is a concern, centring or scaling predictors can help, especially for interactions or polynomial terms. If two variables measure nearly the same thing, consider removing one, combining them, or interpreting the separate coefficients cautiously.

Variance Inflation Factor

VIF stands for variance inflation factor.

VIF checks how much a predictor is explained by the other predictors.

car::vif(model_rt)
age sex 
  1   1

Rough guide:

  • close to 1: little concern
  • around 2 to 5: worth noticing
  • above 5 or 10: potential concern, depending on context

What VIF Does Not Mean

VIF does not tell us whether a predictor matters.

It tells us whether predictors are too entangled for the model to estimate their separate effects cleanly.

VIF is about predictor overlap, not about whether the predictor is important.

How To Check LINE-M

Check How to identify problems How to address problems
Linearity Curves or systematic patterns in residual plots. Add a theoretically justified curve, transformation, interaction, or different model.
Independence Repeated observations, clustered data, time order, or serial patterns. Use a design-aware model, such as multilevel, repeated-measures, or time-series terms.
Normality Strong skew, heavy tails, or clear outliers in residual distribution. Check data quality; consider transformation, robust methods, or a more suitable model family.
Equality of variance Funnel shape or changing residual spread. Consider transformation, robust standard errors, weighted models, or modelling variance directly.
Multicollinearity High VIF values or predictors measuring nearly the same thing. Centre or scale predictors where useful, especially for interactions; remove redundant predictors, combine measures, or interpret separate coefficients cautiously.

Writing Up Model Checks

A short model-checking paragraph can be enough.

Example structure:

I checked the selected model by inspecting the residuals. The histogram suggested that … . The residuals-versus-predicted plot suggested that … . The residuals-versus-age plot suggested that … . The VIF values suggested that … . Overall, these checks suggest that …

Keep the write-up descriptive and honest.

Exercise

From the NOW learning room, open part-2-model-checks.Rmd.

You will:

  1. fit one linear model
  2. calculate predicted values and residuals
  3. inspect the residual distribution
  4. calculate residual skew
  5. plot residuals against predicted values and age
  6. calculate and interpret VIF
  7. write a short model-checking paragraph

Formative Assessment Overview

The formative assessment is one short linear-regression report.

It is worth 0% of the final module grade, but it gives you feedback before the final assessment.

Deadline: Friday 23 October 2026, before 2 pm.

Your report should include:

  • a clear research question
  • a concise data and variables description
  • descriptives and visualisation
  • a fitted linear model
  • model predictions and interpretation
  • model comparison where useful
  • model checks and a brief conclusion

Using the Assessment Criteria

Use the grading matrix as a checklist while writing.

The four criteria are asking whether your report shows that you can:

Criterion What the marker is looking for
Theoretical basis You chose a suitable model for the question and data.
Practical data analysis You carried out a reproducible analysis in R.
Understanding results You understood what the model output means.
Reporting results You communicated the evidence clearly and accurately.

The next slides show how to make each criterion visible in the report.

Criterion: Theoretical Basis

Show why linear regression is appropriate for your research question.

Include:

  • a clear research question with an outcome and predictor or predictors
  • a short explanation that the outcome is continuous
  • a reason for including each predictor in the model
  • a sentence explaining what the model estimates
  • brief reference to model assumptions or checks

A useful sentence shape:

I used linear regression because the outcome variable was continuous and the aim was to estimate how it changed with … .

Criterion: Practical Data Analysis

Show that the analysis can be reproduced from the files you submit.

Include R code that:

  • imports the data from the submitted project folder
  • prepares variables transparently, for example factor coding or filtering
  • calculates descriptive statistics
  • creates at least one useful plot
  • fits the model or models
  • creates predictions or model comparisons where useful
  • runs model checks, including residual plots and VIF where relevant

The RMarkdown file should knit without hidden manual steps.

Criterion: Understanding Results

Show that you understand the model output, not just how to produce it.

Include:

  • interpretation of coefficients in the units of the outcome
  • interpretation of model predictions where these help explain the result
  • interpretation of uncertainty, for example confidence intervals or p-values
  • comparison of models where this helps answer the question
  • interpretation of model checks, including what any problems might mean

Avoid only saying whether a result is significant. Say what the result means.

Criterion: Reporting Results

Show the analysis as a short report that a psychology reader can follow.

Include:

  • a clear structure: question, data, descriptives, model, checks, conclusion
  • labelled tables and figures
  • plots that support the interpretation rather than decorate the report
  • concise statistical reporting in text
  • a conclusion that answers the research question
  • careful wording about limitations and causality

The report should be readable without needing to inspect the R code.

What To Submit

Submit one zip archive to the Dropbox before the deadline, containing:

  • exactly one RMarkdown report file
  • the rendered output, usually PDF
  • the RStudio project file
  • all relevant data files
  • bibliography, CSL, images, or other files needed to knit
  • the GenAI declaration form

Use only your student number in submitted documents.

The submission must be reproducible: we must be able to knit the report from your files.

Formative Report Guide

Use part-2-formative-report-guide.Rmd to work on your formative report.

Check that the report has

  • a research question
  • data and variables
  • descriptives and a plot
  • a fitted model
  • model comparison where useful
  • model results from tidy()
  • predictions and a prediction plot
  • model checks
  • a short conclusion

Final Checklist

Before formative submission, check that:

  • the report knits from a fresh R session
  • the zip contains exactly one .Rmd report
  • the zip contains the knitted output, usually PDF
  • the zip contains the .Rproj file
  • all data and supporting files are included
  • the GenAI declaration form is included
  • submitted documents use your student number, not your name
  • the report answers a clear research question
  • model choice, results, predictions, and checks are explained in context

Next Week

Next week, Thom will continue with binary logistic regression.

This moves from continuous outcomes to binary outcomes, for example:

  • correct or incorrect
  • yes or no
  • present or absent
  • chosen or not chosen

Recommended Reading

References

Baguley, T. (2012). Serious stats: A guide to advanced statistics for the behavioral sciences. Macmillan International Higher Education.

Faraway, J. J. (2015). Linear models with R (Vol. 2). CRC press.

Gelman, A., Hill, J., & Vehtari, A. (2020). Regression and other stories. Cambridge University Press.