Today

  • What a normal linear model estimates
  • Greek symbols and model notation
  • Writing model formulas in RMarkdown
  • Formula syntax in lm()
  • Intercepts and slopes
  • Continuous and categorical predictors
  • tidy() for coefficient tables
  • Estimate, uncertainty, and evidence
  • Short model interpretation

Linear Models

The research question is usually in words:

How many hours do students sleep on a typical night?

A statistical model translates that question into an assumption about the data.

For sleep duration, we might assume:

  • each student’s sleep duration is one observation
  • sleep durations vary naturally between students
  • the observations come from a normal distribution
  • that distribution has an average \(\mu\) and spread \(\sigma\)

The equation is the link between the question and that assumed statistical process.

Start With an Average

The research question is usually in words:

How many hours do students sleep on a typical night?

We describe the assumed process as:

\[ \text{sleep}_i \sim \mathcal{N}(\mu, \sigma) \]

  • sleep duration is assumed to be normally distributed
  • \(\mu\): population mean sleep duration
  • \(\sigma\): population standard deviation

We cannot ask every student. So we use a sample to estimate those population values:

  • \(\hat{\mu}\): estimated mean
  • \(\hat{\sigma}\): estimated standard deviation

Let the Average Depend on Stress

Now the average can depend on a predictor.

If stress predicts sleep:

\[ \mu_i = \beta_0 + \beta_\text{stress} \cdot \text{stress}_i \]

  • \(\mu_i\): predicted average sleep for person \(i\)
  • \(\beta_0\): where the line starts
  • \(\beta_\text{stress}\): the slope of the line
  • \(\sigma\): scatter around the line

Add Sex as a Second Predictor

Now average sleep can depend on stress and sex:

\[ \mu_i = \beta_0 + \beta_\text{stress} \cdot \text{stress}_i + \beta_\text{sex} \cdot \text{sex}_i \]

  • \(\beta_\text{stress}\): slope of each line
  • \(\beta_\text{sex}\): vertical gap between the lines
  • \(\sigma\): scatter around the model lines

This is still one model. It predicts different averages for different predictor values.

Normal Linear Model Notation

A normal linear model can be written as:

\[ \begin{align} y_i &\sim \mathcal{N}(\mu_i, \sigma) \\ \mu_i &= \beta_0 + \beta_1 x_i \end{align} \]

This says:

  • each observed outcome \(y_i\) comes from a normal distribution
  • \(\mu_i\) is the model-predicted mean for observation \(i\)
  • \(\sigma\) is the residual spread around the predicted mean
  • \(\beta_0\) is the intercept
  • \(\beta_1\) is the slope for predictor \(x_i\)

Exercise 1: Formula and Symbols (10 mins)

From the NOW learning room, open part-1-model-formula.Rmd.

Work in pairs / small groups. Completing the first pass, then we discuss the answers together.

You will:

  1. connect the research question to a statistical formula
  2. explain \(\text{rt}_i\), \(\mu_i\), \(\beta_0\), \(\beta_\text{age}\), and \(\text{age}_i\)
  3. arrange symbols into the regression equation
  4. write one short model-description sentence

RMarkdown: write mathematical notation inside dollar signs.

Type this It renders as
$\mu_i$ \(\mu_i\)
$\sigma$ \(\sigma\)
$\beta_0$ \(\beta_0\)

Example: RTs Vary by Age

Hypothesis:

Reaction times vary by participant age.

We adapt the generic model by replacing \(y_i\) with reaction time and \(x_i\) with age.

\[ \text{rt}_i \sim \mathcal{N}(\mu_i, \sigma) \]

\[ \mu_i = \beta_0 + \beta_\text{age} \cdot \text{age}_i \]

This says that each participant’s reaction time varies around a predicted mean, and that predicted mean depends on age.

Implementing the Model in R

The same model is implemented in lm() as:

lm(rt ~ age, data = blomkvist)

Read this as:

  • rt: outcome
  • age: predictor
  • data = blomkvist: dataset containing the variables

Always check before fitting lm(), check that:

  • rows are observations
  • columns are variables
  • the outcome is numeric
  • the outcome has many possible values
  • distances between values are meaningful

rt works because it is numeric reaction time.

sex, smoker, and id are not suitable continuous outcomes.

Model Map

All are linear models. What changes is the question and how the predicted mean is described.

Formula Research question Visual idea
rt ~ 1 What is the average reaction time? one average
rt ~ age Does reaction time change with age? one line
rt ~ sex Does reaction time differ by sex? group means
rt ~ age + sex Does reaction time vary by age and sex? parallel lines
rt ~ age * sex Does the age effect differ by sex? different slopes

Interaction: Different Slopes

Compare two related models:

\[ \mu_i = \beta_0 + \beta_\text{age} \cdot \text{age}_i + \beta_\text{sex} \cdot \text{sex}_i \]

Interaction model adds one more term:

\[ \mu_i = \beta_0 + \beta_\text{age} \cdot \text{age}_i + \beta_\text{sex} \cdot \text{sex}_i + \beta_\text{age:sex} \cdot \text{age}_i \cdot \text{sex}_i \]

Coefficient age + sex age * sex
\(\beta_\text{age}\) shared age effect age effect for reference sex
\(\beta_\text{sex}\) vertical gap gap at baseline age
\(\beta_\text{age:sex}\) not included difference between the two age slopes

The interaction estimate asks: how much different is the age effect for males compared to females? Visually, it is the difference in slopes between the two regression lines.

Quick Check: Core Terminology (2 mins)

Consider this model:

lm(anxiety ~ sleep_hours,
   data = student_data)

And this interpretation:

The intercept is 12.0. The slope for sleep hours is -0.8.

What would a negative slope mean here?

Term What is it in this example?
Outcome
Predictor
Intercept
Slope

Intercept-Only Model

model_0 <- lm(rt ~ 1, data = blomkvist)
coef(model_0)
(Intercept) 
        640 

The intercept is the model’s estimate of the mean outcome.

Model With Age

model_age <- lm(rt ~ age, data = blomkvist)
coef(model_age)
(Intercept)         age 
     314.74        5.77 

The age coefficient estimates how much rt changes for a one-year increase in age.

Age Effect = Age Slope

In the model rt ~ age, the age effect is the coefficient for age.

It answers one question: For a one-year increase in age, how much does the model-predicted reaction time change?

Hypothetical age slopes:

Slope General meaning Example interpretation
10 positive age coefficient reaction time increases by 10 ms for each additional year of age
1 positive age coefficient reaction time increases by 1 ms for each additional year of age
0 coefficient close to zero reaction time does not systematically change with age
-10 negative age coefficient reaction time decreases by 10 ms for each additional year of age

Because higher rt means slower responding, a positive age effect means older participants are predicted to respond more slowly.

Slopes for Continuous Predictors

The slope is a change per one-unit increase in continuous predictor.

Examples:

Model term Outcome Predictor Interpretation
age = 0.01 reaction time age in years predicted reaction time increases by 0.01 for each additional year of age
study_hours = -0.40 anxiety score study hours per week predicted anxiety decreases by 0.40 points for each additional study hour
trial = -2.50 reaction time practice trial predicted reaction time decreases by 2.50 ms for each additional trial

Always ask: one unit of what predictor, and what outcome scale?

Slopes for Categorical Predictors

For a categorical predictor, the slope is a difference from a reference group.

For example, if sex is coded with female as the reference group (intercept):

Model term Meaning
(Intercept) predicted outcome for the reference group when numeric predictors are zero
sexmale = 0.05 predicted difference between male and female participants

A positive categorical coefficient means the comparison group has a higher predicted outcome than the reference group. A negative coefficient means the comparison group has a lower predicted outcome than the reference group.

Model With Age and Sex (2 mins)

model_age_sex <- lm(rt ~ age + sex, data = blomkvist)
coef(model_age_sex)
(Intercept)         age     sexmale 
     339.62        5.74      -59.29 

Turn to the person next to you.

What do these results mean?

  • What does the age coefficient tell us?
  • What does the sex coefficient tell us?
  • What would you need to know before interpreting sex?

Interpreting Model Terms

For rt ~ age + sex:

  • (Intercept) is the predicted outcome when numeric predictors are zero and categorical predictors are at their reference level
  • age is the expected change in rt for a one-year increase in age, holding sex constant
  • sexmale is the expected difference between male and female participants, holding age constant

Do not interpret a coefficient without asking: compared with what, and holding what constant?

Exercise 2: Fit Models and Inspect Coefficients

From the NOW learning room, open part-1-fit-interpret-lm.Rmd.

This is a tutor-guided exercise. We will do the RMarkdown work in one go, then come back to the slides to unpack the tidy() output.

Work through the RMarkdown file until you have run the tidy() output for Your Turn: Age and Sex Model.

You will:

  1. load and inspect the data
  2. compare a verbal description with a model formula
  3. see the same model implemented with lm()
  4. complete the lm() formula for rt ~ age + sex
  5. run tidy() to inspect estimates and confidence intervals

When the tidy() table appears on your screen, pause there. We will explain the columns together before writing the interpretation.

The interaction model rt ~ age * sex is an optional extension for later.

After Running tidy()

tidy(model_age_sex, conf.int = TRUE)
# A tibble: 3 × 7
  term        estimate std.error statistic  p.value conf.low conf.high
  <chr>          <dbl>     <dbl>     <dbl>    <dbl>    <dbl>     <dbl>
1 (Intercept)   340.      27.5       12.3  7.01e-28   285.      394.  
2 age             5.74     0.441     13.0  3.34e-30     4.87      6.61
3 sexmale       -59.3     18.5       -3.20 1.54e- 3   -95.8     -22.8 

In previous years, you may have used summary() for model output.

We will use tidy() because it gives us a cleaner coefficient table and can add confidence intervals.

tidy() returns one row per coefficient.

Reading a tidy() Table

A coefficient table is useful because it separates five pieces of information:

Column What it tells you Plain meaning
term which part of the model this row describes intercept, age effect, sex difference, etc.
estimate point estimate the best guess from this sample, not a perfect value
conf.low, conf.high confidence interval limits the uncertainty around the point estimate
statistic t-value how many standard errors the estimate is away from zero
p.value whether the estimate is statistically different from zero if p < 0.05, treat it as statistically different from zero

Same Estimate, Different Uncertainty

A point estimate is our best guess from the sample, but it is not perfect.

A confidence interval shows the uncertainty around that estimate: a wider interval means more uncertainty, and a narrower interval means less uncertainty.

Imagine three studies all estimate the same positive effect:

Estimate 95% CI What does this mean?
0.30 [0.25, 0.35] precise estimate; zero is not plausible
0.30 [0.05, 0.55] less precise; zero is still not plausible
0.30 [-0.20, 0.80] uncertain; zero is plausible

If a 95% CI does not include zero, we usually reject the null hypothesis that the coefficient is zero (p < .05).

If a 95% CI does include zero, we do not reject that null hypothesis. The estimate can still be positive or negative, but the data are also compatible with no effect.

Quick Check: Estimate and Uncertainty (2 mins)

For each row, what would you say in words?

Estimate 95% CI Evidence
0.03 [0.01, 0.05]
-0.04 [-0.08, -0.01]
0.02 [-0.03, 0.07]

Useful language: evidence for a positive effect, evidence for a negative effect, or uncertain because the interval includes zero.

Transfer Work

If there is time, start part-1-transfer-chinese-ldt.Rmd as individual work. Otherwise, complete it as homework before Session 2.

Use part-1-transfer-own-data-template.Rmd after that if you already have a formative dataset.

The transfer work extends today’s core sequence with one interaction model:

  1. read or write a model description
  2. implement the model in lm()
  3. extract estimates and confidence intervals with tidy()
  4. interpret the interaction as asking whether one effect differs across groups

Homework Before Session 2

Reproduce the main workflow from today with your own formative dataset if you have one.

Try to:

  • identify one continuous outcome
  • identify one or two predictors
  • write a possible lm() formula
  • fit a first normal linear model
  • create a tidy() coefficient table
  • write two or three sentences interpreting the selected model

Use part-1-transfer-own-data-template.Rmd for guidance.

What To Report

For a basic linear model report:

  • research question
  • formal model description
  • final model formula
  • brief explanation of why this model was used
  • coefficient estimates and confidence intervals for the selected model
  • interpretation of the selected model in context
  • one useful coefficient table or plot that supports the selected model

After-Session Reading

Read after the workshop:

  • Required: Mark Andrews, Doing Data Science in R, Chapter 9: Normal Linear Models (Andrews, 2021)
  • Optional/advanced: Gelman, Hill, and Vehtari, Regression and Other Stories, Chapters 7 and 10 on linear regression (Gelman et al., 2020)

Optional refresher:

Use these to revise what lm() estimates, how coefficients are interpreted, and how coefficient estimates support model interpretation.

References

Andrews, M. (2021). Doing data science in R: An introduction for Social Scientists. SAGE Publications Ltd.

Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. Sage publications.

Gelman, A., Hill, J., & Vehtari, A. (2020). Regression and other stories. Cambridge University Press.