BS2004 Mid-Module Survey

Let us know what is/isn’t working.

It only takes a couple minutes (honestly!)

Any Questions?

What is the purpose of sum of squares from an output from anova?

  • Answer #1 (unhelpful): “to calculate your F- and p-values.”

What are SSQ?

Simply, the sum of squared deviations from a mean.
A measure of variation (variance = MS = SSQ / DoF).
But there’s different kinds – remember partitioning of SSQ:

\[TSS = ESS + RSS\]

  • Total, \(TSS = \sum{(y_i-\bar{y})^2}\)
  • Explained, \(ESS = \sum{(\hat{y}-\bar{y})^2}\) – more is better
  • Residual, \(RSS = \sum{(y_i-\hat{y})^2}\) – less is better

\(y_i\) - observed; \(\bar{y}\) - overall mean (‘y-bar’); \(\hat{y}\) - estimated mean (‘y-hat’).

This Week: Interactions

  • Probably the most important thing you’ll learn on BS2004 (or in your BSc degree?)
  • Sound technical, complicated – unnecessary even?
  • Many researchers try to do without them…
    and end up making fools of themselves.

Discuss in your group

  • How would you explain the meaning of ‘interaction’?
  • How is it different from ‘non-orthogonality’?

Hint: Try to define both ideas around “relationships between variables”.

What is an “interaction”?

Two EVs (\(X_1, X_2\)) are said to interact if the effect of \(X_1\) on \(Y\) depends on [the value of] \(X_2\) – and also the other way around.

lm( Y ~ X1 + X2 + X1:X2 )
lm( Y ~ X1*X2 )  # shorthand

 

Note this is different from “\(Y\) depends on both \(X_1\) and \(X_2\).”

lm( Y ~ X1 + X2 )

Interaction vs (non)orthogonality

Interaction

  • The effect of \(X_1\) depends on \(X_2\) and vice versa.
  • “The effect of Aspirin is different in men and women.”

The difference in \(Y\) between Aspirin and placebo is different in men and women: drug effect depends on sex and v.v.

Non-orthogonality

  • The value of \(X_1\) depends on \(X_2\) and vice versa.
  • “More men were given Aspirin, more women were given placebo:”
    drug is informative about sex and vice versa.

Why interactions matter…

There are countless research reports published every year that make unsubstantiated claims because the authors do not know about interactions.

What is going on?

 

Gelman & Stern (2006) The difference between “significant” and “not significant” is not itself statistically significant. The American Statistician 60, 328–331.

A typical fictive example

Drug study: “Does Lowerin lower blood pressure?”

\(N=100\) patients, split by age:

  • 50 young (under 65) and 50 old (over 65)
  • In each age group, half (25) get Lowerin and half don’t.

Well designed… – but really, it asks more than one Q!

In your group: Try and write out the three questions that are at play here.

The results are in!

In your group:

How would you describe the results, based on this plot?

How not to analyse this…

Separate t-tests

  • In young patients: \(P=0.287\);
  • In old patients: \(P=0.00895\).

“The effect of Lowerin depends on age: it lowers blood pressure in old patients (P<0.01) but has no effect in young patients (P>0.05).”

No, no, no!

How would you analyse this?

Given DV bp and EVs age and treat…

m.1 <- lm(bp ~ age + treat + age:treat, data = df)  # or
m.1 <- lm(bp ~ age * treat, data = df)  # or
anova(m.1)
Analysis of Variance Table

Response: bp
          Df Sum Sq Mean Sq F value   Pr(>F)    
age        1 3052.2 3052.23 86.1704 5.17e-15 ***
treat      1  250.9  250.87  7.0824 0.009126 ** 
age:treat  1   43.3   43.27  1.2216 0.271800    
Residuals 96 3400.4   35.42                     
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

There is no evidence that the effect of Drug X depends on age (is different in young and old patients)!

How do I know?

Because I’ve simulated the data, randomly drawing from normal distributions with exactly the same mean difference of 10 between drug and placebo in both age groups.

set.seed(3)  # 3 works a treat
N <- 100
bp <- c(
  rnorm(n=N/4, mean=110, sd=7),  # young: placebo
  rnorm(n=N/4, mean=105, sd=7),  # young: Lowerin
  rnorm(n=N/4, mean=120, sd=7),  # old: placebo
  rnorm(n=N/4, mean=115, sd=7)   # old: Lowerin
  )
age <- factor(
  rep( c("young", "old"), each=N/2), 
  levels=c("young", "old"))
treat <- factor(
  rep( rep(c("Placebo", "Lowerin"), each=N/4), 2),
  levels = c("Placebo", "Lowerin"))

Extra slides

Only if we have time in the session – if we don’t, check them out in your own time.

People really doing this? – Yes…

Grey bars: control; White bars: trained. Senapati B, Treiber CD and Waddell S (2025). Visceral signaling of post-ingestive malaise directs memory updating in Drosophila. BioRxiv doi:https://doi.org/10.1101/2025.10.21.683769.

The problem

What they want to show: the effect of training on performance is different in the mutant flies (C,D) compared to wild-type (B).

Do they show this? Not in their analysis… all they show is that they have evidence for a training effect in the mutant flies, but that they have no evidence for a training effect in the wild-type.

Another way to express the fallacy: “Absence of evidence for an effect is not evidence of absence of that effect.

Let’s see…

I have extracted the data values from the plot and fitted the correct model:

m.flies <- lm(perf.index ~ genotype*training, Flies)
anova(m.flies)
Analysis of Variance Table

Response: perf.index
                  Df   Sum Sq  Mean Sq F value    Pr(>F)    
genotype           2 0.035391 0.017695  7.0756  0.002248 ** 
training           1 0.181241 0.181241 72.4695 1.089e-10 ***
genotype:training  2 0.034199 0.017100  6.8373  0.002688 ** 
Residuals         42 0.105039 0.002501                      
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Phew! The authors are lucky – if correctly analysed, their data would support the claim that the effect of training is different across the three genotypes (significant genotype:starvation interaction).