'data.frame': 20 obs. of 2 variables:
$ ppmCaffeine : Factor w/ 4 levels "ppm50","ppm100",..: 1 2 3 4 1 2 3 4 1 2 ...
$ consumptionDifferenceFromControl: num -0.4 0.01 0.65 0.24 0.34 -0.39 0.53 0.44 0.19 -0.08 ...The Analysis of Variance (ANOVA)
November 27, 2023
Analysis of variance (intro)
Definition: The
analysis of variance (ANOVA) compares the means of multiple groups simultaneously in a single analysis.
ANOVA generalizes two-sample \(t\)-test to more than two groups.
In two-sample \(t\)-test, the test statistic is a ratio of the difference between means and the standard error of the mean:
\[t = \frac{\bar{Y}_{1}-\bar{Y}_{2}}{\mathrm{SE}_{\bar{Y}_{1}-\bar{Y}_{2}}}\]
Analysis of variance (intro)
\[t = \frac{\color{blue}{\bar{Y}_{1}-\bar{Y}_{2}}}{\color{goldenrod}{\mathrm{SE}_{\bar{Y}_{1}-\bar{Y}_{2}}}}\]
Analysis of variance (intro)
By squaring the numerator and denominator of the \(t\)-statistic, we get a ratio of variance components.
\[\frac{\left(\bar{Y}_{1}-\bar{Y}_{2}\right)^2}{\mathrm{SE}^{2}_{\bar{Y}_{1}-\bar{Y}_{2}}}\]
\[\frac{``\mathrm{Variance \ between \ groups}"}{``\mathrm{Variance \ within \ groups}"}\]
Analysis of variance (for real)
Data: Suppose I have one categorical explanatory variable X with \(k > 2\) levels, and a response variable Y.
Hypothesis test:
\[ \begin{eqnarray*} H_{0} & : & \mu_{1} = \mu_{2} = \cdots = \mu_{n}\\ H_{A} & : & \mathrm{At \ least \ one} \ \mu_{i} \ \mathrm{is \ different \ from \ the \ others} \end{eqnarray*} \]
Test statistic:
\[F = \frac{\mathrm{group \ mean \ square}}{\mathrm{error \ mean \ square}} = \frac{\mathrm{MS}_{\mathrm{groups}}}{\mathrm{MS}_{\mathrm{error}}}\]
Analysis of variance (for real)
Test statistic:
\[F = \frac{\mathrm{group \ mean \ square}}{\mathrm{error \ mean \ square}} = \frac{\mathrm{MS}_{\mathrm{groups}}}{\mathrm{MS}_{\mathrm{error}}}\]
Definition: The
group mean square (\(\mathrm{MS}_{\mathrm{groups}}\)) is proportional to the observed amount of variation among the group sample means [between-group variability ].
Definition: The
error mean square (\(\mathrm{MS}_{\mathrm{error}}\)) estimates the variance among subjects that belong to the same group [within-group variability ].
Analysis of variance (for real)
Test statistic:
\[F = \frac{\mathrm{group \ mean \ square}}{\mathrm{error \ mean \ square}} = \frac{\mathrm{MS}_{\mathrm{groups}}}{\mathrm{MS}_{\mathrm{error}}}\]
If \(H_{0}\) is true, then \(\mathrm{MS}_{\mathrm{groups}} = \mathrm{MS}_{\mathrm{error}}\) and \(F = 1\).
If \(H_{0}\) is false, then \(\mathrm{MS}_{\mathrm{groups}} > \mathrm{MS}_{\mathrm{error}}\) and \(F > 1\).
Analysis of variance (example)
Analysis of variance (example)
Practice Problem #1
Many humans like the effect of caffeine, but it occurs in plants as a deterrent to herbivory by animals. Caffeine is also found in flower nectar, and nectar is meant as a reward for pollinators, not a deterrent. How does caffeine in nectar affect visitation by pollinators?
Analysis of variance (example)
Practice Problem #1
Singaravelan et al. (2005) set up feeding stations where bees were offered a choice between a control solution with 20% sucrose or a caffeinated solution with 20% sucrose plus some quantity of caffeine. Over the course of the experiment, four different concentrations of caffeine were provided: 50, 100, 150, and 200 ppm. The response variable was the difference between the amount of nectar consumed from the caffeine feeders and that removed from the control feeders at the same station (grams).
Analysis of variance (example)
Analysis of variance (example)
Singaravelan et al. (2005) set up feeding stations where bees were offered a choice between a control solution with 20% sucrose or a caffeinated solution with 20% sucrose plus some quantity of caffeine. Over the course of the experiment, four different concentrations of caffeine were provided: 50, 100, 150, and 200 ppm. The response variable was the difference between the amount of nectar consumed from the caffeine feeders and that removed from the control feeders at the same station (grams).
Discuss: Describe the experimental design.
Analysis of variance (reminder)
\[t = \frac{\bar{Y}_{1}-\bar{Y}_{2}}{\mathrm{SE}_{\bar{Y}_{1}-\bar{Y}_{2}}}\]
Analysis of variance (derivation)
Data: With \(i\) representing group \(i\), we have
\[Y_{ij} - \bar{Y} = (\bar{Y}_{i} - \bar{Y}) + (Y_{ij} - \bar{Y}_{i})\]
\[\mathrm{SS}_{\mathrm{total}} = \mathrm{SS}_{\mathrm{groups}} + \mathrm{SS}_{\mathrm{error}}\]
Analysis of variance (derivation)
Data: With \(i\) representing group \(i\), we have
\[Y_{ij} - \bar{Y} = (\bar{Y}_{i} - \bar{Y}) + (Y_{ij} - \bar{Y}_{i})\]
\[\mathrm{SS}_{\mathrm{total}} = \sum_{i}\sum_{j}(Y_{ij}-\bar{Y})^2\]
Analysis of variance (derivation)
Data: With \(i\) representing group \(i\), we have
\[Y_{ij} - \bar{Y} = (\bar{Y}_{i} - \bar{Y}) + (Y_{ij} - \bar{Y}_{i})\]
\[\scriptsize{\mathrm{SS}_{\mathrm{total}} = \sum_{i}\sum_{j}(Y_{ij}-\bar{Y})^2 = \sum_{i}n_{i}(\bar{Y}_{i}-\bar{Y})^2 + \sum_{i}\sum_{j}(Y_{ij}-\bar{Y}_{i})^2}\]
Analysis of variance (derivation)
Data: With \(i\) representing group \(i\), we have
\[Y_{ij} - \bar{Y} = (\bar{Y}_{i} - \bar{Y}) + (Y_{ij} - \bar{Y}_{i})\]
\[ \scriptsize{ \begin{eqnarray*} \mathrm{SS}_{\mathrm{total}} = \sum_{i}\sum_{j}(Y_{ij}-\bar{Y})^2 & = & \sum_{i}n_{i}(\bar{Y}_{i}-\bar{Y})^2 + \sum_{i}\sum_{j}(Y_{ij}-\bar{Y}_{i})^2 \\ & = & \mathrm{SS}_{\mathrm{groups}} + \mathrm{SS}_{\mathrm{error}} \end{eqnarray*} } \]
Analysis of variance (derivation)
\[ \scriptsize{ \begin{eqnarray*} \mathrm{SS}_{\mathrm{total}} & = & \sum_{i}\sum_{j}(Y_{ij}-\bar{Y})^2 \\ & = & \sum_{i}\sum_{j}\left[(\bar{Y}_{i} - \bar{Y}) + (Y_{ij} - \bar{Y}_{i})\right]^2 \\ & = & \sum_{i}\sum_{j}\left[(\bar{Y}_{i} - \bar{Y})^2 + (Y_{ij} - \bar{Y}_{i})^2 + 2(\bar{Y}_{i} - \bar{Y})(Y_{ij} - \bar{Y}_{i})\right] \\ & = & \sum_{i}\sum_{j}(\bar{Y}_{i} - \bar{Y})^2 + \sum_{i}\sum_{j}(Y_{ij} - \bar{Y}_{i})^2 + \sum_{i}\sum_{j}2(\bar{Y}_{i} - \bar{Y})(Y_{ij} - \bar{Y}_{i}) \\ & = & \sum_{i}n_{i}(\bar{Y}_{i} - \bar{Y})^2 + \sum_{i}\sum_{j}(Y_{ij} - \bar{Y}_{i})^2 + \sum_{i}\sum_{j}2(\bar{Y}_{i} - \bar{Y})(Y_{ij} - \bar{Y}_{i}) \\ & = & \mathrm{SS}_{\mathrm{groups}} + \mathrm{SS}_{\mathrm{error}} + \sum_{i}\sum_{j}2(\bar{Y}_{i} - \bar{Y})(Y_{ij} - \bar{Y}_{i}) \end{eqnarray*} } \]
Analysis of variance (derivation)
Can show:
\[\sum_{i}\sum_{j}2(\bar{Y}_{i} - \bar{Y})(Y_{ij} - \bar{Y}_{i}) = 0,\]
and thus
\[\mathrm{SS}_{\mathrm{total}} = \mathrm{SS}_{\mathrm{groups}} + \mathrm{SS}_{\mathrm{error}}.\]
Analysis of variance (derivation)
Data: With \(i\) representing group \(i\), we have
\[Y_{ij} - \bar{Y} = (\bar{Y}_{i} - \bar{Y}) + (Y_{ij} - \bar{Y}_{i})\]
\[\scriptsize{\mathrm{SS}_{\mathrm{total}} = \sum_{i}\sum_{j}(Y_{ij}-\bar{Y})^2 = \sum_{i}n_{i}(\bar{Y}_{i}-\bar{Y})^2 + \sum_{i}\sum_{j}(Y_{ij}-\bar{Y}_{i})^2}\]
From sum-of-squares to mean squares
Definition: The
group mean square is given by
\[\mathrm{MS}_{\mathrm{groups}} = \frac{\mathrm{SS}_{\mathrm{groups}}}{df_{\mathrm{groups}}},\] with \(df_{\mathrm{groups}} = k-1\).
Definition: The
error mean square is given by
\[\mathrm{MS}_{\mathrm{error}} = \frac{\mathrm{SS}_{\mathrm{error}}}{df_{\mathrm{error}}},\] with \(df_{\mathrm{error}} = \sum (n_{i}-1) = N-k\).
ANOVA Table
Analysis of variance (example)
Discuss: Is this data tidy or messy?
Definition: Tidy!
Analysis of variance (example)
Analysis of variance (example)
Analysis of variance (example)
Discuss: State the null and alternative hypotheses appropriate for this question.
\[ \begin{eqnarray*} H_{0} & : & \mu_{50} = \mu_{100} = \mu_{150} = \mu_{200} \\ H_{A} & : & \mathrm{At \ least \ one \ of \ the \ means \ is \ different} \end{eqnarray*} \]
Analysis of variance (example)
Short cut using R
Analysis of Variance Table
Response: consumptionDifferenceFromControl
Df Sum Sq Mean Sq F value Pr(>F)
ppmCaffeine 3 1.1344 0.37814 4.1779 0.02308 *
Residuals 16 1.4482 0.09051
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Analysis of variance (example)
Definition: The
\(R^{2}\) value in ANOVA is the “fraction of the variation explained by groups” and is given by
\[R^{2} = \frac{\mathrm{SS}_{\mathrm{groups}}}{\mathrm{SS}_{\mathrm{total}}}.\] Note: \(0 \leq R^2 \leq 1\).
Analysis of variance (example)
Long way
Question: Calculate the following summary statistics for each group: \(n_{i}\), \(\bar{Y}_{i}\), and \(s_{i}\).
Analysis of variance (example)
| ppmCaffeine | n | mean | sd |
|---|---|---|---|
| ppm50 | 5 | 0.008 | 0.2887386 |
| ppm100 | 5 | -0.172 | 0.1694698 |
| ppm150 | 5 | 0.376 | 0.3093218 |
| ppm200 | 5 | 0.378 | 0.3927722 |
Analysis of variance (example)
Compute sum-of-squares
\[ \scriptsize{ \begin{eqnarray*} \mathrm{SS}_{\mathrm{groups}} & = & \sum_{i}n_{i}(\bar{Y}_{i}-\bar{Y})^2 \end{eqnarray*} } \]
Analysis of variance (example)
Compute sum-of-squares
\[ \scriptsize{ \begin{eqnarray*} \mathrm{SS}_{\mathrm{error}} & = & \sum_{i}\sum_{j}(Y_{ij}-\bar{Y}_{i})^2 = \sum_{i}(n_{i}-1)s_{i}^2 \end{eqnarray*} } \]
Analysis of variance (example)
Compute degree of freedom
\[ \begin{equation} \mathrm{df}_{\mathrm{groups}} = k-1 \end{equation} \]
where \(k\) is number of groups.
Analysis of variance (example)
Compute degree of freedom
\[ \begin{equation} \mathrm{df}_{\mathrm{error}} = N-k \end{equation} \]
where \(N\) is number of total observations in data.
Analysis of variance (example)
Compute mean squares
\[ \begin{equation} \mathrm{MS}_{\mathrm{groups}} = \frac{\mathrm{SS}_{\mathrm{groups}}}{\mathrm{df}_{\mathrm{groups}}} \end{equation} \]
Analysis of variance (example)
Compute mean squares
\[ \begin{equation} \mathrm{MS}_{\mathrm{error}} = \frac{\mathrm{SS}_{\mathrm{error}}}{\mathrm{df}_{\mathrm{error}}} \end{equation} \]
Analysis of variance (example)
Compute \(F\)-statistic and \(P\)-value
\[ \begin{equation} F = \frac{\mathrm{MS}_{\mathrm{groups}}}{\mathrm{MS}_{\mathrm{error}}} \end{equation} \]
Analysis of variance (example)
Using Statistical F Table
Analysis of variance (example)
Using Statistical F Table
Analysis of variance (example)
Create manual table and compare
| Df | SumSq | MeanSq | Fval | Pval | |
|---|---|---|---|---|---|
| ppmCaffeine | 3 | 1.134415 | 0.3781383 | 4.177862 | 0.0230776 |
| Residuals | 16 | 1.448160 | 0.0905100 | NA | NA |
Analysis of Variance Table
Response: consumptionDifferenceFromControl
Df Sum Sq Mean Sq F value Pr(>F)
ppmCaffeine 3 1.1344 0.37814 4.1779 0.02308 *
Residuals 16 1.4482 0.09051
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1ANOVA assumptions and robustness
Assumptions (same as 2-sample \(t\)-test)
- Measurements in each group represent a random sample from corresponding population.
- Variable is normally distributed in each of the \(k\) populations.
- Variance is the same in all \(k\) populations.
ANOVA assumptions and robustness
Robustness (same as 2-sample \(t\)-test)
- Robust to deviations in normality.
- Somewhat robust to deviations in equal variances when:
- Sample sizes are “large”, about the same size in each group
- Sample sizes are about the same (balanced)
- Standard deviations are within about a 3-fold difference
Nonparametric alternative to ANOVA
Definition: The
Kruskal-Wallis test is a nonparametric method for mulutiple groups based on ranks.
The Kruskal-Wallis test is similar to the Mann-Whitney \(U\)-test and has the same assumptions:
- Group samples are random samples.
- To use as a test of difference between means or medians, the distributions must have the same shape in every population.
Power of Kruskal-Wallis test is nearly as powerful as ANOVA when sample sizes are large, but has smaller power than ANOVA for small sample sizes.
Planned and unplanned comparisons
So there is a difference between means amongst all groups. Now what? I want to know which means are different from one another!!!
Discuss: Which groups do you think have significantly different means?
Planned comparisons
Definition: A
planned comparison is a comparison between means planned during the design of the study, identified before the data are examined.
A planned comparison must have a strong a priori justification, such as an expectation from theory or a prior study.
Only one or a small number of planned comparisons is allowed, to minimize inflating the Type I error rate.
Planned comparison (details)
A planned comparison is essentially a 2-sample \(t\)-test, except you use the \(\mathrm{MS}_{\mathrm{error}}\) in place of the pooled sample variance when computing the standard error, i.e.
\[\mathrm{SE} = \sqrt{\mathrm{MS}_{\mathrm{error}}\left(\frac{1}{n_{1}} + \frac{1}{n_{2}}\right)}\]
In R, we use the multcomp package to do this (could do it by hand, of course, but ain’t nobody got time for that!)
Planned comparison (example)
ppm200 - ppm50 = 0 is called a contrast. In this case, it simply means “Test if the means between the two groups ppm50 and ppm200 are the same.”
Planned comparison (example)
Simultaneous Confidence Intervals
Multiple Comparisons of Means: User-defined Contrasts
Fit: lm(formula = consumptionDifferenceFromControl ~ ppmCaffeine,
data = strungOutBees)
Quantile = 2.1199
95% family-wise confidence level
Linear Hypotheses:
Estimate lwr upr
ppm200 - ppm50 == 0 0.37000 -0.03336 0.77336Unplanned comparisons
Definition: An
unplanned comparison is one of multiple comparisons, such as between all pairs of means, carried out to help determine where differences between means lie.
Unplanned comparisons are a form of data dredging, so we need to minimize the rising Type I errors that we get from performing many tests.
Unplanned comparison (details)
Definition: With the
Tukey-Kramer method, the probability of making at least one Type I error throughout the course of testing all pairs of means is no greater than the significance level \(\alpha\).
Unplanned comparison (example)
Unplanned comparison (visualization)
Groups in the figure are assigned the same symbol if their means are not significantly different.
Unplanned comparison (visualization)
Groups in the figure are assigned the same symbol if their means are not significantly different.
Tukey-Kramer Assumptions
- Same assumptions as ANOVA.
- \(P\)-value for T-K is exact for balanced designs.
- \(P\)-value for T-K is conservative for unbalanced designs.
- Conservative means real probability of making at least one Type I error is smaller than \(\alpha\), which makes it harder to reject \(H_{0}\).
Fixed-effects ANOVA
Definition:
Fixed-effects ANOVA is ANOVA on fixed groups, i.e. when different categories of the explanatory variable are
- predetermined,
- of direct interest,
- and repeatable.
Any conclusion reached about differences among fixed groups apply only to those fixed groups.
Random-effects ANOVA
Definition:
Random-effects ANOVA is ANOVA on randomly chosen groups, which are groups sampled from a much larger “population” of groups.
Conclusions reached about differences among randomly chosen groups can be generalized to the whole population of groups.
Examples:
- Geographical site for an observational field study
- Subject or individual, in a study involving repeated measures on an individual.
Fixed vs Random-effects
Definition: An explanatory variable is called a
fixed effect if the groups are predefined and are of direct interest. An explanatory variable is called arandom effect if the groups are randomly sampled from a population of possible groups.
Important: The main use of random-effects ANOVA is to estimate
variance components , i.e. the amount of the variance in the data that is among random groups and the amount that is within groups.
Examples:
- Explanatory variables: Genes, Environment
- Response variable: Phenotypic trait
What explains more of the variance in the phenotype - genes or environment?
Fixed vs Random-effects
Important: The main use of random-effects ANOVA is to estimate
variance components , i.e. the amount of the variance in the data that is among random groups and the amount that is within groups.
Examples:
- Explanatory variables: Genes, Environment
- Response variable: Phenotypic trait
Question: What explains more of the variance in the phenotype - genes or environment?
Variance components
Single-factor ANOVA with random effects has two levels of random variation in the response variable \(Y\) - the error and the groups.
Definition: The
variance within groups in the population is written \(\sigma^2\), with
\[\sigma^2 \approx \mathrm{MS}_{\mathrm{error}}.\]
Definition: The
variance between groups (second level of random variation in random-effects ANOVA) is the variance among the group means in the population of groups and is denoted \(\sigma_{A}^{2}\). Thegrand mean \(\mu_{A}\) is the mean of group means.
Variance components
Random-effects ANOVA assumes that the group means are normally distributed, i.e.
\[\mu_{G} \sim N(\mu_{A},\sigma^{2}_{A}).\]
Definition: The parameters \(\sigma^{2}\) and \(\sigma_{A}^{2}\) are called
variance components . They describe all the variance in the response variable \(Y\).
For a balanced design, \(\sigma_{A}^{2}\) can be estimated by
\[s_{A}^{2} = \frac{\mathrm{MS}_{\mathrm{groups}} - \mathrm{MS}_{\mathrm{error}}}{n}.\]
Repeatability
Definition:
Repeatability is the fraction of the summed variance that is present among groups:
\[\mathrm{Repeatability} = \frac{s_{A}^2}{s_{A}^{2} + \mathrm{MS}_{\mathrm{error}}}.\]
A repeatability near zero indicates that most of the variation is within groups.
A repeatability near one indicates that most of the variation is between groups.
Random-effects ANOVA (Example)
Random-effects ANOVA (Example)
Practice Problem #11
One way to assess whether a trait in males has a genetic basis is to determine how similar the measurements of that trait are among his offspring born to different, randomly chosen females. In a lab experiment, Kotiaho et al. (2001) randomly sampled 12 male dung beetles, Onthophagus taurus, and mated each of them to three different virgin females. The average body-condition score of offspring born to each of the three females was measured. ANOVA was used to test whether males differed in the mean condition of their offspring using the three measurements for each male.
Random-effects ANOVA (Example)
Let’s load the data:
Random-effects ANOVA (Example)
Let’s visualize the data (see Chap. 15 R-Pubs):
Random-effects ANOVA (Example)
Let’s get the variance components. We will use a new package called nlme:
male = pdLogChol(1)
Variance StdDev
(Intercept) 0.2361843 0.4859879
Residual 0.1950916 0.4416917Random-effects ANOVA (Example)
male = pdLogChol(1)
Variance StdDev
(Intercept) 0.2361843 0.4859879
Residual 0.1950916 0.4416917(Intercept) corresponds to \(s_{A}^{2}\), which estimates \(\sigma_{A}^{2}\) (variance between groups)
Residual corresponds to \(\mathrm{MS}_{\mathrm{error}}\), which estimates \(\sigma^{2}\) (variance within groups)
Random-effects ANOVA (Example)
Definition: The
heritability of a trait is the fraction of variation in the trait in the population that is genetic rather than environmental.
In our example, differences between males indicate a genetic component, because they were randomly mated and their offspring were raised in a common (lab) environment (if properly designed, of course).
In our case, heritability is the same as the repeatability defined earlier, i.e.
\[\mathrm{Heritability} = \frac{s_{A}^2}{s_{A}^{2} + \mathrm{MS}_{\mathrm{error}}}.\]
Random-effects ANOVA (Example)
Let’s calculate heritability:
[1] 0.5476408So an estimated 55% of the variation in offspring body-condition scores was due to genetic differences between males.
Introduction to Biostatistics, Fall 2023