The Analysis of Variance (ANOVA)

EXTRA CREDIT

You will receive 0.5 points towards your final grade for each of the following DataCamp courses that you complete.

• Importing Data in R (Part 1)
• Data Visualization with ggplot (Part 1)
• Data Visualization with ggplot (Part 2)
• Building Web Applications in R with Shiny

You must get at least 90% XP on an individual course for 0.5 credit.

Due date: December 21, 11:59 pm

• If assumptions of parametric tests are violated, then Type I errors are inflated (leads to false confidence in results).
• Nonparametric tests have less power then parametric tests (throwing away magnitudes and only using ranks).
• When assumptions of two-sample \( t \)-test are met, the Mann-Whitney \( U \)-test has 95% as much power as the two-sample \( t \)-test when samples sizes are large (worse when small).
• Power of Mann-Whitney \( U \)-test is zero when \( n=2 \) (i.e. useless).
• When assumptions of one-sample \( t \)-test are met, the sign test has 64% as much power as the one-sample \( t \)-test when samples sizes are large (worse when small).
• Power of sign test is zero when \( n=5 \).

Definition: A permutation test generates a null distribution for the association between two variables by repeatedly and randomly rearranging the values of one of the two variables in the data.

This is a form of bootstrapping.

In this chapter, we explore a permutation test replacement for the two-sample \( t \)-test.

Variations on this theme can be done for many other tests.

Parametric

Permutation

Algorithm

• Choose a test statistic that measures association between the two variables in the data (e.g. \( \bar{Y}_{1}-\bar{Y}_{2} \); Why not \( t \)?)
• Create a permuted sample of data in which the values of the response variables are randomly reordered.
• Calculate the chosen test statistic for the permuted sample.
• Repeat the permutation process many times – at least 1000 or more.
• Compute the \( P \)-value by comparing the observed test statistic (from original data) to this bootstrapped null distribution.

\( H_{0} \): Mean percent interleukin-17 is the same in both groups.
\( H_{A} \): Mean percent interleukin-17 is NOT the same in both groups.

nPerm <- 10000
permResult <- vector() # initialize vector
for(i in 1:nPerm){
    # step 1: permute the percent interleukin-17
    permSample <- sample(mydata$percentInterleukin17, replace = FALSE)
    # step 2: calculate difference betweeen means
    permMeans <- tapply(permSample, mydata$treatment, mean)
    permResult[i] <- permMeans[2] - permMeans[1]
}

M <- tapply(mydata$percentInterleukin17, mydata$treatment, mean)
(tstat <- M[2]-M[1])

    SPF 
8.04875 

\( P \)-value

(pval <- 2*sum(permResult >= tstat)/nPerm)

[1] 0.0022

Reject hypothesis of equal means.

Definition: The analysis of variance (ANOVA) compares the means of multiple groups simultaneously in a single analysis.

ANOVA generalizes two-sample \( t \)-test to more than two groups.

In two-sample \( t \)-test, the test statistic is a ratio of the difference between means and the standard error of the mean:

\[ t = \frac{\bar{Y}_{1}-\bar{Y}_{2}}{\mathrm{SE}_{\bar{Y}_{1}-\bar{Y}_{2}}} \]

\[ t = \frac{\bar{Y}_{1}-\bar{Y}_{2}}{\mathrm{SE}_{\bar{Y}_{1}-\bar{Y}_{2}}} \]

By squaring the numerator and denominator of the \( t \)-statistic, we get a ratio of variance components.

\[ \frac{\left(\bar{Y}_{1}-\bar{Y}_{2}\right)^2}{\mathrm{SE}^{2}_{\bar{Y}_{1}-\bar{Y}_{2}}} \]

\[ \frac{"\mathrm{Variance \ between \ groups}"}{"\mathrm{Variance \ within \ groups}"} \]

Data: Suppose I have one categorical explanatory variable X with \( k > 2 \) levels, and a response variable Y.

Hypothesis test:

\[ \begin{eqnarray*} H_{0} & : & \mu_{1} = \mu_{2} = \cdots = \mu_{n}\\ H_{A} & : & \mathrm{At \ least \ one} \ \mu_{i} \ \mathrm{is \ different \ from \ the \ others} \end{eqnarray*} \]

Test statistic:

\[ F = \frac{\mathrm{group \ mean \ square}}{\mathrm{error \ mean \ square}} = \frac{\mathrm{MS}_{\mathrm{groups}}}{\mathrm{MS}_{\mathrm{error}}} \]

Test statistic:

\[ F = \frac{\mathrm{group \ mean \ square}}{\mathrm{error \ mean \ square}} = \frac{\mathrm{MS}_{\mathrm{groups}}}{\mathrm{MS}_{\mathrm{error}}} \]

Definition: The group mean square (\( \mathrm{MS}_{\mathrm{groups}} \)) is proportional to the observed amount of variation among the group sample means [between-group variability].

Definition: The error mean square (\( \mathrm{MS}_{\mathrm{error}} \)) estimates the variance among subjects that belong to the same group [within-group variability].

Test statistic:

\[ F = \frac{\mathrm{group \ mean \ square}}{\mathrm{error \ mean \ square}} = \frac{\mathrm{MS}_{\mathrm{groups}}}{\mathrm{MS}_{\mathrm{error}}} \]

If \( H_{0} \) is true, then \( \mathrm{MS}_{\mathrm{groups}} = \mathrm{MS}_{\mathrm{error}} \) and \( F = 1 \).

If \( H_{0} \) is false, then \( \mathrm{MS}_{\mathrm{groups}} > \mathrm{MS}_{\mathrm{error}} \) and \( F > 1 \).

Practice Problem #1

Many humans like the effect of caffeine, but it occurs in plants as a deterrent to herbivory by animals. Caffeine is also found in flower nectar, and nectar is meant as a reward for pollinators, not a deterrent. How does caffeine in nectar affect visitation by pollinators?

Practice Problem #1

Singaravelan et al. (2005) set up feeding stations where bees were offered a choice between a control solution with 20% surcrose or a caffeinated solution with 20% sucrose plus some quantity of caffeine. Over the course of the experiment, four different concentrations of caffeine were provided: 50, 100, 150, and 200 ppm. The response variable was the difference between the amount of nectar consumed from the caffeine feeders and that removed from the control feeders at the same station (grams).

Singaravelan et al. (2005) set up feeding stations where bees were offered a choice between a control solution with 20% surcrose or a caffeinated solution with 20% sucrose plus some quantity of caffeine. Over the course of the experiment, four different concentrations of caffeine were provided: 50, 100, 150, and 200 ppm. The response variable was the difference between the amount of nectar consumed from the caffeine feeders and that removed from the control feeders at the same station (grams).

Discuss: Describe the experimental design.