Comparing two means (Part 2)

• Reading Assignment for Wednesday:
  • Ruxton & Colegrave, Chapter 3: Between-individual variation, replication, and sampling
• Exam #2 this Friday!!
• Lab this week: Review for Exam #2

We will be comparing the means of a numerical variable between two groups.

Definition: In the paired design, both treatments are applied to every sampled unit. In the two-sample design, each treatment group is composed of an independent, random sample of units.

We will be comparing the means of a numerical variable between two groups.

Definition: In the paired design, both treatments are applied to every sampled unit. In the two-sample design, each treatment group is composed of an independent, random sample of units.

Data:

• Response: One numerical variable
• Explanatory: One categorical variable with 2 levels

Paired designs

• The sample size in each group is the same.
• We want to estimate the mean of the differences.

Unpaired designs

• The sample size in each group may not be the same.
• We want to estimate the difference of the means.

\[ \bar{Y}_{1}\sim N(\mu_{1},\sigma_{\bar{Y}_{1}}^2) \ \mathrm{and} \ \bar{Y}_{2}\sim N(\mu_{2},\sigma_{\bar{Y}_{2}}^2) \\ \bar{Y}_{1} - \bar{Y}_{2}\sim N(\mu_{1}-\mu_{2}, \sigma_{\bar{Y}_{1}}^2+\sigma_{\bar{Y}_{2}}^2) \]

Definition: The standard error of the difference of the means between two groups is given by \[ \mathrm{SE}_{\bar{Y_{1}}-\bar{Y_{2}}} = \sqrt{s_{p}^2\left(\frac{1}{n_{1}} + \frac{1}{n_{2}}\right)} \] where pooled sample variance \( s_{p}^{2} \) is given by

\[ s_{p}^2 = \frac{df_{1}s_{1}^2 + df_{2}s_{2}^2}{df_{1}+df_{2}}. \]

Since sampling distribution of \( \bar{Y}_{1} - \bar{Y}_{2} \) is normal

\[ \bar{Y}_{1} - \bar{Y}_{2}\sim N(\mu_{1}-\mu_{2}, \sigma_{\bar{Y}_{1}}^2+\sigma_{\bar{Y}_{2}}^2) \]

the sampling distribution of the statistic

\[ t = \frac{\left(\bar{Y}_{1} - \bar{Y}_{2}\right) - \left(\mu_{1}-\mu_{2}\right)}{\mathrm{SE}_{\bar{Y}_{1} - \bar{Y}_{2}}} \]

has a Student's \( t \)-distribution with total degrees of freedom given by

\[ df = df_{1} + df_{2} = n_{1} + n_{2} - 2. \]

Confidence intervals

Practice Problem #16

A study in West Africa (Lefèvre et al. 2010), working with the mosquito species that carry malaria, wondered whether drinking the local beer influenced attractiveness to mosquitoes. They opened a container holding 50 mosquitoes next to each of 25 alcohol-free participants and measured the proportion of mosquitoes that left the container and flew toward the participants. They repeated this procedure 15 minutes after each of the same participants had consumed a liter of beer, measuring the change in proportion (treatment group).

Practice Problem #16

(cont'd) This procedure was also carried out on another 18 human participants who were given water instead of beer (control group).

mydata <- read.csv("http://whitlockschluter.zoology.ubc.ca/wp-content/data/chapter12/chap12q16BeerAndMosquitoes.csv")
str(mydata)

'data.frame':   43 obs. of  4 variables:
 $ drink      : Factor w/ 2 levels "beer","water": 1 1 1 1 1 1 1 1 1 1 ...
 $ beforeDrink: num  0.13 0.13 0.21 0.25 0.25 0.32 0.43 0.44 0.46 0.5 ...
 $ afterDrink : num  0.49 0.59 0.27 0.43 0.5 0.5 0.37 0.3 0.58 0.89 ...
 $ change     : num  0.36 0.46 0.06 0.18 0.25 0.18 -0.06 -0.14 0.12 0.39 ...

Discuss: What type of plot would be good here?

The long way, first computing the means, and then sample sizes:

(meanChange <- tapply(mydata$change, 
                      mydata$drink, 
                      mean))

       beer       water 
0.154400000 0.007777778 

(n <- tapply(mydata$change, 
             mydata$drink, 
             length))

 beer water 
   25    18 

Now, degree of freedoms and variances:

dfs <- n-1
(vari <- tapply(mydata$change, mydata$drink, var))

      beer      water 
0.02632567 0.01611242 

Pooled sample variance, which is actually a weighted mean (weighted by degrees of freedom):

(sp <- weighted.mean(vari, dfs))

[1] 0.02209091

Finally, find 95% confidence interval:

sderr <- sqrt(sp*sum(1/n)) # Standard error
df <- sum(n) - 2 # Degree of freedom
tcrit <- qt(0.025, df=df, lower.tail=FALSE)
diffMeans <- meanChange["beer"] - meanChange["water"]
lower <- diffMeans - tcrit*sderr
upper <- diffMeans + tcrit*sderr
(CI <- unname(c(lower, upper)))

[1] 0.05383517 0.23940928

Using dplyr:

mydata %>% 
  group_by(drink) %>%
  summarize(mu = mean(change),
            n = n(),
            df = n-1,
            vari = var(change)) %>% 
  summarize(sp = weighted.mean(vari, df),
            sderr = sqrt(sp*sum(1/n)),
            df = sum(df),
            tcrit = qt(0.025, df=df, lower.tail=FALSE),
            diffMeans = first(mu)-last(mu),
            lower = diffMeans - tcrit*sderr,
            upper = diffMeans + tcrit*sderr)

# A tibble: 1 × 7
          sp      sderr    df    tcrit diffMeans      lower     upper
       <dbl>      <dbl> <dbl>    <dbl>     <dbl>      <dbl>     <dbl>
1 0.02209091 0.04594463    41 2.019541 0.1466222 0.05383517 0.2394093

Now the fast way using t.test:

t.test(change ~ drink, 
       data=mydata, 
       mu=0, 
       var.equal=TRUE)$conf.int

[1] 0.05383517 0.23940928
attr(,"conf.level")
[1] 0.95

Two-sample \( t \)-test

\[ H_{0}: \mu_{1} - \mu_{2} = \left(\mu_{1} - \mu_{2}\right)_{0} \] \[ H_{A}: \mu_{1} - \mu_{2} \neq \left(\mu_{1} - \mu_{2}\right)_{0} \]

Test statistic:

\[ t = \frac{\left(\bar{Y}_{1} - \bar{Y}_{2}\right) - \left(\mu_{1} - \mu_{2}\right)_{0}}{SE_{\bar{Y}_{1} - \bar{Y}_{2}}} \]

Assumptions:

• Each sample is randomly sampled from population.
• Numerical variable is normally distributed.

Two-sample \( t \)-test

\[ H_{0}: \mu_{1} - \mu_{2} = \left(\mu_{1} - \mu_{2}\right)_{0} \] \[ H_{A}: \mu_{1} - \mu_{2} \neq \left(\mu_{1} - \mu_{2}\right)_{0} \]

Test statistic:

\[ t = \frac{\left(\bar{Y}_{1} - \bar{Y}_{2}\right) - \left(\mu_{1} - \mu_{2}\right)_{0}}{SE_{\bar{Y}_{1} - \bar{Y}_{2}}} \]

Assumptions:

• Variation is same in both populations.

Practice Problem #16

Two-sample \( t \)-test

\[ H_{0}: \mu_{\mathrm{beer}} - \mu_{\mathrm{water}} = 0 \] \[ H_{A}: \mu_{\mathrm{beer}} - \mu_{\mathrm{water}} \neq 0 \]

\[ H_{0}: \mu_{\mathrm{beer}} = \mu_{\mathrm{water}} \] \[ H_{A}: \mu_{\mathrm{beer}} \neq \mu_{\mathrm{water}} \]

Starting with, as usual, the long way, we need to calculate the test statistic:

\[ t = \frac{\left(\bar{Y}_{1} - \bar{Y}_{2}\right)}{SE_{\bar{Y}_{1} - \bar{Y}_{2}}} \]

(tstat <- unname(meanChange["beer"] - meanChange["water"])/sderr)

[1] 3.191281

tstat > tcrit

[1] TRUE

Let's calculate \( P \)-value as well:

(pval <- 2*pt(abs(tstat), 
              df=df, 
              lower.tail=FALSE))

[1] 0.00271747

Short way, again using t.test (note the var.equal=TRUE):

t.test(change ~ drink, 
       mu=0, 
       var.equal=TRUE,
       data=mydata)

Short way, again using t.test (note the var.equal=TRUE):

    Two Sample t-test

data:  change by drink
t = 3.1913, df = 41, p-value = 0.002717
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.05383517 0.23940928
sample estimates:
 mean in group beer mean in group water 
        0.154400000         0.007777778 

Heuristic for meeting two-sample assumptions:

• Moderate sample sizes (\( n_{1}, n_{2} > 30 \))
• Balanced: \( n_{1} \approx n_{2} \)
• \( |s_{1} - s_{2}| \leq 3 \)

Robust to deviations in normality.

• Moderate sample sizes (\( n_{1}, n_{2} > 30 \))
• Balanced: \( n_{1} \approx n_{2} \)
• \( |s_{1} - s_{2}| \leq 3 \)

n

 beer water 
   25    18 

sqrt(vari)

     beer     water 
0.1622519 0.1269347 

What to do if can't meet assumptions of two-sample \( t \)-test?

Definition: Welch’s t-test compares the mean of two groups and can be used even when the variances of the two groups are not equal.

Standard error and degrees of freedom are calculated differently than two-sample \( t \)-test, but otherwise the same (i.e. uses a \( t \)-distribution).

Same as two-sample in R, except var.equal=FALSE (default).

t.test(change ~ drink, 
       mu=0, 
       var.equal=FALSE, 
       data=mydata)

Same as two-sample in R, except var.equal=FALSE (default).

    Welch Two Sample t-test

data:  change by drink
t = 3.3219, df = 40.663, p-value = 0.001897
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.05746134 0.23578311
sample estimates:
 mean in group beer mean in group water 
        0.154400000         0.007777778 

This topic (replication and pseudoreplication) is covered in Chapter 3 of Ruxton & Colegrave

Question: Do populations differ in the variability of measurements?

Remember, it isn't always about inferring central tendency!

There are two main tests:

• \( F \)-test (Warning: Highly sensitive to departures from normality assumption)
• Levene's test (More robust to departures from normality, but at a cost - loss of power!)

Example 12.4

The brook trout is a species native to eastern North America that has been introduced into streams in the West for sport fishing. Biologists followed the survivorship of a native species, chinook salmon, in a series of 12 streams that either had brook trout introduced or did not (Levin et al. 2002). Their goal was to determine whether the presence of brook trout effected the survivorship of the salmon. In each stream, they released a number of tagged juvenile chinook and then recorded whether or not each chinook survived over one year.

Load data and sneak-a-peek:

'data.frame':   12 obs. of  4 variables:
 $ troutTreatment    : Factor w/ 2 levels "absent","present": 2 1 2 2 1 2 1 2 1 1 ...
 $ nReleased         : int  820 467 960 700 959 545 1029 769 27 998 ...
 $ nSurvivors        : int  166 180 136 153 178 103 326 173 7 120 ...
 $ proportionSurvived: num  0.202 0.385 0.142 0.219 0.186 0.189 0.317 0.225 0.259 0.12 ...

Compute variances in both groups:

(vari <- tapply(chinook$proportionSurvived, 
                chinook$troutTreatment, 
                var))

      absent      present 
0.0107413667 0.0008829667 

var.test(proportionSurvived ~ troutTreatment, 
         data = chinook)

    F test to compare two variances

data:  proportionSurvived by troutTreatment
F = 12.165, num df = 5, denom df = 5, p-value = 0.01589
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
  1.702272 86.936360
sample estimates:
ratio of variances 
          12.16509 

library(car)
leveneTest(chinook$proportionSurvived, 
           group = chinook$troutTreatment, 
           center = mean)

Levene's Test for Homogeneity of Variance (center = mean)
      Df F value   Pr(>F)   
group  1  10.315 0.009306 **
      10                    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

How to compare between two groups with only confidence intervals?

Example 12.5: Mommy's baby, Daddy's maybe

Question: Do babies look more like their fathers or their mothers?

Example 12.5: Mommy's baby, Daddy's maybe

Question: Do babies look more like their fathers or their mothers?

Christenfeld and Hill (1995) predicted that babies more resemble their fathers, due to the hypothesis that this resemblance affords an evolutionary advantage of increased paternal care. They tested this by obtaining pictures of a series of babies and their mothers and fathers. Particpants shown picture of child, and either three possible mothers or three possible fathers (one is correct).

Conclusion: Authors concluded that since fathers turned up statistically significant and mothers did not, that babies more resembled their fathers than their mothers.

Discuss: What’s the mistake here?

Mistake: Misinterpretation of statistical significance

Fallacy: If one test in Group 1 shows with statistical significance that \( \mu_{1} > \mu_{0} \), and the same test in Group 2 does not show \( \mu_{2} > \mu_{0} \), then this shows with statistical significance that \( \mu_{1} > \mu_{2} \).

Fallacy: If one test in Group 1 shows with statistical significance that \( \mu_{1} > \mu_{0} \), and the same test in Group 2 does not show \( \mu_{2} > \mu_{0} \), then this shows with statistical significance that \( \mu_{1} > \mu_{2} \).

Fallacy: If \( \bar{Y}_{1} > \bar{Y}_{2} \), then \( \mu_{1} > \mu_{2} \).

Mistake: Relying on point estimates rather than interval estimates

Conclusion: Comparisons between two groups should always be made directly using the appropriate statistical test, not indirectly by comparing both to the same null hypothesized value.