yearOfChange HigherTaxDeaths lowerTaxDeaths
1 1917 22.21 24.93
2 1917 18.86 20.00
3 1919 28.21 29.93
4 1924 31.64 30.64
5 1926 18.43 20.86
6 1932 9.50 10.14
7 1934 24.29 28.00
8 1935 26.64 25.29
9 1940 35.07 35.00
10 1941 38.86 37.57
11 1942 28.50 34.79Comparing two means
November 6, 2023
Quote-of-the-day
“Statisticians, like artists, have the bad habit of falling in love with their models.”
– George Box
It all starts with experimental design
We will be comparing the means of a numerical variable between two groups.
Definition: In the
paired design , both treatments are applied to every sampled unit. In thetwo-sample design , each treatment group is composed of an independent, random sample of units.
It all starts with experimental design
We will be comparing the means of a numerical variable between two groups.
Definition: In the
paired design , both treatments are applied to every sampled unit. In thetwo-sample design , each treatment group is composed of an independent, random sample of units.
Data:
- Response: One numerical variable
- Explanatory: One categorical variable with 2 levels
Paired vs. unpaired
Paired designs
- The sample size in each group is the same.
- We want to estimate the mean of the differences.
Unpaired designs
- The sample size in each group may not be the same.
- We want to estimate the difference of the means.
Paired design
Remember standard error: \[ \sigma_{\bar{Y}} = \frac{\sigma}{\sqrt{n}} \]
We can increase power and the precision of our estimates by decreasing the standard error through…
- …increasing the sample size (denominator).
- …decreasing the variability \(\sigma\) in our measured variable (numerator).
The paired design mainly effects point 2 above, i.e. reduces variability. How?
Experimental Design
Unpaired Design
Paired Design
Paired vs. Unpaired
Unpaired
Paired
Paired design examples
Discuss: Can you come up with an example of a paired and unpaired design?
From the book:
- Comparing patient weight before and after hospitalization
- Comparing fish species diversity in lakes before and after heavy metal contamination
- Testing effects of sunscreen applied to one arm of each subject compared with a placebo applied to the other arm
- Testing effects of smoking in a sample of smokers, each of which is compared with a nonsmoker closely matched by age, weight, and ethnic background
Paired design: What is our resulting variable?
Definition: Paired measurements are converted to a single measurement by taking the difference between them.
\[d = Y_{T}-Y_{C},\]
where \(Y_{T}\) and \(Y_{C}\) denote the variable in the treatment and control groups, respectively.
Paired design: Estimation
If \(Y_{T}\sim N(\mu_{T},\sigma_{T}^2)\), \(Y_{C}\sim N(\mu_{C},\sigma_{C}^2)\), and \(d = Y_{T}-Y_{C}\), then
\[d \sim N(\mu_{T}-\mu_{C},\sigma_{T}^2 + \sigma_{C}^2)\]
Confidence intervals
\[\bar{d} - t_{\alpha(2),df}\mathrm{SE}_{\bar{d}} < \mu_{d} < \bar{d} + t_{\alpha(2),df}\mathrm{SE}_{\bar{d}}\]
Paired design: Hypothesis testing
Paired \(t\)-test: One-sample \(t\)-test on the difference d
\[H_{0}: \mu_{d} = \mu_{d0}\] \[H_{A}: \mu_{d} \neq \mu_{d0}\]
Test statistic:
\[t = \frac{\bar{d} - \mu_{d0}}{SE_{\bar{d}}}\]
Assumptions: Same as one-sample t-test
- The sampling units are randomly sampled from population.
- Paired differences have normal distribution in population. Original measurements DO NOT have to be normal.
Paired design: Practice Problem #1
Question: Can the death rate be influenced by tax incentives?
Kopczuk and Slemrod (2003) investigated this possibility using data on deaths in the United States in years in which the government announced it was changing (usually raising) the tax rate on inheritance (the estate tax). The authors calculated the death rate during the 14 days before, and the 14 days after, the changes in the estate tax rates took effect. The number of deaths per day for each of these periods was recorded.
Paired design: Practice Problem #1
Paired design: Practice Problem #1
Paired design: Practice Problem #1
with(deathRate,
stripchart(list(HigherTaxDeaths,
lowerTaxDeaths),
vertical = TRUE,
group.names = c("Higher","Lower"),
xlim=c(0.5, 2.5),
pch = 16,
col = "firebrick",
ylab="Death Rate", xlab="Estate tax rate",
cex=1.5, cex.lab=1.5, cex.axis=1.5))
with(deathRate,
segments(1, HigherTaxDeaths,
2, lowerTaxDeaths))Paired design: Practice Problem #1
Question: What are the null and alternate hypotheses?
Answer: \[ \begin{align} H_{0}: & \mathrm{Mean \ change \ in \ death \ rate \ is \ zero}\\ H_{A}: & \mathrm{Mean \ change \ in \ death \ rate \ is \ not \ zero} \end{align} \]
Answer: \[H_{0}: \mu_{d} = 0\] \[H_{A}: \mu_{d} \neq 0\]
Paired design: Practice Problem #1
Let’s do a one-sample \(t\)-test
Paired design: Practice Problem #1
Let’s do a one-sample \(t\)-test
Paired t-test
data: HigherTaxDeaths and lowerTaxDeaths
t = -1.9121, df = 10, p-value = 0.08491
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-2.9408501 0.2244865
sample estimates:
mean of the differences
-1.358182 Two-sample design: Estimation
\[ \bar{Y}_{1}\sim N(\mu_{1},\sigma_{\bar{Y}_{1}}^2) \ \mathrm{and} \ \bar{Y}_{2}\sim N(\mu_{2},\sigma_{\bar{Y}_{2}}^2) \\ \bar{Y}_{1} - \bar{Y}_{2}\sim N(\mu_{1}-\mu_{2}, \sigma_{\bar{Y}_{1}}^2+\sigma_{\bar{Y}_{2}}^2) \]
Definition: The
standard error of the difference of the means between two groups is given by \[ \mathrm{SE}_{\bar{Y_{1}}-\bar{Y_{2}}} = \sqrt{s_{p}^2\left(\frac{1}{n_{1}} + \frac{1}{n_{2}}\right)} \] wherepooled sample variance \(s_{p}^{2}\) is given by \[ s_{p}^2 = \frac{df_{1}s_{1}^2 + df_{2}s_{2}^2}{df_{1}+df_{2}}. \]
Two-sample design: Estimation
Since sampling distribution of \(\bar{Y}_{1} - \bar{Y}_{2}\) is normal
\[\bar{Y}_{1} - \bar{Y}_{2}\sim N(\mu_{1}-\mu_{2}, \sigma_{\bar{Y}_{1}}^2+\sigma_{\bar{Y}_{2}}^2)\]
the sampling distribution of the statistic
\[t = \frac{\left(\bar{Y}_{1} - \bar{Y}_{2}\right) - \left(\mu_{1}-\mu_{2}\right)}{\mathrm{SE}_{\bar{Y}_{1} - \bar{Y}_{2}}} \]
has a Student’s \(t\)-distribution with total degrees of freedom given by
\[ df = df_{1} + df_{2} = n_{1} + n_{2} - 2. \]
Two-sample design: Estimation
Confidence intervals
Two-sample design: Estimation Example
Practice Problem #16
A study in West Africa (Lefèvre et al. 2010), working with the mosquito species that carry malaria, wondered whether drinking the local beer influenced attractiveness to mosquitoes. They opened a container holding 50 mosquitoes next to each of 25 alcohol-free participants and measured the proportion of mosquitoes that left the container and flew toward the participants. They repeated this procedure 15 minutes after each of the same participants had consumed a liter of beer, measuring the change in proportion (treatment group). This procedure was also carried out on another 18 human participants who were given water instead of beer (control group).
Two-sample design: Estimation Example
'data.frame': 43 obs. of 4 variables:
$ drink : chr "beer" "beer" "beer" "beer" ...
$ beforeDrink: num 0.13 0.13 0.21 0.25 0.25 0.32 0.43 0.44 0.46 0.5 ...
$ afterDrink : num 0.49 0.59 0.27 0.43 0.5 0.5 0.37 0.3 0.58 0.89 ...
$ change : num 0.36 0.46 0.06 0.18 0.25 0.18 -0.06 -0.14 0.12 0.39 ...Two-sample design: Estimation Example
Discuss: What type of plot would be good here?
Two-sample design: Estimation Example
The long way, first computing the means, and then sample sizes:
Two-sample design: Estimation Example
Now, degree of freedoms and variances:
Two-sample design: Estimation Example
Finally, find 95% confidence interval:
sderr <- sqrt(sp*sum(1/n)) # Standard error
df <- sum(n) - 2 # Degree of freedom
tcrit <- qt(0.025, df=df, lower.tail=FALSE)
diffMeans <- meanChange["beer"] - meanChange["water"]
lower <- diffMeans - tcrit*sderr
upper <- diffMeans + tcrit*sderr
(CI <- unname(c(lower, upper)))[1] 0.05383517 0.23940928Two-sample design: Estimation Example
Using dplyr:
mydata %>%
group_by(drink) %>%
summarize(mu = mean(change),
n = n(),
df = n-1,
vari = var(change)) %>%
summarize(sp = weighted.mean(vari, df),
sderr = sqrt(sp*sum(1/n)),
df = sum(df),
tcrit = qt(0.025, df=df, lower.tail=FALSE),
diffMeans = first(mu)-last(mu),
lower = diffMeans - tcrit*sderr,
upper = diffMeans + tcrit*sderr)# A tibble: 1 × 7
sp sderr df tcrit diffMeans lower upper
<dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 0.0221 0.0459 41 2.02 0.147 0.0538 0.239Two-sample design: Estimation Example
Now the fast way using t.test:
Two Sample t-test
data: change by drink
t = 3.1913, df = 41, p-value = 0.002717
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
0.05383517 0.23940928
sample estimates:
mean in group beer mean in group water
0.154400000 0.007777778 Two-sample design: Hypothesis testing
Two-sample \(t\)-test
\[H_{0}: \mu_{1} - \mu_{2} = \left(\mu_{1} - \mu_{2}\right)_{0}\] \[H_{A}: \mu_{1} - \mu_{2} \neq \left(\mu_{1} - \mu_{2}\right)_{0}\]
Test statistic:
\[t = \frac{\left(\bar{Y}_{1} - \bar{Y}_{2}\right) - \left(\mu_{1} - \mu_{2}\right)_{0}}{SE_{\bar{Y}_{1} - \bar{Y}_{2}}}\]
Assumptions:
- Each sample is randomly sampled from population.
- Numerical variable is normally distributed.
Two-sample design: Hypothesis testing
Two-sample \(t\)-test
\[H_{0}: \mu_{1} - \mu_{2} = \left(\mu_{1} - \mu_{2}\right)_{0}\] \[H_{A}: \mu_{1} - \mu_{2} \neq \left(\mu_{1} - \mu_{2}\right)_{0}\]
Test statistic:
\[t = \frac{\left(\bar{Y}_{1} - \bar{Y}_{2}\right) - \left(\mu_{1} - \mu_{2}\right)_{0}}{SE_{\bar{Y}_{1} - \bar{Y}_{2}}}\]
Assumptions:
- Variation is same in both populations.
Two-sample design: Testing Example
Practice Problem #16
Two-sample \(t\)-test
\[H_{0}: \mu_{\mathrm{beer}} - \mu_{\mathrm{water}} = 0\] \[H_{A}: \mu_{\mathrm{beer}} - \mu_{\mathrm{water}} \neq 0\]
\[H_{0}: \mu_{\mathrm{beer}} = \mu_{\mathrm{water}}\] \[H_{A}: \mu_{\mathrm{beer}} \neq \mu_{\mathrm{water}}\]
Two-sample design: Testing Example
Starting with, as usual, the long way, we need to calculate the test statistic:
\[t = \frac{\left(\bar{Y}_{1} - \bar{Y}_{2}\right)}{SE_{\bar{Y}_{1} - \bar{Y}_{2}}}\]
Two-sample design: Testing Example
Let’s calculate \(P\)-value as well:
Two-sample design: Testing Example
Short way, again using t.test (note the var.equal=TRUE):
Two-sample design: Testing Example
Short way, again using t.test (note the var.equal=TRUE):
Two Sample t-test
data: change by drink
t = 3.1913, df = 41, p-value = 0.002717
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
0.05383517 0.23940928
sample estimates:
mean in group beer mean in group water
0.154400000 0.007777778 Two-sample design: Assumptions
Heuristic for meeting two-sample assumptions:
- Moderate sample sizes (\(n_{1}, n_{2} > 30\))
- Balanced: \(n_{1} \approx n_{2}\)
- \(1/3 \leq s_{2}/s_{1} \leq 3\)
Robust to deviations in normality.
Two-sample design: Testing Example
- Moderate sample sizes (\(n_{1}, n_{2} > 30\))
- Balanced: \(n_{1} \approx n_{2}\)
- \(1/3 \leq s_{2}/s_{1} \leq 3\)
Two-sample design: Testing Example
What to do if can’t meet assumptions of two-sample \(t\)-test?
Definition:
Welch’s t-test compares the mean of two groups and can be used even when the variances of the two groups are not equal.
Standard error and degrees of freedom are calculated differently than two-sample \(t\)-test, but otherwise the same (i.e. uses a \(t\)-distribution).
Two-sample design: Welch’s t-test
Same as two-sample in R, except var.equal=FALSE (default).
Welch Two Sample t-test
data: change by drink
t = 3.3219, df = 40.663, p-value = 0.001897
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
0.05746134 0.23578311
sample estimates:
mean in group beer mean in group water
0.154400000 0.007777778 Comparing variances
Question: Do populations differ in the variability of measurements?
Remember, it isn’t always about inferring central tendency!
There are two main tests:
- \(F\)-test (Warning: Highly sensitive to departures from normality assumption)
- Levene’s test (More robust to departures from normality, but at a cost - loss of power!)
Comparing variances
Example 12.4
The brook trout is a species native to eastern North America that has been introduced into streams in the West for sport fishing. Biologists followed the survivorship of a native species, chinook salmon, in a series of 12 streams that either had brook trout introduced or did not (Levin et al. 2002). Their goal was to determine whether the presence of brook trout effected the survivorship of the salmon. In each stream, they released a number of tagged juvenile chinook and then recorded whether or not each chinook survived over one year.
Comparing variances
Load data and sneak-a-peek:
'data.frame': 12 obs. of 4 variables:
$ troutTreatment : chr "present" "absent" "present" "present" ...
$ nReleased : int 820 467 960 700 959 545 1029 769 27 998 ...
$ nSurvivors : int 166 180 136 153 178 103 326 173 7 120 ...
$ proportionSurvived: num 0.202 0.385 0.142 0.219 0.186 0.189 0.317 0.225 0.259 0.12 ...Comparing variances
Compute variances in both groups:
Comparing variances - F-test
F test to compare two variances
data: proportionSurvived by troutTreatment
F = 12.165, num df = 5, denom df = 5, p-value = 0.01589
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
1.702272 86.936360
sample estimates:
ratio of variances
12.16509 Comparing variances - Levene’s test
Visualizing significance between groups
How to compare between two groups with only confidence intervals?
The fallacy of indirect comparison
Example 12.5: Mommy’s baby, Daddy’s maybe
Question: Do babies look more like their fathers or their mothers?
The fallacy of indirect comparison
Example 12.5: Mommy’s baby, Daddy’s maybe
Question: Do babies look more like their fathers or their mothers?
Christenfeld and Hill (1995) predicted that babies more resemble their fathers, due to the hypothesis that this resemblance affords an evolutionary advantage of increased paternal care. They tested this by obtaining pictures of a series of babies and their mothers and fathers. Particpants shown picture of child, and either three possible mothers or three possible fathers (one is correct).
The fallacy of indirect comparison
Conclusion: Authors concluded that since fathers turned up statistically significant and mothers did not, that babies more resembled their fathers than their mothers.
Discuss: What’s the mistake here?
Mistake: Misinterpretation of statistical significance
The fallacy of indirect comparison
Fallacy: If one test in Group 1 shows with statistical significance that \(\mu_{1} > \mu_{0}\), and the same test in Group 2 does
not show \(\mu_{2} > \mu_{0}\), then this shows with statistical significance that \(\mu_{1} > \mu_{2}\).
The fallacy of indirect comparison
Fallacy: If one test in Group 1 shows with statistical significance that \(\mu_{1} > \mu_{0}\), and the same test in Group 2 does
not show \(\mu_{2} > \mu_{0}\), then this shows with statistical significance that \(\mu_{1} > \mu_{2}\).
The fallacy of indirect comparison
Fallacy: If \(\bar{Y}_{1} > \bar{Y}_{2}\), then \(\mu_{1} > \mu_{2}\).
Mistake:Relying on point estimates rather than interval estimates
The fallacy of indirect comparison
Conclusion: Comparisons between two groups should always be made directly using the appropriate statistical test, not indirectly by comparing both to the same null hypothesized value.
Introduction to Biostatistics, Fall 2023