Handling violations of assumptions (cont'd)

• Ignore the violations of assumptions
• Transform the data
• Use a nonparametric method
• Use a permutation test (computer-intensive methods)

Definition: A data transformation changes each measurement by the same mathematical formula.

Common transformations:

• Log transformation (data skewed right) \[ Y^{\prime} = \ln[Y] \]
• Arcsine transformation (data are proportions) \[ p^{\prime} = \arcsin[\sqrt{p}] \]
• Square-root transformation (data are counts) \[ Y^{\prime} = \sqrt{Y + 1/2} \]

Other transformations:

• Square transformation (data skewed left) \[ Y^{\prime} = Y^2 \]
• Antilog transformation (data skewed left) \[ Y^{\prime} = e^{Y} \]
• Reciprocal transformation (data skewed right) \[ Y^{\prime} = \frac{1}{Y} \]
• Box-Cox transformation (skew) \[ Y^{\prime}_{\lambda} = \frac{Y^{\lambda} - 1}{\lambda} \]

• Measurements are ratios or products
• Frequency distribution is skewed right
• Group having larger mean also has larger standard deviation
• Data span several orders of magnitude

• Measurements are ratios or products
• Frequency distribution is skewed right
• Group having larger mean also has larger standard deviation
• Data span several orders of magnitude

• Measurements are ratios or products
• Frequency distribution is skewed right
• Group having larger mean also has larger standard deviation
• Data span several orders of magnitude

Hypothesis testing

marine <- read.csv("http://whitlockschluter.zoology.ubc.ca/wp-content/data/chapter13/chap13e1MarineReserve.csv")
shapiro.test(log(marine$biomassRatio))

    Shapiro-Wilk normality test

data:  log(marine$biomassRatio)
W = 0.93795, p-value = 0.06551

Hypothesis testing

hist(log(marine$biomassRatio))

Original statistical hypotheses:

\( H_{0} \): The mean of the biomass ratio of marine reserves is one (\( \mu = 1 \))
\( H_{A} \): The mean of the biomass ratio of marine reserves is not one (\( \mu \neq 1 \))

Transformed statistical hypotheses:

\( H_{0} \): The mean of the log biomass ratio of marine reserves is zero (\( \mu^{\prime} = 0 \))
\( H_{A} \): The mean of the log biomass ratio of marine reserves is not zero (\( \mu^{\prime} \neq 0 \))

t.test(log(marine$biomassRatio), mu=0)

    One Sample t-test

data:  log(marine$biomassRatio)
t = 7.3968, df = 31, p-value = 2.494e-08
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
 0.3470180 0.6112365
sample estimates:
mean of x 
0.4791272 

Estimation

The 95% confidence interval for the log transformed data is

\[ 0.347 < \mu^{\prime} < 0.611. \]

For a 95% confidence interval of the untransformed data, we have

\[ e^{0.347} < \mathrm{geometric \ mean} < e^{0.611}, \]

or

\[ 1.41 < \mathrm{geometric \ mean} < 1.84. \]

Discuss: Conclusion?

• Be careful of sign of your data!!! (i.e. positives and negatives).
• Avoid multiple testing with transformations (i.e. use all transformations and choose one that gives significant result)

• Ignore the violations of assumptions
• Transform the data
• Use a nonparametric method
• Use a permutation test (computer-intensive methods)

Definition: A nonparametric method makes fewer assumptions than standard parametric methods do about the distribution of the variables.

Property: Nonparametric methods are usually based on the ranks of the data points (medians, quartiles, etc.)

Property: Nonparametric tests are typically less powerful than parametric tests.

• A nonparametric alternative to the one-sample \( t \)-test is the sign test.

Definition: The sign test compares the median of a sample to a constant specified in the null hypothesis. It makes no assumptions about the distribution of the measurements in the population.

• A nonparametric alternative to the two-sample \( t \)-test is the Mann-Whitney \( U \)-test.

Definition: The Mann-Whitney \( U \)-test compares the distributions of two groups. It does not require as many assumptions as the two-sample \( t \)-test.

Algorithm:

• First, state a null hypothesized median.
• Label all measurements larger than this median with a “\( + \)”, and all measurements smaller than this median with a “\( - \)”.
• Throw out any measurements exactly equal to the median (sample size is reduced by this amount)
• Use binomial test with the test statistic the number of “\( + \)” values (or \( - \) values), comparing the result to the null proportion \( p_{0}=0.5 \).

Sign test has very little power. If \( n \leq 5 \), then can't use sign test.

Assignment problem #25

Researchers have observed that rainforest areas next to clear-cuts (less than 100 meters away) have a reduced tree biomass compared to rainforest areas far from clear-cuts. To go further, Laurance et al. (1997) tested whether rainforest areas more distant from the clear-cuts were also affected. They compiled data on the biomass change after clear-cutting (in tons/hectare/year) for 36 rainforest areas between 100m and several kilometers from clear-cuts.

Look at data

hist(clearcuts$biomassChange)

hist(exp(clearcuts$biomassChange), main="Exponential transformation")

\( H_{0} \): The median change in biomass is zero.
\( H_{A} \): The median change in biomass is not zero.

# Any biomass equal to zero?
sum(clearcuts$biomassChange == 0)

[1] 0

# How many plots have positive change in biomass?
(X <- sum(clearcuts$biomassChange > 0))

[1] 21

# Perform binomial test
binom.test(X, n=36, p=0.5)

    Exact binomial test

data:  X and 36
number of successes = 21, number of trials = 36, p-value = 0.405
alternative hypothesis: true probability of success is not equal to 0.5
95 percent confidence interval:
 0.4075652 0.7448590
sample estimates:
probability of success 
             0.5833333 

• A nonparametric alternative to the two-sample \( t \)-test is the Mann-Whitney \( U \)-test.

Definition: The Mann-Whitney \( U \)-test compares the distributions of two groups. It does not require as many assumptions as the two-sample \( t \)-test.

Assignment Problem #32

T and B lymphocytes are normal components of the immune system, but in multiple sclerosis they become autoreactive and attack the central nervous system. What triggers the autoimmune process? One hypothesis is that the disease is initiated by environmental factors, especially microbial infection. However, recent work by Berer et al. (2011) on the mouse model of the disease suggests that the autoimmune process is triggered by nonpathogenic microbes living in the gut.

They compared onset of autoimmune encephalomyelitis in two treatment groups of mice from a strain that carries transgenic human CD4\( ^{+} \) T cells, which initiate the disease. One group (GF) was kept free of nonpathogenic gut microbes and all pathogens. The other (SPF) was only pathogen-free and served as controls. They measured percentage of T cells producing the molecule, interleukin-17, in tissue samples from 16 mice in the two groups.

Look at the data

  treatment percentInterleukin17
1       SPF                18.87
2       SPF                15.65
3       SPF                13.45
4       SPF                12.95
5       SPF                 6.01
6       SPF                 5.84

Discuss: Is this data in tidy or messy format?

Answer: Tidy

Look at data

Discuss: Discuss the data with respect to meeting assumptions of statistical tests.

Look at data

Look at data

library(lattice)
histogram( ~ percentInterleukin17 | treatment, data = mydata, layout = c(1,2), col = "firebrick",  type = "count", xlab = "Percent interleukin-17", ylab = "Frequency")

Mann-Whitney \( U \)-test (Wilcoxon rank-sum test)

\( H_{0} \): The distribution of interleukin-17 is the same in the two groups.
\( H_{A} \): The distribution of interleukin-17 is NOT the same in the two groups.

wilcox.test(percentInterleukin17 ~ treatment, data = mydata)

    Wilcoxon rank sum test

data:  percentInterleukin17 by treatment
W = 6, p-value = 0.004662
alternative hypothesis: true location shift is not equal to 0

Discuss: Conlusion?

The Mann-Whitney \( U \)-test tests if the distributions are the same.

If the distributions of the two groups have the same shape (same variance and skew), then the Mann-Whitney \( U \)-test can be used to compare the locations (means or medians) of the two groups (see Example 13.5).

It is for this reason that this test gets misused a lot in the literature.

• If assumptions of parametric tests are violated, then Type I errors are inflated (leads to false confidence in results).
• Nonparametric tests have less power then parametric tests (throwing away magnitudes and only using ranks).
• When assumptions of two-sample \( t \)-test are met, the Mann-Whitney \( U \)-test has 95% as much power as the two-sample \( t \)-test when samples sizes are large (worse when small).
• Power of Mann-Whitney \( U \)-test is zero when \( n=2 \) (i.e. useless).
• When assumptions of one-sample \( t \)-test are met, the sign test has 64% as much power as the one-sample \( t \)-test when samples sizes are large (worse when small).
• Power of sign test is zero when \( n=5 \).

Definition: A permutation test generates a null distribution for the association between two variables by repeatedly and randomly rearranging the values of one of the two variables in the data.

This is a form of bootstrapping.

In this chapter, we explore a permutation test replacement for the two-sample \( t \)-test.

Variations on this theme can be done for many other tests.

Algorithm

• Choose a test statistic that measures association between the two variables in the data (e.g. \( \bar{Y}_{1}-\bar{Y}_{2} \); Why not \( t \)?)
• Create a permuted sample of data in which the values of the response variables are randomly reordered.
• Calculate the chosen test statistic for the permuted sample.
• Repeat the permutation process many times – at least 1000 or more.
• Compute the \( P \)-value by comparing the observed test statistic (from original data) to this bootstrapped null distribution.

\( H_{0} \): Mean percent interleukin-17 is the same in both groups.
\( H_{A} \): Mean percent interleukin-17 is NOT the same in both groups.

nPerm <- 10000
permResult <- vector() # initialize vector
for(i in 1:nPerm){
    # step 1: permute the percent interleukin-17
    permSample <- sample(mydata$percentInterleukin17, replace = FALSE)
    # step 2: calculate difference betweeen means
    permMeans <- tapply(permSample, mydata$treatment, mean)
    permResult[i] <- permMeans[2] - permMeans[1]
}

M <- tapply(mydata$percentInterleukin17, mydata$treatment, mean)
(tstat <- M[2]-M[1])

    SPF 
8.04875 

\( P \)-value

(pval <- 2*sum(permResult >= tstat)/nPerm)

[1] 0.0024

Reject hypothesis of equal means.