• Recap on t-test
• t-test extensions (Wilcoxon test)
• What is a power test?
• The power t-test
• Plotting data in R (bar plots)
• R demonstration continued

• What is the t-statistic (basics of the formula)?
• What is a t-distribution?
• Independent t-test vs. paired t-test
• one vs. two-tailed t-test
• How does the t-value link to the p-value?

We need a metric (a test statistic!) that puts the difference between the samples into perspective with the standard deviations of the two samples !

This is called the t-statistic in a two sample test:

\[t = \frac{\text{observed difference - expected difference}}{\text{estimate of the standard error}}\]

So the larger the numerator or the smaller the denominator, the larger t will turn out. Think what this means in terms of the p-value!

The formula for calculating a t-value makes sure a t-value becomes larger (and is thus more likely to produce a significant p-value) as…

1. the observed difference between samples becomes larger
2. the estimated standard deviations of the samples become smaller
3. the expected difference between samples becomes smaller
4. all of the above

• The two populations that are compared should follow a normal distribution
• The two samples should be taken independently (e.g. not clustered)
• The variances of the two samples can be unequal (set 'var.equal' argument to 'FALSE', which is the default)

But how can we test for normality of the data?

• Using a test (e.g. the Shapiro-Wilk test: shapiro.test())
• Looking at the so-called 'qqplot' using the function qqnorm()

The null hypothesis of the Shapiro-Wilk test is: 'The data follow a normal distribution'

shapiro.test(test$Mark)

    Shapiro-Wilk normality test

data:  test$Mark
W = 0.9624, p-value = 0.02131

What do we conclude?

Statistical power is the probability to detect a significant difference

• This probability should usually not drop below 80%
• If it does, we have little power to detect a difference, should there be one
• By conducting a pilot study, we can estimate the variation and the expected difference between samples (in the case of a t-test)
• This allows us to conduct a power test and plan the required sample size

There is a trade-off between sample size (or standard deviations), expected difference, type-I error probability (\(\alpha\)) and type-II error probability (\(\beta\)) and the power (\(1-\beta\))!

power.t.test(n = 8, delta = 1, sd = 2, sig.level = 0.05)

     Two-sample t test power calculation 

              n = 8
          delta = 1
             sd = 2
      sig.level = 0.05
          power = 0.1522
    alternative = two.sided

NOTE: n is number in *each* group

Again, sample size, expected difference, standard deviations, \(\alpha\) (type I error probability), and \(\beta\) are traded off, you can't have it all!

What can you do to improve your power in a t-test?

1. You can try to reduce unsystematic variation in order to obtain lower standard deviations
2. You can decrease your \(\alpha\)-level
3. You can reduce the sample size
4. All of the above is correct

Create a simple data frame, one categorical, one continuous variable:

d1 <- data.frame(gender = rep(c('M', 'F'), each = 10),
                      height = c(rnorm(n = 10, mean = 177, sd = 9),
                                 rnorm(10, 160, 9)))
head(d1, 11)
   gender height
1       M  171.3
2       M  180.6
3       M  160.3
4       M  173.5
5       M  165.2
6       M  168.2
7       M  179.3
8       M  192.0
9       M  178.9
10      M  191.5
11      F  168.6

• How to perform a Wilcoxon test (and when it makes sense to do so)
• How to test for normality visually or with a test
• What statistical power is
• How to perform a power-t-test to determine the required sample size
• How to plot the results from a t-test quickly
• How to customise a bar plot using some of the many arguments to the function plot()

• Wilcoxon rank-sum test
• qq-plot (qqnorm(), qqline())
• power, sample size, significance level (\(\alpha\)), type-II error probability (1-\(\beta\))
• boxplot()
• barplot()
• segments()

• Passrate: 87%
• Highest mark: 100%
• Lowest mark: 30%
• Median: 70%
• Mean: 70%
• ANOVA on majors: ns, p=0.5