Choosing the correct test

For this exam there are two different tests we have at our disposal, namely t-tests and ANOVA

  1. t-tests are used when we want to compare the means of two different samples

  2. ANOVAs are used when we want to compare the means of more than two different samples

Each test has different variants, dependent on the type of data and the method in which it was collected In addition, each tests has its own non-normal equivalent for data which are non-parametric.

t-tests

Choose a t-test when only two samples are given or when one sample needs to be compared to a given value (one-sample test)

Before a t-test is conducted, we must make sure the data fits certain assumptions. For a t-test this includes Normality and Equal Variances (Homoscedasity)

To test Normality:

qqnorm(cars$speed)
qqline(cars$speed)

Normality is dependent on how well the two dots fit the line. This data seems to fit it quite well, therefore it can be considered Normal.

If you are unsure, try running a Shapiro-Wilks test, were a p value < 0.05 implies non-normal data

shapiro.test(cars$speed)
## 
##  Shapiro-Wilk normality test
## 
## data:  cars$speed
## W = 0.97765, p-value = 0.4576

The p value is greater than 0.05, therefore you are correct in assuming normality.

You need to check Normality for both samples. If just one sample is non-normal then you must conduct a non-parametric alternative. We will discuss this later.

To test Equal Variances:

x <- c(1,2,1,3,5,3,1,2,5,6,8,6,2,1)
y <- c(1,1,1,3,4,3,1,2,5,5,8,6,2,2)
var.test(x, y, alternative = "two.sided")
## 
##  F test to compare two variances
## 
## data:  x and y
## F = 1.1157, num df = 13, denom df = 13, p-value = 0.8465
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
##  0.3581791 3.4755722
## sample estimates:
## ratio of variances 
##           1.115741

The null hypothesis of the variance test is that ‘variance of x’ = ‘variance of y’. So if p < 0.05 then the variances are unequal. In this example p-value = 0.8465, so our assumption of equal variance is fulfilled.

Performing our t-test: Once we are sure our data meet the above assumptions we can run a t-test.

First, however, we must make some decisions concerning our data.

  1. Is it paired or unpaired?
  • paired data comes from the same sources. Some examples include: the same site, person, or family. Usually, but not always, paired data is take ‘before’ and ‘after’ the treatment.
  • data are paired because we expect there to be some dependency between data sets. In other words, there is likely to be some feature contained within the individual that effects the outcome of the data in each sample.
  • If data are taken from different individuals or sites, or sampled randomly, it is likely to be unpaired.
  1. Is our research question one or two-sided?
  • if we simply want to know whether two samples are significantly different, we use a two-sided t-test (also called two-tailed).
  • if we want to know whether one sample is bigger (upper-tailed) or smaller (lower-tailed) than the other, we use a one-sided t-test

Running the t-test

Run your t-test according to the decisions you made above. A t-test reporting a p value less then 0.05 means samples are significantly different.

Asking if a is bigger the b is the same as asking if b is smaller than a.

Is a bigger than b? (paired, upper-tailed)

a <- c(11,36,11,11,5,3,1,22,5,16,8,6,2,81)
b <- c(1,1,1,3,4,3,1,2,5,5,8,6,2,2)
t.test(a, b , paired = TRUE, alternative = "greater")
## 
##  Paired t-test
## 
## data:  a and b
## t = 2.1487, df = 13, p-value = 0.02554
## alternative hypothesis: true mean difference is greater than 0
## 95 percent confidence interval:
##  2.18492     Inf
## sample estimates:
## mean difference 
##        12.42857

Is a bigger than b? (unpaired, upper-tailed)

a <- c(11,36,11,11,5,3,1,22,5,16,8,6,2,81)
b <- c(1,1,1,3,4,3,1,2,5,5,8,6,2,2)
t.test(a, b , paired = FALSE, alternative = "greater")
## 
##  Welch Two Sample t-test
## 
## data:  a and b
## t = 2.2033, df = 13.28, p-value = 0.0229
## alternative hypothesis: true difference in means is greater than 0
## 95 percent confidence interval:
##  2.455042      Inf
## sample estimates:
## mean of x mean of y 
## 15.571429  3.142857

Is a different to b? (paired, two-tailed)

a <- c(11,36,11,11,5,3,1,22,5,16,8,6,2,81)
b <- c(1,1,1,3,4,3,1,2,5,5,8,6,2,2)
t.test(a, b , paired = TRUE, alternative = "two.sided")
## 
##  Paired t-test
## 
## data:  a and b
## t = 2.1487, df = 13, p-value = 0.05108
## alternative hypothesis: true mean difference is not equal to 0
## 95 percent confidence interval:
##  -0.06769983 24.92484269
## sample estimates:
## mean difference 
##        12.42857

Is a different to b? (unpaired, two-tailed)

a <- c(11,36,11,11,5,3,1,22,5,16,8,6,2,81)
b <- c(1,1,1,3,4,3,1,2,5,5,8,6,2,2)
t.test(a, b , paired = FALSE, alternative = "two.sided")
## 
##  Welch Two Sample t-test
## 
## data:  a and b
## t = 2.2033, df = 13.28, p-value = 0.0458
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##   0.2682725 24.5888704
## sample estimates:
## mean of x mean of y 
## 15.571429  3.142857

Running the t-test for non-normal data

If you discover than one of your samples does not meet a required assumption, you must run a Wilcoxen Signed Rank test. In R, you simply replace the function t.test() with wilcox.test()

For example:

#t.test(a, b , paired = TRUE, alternative = "two.sided")

#becomes..
wilcox.test(a, b , paired = TRUE, alternative = "two.sided")
## Warning in wilcox.test.default(a, b, paired = TRUE, alternative = "two.sided"):
## cannot compute exact p-value with ties
## Warning in wilcox.test.default(a, b, paired = TRUE, alternative = "two.sided"):
## cannot compute exact p-value with zeroes
## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  a and b
## V = 36, p-value = 0.01415
## alternative hypothesis: true location shift is not equal to 0

A p value < 0.05 means samples are significantly different.