Content you should have understood before watching this video:

• Number 2, ‘Variables’
• Number 3, ‘Variation in data’
• Number 4, ‘Basic statistical metrics’
• Number 5, ‘Standard deviation and standad error’
• Number 6, ‘Populations, samples, hypotheses’
• Number 7, ‘Distributions’
• Number 8, ‘Quantiles and probabilities’

If we want a more quantitative statement on this question

• We need to know whether our sample is ‘unusual’ or ‘normal’?
• We need to know what ‘unusual’ or ‘normal’ means, so
  • we need to quantify what is usual, normal, rare etc.!
• We need a testable hypothesis

So let’s try

• Female AUT students are NOT different in terms of body height from the average New Zealand female (this is our so-called null hypothesis \(H_0\))
• Because we can stick with this hypothesis or reject it, we need an alternative hypothesis \(H_A\): Our students ARE different from a typical sample of New Zealanders
• Note that the Null hypothesis is negative, which makes it easier to falsify!
• Also note that we can never accept \(H_0\), we can only fail to reject \(H_0\)

Now we need a test statistic and knowledge of the distribution we compare against:

Our test statistic is simply the mean of our sample:

mean(d1$bodyheight[d1$sex == 'F']) 
[1] 163.3333

How does that compare with

pop = rnorm(2000000, mean = 164, sd = 6) #why 2000000?
mean(pop)
[1] 164.0014

YES, our female students are typical New Zealanders in terms of body height!

We can tell by just looking at how we compare to the distribution of NZ female bodyheight. More quantitatively…:

The probability of obtaining a value equal or smaller than the mean we got for our female students when sampling from the NZ population is about 50%:

pnorm(q = mean(d1$bodyheight[d1$sex == 'F']) , mean = 164, sd = 6)
[1] 0.4557641

In other words our sample is NOT unusual and hence we cannot reject our null hypothesis!

Our students are NOT different from a typical sample of New Zealanders

OK, but what if our sample mean had been different, say 160 cm, or 150 cm…?

pnorm(q = 160, mean = 164, sd = 6)
[1] 0.2524925

Is a value that we’d get 25% of the time by chance rare?

pnorm(q = 150, mean = 164, sd = 6)
[1] 0.009815329

Is a value that we’d get 1% of the time by chance rare?

• Those probabilities (50%, 25%, 1%) are our p-values. They always are to be interpreted as ‘the probability of obtaining such an extreme value by chance’
• We need a threshold to objectively distinguish from ‘rare’ and ‘unusually rare’:
  • Normally, we say that if our sample is more extreme than what we would find 5% of the time by chance, then our p-value is significant (in a statistical sense)
  • This threshold is also called the \(\alpha\)-threshold

In all of this, we can make 2 types of errors!

• A type I error is when we falsly reject the null hypothesis \(H_0\)
  • In plain language, this means that we call something ‘significant’ (e.g. a difference, 2 samples, etc.) while in reality there is no significant difference (or, more generally ‘nothing going on’)
• A type II error is when we falsly fail to reject the null hypothesis \(H_0\)
  • In plain language, this means that “we don’t see anything where in reality things (e.g. samples) are different”

Note that the ‘plain language’ definitions are inexact, but hopefully help you to understand the principle of type I/II errors

Maybe easier to remember…:

• How to formulate a null and an alternative hypothesis
• The principle of a statistical test, using a simple test statistic, e.g. a mean
• Using that test statistic to make a frequentist statistical decision
• Understand that in this decision, we can make two correct decisions, and two errors, namely a type I and a type II error