Hypothesis testing

• Reading assignment for Friday, Whitlock & Schluter, Chapter 7 (QUIZ = YES)
• Timeline - https://qubeshub.org/groups/introbiostats/collections/timeline

Definition: Hypothesis testing compares data to what we would expect to see if a specific null hypothesis were true. If the data are too unusual, compared to what we would expect to see if the null hypothesis were true, then the null hypothesis is rejected.

Definition: A null hypothesis is a specific statement about a population parameter made for the purpose of argument.

Definition: The alternative hypothesis includes all other feasible values for the population parameter besides the value stated in the null hypothesis.

Can parents distinguish their own children by smell alone? To investigate, Porter and Moore (1981) gave new T-shirts to children of nine mothers. Each child wore his or her shirt to bed for three consecutive nights. During the day, from waking until bedtime, the shirts were kept in individually sealed plastic bags. No scented soaps or perfumes were used during the study. Each mother was then given the shirt of her child and that of another, randomly chosen child and asked to identify her own by smell.

Discuss: What is the null hypothesis? alternative hypothesis?

Can parents distinguish their own children by smell alone? To investigate, Porter and Moore (1981) gave new T-shirts to children of nine mothers. Each child wore his or her shirt to bed for three consecutive nights. During the day, from waking until bedtime, the shirts were kept in individually sealed plastic bags. No scented soaps or perfumes were used during the study. Each mother was then given the shirt of her child and that of another, randomly chosen child and asked to identify her own by smell.

Discuss: What is the null hypothesis? alternative hypothesis?

Answer: With \( p \) the probability of choosing correctly,
\[ H_{0}: \ p = 0.5 \] \[ H_{A}: \ p \neq 0.5 \]

Definition: The test statistic is a number calculated from the data that is used to evaluate how compatible the data are with the result expected under the null hypothesis.

Definition: The null distribution is the sampling distribution of outcomes for a test statistic under the assumption that the null hypothesis is true.

Definition: A  \( P \)-value is the probability of obtaining the data (or data showing as great or greater difference from the null hypothesis) if the null hypothesis were true.

• State the hypotheses.
• Compute the test statistic.
• Determine the \( P \)-value.
• Draw the appropriate conclusions.

Can parents distinguish their own children by smell alone? To investigate, Porter and Moore (1981) gave new T-shirts to children of nine mothers. Each child wore his or her shirt to bed for three consecutive nights. During the day, from waking until bedtime, the shirts were kept in individually sealed plastic bags. No scented soaps or perfumes were used during the study. Each mother was then given the shirt of her child and that of another, randomly chosen child and asked to identify her own by smell. Eight of nine mothers identified their children correctly.

Discuss: What test statistic should you use?

Answer: The number of mothers with correct identifications.

So, P = 0.04. Is that good?

https://youtu.be/7jSE3JANx14?t=4m29s

So, P = 0.04. Is that good?

Definition: The significance level, \( \alpha \), is the probability used as a criterion for rejecting the null hypothesis. If the \( P \)-value is less than or equal to \( \alpha \), then the null hypothesis is rejected. If the \( P \)-value is greater than \( \alpha \), then the null hypothesis is not rejected

Definition: A result is considered statistically significant when \( P \)-value \( < \alpha \).

Definition: A result is considered not statistically significant when \( P \)-value \( \geq \alpha \).

Can parents distinguish their own children by smell alone? To investigate, Porter and Moore (1981) gave new T-shirts to children of nine mothers. Each child wore his or her shirt to bed for three consecutive nights. During the day, from waking until bedtime, the shirts were kept in individually sealed plastic bags. No scented soaps or perfumes were used during the study. Each mother was then given the shirt of her child and that of another, randomly chosen child and asked to identify her own by smell. Eight of nine mothers identified their children correctly.

Discuss: Given \( \alpha = 0.05 \), \( \{H_{0}: \ p = 0.5\} \), and \( P \)-value of 0.04, what is the appropriate conclusion?

Answer: Reject \( H_{0} \). There is evidence that mothers consistently identify own children correctly by smell.

“We want to know if results are right, but a p-value doesn’t measure that. It can’t tell you the magnitude of an effect, the strength of the evidence or the probability that the finding was the result of chance.”

Christie Aschwanden

http://fivethirtyeight.com/pvalue

“Belief that "statistical significance” can alone discriminate between truth and falsehood borders on magical thinking.“

Cohen

A Dirty Dozen: Twelve P-Value Misconceptions, by Steven Goodman
    [http://dx.doi.org/10.1053/j.seminhematol.2008.04.003]

Measure and report precision and effect size separately (the \( P \)-value is a summary measure that mixes them):

• Present the magnitude of effect through the use of measures such as rates, risk differences, and odds ratios.
• Report precision with standard errors or confidence intervals.

• Statistical significance is NOT the same as biological importance.
• Effect sizes are important. Large sample sizes can lead to statistically significant results, even though the effect size is small!