• A fundamental concept in statistics that stands for probability value
• Formally Defined as:
  • The probability of obtaining test results at least as extreme as the result actually observed, under the assumption that the null hypothesis is correct
• More Simply put:
  • It is the likelihood that the data supports the null hypothesis
  • Another way to envision it: Assuming our null (base) hypothesis is true, what is the likelihood that there is evidence against it
    • high p - value indicates the data would likely support the null hypothesis
    • low p - value would not support the null or rather would hint at the support of the alternate hypothesis

Definitions referenced from: https://wmed.edu/sites/default/files/P-VALUES%20SIMPLIFIED.pdf

Before computing p-values, we need to form the hypotheses we are trying to support or refute.

• Null Hypothesis (\(H_0\)): Our default assumption, or base statement. It’s the hypothesis we’re trying to find evidence against.
  • Example: The average length of a certain tree’s branch is 15 cm.
  • \[H_0: \mu = 15\]
• Alternative Hypothesis (\(H_a\) OR \(H_1\)): What we believe might be true instead.
  • Example: The average length is NOT 15 cm.
  • \[H_a: \mu \neq 15\]
    
  • Usually what we hope to accept in most practice problems or research

Suppose \(T\) is the test statistic we are looking at in this case and \(t_{obs}\) shall be the observed value.

For a one-sided test (greater than in this case):

\[\text{p-value} = P(T \ge t_{obs} | H_0 \text{ is true})\] Translating this into English, you flip the order you read the equation in: Assuming our null hypothesis is true (corresponds to after “|” in the equation), what is the probability that our test statistic is higher than the observed value?

For a two-sided test:

\[\text{p-value} = 2 \times P(T \ge |t_{obs}| | H_0 \text{ is true})\]

With the inclusion of the absolute value in the equation, it works for both ends of a symmetric distribution curve and thus is being multiplied by 2

Let’s assume we have a z-statistic of 1.96 calculated from a one-tailed test. The p-value is the shaded area under the curve to the right of our statistic. If it were to be -1.96, The area would be reflected across z-score = 0 and end up on the left side of the graph. That would correlate to a less-than one-sided test instead.

Now that we have it, what does it do for us?

1. A significance level (\(\alpha\)) is chosen BEFORE we conduct the test. This is our threshold for the disproving evidence. The most common choices are 0.05, 0.025, and 0.1.
2. We compare our p-value to \(\alpha\).

-If p-value \(\le \alpha\): We REJECT the null hypothesis (\(H_0\)). The result is deemed statistically significant. Thus, we have enough evidence to say our data was highly unlikely to occur by chance alone under the assumption of a true null hypothesis, providing evidence to reject it. In our example, the p value is less than or equal meaning that we do reject the null hypothesis and accept the alternate (tree branch length is on average NOT 15 cms)

-If p-value \(> \alpha\): We FAIL TO REJECT the null hypothesis (\(H_0\)). The result is not statistically significant. We don’t have enough evidence to reject the default assumption. In other words, it is possible our data could have occurred by chance under a true null hypothesis leaving us with no basis to refute it.

Here is the ggplot2 code used to create the visualization for the one-tail test:

x <- seq(-4, 4, length.out = 1000)
y <- dnorm(x)
df <- data.frame(x, y)

ggplot(df, aes(x, y)) +
  geom_line() +
  geom_area(
            data = subset(df, x >= 1.96), aes(y = y), 
            fill = "red", alpha = 0.5
            ) +
  labs(
       title = "P-value for a One-Tailed Test (z = 1.96)",
       x = "Z-statistic",
       y = "Density (No proper label for this required)"
       ) +
  annotate(
           "text", x = 3, y = 0.05, 
           label = "p-value\n(Area = 0.025)", 
           color = "red"
           ) +
  theme_minimal()

For a two-tailed test, we’re interested in extreme values in BOTH direction. Note the absolute value for the z score. The p-value is the SUM of the areas in both tails.

This shows how the p-value changes as the sample size increases, assuming the the true difference between groups or effect size is constant. This is regards to a t-test that makes use of true differences between groups rather than one data set.

Notice that with more data (ie. larger sample sizes incremented by 5), we are more likely to get statistically significant results (p-values > 0.05).

• FALSE: The p-value is the probability that the null hypothesis is true.
  • TRUE: The p-value is calculated ASSUMING the null hypothesis is true. The probability of that assumption being correct is completely unknown to us.
• FALSE: A high p-value proves that the null hypothesis is true.
  • TRUE: A high p-value only means we LACK sufficient evidence to REJECT the null. We cannot make the decision that our baseline is false (essentially, not a complete opposite). To say it in another way, not having proof does not mean that we have evidence nothing is there.
• FALSE: A p-value of 0.05 means there is a 5% chance of making a mistake.
  • TRUE: That value translates to if our null hypothesis were true, there would be a 5% chance of getting a sample result at least as extreme as the one just seen.

• The p-value measures the strength of evidence against a null hypothesis (\(H_0\)).
• It’s the probability of getting your observed result (or more extreme) if \(H_0\) were true.
• A small p-value (e.g., \(\le 0.05\) OR \(\le 0.025\)) is an extremely low chance to get that sample result again within the given significance level (\(\alpha\)), leading us to reject \(H_0\).
• A large p-value means our result is consistent with \(H_0\), so we fail to reject it.
• P-value in the context of your scenario along with sample size and effect size is very significant and can lead to major decisions being taken