In statistics, the p-value is a number describing the likelihood of obtaining the observed data under the null hypothesis of a statistical test.

A smaller p-value provides stronger evidence against the null hypothesis.

Key Topics:

• Null Hypothesis: it is assumed there is no change until there is enough statistical evidence to reject it

• Alternative Hypothesis: proposes that the relationship does exist, contradicting the null hypothesis

• Significance Level: probability of incorrectly rejecting a true null hypothesis

The p-value for a one-tailed test measures the probability of observing results in one direction when the null hypothesis is true:

\[ P = P(T \geq t \mid H_0) \]

The p-value for a two-tailed test measures the porbability of observing results in both directions: \[ P = 2 \times \min[P(T \geq t \mid H_0), P(T \leq t \mid H_0)] \]

• \(T\) is the test statistic

• \(t\) is the observed value of the test statistic

• \(H_0\) is the null hypothesis

The decision rule is determined by: \[ \text{If } p \leq \alpha, \text{ reject } H_0 \]

A One-Tailed test is testing for the possibility of the relationship in one direction and completely disregarding the possibility of a relationship in the other direction. It is appropriate to use when the consequences of not testing in the other direction have been acknowledged and are negligible.

One-Tailed Test (\(\alpha = 0.05\)): \[ P(\text{Type I Error}) = \alpha = 0.05 \text{ (all in one direction)} \] A Two-Tailed test is testing for the possibility of the relationship in both directions. It is appropriate to use when both directions have effects that are meaningful to the results.

Two-Tailed Test (\(\alpha = 0.05\)): \[ P(\text{Type I Error}) = \alpha/2 + \alpha/2 = 0.025 + 0.025 \]

Larger sample sizes increase the ability to detect real effects. The same underlying difference becomes statistically significant with more data.

The red shaded areas represent the rejection regions where we would reject the null hypothesis if our test statistic falls in these regions.

This 3D surface shows how p-values depend on both effect size and sample size. Larger effects and larger samples both lead to smaller, more significant p-values.

Earlier, we made a plot with a two-tailed test. Here, I will show the code for the same plot with a one-tailed test:

# normal distribution data
x = seq(-15, 15, length.out = 1000)
y = dnorm(x)
df = data.frame(x=x, y=y)

# critical value for (right-tailed) with alpha = 0.05
critical_value = qnorm(0.95)  # 95th percentile

# create the plot
ggplot(df, aes(x = x, y = y)) +
  # plotting the curve
  geom_line(color = "navy", linewidth = 1.2) +
  # plotting critical section
  geom_area(data = df %>% filter(x >= critical_value),
            aes(x = x, y = y), fill = "red", alpha = 0.6) +
  # dashed lines to distinguish critical area
  geom_vline(xintercept = critical_value,
             linetype = "dashed", color = "darkred", linewidth = 1) +
  # labels
  labs(title = "One-Tailed Test (Right-Tailed): P-Value Region (α = 0.05)",
       x = "Test Statistic",
       y = "Probability Density") +
  theme_minimal()