Hypothesis testing is a statistical method that helps determine if there is enough evidence to support a specific hypothesis. It is widely used in various fields such as:

• Biology
• Engineering
• Social Sciences

The two types of hypotheses in a hypothesis test are:

• Null Hypothesis (\(H_0\)): Assumes no effect or no difference.
• Alternative Hypothesis (\(H_1\)): Assumes an effect or a difference.

Mathematical Notation

\[ H_0: \mu = \mu_0 \quad \text{vs} \quad H_1: \mu \neq \mu_0 \]

In hypothesis testing, we encounter two types of errors:

• Type I Error: Rejecting \(H_0\) when it is true (\(\alpha\))
• Type II Error: Failing to reject \(H_0\) when it is false (\(\beta\))

\[ P(\text{Type I Error}) = \alpha \quad \text{and} \quad P(\text{Type II Error}) = \beta \]

A test statistic is calculated to compare the observed data to what is expected under the null hypothesis. The p-value represents the probability of observing data at least as extreme as the observed results, given that \(H_0\) is true.

For example, suppose we want to test if there is a significant difference in the mean heights of two groups. Here’s a sample distribution of data from each group.

This plot shows the significance level (\(\alpha\)) needed to achieve a target power at different effect sizes and sample sizes. This visualization helps to understand how effect size and sample size impact the required significance level.

library(plotly)

target_power <- 0.8
effect_sizes <- seq(0.1, 1, by = 0.1)
sample_sizes <- seq(10, 200, by = 10)

alpha_levels <- expand.grid(effect_size = effect_sizes, sample_size = sample_sizes)

# alpha calc
alpha_levels$alpha <- mapply(function(e, n) {
  qnorm(target_power) - e * sqrt(n)
}, alpha_levels$effect_size, alpha_levels$sample_size)

# plot
plot_ly(alpha_levels, x = ~effect_size, y = ~sample_size, z = ~alpha, type = "scatter3d", mode = "markers",
        marker = list(size = 3, color = ~alpha, colorscale = "Viridis")) %>%
  layout(scene = list(xaxis = list(title = "Effect Size"),
                      yaxis = list(title = "Sample Size"),
                      zaxis = list(title = "Required Alpha Level")))