2026-02-08

What Is Hypothesis Testing?

  • Statistical method for decision making using data
  • Widely used in UI/UX through A/B testing
  • Helps determine if design changes actually work

UI/UX A/B Testing Example

  • Version A: Old checkout button
  • Version B: New checkout button
  • Metric: Click-through rate (CTR) ## Hypotheses

\[ H_0: p_A = p_B \]

\[ H_1: p_A \neq p_B \]

Test Statistic

\[ z = \frac{\hat{p}_A - \hat{p}_B}{\sqrt{p(1-p)(1/n_A + 1/n_B)}} \]

Simulated Data

Click-Through Rate Plot

R Code: Creating the Click-Through Rate Plot

library(ggplot2)

ctr <- aggregate(clicks ~ version, data, mean)

ggplot(ctr, aes(x = version, y = clicks, fill = version)) +
  geom_col() +
  labs(
    title = "Click-Through Rate by Design Version",
    x = "Design Version",
    y = "Click-Through Rate"
  ) +
  theme_minimal()

Distribution of Clicks

Hypothesis Test in R

## [1] 0.2306148
  • p-value > 0.05
  • We fail to reject the null hypothesis
  • The observed difference may be due to chance

p-value Interpretation

  • Small p-value → strong evidence against H₀
  • In UI/UX, this means the design change likely matters
  • Statistical significance ≠ good user experience

3D UX Performance Scatter

Key Takeaways

-Hypothesis testing supports UX decisions

-A/B testing prevents guesswork

-Statistics improves product outcomes