1 Glossary of Key Terms

Term Definition
Non-parametric A statistical test that does NOT assume the data follow a normal distribution.
Rank The position of a value when all values are sorted from smallest to largest.
Median The middle value of a sorted dataset — used instead of the mean in non-parametric tests.
Distribution The pattern of how data values are spread out.
Skewed data Data with a long tail on one side — not symmetric like a bell curve.
Outlier A value that is far away from the rest of the data.
Wilcoxon test A non-parametric alternative to the t-test, based on ranked data.
Mann-Whitney U test The non-parametric alternative to the independent t-test.
Kruskal-Wallis test The non-parametric alternative to one-way ANOVA.
Spearman correlation A rank-based measure of how two variables move together.
Sign test A simple test of whether values tend to be above or below a target.
Tied ranks When two or more values are equal and share a rank.

That’s all the new vocabulary you need. Let’s get started.

2 What Does “Non-Parametric” Mean?

Non-parametric means that a statistical test does not assume the data follow a specific shape — like the bell curve. Instead of working with the raw numbers directly, non-parametric tests convert the data into ranks (1st, 2nd, 3rd …) and analyse those ranks.

This makes them safer to use when:

  • Your sample is small (fewer than ~20 observations per group)
  • The data are clearly skewed (long tail on one side)
  • There are outliers that would distort a mean
  • Your data are ordinal (e.g. survey ratings: 1–5)
par(mfrow = c(1, 2))

# Normal-looking data
set.seed(1)
normal_data <- rnorm(200, mean = 50, sd = 10)
hist(normal_data, col = "#2a9d8f", border = "white",
     main = "Normal Data\n(parametric tests safe)",
     xlab = "Value", breaks = 20)

# Skewed data (log-normal)
skewed_data <- rlnorm(200, meanlog = 3.5, sdlog = 0.9)
hist(skewed_data, col = "#e76f51", border = "white",
     main = "Skewed Data\n(use non-parametric tests)",
     xlab = "Value", breaks = 30)

par(mfrow = c(1, 1))

The left histogram is roughly bell-shaped — parametric tests are appropriate. The right histogram has a long right tail — a sign that non-parametric tests will give more reliable results.

2.1 How Ranking Works

Before any non-parametric test runs, the raw values are replaced by their ranks:

# Raw values
raw <- c(14, 3, 27, 8, 3, 19)

# Ranks (ties get the average of their shared positions)
ranked <- rank(raw)

data.frame(Raw_Value = raw, Rank = ranked)

Notice that the two 3s share positions 1 and 2, so they both get rank 1.5 (the average of 1 and 2). Once the data are converted to ranks, the test proceeds — and outliers lose much of their power to distort the result.

2.2 Which Test Do I Need?

Here is the same decision diagram as Lesson 1, but for non-parametric data:

Each box above is the rank-based counterpart of the parametric test you learned in Lesson 1. The logic is identical — only the method changes.

3 Test 1: Wilcoxon Signed-Rank Test (One Sample)

When to use it: you have one group of numbers and want to know if their median is different from some specific target value — the non-parametric version of the one-sample t-test.

Example: Is the median sepal length of setosa iris flowers different from 5.0 cm?

data(iris)
setosa_length <- iris$Sepal.Length[iris$Species == "setosa"]

wilcox.test(setosa_length, mu = 5.0)
## Warning in wilcox.test.default(setosa_length, mu = 5): cannot compute exact
## p-value with ties
## Warning in wilcox.test.default(setosa_length, mu = 5): cannot compute exact
## p-value with zeroes
## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  setosa_length
## V = 453.5, p-value = 0.985
## alternative hypothesis: true location is not equal to 5
hist(setosa_length, col = "#a8dadc", border = "white",
     main = "Setosa Sepal Length vs. Hypothesized Median",
     xlab = "Sepal Length (cm)")
abline(v = 5.0,                       col = "red",       lwd = 2, lty = 2)
abline(v = median(setosa_length),     col = "darkgreen", lwd = 2)
legend("topright",
       legend = c("Hypothesized median (5.0)", "Sample median"),
       col    = c("red", "darkgreen"),
       lty    = c(2, 1), lwd = 2, bty = "n")

Reading the output: the p-value tests whether the sample median (green line) is far enough from 5.0 (red dashed line) to be surprising. A p-value below 0.05 would mean the true median is unlikely to be 5.0 cm.

4 Test 2: Mann-Whitney U Test (Two Independent Groups)

When to use it: you have two separate, unrelated groups and want to compare their distributions — the non-parametric version of the independent two-sample t-test.

Example: Do versicolor and virginica iris flowers differ in sepal length, without assuming normality?

iris2 <- subset(iris, Species %in% c("versicolor", "virginica"))
wilcox.test(Sepal.Length ~ Species, data = iris2)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  Sepal.Length by Species
## W = 526, p-value = 5.869e-07
## alternative hypothesis: true location shift is not equal to 0
par(mfrow = c(1, 2))

# Boxplot showing medians
boxplot(Sepal.Length ~ Species, data = iris2,
        col = c("#e76f51", "#2a9d8f"),
        main = "Sepal Length by Species\n(Medians shown)",
        xlab = "Species", ylab = "Sepal Length (cm)")

# Stripchart showing individual points
stripchart(Sepal.Length ~ Species, data = iris2,
           method = "jitter", jitter = 0.1,
           pch = 19, col = c("#e76f51", "#2a9d8f"),
           vertical = TRUE,
           main = "Individual Data Points",
           xlab = "Species", ylab = "Sepal Length (cm)")

par(mfrow = c(1, 1))

Reading the output: the Mann-Whitney U test (wilcox.test() with two groups) asks: if we randomly pick one value from each group, how often is the versicolor value larger? A p-value far below 0.05 — as here — means the two groups clearly differ.

5 Test 3: Wilcoxon Signed-Rank Test (Paired)

When to use it: you have two measurements on the same subjects — before/after, drug A vs. drug B — and the data are skewed or the sample is small. This is the non-parametric version of the paired t-test.

Example: The sleep dataset records extra hours of sleep for 10 patients under two drugs. We test the difference without assuming normality.

data(sleep)
sleep_wide <- reshape(sleep, idvar = "ID", timevar = "group",
                      direction = "wide")

wilcox.test(sleep_wide$extra.1, sleep_wide$extra.2, paired = TRUE)
## Warning in wilcox.test.default(sleep_wide$extra.1, sleep_wide$extra.2, paired =
## TRUE): cannot compute exact p-value with ties
## Warning in wilcox.test.default(sleep_wide$extra.1, sleep_wide$extra.2, paired =
## TRUE): cannot compute exact p-value with zeroes
## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  sleep_wide$extra.1 and sleep_wide$extra.2
## V = 0, p-value = 0.009091
## alternative hypothesis: true location shift is not equal to 0
par(mfrow = c(1, 2))

# Slopegraph: same patient, two drugs
pair_mat <- rbind(sleep_wide$extra.1, sleep_wide$extra.2)
matplot(pair_mat, type = "b", pch = 19, lty = 1, col = "grey50",
        xaxt = "n", xlim = c(0.8, 2.2),
        xlab = "Drug", ylab = "Extra Hours of Sleep",
        main = "Each Line = One Patient")
axis(1, at = 1:2, labels = c("Drug 1", "Drug 2"))

# Distribution of differences
diffs <- sleep_wide$extra.2 - sleep_wide$extra.1
hist(diffs, col = "#c77dff", border = "white", breaks = 6,
     main = "Differences (Drug 2 − Drug 1)",
     xlab = "Extra Hours Gained")
abline(v = 0, col = "red", lty = 2, lwd = 2)

par(mfrow = c(1, 1))

Reading the output: the signed-rank test looks at the direction and magnitude of each patient’s difference. Most lines slope upward (Drug 2 gives more sleep), matching the small p-value. The histogram of differences confirms most values are positive (right of the red zero line).

6 Test 4: Kruskal-Wallis Test (3+ Groups)

When to use it: you have three or more independent groups and normality cannot be assumed — the non-parametric version of one-way ANOVA.

Example: Do all three iris species differ in sepal length, using a rank-based test?

kruskal.test(Sepal.Length ~ Species, data = iris)
## 
##  Kruskal-Wallis rank sum test
## 
## data:  Sepal.Length by Species
## Kruskal-Wallis chi-squared = 96.937, df = 2, p-value < 2.2e-16
par(mfrow = c(1, 2))

boxplot(Sepal.Length ~ Species, data = iris,
        col = c("#f4a261", "#2a9d8f", "#264653"),
        main = "Sepal Length by Species",
        xlab = "Species", ylab = "Sepal Length (cm)")

# Visualise the ranks
iris$rank_sl <- rank(iris$Sepal.Length)
boxplot(rank_sl ~ Species, data = iris,
        col = c("#f4a261", "#2a9d8f", "#264653"),
        main = "Ranked Sepal Length by Species",
        xlab = "Species", ylab = "Rank")

par(mfrow = c(1, 1))

Reading the output: find Kruskal-Wallis chi-squared and p-value. A tiny p-value — as here — tells you at least one species has a different median rank from the others. Just like ANOVA, this test only says that a difference exists, not which pair differs. Follow up with pairwise Wilcoxon tests to find out which pairs:

pairwise.wilcox.test(iris$Sepal.Length, iris$Species,
                     p.adjust.method = "bonferroni")
## 
##  Pairwise comparisons using Wilcoxon rank sum test with continuity correction 
## 
## data:  iris$Sepal.Length and iris$Species 
## 
##            setosa  versicolor
## versicolor 2.5e-13 -         
## virginica  < 2e-16 1.8e-06   
## 
## P value adjustment method: bonferroni

The Bonferroni correction guards against false positives when making multiple comparisons at once. Adjusted p-values below 0.05 indicate pairs that genuinely differ.

7 Test 5: Spearman Rank Correlation

When to use it: you have two numeric variables and want to know if they move together — but the relationship may not be a straight line, or the data are skewed. The non-parametric version of Pearson correlation.

Example: Do heavier cars get worse gas mileage? (Same question as Lesson 1, now without the linearity assumption.)

cor.test(mtcars$mpg, mtcars$wt, method = "spearman")
## Warning in cor.test.default(mtcars$mpg, mtcars$wt, method = "spearman"): Cannot
## compute exact p-value with ties
## 
##  Spearman's rank correlation rho
## 
## data:  mtcars$mpg and mtcars$wt
## S = 10292, p-value = 1.488e-11
## alternative hypothesis: true rho is not equal to 0
## sample estimates:
##       rho 
## -0.886422
par(mfrow = c(1, 2))

# Raw scatter (same as Lesson 1)
plot(mtcars$wt, mtcars$mpg, pch = 19, col = "steelblue",
     xlab = "Weight (1000 lbs)", ylab = "Miles per Gallon",
     main = "Raw Data")
abline(lm(mpg ~ wt, data = mtcars), col = "red", lwd = 2)

# Ranked scatter (what Spearman actually uses)
plot(rank(mtcars$wt), rank(mtcars$mpg),
     pch = 19, col = "#c77dff",
     xlab = "Rank of Weight", ylab = "Rank of MPG",
     main = "Ranked Data (Spearman)")
abline(lm(rank(mpg) ~ rank(wt), data = mtcars), col = "red", lwd = 2)

par(mfrow = c(1, 1))

Reading the output: the rho (ρ) value ranges from −1 to +1, just like Pearson’s r. Here it’s strongly negative: as weight rank goes up, MPG rank goes down. The right plot shows the ranked data — notice the relationship is even cleaner as a straight line after ranking, because Spearman captures any consistent increase or decrease, not just a linear one.

8 Bonus: Generating Your Own Skewed Example Data

Just as in Lesson 1, generating controlled data helps you see a clean result free from real-world noise. For non-parametric lessons, the trick is to simulate skewed or small samples where parametric tests would be unreliable.

8.1 A Skewed Two-Group Comparison

set.seed(123)
# Log-normal data: always positive and right-skewed
group_a <- rlnorm(25, meanlog = 3.0, sdlog = 0.5)
group_b <- rlnorm(25, meanlog = 3.6, sdlog = 0.5)

# Parametric test (not ideal here)
t.test(group_a, group_b)
## 
##  Welch Two Sample t-test
## 
## data:  group_a and group_b
## t = -4.3994, df = 37.472, p-value = 8.68e-05
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -30.00857 -11.08864
## sample estimates:
## mean of x mean of y 
##  22.07971  42.62831
# Non-parametric test (more appropriate)
wilcox.test(group_a, group_b)
## 
##  Wilcoxon rank sum exact test
## 
## data:  group_a and group_b
## W = 94, p-value = 7.668e-06
## alternative hypothesis: true location shift is not equal to 0
par(mfrow = c(1, 2))
hist(group_a, col = rgb(0.2, 0.5, 0.8, 0.6), xlim = c(0, 120),
     main = "Group A vs. Group B\n(Skewed)", xlab = "Value", breaks = 15)
hist(group_b, col = rgb(0.9, 0.4, 0.2, 0.6), add = TRUE, breaks = 15)
legend("topright",
       legend = c("Group A", "Group B"),
       fill = c(rgb(0.2, 0.5, 0.8, 0.6), rgb(0.9, 0.4, 0.2, 0.6)),
       bty = "n")
boxplot(group_a, group_b, names = c("A", "B"),
        col = c("#457b9d", "#e76f51"),
        main = "Boxplots (medians shown)", ylab = "Value")

par(mfrow = c(1, 1))

Both tests detect the difference here, but with skewed data the Wilcoxon p-value is more trustworthy — it is not distorted by the long right tail the way the t-test is.

8.2 A Small-Sample Paired Comparison

set.seed(42)
# Only 8 pairs — too few for normality to be assumed
before <- c(12, 18, 9, 24, 15, 11, 20, 8)
after  <- before + sample(c(-1, 0, 2, 4, 6), 8, replace = TRUE)

wilcox.test(before, after, paired = TRUE)
## Warning in wilcox.test.default(before, after, paired = TRUE): cannot compute
## exact p-value with ties
## Warning in wilcox.test.default(before, after, paired = TRUE): cannot compute
## exact p-value with zeroes
## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  before and after
## V = 6, p-value = 0.7835
## alternative hypothesis: true location shift is not equal to 0
matplot(rbind(before, after), type = "b", pch = 19, lty = 1, col = "grey40",
        xaxt = "n", xlim = c(0.8, 2.2),
        xlab = "Time", ylab = "Value",
        main = "Small Paired Sample (each line = one subject)")
axis(1, at = 1:2, labels = c("Before", "After"))

With only 8 subjects, the Wilcoxon signed-rank test is the safe choice. It asks: do the differences between pairs consistently point in one direction? Most lines slope upward, supporting a real effect.

8.3 A Skewed Three-Group Comparison

set.seed(7)
sim_data <- data.frame(
  value = c(rlnorm(20, 2.5, 0.4), rlnorm(20, 3.0, 0.4), rlnorm(20, 3.5, 0.4)),
  group = factor(rep(c("Low", "Medium", "High"), each = 20))
)

kruskal.test(value ~ group, data = sim_data)
## 
##  Kruskal-Wallis rank sum test
## 
## data:  value by group
## Kruskal-Wallis chi-squared = 26.229, df = 2, p-value = 2.016e-06
boxplot(value ~ group, data = sim_data,
        col = c("#a8dadc", "#457b9d", "#1d3557"),
        main = "Skewed Three-Group Comparison", xlab = "Group", ylab = "Value")

9 Parametric vs. Non-Parametric: Side-by-Side

Situation Parametric test (Lesson 1) Non-parametric test (Lesson 2)
1 group vs. a target One-sample t-test Wilcoxon signed-rank
2 independent groups Independent t-test Mann-Whitney U
2 paired measurements Paired t-test Wilcoxon signed-rank (paired)
3+ independent groups One-way ANOVA Kruskal-Wallis
2 numeric variables Pearson correlation Spearman correlation

Key difference: parametric tests use means and assume a normal distribution. Non-parametric tests use medians and ranks — no shape assumption required.

10 Beginner Cheat Sheet

I want to… Use R function
Compare one group’s median to a target Wilcoxon signed-rank wilcox.test(x, mu = ...)
Compare two separate groups (skewed / small) Mann-Whitney U wilcox.test(y ~ group)
Compare before/after on the same subjects Paired Wilcoxon wilcox.test(x, y, paired = TRUE)
Compare 3+ separate groups Kruskal-Wallis kruskal.test(y ~ group)
See if two variables move together (any shape) Spearman correlation cor.test(x, y, method = "spearman")

Golden rule: when in doubt, check normality first (shapiro.test()). If p > 0.05 and your sample is large enough, the parametric test from Lesson 1 is fine. If your data are clearly skewed, your sample is small, or you have strong outliers — use the non-parametric test from this lesson.

11 Practice Question

Try this yourself: The built-in ToothGrowth dataset records tooth length (len) in guinea pigs given two supplement types (supp: OJ or VC). The sample is small and the data can be skewed.

  1. Check normality for each supplement group with shapiro.test().
  2. If normality is in doubt, run the appropriate non-parametric test from this lesson to compare the two groups.
# Hints:
data(ToothGrowth)
shapiro.test(ToothGrowth$len[ToothGrowth$supp == "OJ"])
shapiro.test(ToothGrowth$len[ToothGrowth$supp == "VC"])
boxplot(len ~ supp, data = ToothGrowth)
# Two independent groups -> which test from this lesson fits?
wilcox.test(len ~ supp, data = ToothGrowth)