Non-Parametric Hypothesis Test: Sign Test

Introduction

The Sign Test is a simple nonparametric method used to test hypotheses about: - The median in a one-sample setup. - The median difference in a paired setup.

It is particularly useful when: - Data are ordinal or unsuitable for parametric methods. - Quantitative measurements may be infeasible or prone to error. - The sample size is small, or normality assumptions are questionable.

Unlike parametric tests (e.g., the paired t-test), the Sign Test only considers the sign of the differences, making it robust against outliers and non-normal distributions.

1. Sign Test for a Single Sample

Theory

Hypothesis (One-Sample Case)

Given a sample:

\[ \{X_1, X_2, \dots, X_n\} \]

we test whether the true median \(m\) equals some hypothesized value \(m_0\):

Null Hypothesis (\(H_0\)): \(m = m_0\)
Alternative Hypothesis (\(H_A\)): \(m \neq m_0\) (two-sided test)

Test Statistic

We define:

\(S^+\) = Number of positive differences, where \(X_i - m_0 > 0\).
\(S^-\) = Number of negative differences, where \(X_i - m_0 < 0\).

Observations where \(X_i - m_0 = 0\) are discarded.

Under \(H_0\), each observation has an equal probability (\(0.5\)) of being above or below \(m_0\). Thus, \(S^+\) follows a Binomial distribution:

\[ S^+ \sim \text{Binomial}(n^*, 0.5) \]

where \(n^*\) is the number of non-zero differences.

Example: Simulated Data (One-Sample)

Scenario

Suppose we suspect that the median weight of a certain population is 70 kg. We collect a small sample of 12 individuals and measure their weights.

Sample R Code

# Simulated sample of weights (kg)
set.seed(123)
weights <- c(68, 72, 71, 69, 70, 74, 75, 66, 71, 73, 67, 69)

# We want to test if the median = 70
hypothesized_median <- 70

# Use DescTools::SignTest
sign_test_one_sample <- SignTest(weights, mu = hypothesized_median)

# Print results
sign_test_one_sample

## 
##  One-sample Sign-Test
## 
## data:  weights
## S = 6, number of differences = 11, p-value = 1
## alternative hypothesis: true median is not equal to 70
## 96.1 percent confidence interval:
##  68 73
## sample estimates:
## median of the differences 
##                      70.5

Interpretation

If p-value < 0.05, we reject H₀:

H₀ → the median is significantly different from 70.

If p-value > 0.05, we fail to reject H₀:

H₀ → there is no evidence that the median differs from 70.

df_weights <- data.frame(Weight = weights)
ggplot(df_weights, aes(x = Weight)) +
  geom_histogram(bins = 8, fill = "steelblue", alpha = 0.8, color = "black") +
  geom_vline(xintercept = hypothesized_median, color = "red", size = 1.2) +
  labs(title = "Histogram of Weights", x = "Weight (kg)", y = "Frequency") +
  theme_minimal()

### 2. Sign Test for Paired Data

Theory

The Sign Test for Paired Data is used to determine whether the median difference between two paired variables is zero.

Hypothesis (Paired Case)

Given paired observations:

\[ \{(X_1, Y_1), (X_2, Y_2), \dots, (X_n, Y_n)\} \]

we define the difference:

\[ D_i = X_i - Y_i \]

We then test whether the median difference is zero:

Null Hypothesis (\(H_0\)): \(\text{median}(D) = 0\)
Alternative Hypothesis (\(H_A\)): \(\text{median}(D) \neq 0\) (two-sided test)

Test Statistic

We examine the sign of each difference \(D_i\):

\(S^+\) = Number of positive differences.
\(S^-\) = Number of negative differences.
Observations where \(D_i = 0\) are discarded.

Under \(H_0\), each nonzero difference is equally likely to be positive or negative, meaning:

\[ S^+ \sim \text{Binomial}(n^*, 0.5) \]

where \(n^*\) is the number of nonzero differences.

Example: Simulated Paired Data

Scenario

A company wants to compare two production methods (Method A vs. Method B) by measuring the assembly time (in minutes) for 10 parts using both methods.

set.seed(2023)
# Simulate paired times
methodA <- c(12.3, 11.8, 14.0, 15.2, 10.9, 13.1, 12.7, 14.5, 13.0, 15.0)
methodB <- methodA - rnorm(10, mean = 0.8, sd = 0.5)  # Usually B is faster by ~0.8 min

# Display a portion of data
df_methods <- data.frame(
  Part = 1:10,
  TimeA = methodA,
  TimeB = methodB,
  Difference = methodA - methodB
)
kable(df_methods, caption = "Paired Assembly Times: Method A vs. Method B")

Paired Assembly Times: Method A vs. Method B
Part	TimeA	TimeB	Difference
1	12.3	11.54189	0.7581078
2	11.8	11.49147	0.3085281
3	14.0	14.13753	-0.1375337
4	15.2	14.49307	0.7069277
5	10.9	10.41674	0.4832572
6	13.1	11.75460	1.3453987
7	12.7	12.35686	0.3431364
8	14.5	13.19918	1.3008199
9	13.0	12.39963	0.6003667
10	15.0	14.43406	0.5659385

Now we perform the Sign Test on these paired differences:

sign_test_paired <- SignTest(methodA, methodB)
sign_test_paired

## 
##  Dependent-samples Sign-Test
## 
## data:  methodA and methodB
## S = 9, number of differences = 10, p-value = 0.02148
## alternative hypothesis: true median difference is not equal to 0
## 97.9 percent confidence interval:
##  0.3085281 1.3008199
## sample estimates:
## median of the differences 
##                 0.5831526

Interpretation

If p-value < 0.05, we reject H₀:

H₀ → the median is significantly different from 70.

If p-value > 0.05, we fail to reject H₀:

H₀ → there is no evidence that the median differs from 70.

df_methods$DiffSign <- ifelse(df_methods$Difference > 0, "Positive", "Negative")

ggplot(df_methods, aes(x = Part, y = Difference, fill = DiffSign)) +
  geom_bar(stat = "identity") +
  geom_hline(yintercept = 0, color = "red", size = 1) +
  labs(title = "Differences (Method A - Method B)", y = "Time Difference (min)") +
  scale_fill_manual(values = c("Positive" = "steelblue", "Negative" = "tomato")) +
  theme_minimal()

3. Example with a Real Dataset: `sleep` in R

The built-in sleep dataset in R has 10 patients, each receiving two treatments for extra hours of sleep. A paired sign test can be used to see if there’s a median difference in sleep improvement between the two treatments.

# Load the built-in 'sleep' dataset
data("sleep")

# 'sleep' has "extra" (extra hours of sleep), "group" (treatment 1 or 2),
# and "ID" (subject ID). We need a wide format for paired differences.
sleep_wide <- reshape(sleep, timevar = "group", idvar = "ID", direction = "wide")

# Check the wide format
kable(sleep_wide, caption = "Sleep Data in Wide Format (extra.1 vs. extra.2)")

Sleep Data in Wide Format (extra.1 vs. extra.2)
ID	extra.1	extra.2
1	0.7	1.9
2	-1.6	0.8
3	-0.2	1.1
4	-1.2	0.1
5	-0.1	-0.1
6	3.4	4.4
7	3.7	5.5
8	0.8	1.6
9	0.0	4.6
10	2.0	3.4

sign_test_sleep <- SignTest(sleep_wide$extra.1, sleep_wide$extra.2)
sign_test_sleep

## 
##  Dependent-samples Sign-Test
## 
## data:  sleep_wide$extra.1 and sleep_wide$extra.2
## S = 0, number of differences = 9, p-value = 0.003906
## alternative hypothesis: true median difference is not equal to 0
## 97.9 percent confidence interval:
##  -2.4 -0.8
## sample estimates:
## median of the differences 
##                      -1.3

Interpretation

If p-value < 0.05, reject H₀:

H₀ → There is a median difference in extra hours of sleep between treatments.

If p-value ≥ 0.05, fail to reject H₀:

H₀ → There is no significant median difference in extra hours of sleep between treatments.

sleep_wide$Difference <- sleep_wide$extra.1 - sleep_wide$extra.2
ggplot(sleep_wide, aes(x = factor(ID), y = Difference)) +
  geom_bar(stat = "identity", fill = "purple", alpha = 0.7) +
  geom_hline(yintercept = 0, color = "red") +
  labs(title = "Sleep Difference: Treatment 1 - Treatment 2", x = "Subject ID", y = "Difference in Extra Hours") +
  theme_minimal()

4. Hands-On Exercise

Scenario

A researcher measures reaction times (in seconds) for 8 participants before and after a mindfulness training program. The goal is to determine whether the training significantly reduces reaction time.

Dataset

The measured reaction times are:

\[ \begin{array}{|c|c|c|} \hline \textbf{Participant} & \textbf{Before} & \textbf{After} \\ \hline 1 & 12.5 & 11.2 \\ 2 & 13.0 & 11.8 \\ 3 & 15.2 & 15.0 \\ 4 & 10.8 & 10.5 \\ 5 & 14.0 & 13.2 \\ 6 & 11.5 & 10.9 \\ 7 & 13.7 & 12.4 \\ 8 & 16.0 & 15.3 \\ \hline \end{array} \]

Task

Create vectors in R for the Before and After reaction times.
Perform a Sign Test for paired data (Before vs. After).
Plot a bar chart of the differences (Before - After).
Interpret your findings based on statistical results.

Hint Code

before <- c(12.5, 13.0, 15.2, 10.8, 14.0, 11.5, 13.7, 16.0)
after  <- c(11.2, 11.8, 15.0, 10.5, 13.2, 10.9, 12.4, 15.3)

# Sign test
sign_test_ex <- SignTest(before, after)

# Print results
sign_test_ex

## 
##  Dependent-samples Sign-Test
## 
## data:  before and after
## S = 8, number of differences = 8, p-value = 0.007812
## alternative hypothesis: true median difference is not equal to 0
## 99.2 percent confidence interval:
##  0.2 1.3
## sample estimates:
## median of the differences 
##                      0.75

Sample R Code

# Create vectors for Before and After reaction times
before <- c(12.5, 13.0, 15.2, 10.8, 14.0, 11.5, 13.7, 16.0)
after  <- c(11.2, 11.8, 15.0, 10.5, 13.2, 10.9, 12.4, 15.3)

# Compute the differences (Before - After)
differences <- before - after

# Load BSDA package for Sign Test
library(BSDA)

# Perform Sign Test for Paired Data
SIGN.test(differences, md = 0, alternative = "greater")

## 
##  One-sample Sign-Test
## 
## data:  differences
## s = 8, p-value = 0.003906
## alternative hypothesis: true median is greater than 0
## 95 percent confidence interval:
##  0.3407143       Inf
## sample estimates:
## median of x 
##        0.75 
## 
## Achieved and Interpolated Confidence Intervals: 
## 
##                   Conf.Level L.E.pt U.E.pt
## Lower Achieved CI     0.8555 0.6000    Inf
## Interpolated CI       0.9500 0.3407    Inf
## Upper Achieved CI     0.9648 0.3000    Inf

# Plot a bar chart of differences
barplot(differences, names.arg = 1:length(differences),
        col = "skyblue", main = "Reaction Time Differences (Before - After)",
        xlab = "Participant", ylab = "Difference in Reaction Time")
abline(h = 0, col = "red", lwd = 2, lty = 2)  # Add reference line at zero

Non-Parametric Hypothesis Test: Sign Test

Dr. Debashis Chatterjee

2025-02-19

Introduction

1. Sign Test for a Single Sample

Theory

Hypothesis (One-Sample Case)

Test Statistic

Example: Simulated Data (One-Sample)

Scenario

Sample R Code

Interpretation

Theory

Hypothesis (Paired Case)

Test Statistic

Example: Simulated Paired Data

Scenario

Now we perform the Sign Test on these paired differences:

Interpretation

3. Example with a Real Dataset: `sleep` in R

Interpretation

4. Hands-On Exercise

Scenario

Dataset

Task

Hint Code

Sample R Code

Non-Parametric Hypothesis Test: Sign Test

Dr. Debashis Chatterjee

2025-02-19

Introduction

1. Sign Test for a Single Sample

Theory

Hypothesis (One-Sample Case)

Test Statistic

Example: Simulated Data (One-Sample)

Scenario

Sample R Code

Interpretation

Theory

Hypothesis (Paired Case)

Test Statistic

Example: Simulated Paired Data

Scenario

Now we perform the Sign Test on these paired differences:

Interpretation

3. Example with a Real Dataset: sleep in R

Interpretation

4. Hands-On Exercise

Scenario

Dataset

Task

Hint Code

Sample R Code

3. Example with a Real Dataset: `sleep` in R