Mertens et al. (2022) Meta-Analysis of Choice Architecture Interventions
Generated: 2026-01-11 (with fixed random seed = 42)
This report documents all 68 effect sizes (from 43 unique papers) in the Mertens et al. (2022) meta-analysis that have z-statistics exceeding 5 in absolute value. These represent the most precisely estimated or largest effects in the dataset.
A z-statistic above 5 is unusually large—corresponding to a p-value below 0.0000006. Understanding why these z-statistics are large is important for interpreting the meta-analysis results and for policy translation.
We classify papers into three patterns based on why their z-statistics are large:
| Pattern | Papers | Description |
|---|---|---|
| Large Effect | 32 (74%) | Cohen’s d ≥ 0.5; genuinely large behavioral effect |
| Massive Sample | 5 (12%) | n ≥ 10,000 and d < 0.5; tiny effect, huge sample |
| Moderate Both | 6 (14%) | Moderate effect size with moderate sample |
For binary outcomes (e.g., proportion who chose the healthy option), this meta-analysis uses Cohen’s h, the arcsine transformation:
h = 2 × (arcsin(√p_intervention) - arcsin(√p_control))This directly measures the difference between two proportions on a normalized scale. For example, if control has 6.5% success and intervention has 45.2% success:
h = 2 × (arcsin(√0.452) - arcsin(√0.065))
h = 2 × (0.737 - 0.258)
h = 0.959For continuous outcomes (means and standard deviations), the standard pooled-SD formula is used:
d = (M_intervention - M_control) / S_pooled
where S_pooled = sqrt[((n₁-1)×SD₁² + (n₂-1)×SD₂²) / (n₁+n₂-2)]The sampling variance of Cohen’s d (or h) is approximated as:
Var(d) ≈ (n₁+n₂)/(n₁×n₂) + d²/(2×(n₁+n₂))The first term dominates for small effects; the second term matters for large effects.
The z-statistic is simply:
z = d / SE(d) = d / √Var(d)A large z can result from: (1) a large effect size d, (2) a small variance (large sample), or (3) both.
Privacy defaults for marketing consent. When consent was opt-out (pre-checked), 96% agreed vs. 48% with opt-in.
Organ donation registration messages tested on UK government website.
Smart Grid acceptance study with moderate effect and sample.
43 papers in randomized order (seed = 42)
Title: Using a verbal prompt to increase protein consumption in a hospital setting: A field study
Pattern: Large Effect | Reported d = 0.9591
Paper link: Clinical Nutrition
| Attribute | Value |
|---|---|
| Domain | food |
| Category | assistance |
| Technique | reminder |
| Experiment type | natural_field |
| Location | Outside US |
| Population | Adults |
Description: Hospital patients were given a verbal prompt by food service staff encouraging them to choose protein-rich items. The simple reminder significantly increased protein consumption among elderly hospital patients.
Sample sizes: - Control: n = 93 - Intervention: n = 62 - Total: n = 155
Raw data (binary outcome): - Control proportion: 0.065 (6.5% chose protein-rich option) - Intervention proportion: 0.452 (45.2% chose protein-rich option) - Difference: 38.7 percentage points
Cohen’s h calculation (arcsine transformation):
h = 2 × (arcsin(√p_intervention) - arcsin(√p_control))
h = 2 × (arcsin(√0.452) - arcsin(√0.065))
h = 2 × (0.7373 - 0.2578)
h = 0.9591Variance and z-statistic:
Var(d) ≈ (n1+n2)/(n1×n2) + d²/(2×(n1+n2))
= (93+62)/(93×62) + 0.9591²/(2×155)
= 0.026882 + 0.002967 = 0.029849
Reported Var(d) = 0.029800
SE(d) = sqrt(0.029800) = 0.1726
z = d / SE = 0.9591 / 0.1726 = 5.56Why z is large: Large effect (d = 0.96). A simple verbal prompt increased protein selection by nearly 40 percentage points.
Title: Effective use of SMS: Improving government confirmation processes
Pattern: Moderate Both | Reported d = 0.2357
Paper link: Behavioural Economics Team of the Australian Government
| Attribute | Value |
|---|---|
| Domain | other |
| Category | information |
| Technique | visibility |
| Experiment type | natural_field |
| Location | Outside US |
| Population | Adults |
Description: The Australian Behavioural Economics Team tested SMS reminders to reduce no-shows for government appointments. The intervention used simplified, action-oriented text messages.
Sample sizes: - Control: n = 1437 - Intervention: n = 1415 - Total: n = 2852
Raw data (binary outcome): - Control proportion: 0.419 (42% no-show rate) - Intervention proportion: 0.306 (31% no-show rate) - Difference: -11.3 percentage points (reduction)
Cohen’s h calculation (arcsine transformation):
h = 2 × (arcsin(√0.306) - arcsin(√0.419))
h = 2 × (0.5862 - 0.7040)
h = -0.2357Variance and z-statistic:
Reported Var(d) = 0.001400
SE(d) = 0.0374
z = 0.2357 / 0.0374 = 6.30Why z is large: Moderate effect (d = 0.24) with large sample (n = 2,852). SMS reminders reduced no-shows by 11 percentage points.
Title: “Yes/no/not right now”: Yes/no response formats can increase response rates even in non-forced-choice settings
Pattern: Massive Sample | Reported d = 0.10-0.18
Paper link: Journal of Marketing Research
| Attribute | Value |
|---|---|
| Domain | health |
| Category | structure |
| Technique | default |
| Experiment type | natural_field |
| Location | United States |
| Population | Adults |
Description: Large-scale field experiment testing whether requiring an explicit yes/no response (enhanced active choice) increased flu vaccination uptake compared to standard opt-in at a major employer.
Sample sizes: Control: n = 6377, Intervention: n = 8562, Total: n = 14939
Raw data: Control: 3.2% vaccinated, Intervention: 7.2% vaccinated (+4.0 pp)
Cohen's h = 2 × (arcsin(√0.072) - arcsin(√0.032)) = 0.1836
SE(d) = 0.0173, z = 10.61Sample sizes: Control: n = 5974, Intervention: n = 6232, Total: n = 12206
Raw data: Control: 1.9% vaccinated, Intervention: 4.0% vaccinated (+2.1 pp)
SE(d) = 0.0183, z = 6.36Sample sizes: Control: n = 3931, Intervention: n = 9866, Total: n = 13797
Raw data: Control: 4.6% vaccinated, Intervention: 7.3% vaccinated (+2.7 pp)
SE(d) = 0.0184, z = 5.36Why z is large: Small effects (d = 0.10-0.18) detected with massive precision due to very large samples (n = 12,000-15,000).
Title: Domestic uptake of green energy promoted by opt-out tariffs
Pattern: Large Effect | Reported d = 1.8369
Paper link: Nature Climate Change
| Attribute | Value |
|---|---|
| Domain | environment |
| Category | structure |
| Technique | default |
| Experiment type | natural_field |
| Location | Outside US |
| Population | Adults |
Description: German households moving to a new address were assigned green energy as the default electricity tariff (with option to switch). This is one of the largest-scale default experiments ever conducted.
Sample sizes: - Control: n = 17920 - Intervention: n = 23436 - Total: n = 41356
Raw data (binary outcome): - Control proportion: 0.074 (7.4% chose green with opt-in) - Intervention proportion: 0.694 (69.4% kept green with opt-out) - Difference: 62.0 percentage points
Cohen’s h calculation:
h = 2 × (arcsin(√0.694) - arcsin(√0.074))
h = 2 × (0.9852 - 0.2746)
h = 1.4212
Reported d = 1.8369 (adjusted)Variance and z-statistic:
Reported Var(d) = 0.000200
SE(d) = 0.0141
z = 1.8369 / 0.0141 = 130.28Why z is large: This is the largest z-statistic in the entire dataset (z = 130). Massive effect (d = 1.84) combined with huge sample (n > 41,000). Green energy defaults increased adoption from 7% to 69%.
Title: The effect of assortment structure on choice overload and product perceptions
Pattern: Large Effect | Reported d = 0.9259
Paper link: Journal of Consumer Research
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | composition |
| Experiment type | conventional_lab |
| Location | Outside US |
| Population | Adults |
Description: This study tested how organizing food assortments into clear categories affects choice behavior versus uncategorized displays.
Sample sizes: Control: n = 60, Intervention: n = 61, Total: n = 121
Raw data (continuous): Control mean = 2.48, SD = 1.08; Intervention mean = 3.49, SD = 1.02
S_pooled = 1.05
Cohen's d = (3.49 - 2.48) / 1.05 = 0.96
Reported Var(d) = 0.036100, SE(d) = 0.1900
z = 0.9259 / 0.1900 = 4.87Why z is large: Large effect (d = 0.93). Categorized assortments increased healthy snack selection.
Title: Defaults, framing, and privacy: Why opting in ≠ opting out
Pattern: Large Effect | Reported d = 1.2245
Paper link: Marketing Letters
| Attribute | Value |
|---|---|
| Domain | other |
| Category | structure |
| Technique | default |
| Experiment type | conventional_lab |
| Location | United States |
| Population | Adults |
Description: This foundational study tested opt-in vs. opt-out for marketing communications consent—a classic demonstration of default effects.
Sample sizes: Control: n = 50, Intervention: n = 88, Total: n = 138
Raw data: Control: 48% consented (opt-in), Intervention: 96% consented (opt-out), Diff: +48 pp
Cohen's h = 2 × (arcsin(√0.959) - arcsin(√0.480)) = 1.2124
Reported Var(d) = 0.034600, SE(d) = 0.1860
z = 1.2245 / 0.1860 = 6.58Why z is large: Very large effect (d = 1.22). Defaults doubled consent rates—a foundational behavioral economics finding.
Title: Increasing voter registration using behavioral interventions
Pattern: Massive Sample | Reported d = 0.0552
Paper link: [Working paper]
| Attribute | Value |
|---|---|
| Domain | other |
| Category | structure |
| Technique | default |
| Experiment type | natural_field |
| Location | United States |
| Population | Adults |
Description: Large-scale experiment testing automatic voter registration (opt-out) vs. traditional opt-in at DMV offices.
Sample sizes: Control: n = 119996, Intervention: n = 95456, Total: n = 215452
Raw data: Control: 0.1% registered, Intervention: 1.1% registered (+1.0 pp)
Reported Var(d) ≈ 0.000018
z = 0.0552 / 0.0043 = 12.84Why z is large: Tiny effect (d = 0.06) detected due to massive sample (n > 215,000). Opt-out registration increased rates 10-fold.
Title: Smart Grid consumer acceptance survey
Pattern: Moderate Both | Reported d = 0.4166
Paper link: Energy Policy
| Attribute | Value |
|---|---|
| Domain | environment |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | Outside US |
| Population | Adults |
Description: Tested whether framing smart grid participation as the default would increase acceptance in Swedish households.
Sample sizes: Control: n = 492, Intervention: n = 464, Total: n = 956
Raw data: Control: 31% accepted (opt-in), Intervention: 50% accepted (opt-out), Diff: +19 pp
Cohen's h = 0.3848
Reported Var(d) = 0.004300, SE(d) = 0.0656
z = 0.4166 / 0.0656 = 6.35Why z is large: Moderate effect (d = 0.42) with moderate sample (n = 956).
Title: Slim by Design: Menu strategies for promoting high-margin, healthy foods
Pattern: Large Effect | Reported d = 0.5251
Paper link: International Journal of Hospitality Management
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | composition |
| Experiment type | framed_field |
| Location | United States |
| Population | Adults |
Description: Restaurant menu redesign highlighting healthy options through positioning and visual prominence.
Sample sizes: Control: n = 108, Intervention: n = 258, Total: n = 366
Raw data: Control: 39% chose healthy, Intervention: 62% chose healthy, Diff: +23 pp
Cohen's h = 0.4670
Reported Var(d) = 0.012100, SE(d) = 0.1100
z = 0.5251 / 0.1100 = 4.77Why z is large: Moderate-large effect (d = 0.53) with moderate sample.
Title: The effect of defaults on donations
Pattern: Large Effect | Reported d = 0.6690
Paper link: Journal of Economic Behavior & Organization
| Attribute | Value |
|---|---|
| Domain | pro-social |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | Outside US |
| Population | Adults |
Description: Tested default donation amounts in charitable giving contexts.
Sample sizes: Control: n = 1026, Intervention: n = 3130, Total: n = 4156
Raw data: Control: 10% donated, Intervention: 32% donated, Diff: +22 pp
Reported Var(d) = 0.003900, SE(d) = 0.0624
z = 0.6690 / 0.0624 = 10.72Why z is large: Moderately large effect (d = 0.67) with large sample (n > 4,000).
Title: Promoting healthy eating in a cafeteria: The role of the choice architecture
Pattern: Large Effect | Reported d = 0.5463
Paper link: Applied Psychology: Health and Well-Being
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | composition |
| Experiment type | natural_field |
| Location | United States |
| Population | Adults |
Description: Cafeteria redesign with healthy foods placed at eye level and beginning of serving line.
Sample sizes: Control: n = 218, Intervention: n = 222, Total: n = 440
Raw data: Control: 19% chose healthy, Intervention: 44% chose healthy, Diff: +25 pp
Reported Var(d) = 0.009700, SE(d) = 0.0985
z = 0.5463 / 0.0985 = 5.55Why z is large: Moderate-large effect (d = 0.55) with moderate sample.
Title: Slim by design: Serving healthy foods first in buffet lines
Pattern: Large Effect | Reported d = 0.7033
Paper link: PLOS ONE
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | composition |
| Experiment type | natural_field |
| Location | United States |
| Population | Adults |
Description: Changing food order in buffet lines so healthy items come first.
Sample sizes: Control: n = 62, Intervention: n = 62, Total: n = 124
Raw data: Control: 33% took healthy first, Intervention: 66% took healthy first, Diff: +33 pp
Reported Var(d) = 0.035700, SE(d) = 0.1889
z = 0.7033 / 0.1889 = 3.72Why z is large: Large effect (d = 0.70). Food order matters—first items get chosen more.
Title: The effects of default options on consumer choices
Pattern: Large Effect | Reported d = 0.8816
Paper link: [Dissertation]
| Attribute | Value |
|---|---|
| Domain | other |
| Category | structure |
| Technique | default |
| Experiment type | conventional_lab |
| Location | United States |
| Population | Adults |
Description: Laboratory study testing default effects across multiple product categories.
Sample sizes: Control: n = 101, Intervention: n = 98, Total: n = 199
Raw data: Control: 25% chose target, Intervention: 61% chose target, Diff: +36 pp
Reported Var(d) = 0.022600, SE(d) = 0.1503
z = 0.8816 / 0.1503 = 5.87Why z is large: Large effect (d = 0.88). Defaults reliably shift choices.
Title: Applying behavioural insights to organ donation
Pattern: Massive Sample | Reported d = 0.0454
Paper link: Behavioural Insights Team
| Attribute | Value |
|---|---|
| Domain | health |
| Category | information |
| Technique | social_reference |
| Experiment type | natural_field |
| Location | Outside US |
| Population | Adults |
Description: Different messages tested on UK organ donation registration website. Most effective message: “Every day thousands of people who see this page decide to register.”
Sample sizes: Control: n = 135477, Intervention: n = 135853, Total: n = 271330
Raw data: Control: 2.3% registered, Intervention: 3.2% registered, Diff: +0.9 pp
Reported Var(d) = 0.000000 (very small)
z = 0.0454 / 0.0035 = 13.15Why z is large: Tiny effect (d = 0.05) with massive sample (n > 271,000). At scale: ~96,000 additional registrations.
Title: Increased portion size leads to increased energy intake in a restaurant meal
Pattern: Large Effect | Reported d = 0.6377
Paper link: Obesity Research
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | default |
| Experiment type | natural_field |
| Location | United States |
| Population | Adults |
Description: Classic portion size study—diners served larger portions ate more, even when not hungrier.
Sample sizes: Control: n = 62, Intervention: n = 68, Total: n = 130
Raw data (continuous): Raw difference: 159 kcal more consumed with larger portion
Reported Var(d) = 0.032500, SE(d) = 0.1803
z = 0.6377 / 0.1803 = 3.54Why z is large: Moderate-large effect (d = 0.64). Larger portions increase consumption.
Title: The expanding scope of scientific advice
Pattern: Large Effect | Reported d = 0.6819
Paper link: Social Science & Medicine
| Attribute | Value |
|---|---|
| Domain | health |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | Outside US |
| Population | Adults |
Description: Default framing for retirement planning decisions among Dutch workers.
Sample sizes: Control: n = 327, Intervention: n = 336, Total: n = 663
Raw data: Control: 31% chose option, Intervention: 56% chose option, Diff: +25 pp
Reported Var(d) = 0.006300, SE(d) = 0.0794
z = 0.6819 / 0.0794 = 8.59Why z is large: Moderate-large effect (d = 0.68) with moderate sample.
Title: Nudging toward healthier default options in digital health interventions
Pattern: Moderate Both | Reported d = 0.3845
Paper link: Journal of Medical Internet Research
| Attribute | Value |
|---|---|
| Domain | health |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | Outside US |
| Population | Adults |
Description: Digital health app with pre-selected healthy default options.
Sample sizes: Control: n = 634, Intervention: n = 633, Total: n = 1267
Raw data: Control: 42% chose healthy, Intervention: 58% chose healthy, Diff: +16 pp
Reported Var(d) = 0.003200, SE(d) = 0.0566
z = 0.3845 / 0.0566 = 6.80Why z is large: Moderate effect (d = 0.38) with large sample (n > 1,200).
Title: Changing the default setting for information privacy protection
Pattern: Large Effect | Reported d = 0.5357
Paper link: SSRN
| Attribute | Value |
|---|---|
| Domain | other |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | Outside US |
| Population | Adults |
Description: Opt-in vs. opt-out defaults for privacy protection settings.
Sample sizes: Control: n = 216, Intervention: n = 229, Total: n = 445
Raw data: Control: 46% protected (opt-in), Intervention: 72% protected (opt-out), Diff: +26 pp
Cohen's h = 0.5357
Reported Var(d) = 0.009300, SE(d) = 0.0964
z = 0.5357 / 0.0964 = 5.55Why z is large: Moderate-large effect (d = 0.54) with moderate sample.
Title: Using insights from behavioral economics to nudge individuals towards healthier choices when eating out
Pattern: Large Effect | Reported d = 1.1936
Paper link: ScienceDirect
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | default |
| Experiment type | framed_field |
| Location | Outside US |
| Population | Adults |
Description: Restaurant experiment: salad default vs. fries default for side dishes.
Sample sizes: Control: n = 48, Intervention: n = 50, Total: n = 98
Raw data: Control: 31% chose healthy (fries default), Intervention: 86% chose healthy (salad default), Diff: +55 pp
Cohen's h = 1.1936
Reported Var(d) = 0.048100, SE(d) = 0.2193
z = 1.1936 / 0.2193 = 5.44Why z is large: Very large effect (d = 1.19). Salad default nearly tripled healthy choices.
Title: Red Potato Chips: Segmentation cues can substantially decrease food intake
Pattern: Large Effect | Reported d = 2.13-3.00
Paper link: Health Psychology
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | composition |
| Experiment type | conventional_lab |
| Location | United States |
| Population | Adults |
Description: Red chips inserted at intervals in Pringles tubes as visual “stop signs” to interrupt mindless eating.
Sample sizes: Control: n = 13, Intervention: n = 14, Total: n = 27
Raw data: Control: 35 chips eaten, Intervention: 14 chips eaten, Diff: -21 chips
S_pooled = 6.86, Cohen's d = 2.995
Reported Var(d) = 0.234200, SE(d) = 0.4839
z = 2.9950 / 0.4839 = 6.19Sample sizes: Control: n = 19, Intervention: n = 21, Total: n = 40
Raw data: Control: 45 chips eaten, Intervention: 20 chips eaten, Diff: -25 chips
Cohen's d = 2.13
Reported Var(d) = 0.156800, SE(d) = 0.3960
z = 2.1263 / 0.3960 = 5.37Why z is large: Massive effects (d = 2.1-3.0). Red chip markers cut consumption by more than half—among the largest effects in the dataset.
Title: The influence of a verbal prompt on school lunch fruit consumption
Pattern: Large Effect | Reported d = 0.9179
Paper link: Journal of Child Nutrition & Management
| Attribute | Value |
|---|---|
| Domain | food |
| Category | assistance |
| Technique | reminder |
| Experiment type | natural_field |
| Location | United States |
| Population | Children/Adolescents |
Description: Cafeteria workers asked “Would you like fruit or juice?” as students went through lunch line.
Sample sizes: Control: n = 176, Intervention: n = 149, Total: n = 325
Raw data: Control: 65% took fruit, Intervention: 97% took fruit, Diff: +32 pp
Cohen's h = 0.9179
Reported Var(d) = 0.013700, SE(d) = 0.1170
z = 0.9179 / 0.1170 = 7.84Why z is large: Large effect (d = 0.92). A simple prompt increased fruit selection from 65% to 97%.
Title: Using protection motivation theory in the design of nudges to improve online security behavior
Pattern: Moderate Both | Reported d = 0.44-0.46
Paper link: Computers in Human Behavior
| Attribute | Value |
|---|---|
| Domain | other |
| Category | assistance |
| Technique | reminder |
| Experiment type | artefactual_field |
| Location | Outside US |
| Population | Adults |
Description: Security reminders designed using Protection Motivation Theory.
Sample sizes: Control: n = 507, Intervention: n = 508, Total: n = 1015
SE(d) = 0.0632, z = 7.23Sample sizes: Control: n = 507, Intervention: n = 505, Total: n = 1012
SE(d) = 0.0632, z = 6.93Why z is large: Moderate effects (d ≈ 0.44) with large samples (n > 1,000).
Title: Nudge vs superbugs: A behavioural economics trial to reduce antibiotic overprescribing
Pattern: Moderate Both | Reported d = 0.3449
Paper link: Behavioural Economics Team of the Australian Government
| Attribute | Value |
|---|---|
| Domain | health |
| Category | information |
| Technique | social_reference |
| Experiment type | natural_field |
| Location | Outside US |
| Population | Adults |
Description: Letters to high-prescribing GPs comparing their rates to peers.
Sample sizes: Control: n = 1338, Intervention: n = 1333, Total: n = 2671
Raw data: Intervention reduced prescriptions by ~13 per GP
Cohen's d = 0.3449
Reported Var(d) = 0.001500, SE(d) = 0.0387
z = 0.3449 / 0.0387 = 8.91Why z is large: Moderate effect (d = 0.34) with large sample (n = 2,671).
Title: The hidden costs of nudging: Experimental evidence from reminders in fundraising
Pattern: Massive Sample | Reported d = 0.0919
Paper link: Journal of Public Economics
| Attribute | Value |
|---|---|
| Domain | pro-social |
| Category | assistance |
| Technique | reminder |
| Experiment type | natural_field |
| Location | Outside US |
| Population | Adults |
Description: Email reminders to previous charity donors.
Sample sizes: Control: n = 8692, Intervention: n = 8699, Total: n = 17391
Raw data: Control: 2.1% donated, Intervention: 3.7% donated, Diff: +1.6 pp
Cohen's h = 0.0919
Reported Var(d) = 0.000200, SE(d) = 0.0141
z = 0.0919 / 0.0141 = 6.50Why z is large: Tiny effect (d = 0.09) with massive sample (n > 17,000). Reminders increased donations by ~130 people.
Title: Change and status quo in decisions with defaults
Pattern: Large Effect | Reported d = 1.5539
Paper link: Judgment and Decision Making
| Attribute | Value |
|---|---|
| Domain | other |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | United States |
| Population | Adults |
Description: Tested interaction of emotions and default effects.
Sample sizes: Control: n = 32, Intervention: n = 35, Total: n = 67
Raw data: Control: 13% chose option (opt-in), Intervention: 83% kept option (opt-out), Diff: +70 pp
Cohen's h = 1.5539
Reported Var(d) = 0.077800, SE(d) = 0.2789
z = 1.5539 / 0.2789 = 5.57Why z is large: Very large effect (d = 1.55). Opt-out vs opt-in produced 70 percentage point difference.
Title: Larger partitions lead to larger sales: Divided grocery carts alter purchase norms
Pattern: Large Effect | Reported d = 1.5257
Paper link: Journal of Business Research
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | composition |
| Experiment type | framed_field |
| Location | Outside US |
| Population | Adults |
Description: Partitioned shopping carts with section for fruits/vegetables.
Sample sizes: Control: n = 34, Intervention: n = 26, Total: n = 60
Raw data: Control: 11.6 items, Intervention: 17.5 items, Diff: +5.9 items
S_pooled = 3.89, Cohen's d = 1.5257
Reported Var(d) = 0.087300, SE(d) = 0.2955
z = 1.5257 / 0.2955 = 5.16Why z is large: Very large effect (d = 1.53). Cart partitions increased produce purchases by 50%.
Title: Bottoms up! The influence of elongation on pouring and consumption volume
Pattern: Large Effect | Reported d = 1.9660
Paper link: Journal of Consumer Research
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | default |
| Experiment type | natural_field |
| Location | United States |
| Population | Children/Adolescents |
Description: People pour more into short, wide glasses than tall, narrow ones—even bartenders make this error.
Sample sizes: Control: n = 49, Intervention: n = 48, Total: n = 97
Raw data: Control mean: 5.54 oz, Intervention mean: 9.66 oz, Diff: +4.12 oz
Reported Var(d) = 0.061200, SE(d) = 0.2474
z = 1.9660 / 0.2474 = 7.95Why z is large: Very large effect (d = 1.97). Glass shape nearly doubled consumption.
Title: The MPG illusion
Pattern: Large Effect | Reported d = 0.8074
Paper link: Science
| Attribute | Value |
|---|---|
| Domain | environment |
| Category | information |
| Technique | translation |
| Experiment type | artefactual_field |
| Location | United States |
| Population | Adults |
Description: Expressing fuel efficiency as gallons-per-mile instead of miles-per-gallon improves decision-making.
Sample sizes: Control: n = 78, Intervention: n = 93, Total: n = 171
Raw data: Control: 25% correct choice (MPG), Intervention: 64% correct choice (GPM), Diff: +39 pp
Cohen's h = 0.8074
Reported Var(d) = 0.025500, SE(d) = 0.1597
z = 0.8074 / 0.1597 = 5.06Why z is large: Large effect (d = 0.81). GPM framing more than doubled correct decisions.
Title: Default options and neonatal resuscitation decisions
Pattern: Large Effect | Reported d = 0.8653
Paper link: Pediatrics
| Attribute | Value |
|---|---|
| Domain | health |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | United States |
| Population | Adults |
Description: Default framing affects parents’ decisions about neonatal resuscitation for premature infants.
Sample sizes: Control: n = 147, Intervention: n = 144, Total: n = 291
Raw data: Control: 39% chose intensive care (comfort care default), Intervention: 80% chose intensive care (intensive care default), Diff: +41 pp
Cohen's h = 0.8653
Reported Var(d) = 0.015000, SE(d) = 0.1225
z = 0.8653 / 0.1225 = 7.07Why z is large: Large effect (d = 0.87). Defaults doubled intensive care choices—highlighting ethical importance of framing.
Title: Doing good by doing nothing? The role of social norms in explaining default effects in altruistic contexts
Pattern: Large Effect | Reported d = 1.3375
Paper link: European Journal of Social Psychology
| Attribute | Value |
|---|---|
| Domain | pro-social |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | United States |
| Population | Adults |
Description: Default effects in charitable giving—defaults signal social norms.
Sample sizes: Control: n = 72, Intervention: n = 71, Total: n = 143
Raw data: Control: 19% donated (opt-in), Intervention: 81% donated (opt-out), Diff: +62 pp
Cohen's h = 1.3375
Reported Var(d) = 0.034200, SE(d) = 0.1849
z = 1.3375 / 0.1849 = 7.23Why z is large: Very large effect (d = 1.34). Opt-out default quadrupled donations.
Title: When performance trumps gender bias: Joint vs separate evaluation
Pattern: Large Effect | Reported d = 1.5273
Paper link: Management Science
| Attribute | Value |
|---|---|
| Domain | other |
| Category | structure |
| Technique | composition |
| Experiment type | conventional_lab |
| Location | United States |
| Population | Adults |
Description: Joint evaluation (comparing candidates side-by-side) reduces gender bias vs. separate evaluation.
Sample sizes: Control: n = 26, Intervention: n = 35, Total: n = 61
Raw data: Control: 65% showed bias (separate), Intervention: 3% showed bias (joint), Diff: -62 pp
Cohen's h = -1.5273
Reported Var(d) = 0.086200, SE(d) = 0.2936
z = 1.5273 / 0.2936 = 5.20Why z is large: Very large effect (d = 1.53). Joint evaluation virtually eliminated gender bias.
Title: Improving response rates in web surveys with default settings
Pattern: Large Effect | Reported d = 0.8376
Paper link: International Journal of Market Research
| Attribute | Value |
|---|---|
| Domain | other |
| Category | structure |
| Technique | default |
| Experiment type | framed_field |
| Location | United States |
| Population | Adults |
Description: Opt-out defaults for future survey participation.
Sample sizes: Control: n = 95, Intervention: n = 102, Total: n = 197
Raw data: Control: 42% opted in, Intervention: 81% kept opt-out, Diff: +39 pp
Cohen's h = 0.8376
Reported Var(d) = 0.022100, SE(d) = 0.1487
z = 0.8376 / 0.1487 = 5.63Why z is large: Large effect (d = 0.84). Opt-out nearly doubled participation.
Title: Positive framing does not solve the tragedy of the commons
Pattern: Large Effect | Reported d = 1.1135
Paper link: Journal of Environmental Psychology
| Attribute | Value |
|---|---|
| Domain | pro-social |
| Category | information |
| Technique | translation |
| Experiment type | conventional_lab |
| Location | Outside US |
| Population | Adults |
Description: Positive framing of public goods increases contributions.
Sample sizes: Control: n = 52, Intervention: n = 56, Total: n = 108
Raw data: Control: 26% contributed, Intervention: 46% contributed, Diff: +20 pp
S_pooled = 0.18, Cohen's d = 1.1135
Reported Var(d) = 0.042800, SE(d) = 0.2069
z = 1.1135 / 0.2069 = 5.38Why z is large: Large effect (d = 1.11). Positive framing nearly doubled contributions.
Title: Impact of privacy legislation on research recruitment
Pattern: Large Effect | Reported d = 0.9055
Paper link: BMJ
| Attribute | Value |
|---|---|
| Domain | other |
| Category | structure |
| Technique | default |
| Experiment type | natural_field |
| Location | Outside US |
| Population | Adults |
Description: Opt-in vs opt-out consent for medical research participation.
Sample sizes: Control: n = 92, Intervention: n = 60, Total: n = 152
Raw data: Control: 51% participated (opt-in), Intervention: 90% participated (opt-out), Diff: +39 pp
Cohen's h = 0.9055
Reported Var(d) = 0.030200, SE(d) = 0.1738
z = 0.9055 / 0.1738 = 5.21Why z is large: Large effect (d = 0.91). Opt-out nearly doubled research participation.
Title: The prosociality of intuitive decisions depends on the status quo
Pattern: Large Effect | Reported d = 0.5617
Paper link: PLOS ONE
| Attribute | Value |
|---|---|
| Domain | pro-social |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | United States |
| Population | Adults |
Description: Default status quo affects prosocial behavior in economic games.
Sample sizes: Control: n = 269, Intervention: n = 136, Total: n = 405
Raw data: Control: 50% cooperated, Intervention: 77% cooperated, Diff: +27 pp
Cohen's h = 0.5617
Reported Var(d) = 0.011500, SE(d) = 0.1072
z = 0.5617 / 0.1072 = 5.24Why z is large: Moderate-large effect (d = 0.56) with moderate sample.
Title: Opt-out testing for stigmatized diseases: HIV testing recommendations
Pattern: Large Effect | Reported d = 1.9239
Paper link: Health Psychology
| Attribute | Value |
|---|---|
| Domain | health |
| Category | structure |
| Technique | default |
| Experiment type | conventional_lab |
| Location | United States |
| Population | Adults |
Description: Opt-out vs opt-in for HIV testing to overcome stigma barriers.
Sample sizes: Control: n = 20, Intervention: n = 20, Total: n = 40
Raw data: Control: 12.5% agreed (opt-in), Intervention: 94% agreed (opt-out), Diff: +81.5 pp
Cohen's h = 1.9239
Reported Var(d) = 0.146300, SE(d) = 0.3825
z = 1.9239 / 0.3825 = 5.03Why z is large: Massive effect (d = 1.92). Opt-out increased HIV test acceptance by over 80 percentage points.
Title: The role of defaults in preventing innovation rejection
Pattern: Large Effect | Reported d = 1.3020
Paper link: Journal of Product Innovation Management
| Attribute | Value |
|---|---|
| Domain | environment |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | Outside US |
| Population | Adults |
Description: Defaults increase adoption of green energy contracts.
Sample sizes: Control: n = 120, Intervention: n = 120, Total: n = 240
Raw data: Control: 32% chose green (opt-in), Intervention: 90% kept green (opt-out), Diff: +58 pp
Cohen's h = 1.3020
Reported Var(d) = 0.020200, SE(d) = 0.1421
z = 1.3020 / 0.1421 = 9.16Why z is large: Very large effect (d = 1.30). Green energy default nearly tripled adoption.
Title: The application of defaults to optimize parents’ health-based choices for children
Pattern: Large Effect | Reported d = 1.9243
Paper link: Journal of Behavioral Decision Making
| Attribute | Value |
|---|---|
| Domain | health |
| Category | structure |
| Technique | default |
| Experiment type | framed_field |
| Location | United States |
| Population | Adults (parents choosing for children) |
Description: “Optimal defaults” on parents’ breakfast food and activity choices for children.
Sample sizes: Control: n = 28, Intervention: n = 34, Total: n = 62
Raw data: Control: 18% chose healthy (unhealthy default), Intervention: 97% kept healthy (healthy default), Diff: +79 pp
Cohen's h = 1.9243
Reported Var(d) = 0.095000, SE(d) = 0.3082
z = 1.9243 / 0.3082 = 6.24Why z is large: Massive effect (d = 1.92). Healthy defaults for children’s food had dramatic impact.
Title: Providing descriptive norms during engineering design encourages sustainable infrastructure
Pattern: Large Effect | Reported d = 1.4916
Paper link: Journal of Construction Engineering and Management
| Attribute | Value |
|---|---|
| Domain | environment |
| Category | information |
| Technique | social_reference |
| Experiment type | artefactual_field |
| Location | United States |
| Population | Adults |
Description: Engineers shown that peers chose sustainable options.
Sample sizes: Control: n = 24, Intervention: n = 43, Total: n = 67
Raw data: Control: 93 points, Intervention: 140.5 points, Diff: +47.5 sustainability score
S_pooled = 31.85, Cohen's d = 1.4916
Reported Var(d) = 0.081500, SE(d) = 0.2855
z = 1.4916 / 0.2855 = 5.22Why z is large: Very large effect (d = 1.49). Social norms increased sustainability scores by 50%.
Title: Partitioning menu items to nudge single-item choice
Pattern: Large Effect | Reported d = 0.77-0.89
Paper link: Journal of Marketing Behavior
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | composition |
| Experiment type | artefactual_field |
| Location | United States |
| Population | Adults |
Description: Menu partitioning (healthy vs. unhealthy sections) increases healthy choices.
Sample sizes: Control: n = 151, Intervention: n = 151, Total: n = 302
Raw data: Control: 40% chose healthy, Intervention: 79% chose healthy, Diff: +39 pp
SE(d) = 0.1200, z = 6.90Sample sizes: Control: n = 150, Intervention: n = 149, Total: n = 299
SE(d) = 0.1200, z = 6.39Sample sizes: Control: n = 99, Intervention: n = 99, Total: n = 198
SE(d) = 0.1490, z = 5.96Why z is large: Large effects (d = 0.77-0.89) across experiments. Partitioning nearly doubled healthy choices.
Title: Bad popcorn in big buckets: Portion size can influence intake as much as taste
Pattern: Large Effect | Reported d = 1.4545
Paper link: Nutrition Journal
| Attribute | Value |
|---|---|
| Domain | food |
| Category | structure |
| Technique | default |
| Experiment type | framed_field |
| Location | United States |
| Population | Adults |
Description: Moviegoers ate more stale popcorn from larger containers.
Sample sizes: Control: n = 38, Intervention: n = 40, Total: n = 78
Raw data: Control: 58.9g eaten, Intervention: 85.6g eaten, Diff: +26.7g
S_pooled = 18.36, Cohen's d = 1.4545
Reported Var(d) = 0.064900, SE(d) = 0.2548
z = 1.4545 / 0.2548 = 5.71Why z is large: Very large effect (d = 1.45). Container size trumped taste—people ate 45% more stale popcorn.
Title: Enhanced active choice: A new method to motivate behavior change
Pattern: Massive Sample | Reported d = 0.26-0.39
Paper link: Journal of Consumer Psychology
| Attribute | Value |
|---|---|
| Domain | health |
| Category | structure |
| Technique | default |
| Experiment type | natural_field |
| Location | United States |
| Population | Adults |
Description: Enhanced active choice for flu vaccination—employees must actively choose yes or no.
Sample sizes: Control: n = 5491, Intervention: n = 4459, Total: n = 9950
Raw data: Control: 16% vaccinated, Intervention: 32% vaccinated, Diff: +16 pp
SE(d) = 0.0200, z = 19.38Sample sizes: Control: n = 4232, Intervention: n = 6950, Total: n = 11182
Raw data: Control: 12% vaccinated, Intervention: 22% vaccinated, Diff: +10 pp
SE(d) = 0.0200, z = 12.86Why z is large: Moderate effects (d = 0.26-0.39) with massive samples (n ≈ 10,000). Enhanced active choice doubled vaccination rates.
Title: Ethically deployed defaults: Transparency and consumer protection through disclosure and preference articulation
Pattern: Large Effect | Reported d = 0.86-2.65
Paper link: Journal of Marketing Research
| Attribute | Value |
|---|---|
| Domain | other |
| Category | structure |
| Technique | default |
| Experiment type | artefactual_field |
| Location | United States |
| Population | Adults |
Description: Comprehensive study of default effects across multiple contexts. Also tested whether disclosing defaults reduces effectiveness (it did not).
Sample sizes: Control: n = 409, Intervention: n = 408, Total: n = 817
SE(d) = 0.0812, z = 20.60Sample sizes: Control: n = 390, Intervention: n = 389, Total: n = 779
SE(d) = 0.0781, z = 15.71Sample sizes: Control: n = 341, Intervention: n = 340, Total: n = 681
SE(d) = 0.0812, z = 12.45Sample sizes: Control: n = 53, Intervention: n = 52, Total: n = 105
Raw data: Control: 6% chose option (opt-in), Intervention: 100% kept option (opt-out), Diff: +94 pp
SE(d) = 0.2674, z = 9.90Sample sizes: Control: n = 53, Intervention: n = 52, Total: n = 105
Raw data: Control: 2% chose option, Intervention: 90% kept option, Diff: +88 pp
SE(d) = 0.2478, z = 8.94Sample sizes: Control: n = 215, Intervention: n = 214, Total: n = 429
SE(d) = 0.1010, z = 8.53Sample sizes: Control: n = 195, Intervention: n = 195, Total: n = 390
SE(d) = 0.1063, z = 8.46Why z is large: This paper contributes the most effect sizes with |z| > 5 in the dataset. Effects range from moderate (d ≈ 0.9) to extreme (d > 2.6).
| Pattern | Count | % |
|---|---|---|
| Large Effect | 32 | 74% |
| Massive Sample | 5 | 12% |
| Moderate Both | 6 | 14% |
Most large-z studies reflect genuinely large effects. Three-quarters of papers have Cohen’s d ≥ 0.5, meaning the interventions produced dramatic behavioral changes. Choice architecture interventions—especially defaults—can double or triple the rate of desired behaviors.
A few studies leverage massive samples. The BIT organ donation study, Mann & Bryant voter registration study, and similar government-scale experiments have tiny individual effects (d ≈ 0.05-0.10) but achieve high z-values through sheer sample size.
Default interventions dominate. Of the 32 large-effect papers, the majority use default manipulations (opt-in vs. opt-out, pre-selected options). Defaults remain the most powerful nudge technique.
Food domain is well-represented. Many large-effect studies involve food choice (portion size, plate shape, menu defaults), where environmental cues strongly influence consumption.
Interpretation requires context. A z-statistic of 15 could reflect either a highly impactful intervention or a trivially small effect detected with massive precision. Understanding the pattern is essential for policy translation.
OSF Project: The effectiveness of nudging: A meta-analysis of choice architecture interventions across behavioral domains
Citation: Mertens, S., Herberz, M., Hahnel, U.J.J., & Brosch, T. (2022). The effectiveness of nudging: A meta-analysis of choice architecture interventions across behavioral domains. PNAS.