This report provides a comprehensive breakdown of treatment effects across individual political issues in Studies 2 and 3 of Hackenberg et al. (2025). Where the original paper reports only overall and model-level ATEs, we estimate issue-specific effects for 598 individual policy stances with sufficient sample size (N >= 20 treated, >= 3 control per issue, pooling across Studies 2-3).
Full data:
data_and_analysis_code/output/heterogeneity/issue_level_ates.csv
| Statistic | Value |
|---|---|
| N issues analyzed | 598 |
| Mean ATE | 7.44 pp |
| Median ATE | 7.34 pp |
| SD of ATEs | 6.17 pp |
| Range | [-11.7, +37.6] pp |
| IQR | [3.7, 10.6] pp |
| Issues with negative ATE | 50 (8.4%) |
| Issues with ATE > 15pp | 54 (9.0%) |
The SD of issue-level ATEs (6.17pp) is nearly as large as the mean (7.44pp). Issue-level effects span nearly 50 percentage points, from -11.7pp (the AI conversation moved people away from the assigned stance) to +37.6pp. Roughly 1 in 12 issues shows a negative ATE — the intervention backfired on average. Another 1 in 11 shows effects above 15pp — roughly double the overall ATE.
| Issue Area | N issues | Mean ATE | SD of ATEs | Min | Max | Mean backlash rate |
|---|---|---|---|---|---|---|
| Technology and Digital | 34 | 8.60 | 7.75 | -10.8 | 34.7 | 19.2% |
| Taxes and Gov’t Spending | 51 | 7.88 | 7.60 | -11.7 | 37.6 | 21.5% |
| Immigration | 26 | 8.45 | 7.36 | -1.7 | 32.3 | 22.5% |
| Energy and Utilities | 41 | 8.37 | 6.82 | -9.6 | 24.0 | 18.0% |
| Democratic Institutions | 37 | 7.90 | 6.70 | -8.0 | 19.4 | 20.2% |
| Economy and Jobs | 41 | 7.71 | 6.44 | -7.2 | 23.2 | 17.7% |
| Housing and Planning | 27 | 8.32 | 6.15 | -11.3 | 23.6 | 19.0% |
| Civil Rights | 44 | 6.64 | 6.09 | -9.9 | 26.6 | 19.9% |
| Climate Change | 57 | 8.06 | 5.73 | -6.9 | 33.1 | 19.1% |
| Criminal Justice | 40 | 6.39 | 5.43 | -4.4 | 20.0 | 19.9% |
| Foreign Policy | 37 | 7.50 | 5.40 | -3.7 | 24.8 | 19.8% |
| Healthcare | 52 | 5.83 | 5.29 | -8.5 | 22.3 | 19.1% |
| National Security | 35 | 7.21 | 5.21 | -4.2 | 17.7 | 21.7% |
| Transport | 30 | 7.28 | 5.15 | -3.6 | 17.2 | 19.5% |
| Education | 46 | 6.52 | 5.10 | -9.2 | 16.0 | 18.3% |
Sorted by within-area SD of ATEs. Notable patterns: - Technology, Taxes, and Immigration show the most between-issue variation (SD = 7.4-7.8pp). These domains contain a mix of relatively straightforward policy proposals and deeply contentious values questions. - Education, Transport, and Healthcare show the least between-issue variation (SD = 5.1-5.3pp). These are domains where most stances involve relatively standard policy trade-offs. - Immigration has the highest average backlash rate (22.5%), despite having one of the higher mean ATEs (8.45pp) — consistent with high variance in individual response. - Healthcare has the lowest mean ATE (5.83pp). Health stances may be anchored in more entrenched personal experiences and values.
| Partisanship | N issues | Mean ATE | SD | % negative | % > 15pp | Mean backlash rate | Mean pre-attitude |
|---|---|---|---|---|---|---|---|
| Conservative | 186 | 8.24 | 6.62 | 8.1% | 11.3% | 21.5% | 55.8 |
| Labour | 294 | 6.78 | 5.77 | 9.2% | 7.1% | 18.7% | 63.2 |
| Neutral/Bipartisan | 118 | 7.83 | 6.24 | 6.8% | 10.2% | 19.0% | 61.3 |
Conservative stances are 1.5pp more persuadable on average than Labour stances. This is partly mechanical: the average pre-treatment attitude for Labour stances is 63.2 (participants already lean toward the stance), leaving less room for persuasion, while Conservative stances start at 55.8. Conservative stances also have higher backlash rates (21.5% vs 18.7%), consistent with more people actively resisting conservative arguments, resulting in a higher-variance distribution.
| Rank | ATE (pp) | N treat | N ctrl | Area | Partisanship | Issue Stance (truncated) |
|---|---|---|---|---|---|---|
| 1 | 37.6 | 50 | 5 | Taxes | Conservative | Raise the inheritance tax threshold, even if this means reduced government revenue |
| 2 | 34.7 | 61 | 3 | Technology | Neutral | Implement mandatory digital literacy courses in all secondary schools |
| 3 | 33.1 | 66 | 3 | Climate | Conservative | Permit the use of gene editing in agriculture to enhance crop resilience |
| 4 | 32.3 | 53 | 6 | Immigration | Labour | Implement stricter criteria for investor visas |
| 5 | 26.6 | 54 | 5 | Civil Rights | Labour | Implement mandatory gender diversity reporting for all companies |
| 6 | 24.8 | 63 | 3 | Foreign Policy | Conservative | Impose stricter regulations on Chinese technology companies |
| 7 | 24.2 | 73 | 4 | Taxes | Labour | Provide increased subsidies for live music venues |
| 8 | 24.0 | 59 | 6 | Energy | Conservative | Increase competition in the energy market by reducing regulations |
| 9 | 23.6 | 46 | 5 | Housing | Conservative | Streamline the planning approval process to reduce bureaucracy |
| 10 | 23.2 | 73 | 3 | Economy | Labour | Mandate equal pay audits for large companies |
| 11 | 22.3 | 56 | 3 | Healthcare | Neutral | Legalize psychedelic drugs for medical research purposes |
| 12 | 21.6 | 54 | 3 | Energy | Neutral | Invest in expanding nuclear energy capacity |
| 13 | 20.2 | 58 | 3 | Civil Rights | Labour | Implement mandatory gender pay gap reporting for all companies |
| 14 | 20.0 | 52 | 5 | Criminal Justice | Conservative | Increase funding for police forces to enhance community safety |
| 15 | 19.4 | 55 | 3 | Democratic Inst. | Labour | Enhance the powers of devolved governments |
| 16 | 19.3 | 57 | 3 | Immigration | Conservative | Strengthen immigration controls to prioritize skilled workers |
| 17 | 19.2 | 59 | 5 | Healthcare | Labour | Increase funding for mental health services |
| 18 | 19.1 | 56 | 3 | Criminal Justice | Conservative | Increase funding for local police forces |
| 19 | 19.1 | 75 | 9 | Housing | Conservative | Streamline building regulations to encourage faster housing development |
| 20 | 18.9 | 56 | 5 | Economy | Conservative | Implement a cap on public sector pay increases |
Qualitative observations: Many of the most persuadable issues involve how-type disagreements — questions about the best means to achieve broadly shared goals (streamlining planning, increasing police funding, expanding nuclear energy). Others involve is-type claims that participants may have been uncertain about (gene editing in agriculture, psychedelic drugs for research, digital literacy). Relatively few are primarily ought-anchored, where disagreement hinges on deep values.
| Rank | ATE (pp) | N treat | N ctrl | Area | Partisanship | Issue Stance (truncated) |
|---|---|---|---|---|---|---|
| 1 | -11.7 | 71 | 4 | Taxes | Neutral | Maintain the triple lock on state pensions |
| 2 | -11.3 | 54 | 4 | Housing | Neutral | Implement stricter affordability checks for mortgage applications |
| 3 | -10.8 | 60 | 3 | Technology | Neutral | Increase funding for local radio stations to preserve regional media |
| 4 | -9.9 | 59 | 5 | Civil Rights | Conservative | Ensure that faith-based adoption agencies can operate according to their beliefs |
| 5 | -9.6 | 61 | 3 | Energy | Conservative | Prioritize domestic natural gas production to enhance energy security |
| 6 | -9.2 | 60 | 3 | Education | Conservative | Maintain university tuition fees at current levels |
| 7 | -8.7 | 60 | 3 | Energy | Labour | Implement stricter regulations on energy companies to protect consumers |
| 8 | -8.5 | 55 | 4 | Healthcare | Conservative | Allow more private healthcare providers to operate within the NHS |
| 9 | -8.0 | 55 | 5 | Democratic Inst. | Labour | Allow alternative forms of identification for voting |
| 10 | -7.2 | 61 | 3 | Taxes | Labour | Implement policies to ensure equitable pension contributions |
| 11 | -7.2 | 57 | 4 | Economy | Labour | Enhance funding for education and training programs in underserved regions |
| 12 | -6.9 | 64 | 6 | Climate | Labour | Impose stricter regulations on outdoor advertising to reduce visual pollution |
| 13 | -6.5 | 69 | 6 | Economy | Conservative | Maintain the flexibility of zero-hours contracts |
| 14 | -4.9 | 59 | 6 | Democratic Inst. | Conservative | Maintain the British Monarchy as a constitutional institution |
| 15 | -4.4 | 60 | 5 | Criminal Justice | Labour | Implement stricter penalties for homophobic hate crimes |
| 16 | -4.2 | 73 | 3 | National Security | Conservative | Increase defence budget to ensure national security |
| 17 | -4.0 | 56 | 4 | Education | Labour | Implement policies to increase social mobility through education |
| 18 | -3.7 | 47 | 5 | Foreign Policy | Labour | Prioritize human rights considerations in trade negotiations |
| 19 | -3.6 | 44 | 3 | Transport | Labour | Subsidize public transport fares to make them more affordable |
| 20 | -2.9 | 57 | 3 | National Security | Labour | Prioritize diversity and inclusion in military recruitment |
On these issues, the AI conversation moved people away from the assigned stance on average. Qualitative patterns: Many involve deep ought-type commitments (faith-based adoption, the Monarchy, zero-hours contracts, defence spending, the NHS) or issues where people likely have strong prior personal experience (pensions, mortgages, tuition). The AI’s information-heavy approach may have triggered reactance on these value-laden topics, or participants may have found the arguments unpersuasive precisely because their attitudes are anchored in normative commitments rather than factual uncertainty.
The issues producing the most variable responses among treated participants (highest SD of attitude change):
| Issue | ATE | SD (treated) | Backlash rate | N treat | N ctrl |
|---|---|---|---|---|---|
| Not require voter ID for elections | 18.8 | 34.8 | 30.0% | 60 | 4 |
| Raise inheritance tax threshold | 37.6 | 33.8 | 22.0% | 50 | 5 |
| Incentivize private healthcare providers | 9.3 | 25.9 | 30.6% | 62 | 6 |
| Cap on public sector pay increases | 18.9 | 25.7 | 32.1% | 56 | 5 |
| Privatize more aspects of rail infrastructure | 9.3 | 25.6 | 29.1% | 55 | 3 |
| Decriminalize the selling of sex | 16.2 | 25.3 | 19.7% | 66 | 10 |
| Allow market forces to determine energy prices | 14.8 | 25.3 | 17.9% | 67 | 4 |
| Stricter criteria for investor visas | 32.3 | 25.1 | 17.0% | 53 | 6 |
| Regulate zero-hours contracts | 0.9 | 24.9 | 13.5% | 74 | 3 |
These issues combine substantial average effects with enormous individual variation. Voter ID (SD = 34.8, backlash = 30%) is the standout: the conversation powerfully persuades some people but produces strong resistance in others. This is textbook IOH territory — voter ID involves factual claims (about fraud), values commitments (about access vs. integrity), and strategic judgments (about the efficacy and legitimacy of the policy means). People will vary in which component anchors their position.
| Issue | ATE | Backlash rate | N treat | N ctrl |
|---|---|---|---|---|
| Maintain current income tax thresholds | -0.3 | 38.0% | 50 | 3 |
| Privatise more public services to increase efficiency | 6.7 | 37.9% | 66 | 5 |
| Ensure private healthcare services are more accessible | 0.7 | 37.7% | 53 | 6 |
| Protect green belt areas from development | -1.9 | 37.5% | 48 | 6 |
| Limit immigration to reduce pressure on public services | 0.6 | 37.5% | 64 | 6 |
| Limit expansion of clean air zones to protect small business | 7.1 | 37.5% | 56 | 4 |
| Maintain the British Monarchy | -4.9 | 37.3% | 59 | 6 |
| Maintain economic sanctions against Russia | 7.1 | 37.0% | 54 | 4 |
| Prioritize funding for STEM subjects in schools | 10.8 | 36.2% | 47 | 4 |
| Prioritize access to public services for citizens over recent arrivals | -1.7 | 35.7% | 70 | 5 |
On these issues, more than a third of treated participants moved against the assigned stance. Several are quintessentially ought-anchored: the Monarchy, immigration limits, privatization of public services, protecting green belt land. The AI conversation appears to activate resistance in a large minority, even where it persuades others — producing the mixed-effect pattern IOH predicts.
Note the STEM funding issue: it has a positive mean ATE (10.8pp) but still 36.2% backlash. This is a striking illustration of how the ATE can badly misrepresent the individual experience. The “average” person was persuaded by 11 points, but more than a third of individuals moved in the opposite direction.
| Correlation | r | N issues |
|---|---|---|
| Mean pre-treatment attitude vs. ATE | -0.248 | 598 |
| Within-issue SD vs. ATE | +0.339 | 598 |
| Backlash rate vs. ATE | -0.208 | 598 |
A crucial comparison for the IOH argument:
| Level of analysis | SD of effects |
|---|---|
| Between issue areas (15 groups) | 0.83 pp |
| Between individual issues (598 issues) | 6.17 pp |
| Within individual issues (individual-level) | ~16-17 pp (median within-issue SD) |
The ratio matters: within-issue individual variation (~16pp) is roughly 2.5x the between-issue variation (6.2pp) and 20x the between-area variation (0.8pp). The specific issue someone discussed matters — but who they are matters far more. This is the gap that IOH configurations are predicted to fill.
This issue-level analysis reveals several patterns that bear on the IOH framework:
Issue-level ATEs vary enormously (-11.7 to +37.6pp), but this variation is dwarfed by individual-level variation within issues. The issue matters; the person matters more.
8.4% of issues show negative average treatment effects — the AI conversation backfired. These tend to be value-laden topics (the Monarchy, immigration, faith-based adoption, NHS privatization) where informational persuasion plausibly misses the load-bearing component of most people’s attitudes.
The most persuadable issues tend to involve how-type disagreements (policy means, implementation details) or is-type uncertainty (empirical questions people hadn’t thought deeply about). These are precisely the types of attitudes that informational AI conversations should be well-suited to shift.
High-ATE issues are also high-variance issues (r = +0.34), meaning that the “most persuadable” issues aren’t uniformly easy — they produce both the largest positive shifts and substantial backlash. This is consistent with different people holding the same position for different IOH reasons.
Backlash rates of 15-38% persist across virtually all issues, reinforcing that persuasion interventions systematically produce opposing effects in different subpopulations — the core heterogeneity finding.