Retora vs Human Message Evaluations
Total-index audience analysis with dimension diagnostics
Executive Summary
Retora coverage
100%
Participant-message rows
236 / 236
Total-index MAE
1.12
Average signed bias
-0.67
Retora coverage
100%
Participant-message rows
236 / 236
Total-index MAE
1.12
Average signed bias
-0.67
Plain-language readout: On the 1-7 total message-evaluation index, Retora lands within 1 point of the human average in 58.5% of participant-message cases. Using a looser band, it is within 1.5 points in 75.0% of cases and within 2 points in 86.0%.
Overall, Retora is not wildly off, but it is systematically more negative: its total score is on average 0.67 points lower than humans. The average miss is 1.12 points on the same 1-7 scale.
The total message-evaluation index averages the dimensions each participant actually rated. Retora is compared on that same set of dimensions for each participant-message pair.
Best matched audience groups:
- Gender x age: Female, 45-54 (24 participant-messages, MAE 0.71)
- Gender x age: Male, 55-64 (20 participant-messages, MAE 0.80)
- Gender x progressive position: Male, Conservative / low progressive (16 participant-messages, MAE 0.83)
Largest audience deviations:
- Gender x age: Female, 35-44 (24 participant-messages, MAE 1.46)
- Gender x age: Female, 18-24 (20 participant-messages, MAE 1.44)
- Trust: High trust (20 participant-messages, MAE 1.35)
Direction of mismatch: The largest directional bias is for Gender x age: Female, 35-44, where Retora is more negative than humans on average (signed bias -1.38).
Trust pattern: Retora is least accurate for High trust participants (MAE 1.35, 20 participant-messages) and most accurate for Medium trust participants (MAE 1.02). The strongest directional trust-bin mismatch is for High trust, where Retora scores are on average lower than humans by 0.92 points. This is a descriptive subgroup pattern; the report avoids treating a nearly flat trend line as evidence of a strong linear trust relationship.
Total Evaluation Index
The total index is the average of existing message-rating dimensions. Human and Retora totals are paired row by row, using only dimensions the human participant actually rated.
Audience Index Diagnostics
| Progressive index mean | Progressive index SD | Trust index mean | Trust index SD |
|---|---|---|---|
| 5.89 | 2.00 | 3.19 | 2.03 |
| Index | Group | Participant-messages | Participants | Total-index MAE | Signed bias |
|---|---|---|---|---|---|
| Trust | High trust | 20 | 5 | 1.35 | -0.92 |
| Trust | Low trust | 124 | 31 | 1.15 | -0.73 |
| Trust | Medium trust | 92 | 23 | 1.02 | -0.52 |
| Progressive position | Conservative / low progressive | 20 | 5 | 1.01 | -0.76 |
| Progressive position | Mixed | 92 | 23 | 1.28 | -0.82 |
| Progressive position | Progressive | 124 | 31 | 1.01 | -0.54 |
Human Vs Retora Dimension Averages
These interactive bars compare average human and Retora ratings by client-readable audience groups. Error bars show 95% normal confidence intervals around the mean.
Dimension Diagnostics
The charts below decompose the total index into the original moral-rating dimensions. They are diagnostic detail rather than the headline audience result.
Message Diagnostics
Best And Worst Matched Participant-Messages
| Section | Participant | Topic | Format | Quality | Message ID | Message | Human total | Retora total | Abs. error | Signed bias |
|---|---|---|---|---|---|---|---|---|---|---|
| Closest predictions | 5483702ffdf99b24c654b810 | climate_change | pitch | decent | climate_change_pitch_authority_q2_v1 | A clean-energy transition needs stable rules, not last-minute improvisation. Clear targets help firms invest, workers... | 4.25 | 4.25 | 0.00 | 0.00 |
| Closest predictions | 58ff31a1d10e2b000108579e | immigration | motto | strong | immigration_motto_fairness_q3_v1 | When newcomers follow the process, timely decisions and shared duties make fair treatment visible. | 3.75 | 3.75 | 0.00 | 0.00 |
| Closest predictions | 5e89ba09ec9b3364b1fc0bd1 | immigration | pitch | strong | immigration_pitch_fairness_q3_v2 | A credible migration system should not punish patience or hide costs in a few districts. It should give newcomers lan... | 4.75 | 4.75 | 0.00 | 0.00 |
| Closest predictions | 5ee92b925f0f482bb1e73668 | taxation_and_redistribution | pitch | weak | taxation_and_redistribution_pitch_freedom_q1_v2 | Progressive taxation can help people have more options. If housing, care, and education are out of reach, life gets n... | 4.50 | 4.50 | 0.00 | 0.00 |
| Closest predictions | 6658b1507df7606690e4d685 | gender_pay_gap | motto | decent | gender_pay_gap_motto_decency_q2_v2 | Basic workplace standards require honest reviews of gender-based pay gaps. | 4.50 | 4.50 | 0.00 | 0.00 |
| Closest predictions | 66bb63dab2b110041a61af00 | gender_pay_gap | motto | weak | gender_pay_gap_motto_freedom_q1_v2 | No one should have fewer options because pay decisions are biased. | 4.50 | 4.50 | 0.00 | 0.00 |
| Closest predictions | 66ef27cedb6307178a650d79 | immigration | motto | decent | immigration_motto_authority_q2_v2 | Orderly routes and consistent decisions make humane migration easier to sustain. | 3.25 | 3.25 | 0.00 | 0.00 |
| Closest predictions | 678ff81c8c6b56f93355e0bb | immigration | motto | strong | immigration_motto_authority_q3_v2 | Good rules make migration safer, more orderly, and easier for communities to manage. | 4.25 | 4.25 | 0.00 | 0.00 |
| Closest predictions | 69cc28cd29af8d640be609c5 | immigration | motto | strong | immigration_motto_fairness_q3_v2 | A fair system asks newcomers to contribute and gives them real tools to do so. | 4.50 | 4.50 | 0.00 | 0.00 |
| Closest predictions | 69cc28cd29af8d640be609c5 | climate_change | motto | decent | climate_change_motto_harm_q2_v1 | Cleaner power and better buildings reduce risks from heat, floods, and dirty air. | 4.00 | 4.00 | 0.00 | 0.00 |
| Closest predictions | 69ec0801854129d6e09b1a0b | taxation_and_redistribution | motto | strong | taxation_and_redistribution_motto_fairness_q3_v2 | A fair economy asks great wealth to support the foundations that made success possible. | 3.75 | 3.75 | 0.00 | 0.00 |
| Closest predictions | 69ec0801854129d6e09b1a0b | immigration | pitch | decent | immigration_pitch_belonging_q2_v1 | Humane migration policy should help newcomers become active members of local life. Language learning, work, and commu... | 4.50 | 4.50 | 0.00 | 0.00 |
| Closest predictions | 69eeab43b5b5e976fb68e27a | climate_change | motto | decent | climate_change_motto_fairness_q2_v2 | A clean-energy transition must share costs according to responsibility and ability to pay. | 4.00 | 4.00 | 0.00 | 0.00 |
| Closest predictions | 69efb336c58d4100c1b4a231 | immigration | pitch | strong | immigration_pitch_belonging_q3_v1 | Integration works best when people stop living parallel lives. Language classes, work routes, and local meeting place... | 3.25 | 3.25 | 0.00 | 0.00 |
| Closest predictions | 69f5b0fdb36c29c2c4ae7c07 | gender_pay_gap | motto | strong | gender_pay_gap_motto_harm_q3_v1 | One lower offer can compound into smaller raises, weaker savings, and a thinner pension. | 5.25 | 5.25 | 0.00 | 0.00 |
| Closest predictions | 69ff71b9ee7aed92a102e70a | immigration | motto | strong | immigration_motto_fairness_q3_v1 | When newcomers follow the process, timely decisions and shared duties make fair treatment visible. | 4.00 | 4.00 | 0.00 | 0.00 |
| Closest predictions | 69ff71b9ee7aed92a102e70a | climate_change | motto | weak | climate_change_motto_fairness_q1_v1 | Big polluters should pay more toward the clean-energy transition. | 4.50 | 4.50 | 0.00 | 0.00 |
| Closest predictions | 6a00893df8dbc36e2be09c1c | gender_pay_gap | motto | weak | gender_pay_gap_motto_fairness_q1_v2 | Pay should be fair when people are doing similar work. | 3.75 | 3.75 | 0.00 | 0.00 |
| Closest predictions | 6a042d9260c03aa95f88992e | taxation_and_redistribution | pitch | strong | taxation_and_redistribution_pitch_freedom_q3_v1 | A person is not truly choosing if every option is blocked by rent, care costs, or a school place they cannot access. ... | 5.50 | 5.50 | 0.00 | 0.00 |
| Closest predictions | 6a232d1401eefd2809ecd27d | climate_change | motto | decent | climate_change_motto_fairness_q2_v2 | A clean-energy transition must share costs according to responsibility and ability to pay. | 3.00 | 3.00 | 0.00 | 0.00 |
| Largest deviations | 69fb7b9b615d3fefc0ec2abb | immigration | motto | weak | immigration_motto_fairness_q1_v2 | A fair migration system should treat people with basic respect. | 7.00 | 3.50 | 3.50 | -3.50 |
| Largest deviations | 6a183c4b7720c4cf5c72ee47 | climate_change | pitch | strong | climate_change_pitch_freedom_q3_v2 | A country that depends on expensive imported fuels gives up too much control over its future. Renewables, storage, an... | 7.00 | 3.50 | 3.50 | -3.50 |
| Largest deviations | 637fad91c9976f9e66a65840 | climate_change | motto | weak | climate_change_motto_harm_q1_v2 | Clean energy can help reduce climate risks for families and communities. | 7.00 | 3.75 | 3.25 | -3.25 |
| Largest deviations | 69ac4273c42aa6de27024381 | climate_change | pitch | weak | climate_change_pitch_fairness_q1_v1 | Climate action should be fair. People who pollute more should probably do more. Ordinary households should not be lef... | 7.00 | 3.75 | 3.25 | -3.25 |
| Largest deviations | 69f3c78e4b6b2b837ad4384f | climate_change | motto | strong | climate_change_motto_authority_q3_v2 | Stable targets and visible follow-through prevent a rushed energy scramble later. | 5.75 | 2.50 | 3.25 | -3.25 |
| Largest deviations | 69fcb26251b7047bcc4538cd | gender_pay_gap | motto | weak | gender_pay_gap_motto_fairness_q1_v1 | People should receive fair pay for equal work. | 6.50 | 3.25 | 3.25 | -3.25 |
| Largest deviations | 6a183c4b7720c4cf5c72ee47 | immigration | pitch | weak | immigration_pitch_belonging_q1_v2 | Integration should bring people together. If newcomers and local residents rarely meet, trust becomes harder. A more ... | 7.00 | 3.75 | 3.25 | -3.25 |
| Largest deviations | 6a183c4b7720c4cf5c72ee47 | gender_pay_gap | pitch | weak | gender_pay_gap_pitch_decency_q1_v2 | A good employer should care about respectful treatment. If pay differences are hidden or ignored, that does not look ... | 7.00 | 3.75 | 3.25 | -3.25 |