Background and Method

A nationally representative sample of American adults, from the AmeriSpeak Panel, completed a study conducted by NORC.

In a three-cell experimental design, participants listed, in open-text form, five values that they believe guide the U.S. on paper (i.e., the Constitution). Then, they indicated which of the values they listed is most important to the U.S. on paper.

This value was then embedded into the experimental manipulation. Participants in the promise kept condition wrote 2-3 sentences about the ways in which the U.S. is living up to its promise of that value. Participants in the promise broken condition wrote 2-3 sentences about the ways in which the U.S. is NOT living up to its promise of that value.Participants in the control condition provided a definition of that value.

Then, they completed five randomly-ordered DV’s: (1) anti-establishment sentiment; (2) American pride; (3) trust in government; (4) satisfaction with American democracy; and (5) support for radical change.

Participation

Total N

The sample size that was agreed upon was 1800. Due to some issues with data collection, NORC collected 1823 responses.

Eligibility

elg_n = nrow(df_bsc_elg)

I read through the open responses and dropped the ones that are obviously nonsensical. That leaves us with 1778 eligible responses.

Demographics

Race and ethnicity

df_bsc_elg %>% 
  group_by(RACETHNICITY) %>% 
  summarise(N = n()) %>% 
  ungroup() %>% 
  mutate(Perc = round(100*(N/sum(N)),2)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

RACETHNICITY	N	Perc
asian/non-hisp	65	3.66
black/non-hisp	200	11.25
hisp	358	20.13
multi/non-hisp	61	3.43
other/non-hisp	26	1.46
white/non-hisp	1068	60.07

Gender

df_bsc_elg %>% 
  group_by(GENDER) %>% 
  summarise(N = n()) %>% 
  ungroup() %>% 
  mutate(Perc = round(100*(N/sum(N)),2)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

GENDER	N	Perc
female	903	50.79
male	875	49.21

Age

df_bsc_elg %>% 
  summarise(age_mean = round(mean(AGE,na.rm = T),2),
            age_sd = round(sd(AGE,na.rm = T),2)) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

age_mean	age_sd
48.45	17.48

Education

df_bsc_elg %>% 
  group_by(edu) %>% 
  summarise(N = n()) %>% 
  ungroup() %>% 
  mutate(Perc = round(100*(N/sum(N)),2)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

edu	N	Perc
less HS	92	5.17
HS	327	18.39
some coll	727	40.89
bachelors	368	20.70
masters	264	14.85

Income

income_med <- median(df_bsc_elg$income_num,na.rm = T)
all_incomes <- c("under $5,000",
                 "$5,000 to $9,999",
                 "$10,000 to $14,999",
                 "$15,000 to $19,999",
                 "$20,000 to $24,999",
                 "$25,000 to $29,999",
                 "$30,000 to $34,999",
                 "$35,000 to $39,999",
                 "$40,000 to $49,999",
                 "$50,000 to $59,999",
                 "$60,000 to $74,999",
                 "$75,000 to $84,999",
                 "$85,000 to $99,999",
                 "$100,000 to $124,999",
                 "$125,000 to $149,999",
                 "$150,000 to $174,999",
                 "$175,000 to $199,999",
                 "$200,000 or more")
income_med_char <- all_incomes[income_med]

df_bsc_elg %>% 
  ggplot(aes(x = income)) +
  geom_bar() +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_blank(),
        axis.title.y = element_blank()) +
  coord_flip()

Median = $60,000 to $74,999

Measures

Value lists

Top 50 most mentioned:

df_bsc_elg %>%
  select(PID,val_1:val_5) %>%
  pivot_longer(-PID,
               names_to = "val_num",
               values_to = "value") %>% 
  filter(value != "98") %>% 
  filter(value != "") %>%
  filter(!is.na(value)) %>% 
  group_by(value) %>% 
  summarise(N = n()) %>% 
  ungroup() %>% 
  mutate(Perc = round(100*(N/sum(N)),2)) %>% 
  ungroup() %>% 
  arrange(desc(N)) %>% 
  slice(1:50) %>% 
  dplyr::select(N,Perc,value) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                position = "left") %>% 
  scroll_box(width = "100%", height = "400px")

N	Perc	value
584	6.61	Freedom
341	3.86	Equality
271	3.07	Liberty
251	2.84	Freedom of speech
189	2.14	Justice
182	2.06	Democracy
145	1.64	Freedom of religion
124	1.40	Independence
117	1.32	freedom
106	1.20	Opportunity
86	0.97	Pursuit of happiness
84	0.95	Honesty
80	0.91	Integrity
79	0.89	equality
67	0.76	Life
63	0.71	Safety
62	0.70	Diversity
61	0.69	liberty
60	0.68	freedom of speech
59	0.67	Family
52	0.59	Respect
51	0.58	Money
50	0.57	Fairness
49	0.55	Right to bear arms
45	0.51	Individualism
43	0.49	Freedom of Speech
43	0.49	Unity
41	0.46	Honor
41	0.46	democracy
40	0.45	Free speech
39	0.44	freedom of religion
38	0.43	Happiness
38	0.43	Trust
36	0.41	Prosperity
35	0.40	justice
34	0.38	Love
33	0.37	Freedom of Religion
33	0.37	Religious freedom
32	0.36	Justice for all
32	0.36	Loyalty
32	0.36	Peace
31	0.35	Truth
30	0.34	God
27	0.31	opportunity
26	0.29	Security
25	0.28	Equal rights
25	0.28	Freedom of choice
24	0.27	Education
23	0.26	Compassion
22	0.25	Capitalism

Top Values

Top 50 most mentioned:

df_bsc_elg %>%
  group_by(top_value_char) %>% 
  summarise(N = n()) %>% 
  ungroup() %>% 
  mutate(Perc = round(100*(N/sum(N)),2)) %>% 
  ungroup() %>% 
  arrange(desc(N)) %>% 
  dplyr::select(N,Perc,top_value_char) %>% 
  slice(1:50) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                position = "left") %>% 
  scroll_box(width = "100%", height = "400px")

N	Perc	top_value_char
356	20.02	Freedom
75	4.22	Democracy
70	3.94	freedom
68	3.82	Liberty
65	3.66	Equality
60	3.37	Freedom of speech
28	1.57	Freedom of religion
24	1.35	Independence
19	1.07	Integrity
19	1.07	Opportunity
18	1.01	freedom of speech
17	0.96	God
17	0.96	Honesty
14	0.79	Freedom of Speech
13	0.73	Money
13	0.73	democracy
13	0.73	liberty
12	0.67	Pursuit of happiness
12	0.67	equality
10	0.56	Respect
9	0.51	Free speech
9	0.51	Freedom of choice
9	0.51	Life
8	0.45	Economy
8	0.45	Family
8	0.45	Justice
8	0.45	Right to bear arms
7	0.39	Equal rights
7	0.39	Equality for all
7	0.39	Freedom from oppression
7	0.39	Freedom of Religion
6	0.34	Love
5	0.28	FREEDOM
5	0.28	Honor
5	0.28	Limited government
5	0.28	Loyalty
5	0.28	Prosperity
5	0.28	Religious freedom
5	0.28	Rights
5	0.28	Safety
5	0.28	Trust
4	0.22	Capitalism
4	0.22	Free Speech
4	0.22	Freedom for all
4	0.22	Honest
4	0.22	Rule of law
4	0.22	equal rights
4	0.22	freedom of religion
4	0.22	life
4	0.22	rule of law

Manipulation responses

Promise Kept Condition

In 2-3 sentences, please describe the ways in which the U.S. is living up to its promise of [TOP-VALUE].

df_bsc_elg %>%
  filter(cond == "kept") %>% 
  mutate(response = iconv(response,from =  "UTF-8", to = "ASCII", sub = "")) %>% 
  rename(top_value = top_value_char) %>% 
  select(PID,top_value,response) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                position = "left") %>% 
  scroll_box(width = "100%", height = "400px")

Promise Broken Condition

In 2-3 sentences, please describe the ways in which the U.S. is NOT living up to its promise of [TOP-VALUE].

df_bsc_elg %>%
  filter(cond == "brkn") %>% 
  mutate(response = iconv(response,from =  "UTF-8", to = "ASCII", sub = "")) %>% 
  rename(top_value = top_value_char) %>% 
  select(top_value,response) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                position = "left") %>% 
  scroll_box(width = "100%", height = "400px")

Control Condition

In 2-3 sentences, please define [TOP-VALUE].

df_bsc_elg %>%
  filter(cond == "ctrl") %>% 
  mutate(response = iconv(response,from =  "UTF-8", to = "ASCII", sub = "")) %>% 
  rename(top_value = top_value_char) %>% 
  select(top_value,response) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                position = "left") %>% 
  scroll_box(width = "100%", height = "400px")

Anti-Establishment Sentiment

The US’s economy is rigged to advantage the rich and powerful
Traditional politicians and parties don’t care about people like me
Experts in this country don’t understand the lives of people like me
Most of the time we can trust people in the government to do what is right [R]
alpha = 0.7

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = antiest)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,7,1),
                     limits = c(0,8)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$antiest,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

American Pride

How proud are you to be an American?

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = ampride)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,5,1),
                     limits = c(0,6)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$ampride,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

Trust in government

To what extent do you trust the government in Washington, across parties and administrations, to do what is right?

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = trustgov)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,7,1),
                     limits = c(0,8)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$trustgov,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

Satisfaction with American democracy

How satisfied are you with the way democracy is working in the United States?

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = demsatis)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,7,1),
                     limits = c(0,8)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$demsatis,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

Support for radical change

The way this country works needs to be radically changed.

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = radchange)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,7,1),
                     limits = c(0,8)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$radchange,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

Political discontent

I also took a mean score of anti-establishment sentiment, distrust in government (reverse-scored trust in government), and support for radical change.

alpha = 0.68

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = discontent)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,7,1),
                     limits = c(0,8)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$discontent,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

Analysis

Anti-establishment sentiment

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(antiest,na.rm = T),
            sd = sd(antiest,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	5.184322	1.142550
ctrl	613	4.940087	1.082434
kept	574	4.926883	1.095320

Omnibus Effect:

F(2, 1770) = 10.11, p < .001, $\eta^2_p$ = .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	-0.2442349	-0.3940392	-0.0944306	0.0003987
kept-brkn	-0.2574394	-0.4098607	-0.1050181	0.0002281
kept-ctrl	-0.0132045	-0.1642724	0.1378634	0.9770924

m1 <- t.test(antiest ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))
d1 <- cohen.d(antiest ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))

kept-brkn: t(1158.9) = 3.92; d = -0.23.

m1 <- t.test(antiest ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))
d1 <- cohen.d(antiest ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))

ctrl-brkn: t(1190.23) = 3.8; d = -0.22.

m1 <- t.test(antiest ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))
d1 <- cohen.d(antiest ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))

kept-ctrl: t(1173.26) = 0.21; d = -0.01.

American pride

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(ampride,na.rm = T),
            sd = sd(ampride,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	3.417377	1.304196
ctrl	613	3.453048	1.269347
kept	574	3.461403	1.300028

Omnibus Effect:

F(2, 1761) = 0.19, p = .826, $\eta^2_p$ < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	0.0356713	-0.1396235	0.2109661	0.8820131
kept-brkn	0.0440270	-0.1340428	0.2220969	0.8308621
kept-ctrl	0.0083557	-0.1682630	0.1849744	0.9932336

m1 <- t.test(ampride ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))
d1 <- cohen.d(ampride ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))

kept-brkn: t(1154.21) = -0.58; d = 0.03.

m1 <- t.test(ampride ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))
d1 <- cohen.d(ampride ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))

ctrl-brkn: t(1187.64) = -0.48; d = 0.03.

m1 <- t.test(ampride ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))
d1 <- cohen.d(ampride ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))

kept-ctrl: t(1166.21) = -0.11; d = 0.01.

Trust in government

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(trustgov,na.rm = T),
            sd = sd(trustgov,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	2.852041	1.336251
ctrl	613	3.018062	1.333279
kept	574	3.012281	1.355140

Omnibus Effect:

F(2, 1764) = 2.91, p = .055, $\eta^2_p$ < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	0.1660216	-0.0158910	0.3479342	0.0820727
kept-brkn	0.1602399	-0.0247045	0.3451843	0.1048161
kept-ctrl	-0.0057817	-0.1891498	0.1775864	0.9969887

m1 <- t.test(trustgov ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))
d1 <- cohen.d(trustgov ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))

kept-brkn: t(1153.65) = -2.03; d = 0.12.

m1 <- t.test(trustgov ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))
d1 <- cohen.d(trustgov ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))

ctrl-brkn: t(1193.34) = -2.15; d = 0.12.

m1 <- t.test(trustgov ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))
d1 <- cohen.d(trustgov ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))

kept-ctrl: t(1169.04) = 0.07; d = 0.

Satisfaction with American democracy

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(demsatis,na.rm = T),
            sd = sd(demsatis,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	3.254237	1.735412
ctrl	613	3.415435	1.776998
kept	574	3.458988	1.752264

Omnibus Effect:

F(2, 1769) = 2.21, p = .110, $\eta^2_p$ < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	0.1611979	-0.0766393	0.3990350	0.2502706
kept-brkn	0.2047505	-0.0367351	0.4462361	0.1152253
kept-ctrl	0.0435526	-0.1960698	0.2831751	0.9046851

m1 <- t.test(demsatis ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))
d1 <- cohen.d(demsatis ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))

kept-brkn: t(1159.24) = -2; d = 0.12.

m1 <- t.test(demsatis ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))
d1 <- cohen.d(demsatis ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))

ctrl-brkn: t(1196.92) = -1.59; d = 0.09.

m1 <- t.test(demsatis ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))
d1 <- cohen.d(demsatis ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))

kept-ctrl: t(1177.4) = -0.42; d = 0.02.

Support for radical change

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(radchange,na.rm = T),
            sd = sd(radchange,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	5.062925	1.608470
ctrl	613	4.707038	1.659824
kept	574	4.836555	1.677208

Omnibus Effect:

F(2, 1765) = 7.13, p < .001, $\eta^2_p$ < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	-0.3558875	-0.5792878	-0.1324873	0.0005635
kept-brkn	-0.2263698	-0.4537775	0.0010379	0.0513667
kept-ctrl	0.1295177	-0.0957752	0.3548106	0.3685290

m1 <- t.test(radchange ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))
d1 <- cohen.d(radchange ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))

kept-brkn: t(1148.6) = 2.34; d = -0.14.

m1 <- t.test(radchange ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))
d1 <- cohen.d(radchange ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))

ctrl-brkn: t(1196.94) = 3.77; d = -0.22.

m1 <- t.test(radchange ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))
d1 <- cohen.d(radchange ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))

kept-ctrl: t(1170.19) = -1.33; d = 0.08.

Political discontent

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(discontent,na.rm = T),
            sd = sd(discontent,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	5.132191	1.092530
ctrl	613	4.876087	1.064480
kept	574	4.917804	1.089992

Omnibus Effect:

F(2, 1775) = 9.60, p < .001, $\eta^2_p$ = .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	-0.2561037	-0.4024325	-0.1097748	0.0001249
kept-brkn	-0.2143873	-0.3631363	-0.0656383	0.0021316
kept-ctrl	0.0417164	-0.1057116	0.1891443	0.7845652

m1 <- t.test(discontent ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))
d1 <- cohen.d(discontent ~ cond,data = df_bsc_elg %>% filter(cond != "ctrl"))

kept-brkn: t(1162.16) = 3.35; d = -0.2.

m1 <- t.test(discontent ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))
d1 <- cohen.d(discontent ~ cond,data = df_bsc_elg %>% filter(cond != "kept"))

ctrl-brkn: t(1197.31) = 4.12; d = -0.24.

m1 <- t.test(discontent ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))
d1 <- cohen.d(discontent ~ cond,data = df_bsc_elg %>% filter(cond != "brkn"))

kept-ctrl: t(1175.59) = -0.67; d = 0.04.

Analysis without exclusions

I’ll repeat all these analyses, but this time without exclusions.

Anti-establishment sentiment

df_bsc %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(antiest,na.rm = T),
            sd = sd(antiest,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	609	5.170641	1.145946
ctrl	626	4.940133	1.080054
kept	588	4.909972	1.106247

Omnibus Effect:

F(2, 1815) = 9.96, p < .001, $\eta^2_p$ = .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	-0.2305081	-0.3789327	-0.0820835	0.0008095
kept-brkn	-0.2606699	-0.4115760	-0.1097639	0.0001566
kept-ctrl	-0.0301618	-0.1800581	0.1197345	0.8844829

m1 <- t.test(antiest ~ cond,data = df_bsc %>% filter(cond != "ctrl"))
d1 <- cohen.d(antiest ~ cond,data = df_bsc %>% filter(cond != "ctrl"))

kept-brkn: t(1190.99) = 4; d = -0.23.

m1 <- t.test(antiest ~ cond,data = df_bsc %>% filter(cond != "kept"))
d1 <- cohen.d(antiest ~ cond,data = df_bsc %>% filter(cond != "kept"))

ctrl-brkn: t(1221.82) = 3.63; d = -0.21.

m1 <- t.test(antiest ~ cond,data = df_bsc %>% filter(cond != "brkn"))
d1 <- cohen.d(antiest ~ cond,data = df_bsc %>% filter(cond != "brkn"))

kept-ctrl: t(1198.27) = 0.48; d = -0.03.

American pride

df_bsc %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(ampride,na.rm = T),
            sd = sd(ampride,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	609	3.425497	1.302916
ctrl	626	3.456452	1.261986
kept	588	3.467466	1.296110

Omnibus Effect:

F(2, 1805) = 0.17, p = .843, $\eta^2_p$ < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	0.0309549	-0.1416095	0.2035194	0.9070413
kept-brkn	0.0419691	-0.1332003	0.2171385	0.8402986
kept-ctrl	0.0110141	-0.1630406	0.1850689	0.9879273

m1 <- t.test(ampride ~ cond,data = df_bsc %>% filter(cond != "ctrl"))
d1 <- cohen.d(ampride ~ cond,data = df_bsc %>% filter(cond != "ctrl"))

kept-brkn: t(1185.04) = -0.56; d = 0.03.

m1 <- t.test(ampride ~ cond,data = df_bsc %>% filter(cond != "kept"))
d1 <- cohen.d(ampride ~ cond,data = df_bsc %>% filter(cond != "kept"))

ctrl-brkn: t(1217.89) = -0.42; d = 0.02.

m1 <- t.test(ampride ~ cond,data = df_bsc %>% filter(cond != "brkn"))
d1 <- cohen.d(ampride ~ cond,data = df_bsc %>% filter(cond != "brkn"))

kept-ctrl: t(1193.07) = -0.15; d = 0.01.

Trust in government

df_bsc %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(trustgov,na.rm = T),
            sd = sd(trustgov,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	609	2.846535	1.332820
ctrl	626	3.019293	1.333193
kept	588	3.011986	1.356590

Omnibus Effect:

F(2, 1809) = 3.22, p = .040, $\eta^2_p$ < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	0.1727580	-0.0067340	0.3522499	0.0621897
kept-brkn	0.1654516	-0.0168993	0.3478026	0.0844698
kept-ctrl	-0.0073063	-0.1885026	0.1738900	0.9950802

m1 <- t.test(trustgov ~ cond,data = df_bsc %>% filter(cond != "ctrl"))
d1 <- cohen.d(trustgov ~ cond,data = df_bsc %>% filter(cond != "ctrl"))

kept-brkn: t(1184.46) = -2.12; d = 0.12.

m1 <- t.test(trustgov ~ cond,data = df_bsc %>% filter(cond != "kept"))
d1 <- cohen.d(trustgov ~ cond,data = df_bsc %>% filter(cond != "kept"))

ctrl-brkn: t(1225.18) = -2.27; d = 0.13.

m1 <- t.test(trustgov ~ cond,data = df_bsc %>% filter(cond != "brkn"))
d1 <- cohen.d(trustgov ~ cond,data = df_bsc %>% filter(cond != "brkn"))

kept-ctrl: t(1196.25) = 0.09; d = -0.01.

Satisfaction with American democracy

df_bsc %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(demsatis,na.rm = T),
            sd = sd(demsatis,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	609	3.264803	1.727940
ctrl	626	3.409968	1.771331
kept	588	3.463373	1.765213

Omnibus Effect:

F(2, 1814) = 2.06, p = .127, $\eta^2_p$ < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	0.1451652	-0.0895966	0.3799270	0.3153440
kept-brkn	0.1985705	-0.0396258	0.4367667	0.1237498
kept-ctrl	0.0534052	-0.1834706	0.2902811	0.8571984

m1 <- t.test(demsatis ~ cond,data = df_bsc %>% filter(cond != "ctrl"))
d1 <- cohen.d(demsatis ~ cond,data = df_bsc %>% filter(cond != "ctrl"))

kept-brkn: t(1189.2) = -1.96; d = 0.11.

m1 <- t.test(demsatis ~ cond,data = df_bsc %>% filter(cond != "kept"))
d1 <- cohen.d(demsatis ~ cond,data = df_bsc %>% filter(cond != "kept"))

ctrl-brkn: t(1228) = -1.45; d = 0.08.

m1 <- t.test(demsatis ~ cond,data = df_bsc %>% filter(cond != "brkn"))
d1 <- cohen.d(demsatis ~ cond,data = df_bsc %>% filter(cond != "brkn"))

kept-ctrl: t(1203.42) = -0.52; d = 0.03.

Support for radical change

df_bsc %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(radchange,na.rm = T),
            sd = sd(radchange,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	609	5.049505	1.603980
ctrl	626	4.721154	1.655518
kept	588	4.831904	1.676360

Omnibus Effect:

F(2, 1810) = 6.31, p = .002, $\eta^2_p$ < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	-0.3283511	-0.5484520	-0.1082502	0.0013870
kept-brkn	-0.2176010	-0.4414825	0.0062805	0.0589039
kept-ctrl	0.1107501	-0.1115424	0.3330426	0.4721265

m1 <- t.test(radchange ~ cond,data = df_bsc %>% filter(cond != "ctrl"))
d1 <- cohen.d(radchange ~ cond,data = df_bsc %>% filter(cond != "ctrl"))

kept-brkn: t(1178.92) = 2.29; d = -0.13.

m1 <- t.test(radchange ~ cond,data = df_bsc %>% filter(cond != "kept"))
d1 <- cohen.d(radchange ~ cond,data = df_bsc %>% filter(cond != "kept"))

ctrl-brkn: t(1227.99) = 3.53; d = -0.2.

m1 <- t.test(radchange ~ cond,data = df_bsc %>% filter(cond != "brkn"))
d1 <- cohen.d(radchange ~ cond,data = df_bsc %>% filter(cond != "brkn"))

kept-ctrl: t(1197.24) = -1.15; d = 0.07.

Political discontent

df_bsc %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(discontent,na.rm = T),
            sd = sd(discontent,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	609	5.125000	1.086728
ctrl	626	4.880391	1.062399
kept	588	4.910738	1.091416

Omnibus Effect:

F(2, 1820) = 9.31, p < .001, $\eta^2_p$ = .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	-0.2446086	-0.3887883	-0.1004290	0.0002118
kept-brkn	-0.2142621	-0.3607211	-0.0678031	0.0017755
kept-ctrl	0.0303465	-0.1151323	0.1758254	0.8764152

m1 <- t.test(discontent ~ cond,data = df_bsc %>% filter(cond != "ctrl"))
d1 <- cohen.d(discontent ~ cond,data = df_bsc %>% filter(cond != "ctrl"))

kept-brkn: t(1193.15) = 3.4; d = -0.2.

m1 <- t.test(discontent ~ cond,data = df_bsc %>% filter(cond != "kept"))
d1 <- cohen.d(discontent ~ cond,data = df_bsc %>% filter(cond != "kept"))

ctrl-brkn: t(1229.9) = 4; d = -0.23.

m1 <- t.test(discontent ~ cond,data = df_bsc %>% filter(cond != "brkn"))
d1 <- cohen.d(discontent ~ cond,data = df_bsc %>% filter(cond != "brkn"))

kept-ctrl: t(1202.35) = -0.49; d = 0.03.

Broken Social Contract

November 2024: Analysis

Background and Method

Participation

Total N

Eligibility

Demographics

Race and ethnicity

Gender

Age

Education

Income

Measures

Value lists

Top Values

Manipulation responses

Promise Kept Condition

Promise Broken Condition

Control Condition

Anti-Establishment Sentiment

Distribution

American Pride

Distribution

Trust in government

Distribution

Satisfaction with American democracy

Distribution

Support for radical change

Distribution

Political discontent

Distribution

Analysis

Anti-establishment sentiment

American pride

Trust in government

Satisfaction with American democracy

Support for radical change

Political discontent

Analysis without exclusions

Anti-establishment sentiment

American pride

Trust in government

Satisfaction with American democracy

Support for radical change

Political discontent