Broken Social Contract

November 2024: Analysis

Background and Method

A nationally representative sample of American adults, from the AmeriSpeak Panel, completed a study conducted by NORC.

In a three-cell experimental design, participants listed, in open-text form, five values that they believe guide the U.S. on paper (i.e., the Constitution). Then, they indicated which of the values they listed is most important to the U.S. on paper.

This value was then embedded into the experimental manipulation. Participants in the promise kept condition wrote 2-3 sentences about the ways in which the U.S. is living up to its promise of that value. Participants in the promise broken condition wrote 2-3 sentences about the ways in which the U.S. is NOT living up to its promise of that value.Participants in the control condition provided a definition of that value.

Then, they completed five randomly-ordered DV’s: (1) anti-establishment sentiment; (2) American pride; (3) trust in government; (4) satisfaction with American democracy; and (5) support for radical change.

Participation

Total N

The sample size that was agreed upon was 1800. Due to some issues with data collection, NORC collected 1823 responses.

Eligibility

elg_n = nrow(df_bsc_elg)

I read through the open responses and dropped the ones that are obviously nonsensical. That leaves us with 1778 eligible responses.

Demographics

Race and ethnicity

df_bsc_elg %>% 
  group_by(RACETHNICITY) %>% 
  summarise(N = n()) %>% 
  ungroup() %>% 
  mutate(Perc = round(100*(N/sum(N)),2)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

RACETHNICITY	N	Perc
asian/non-hisp	65	3.66
black/non-hisp	200	11.25
hisp	358	20.13
multi/non-hisp	61	3.43
other/non-hisp	26	1.46
white/non-hisp	1068	60.07

Gender

df_bsc_elg %>% 
  group_by(GENDER) %>% 
  summarise(N = n()) %>% 
  ungroup() %>% 
  mutate(Perc = round(100*(N/sum(N)),2)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

GENDER	N	Perc
female	903	50.79
male	875	49.21

Age

df_bsc_elg %>% 
  summarise(age_mean = round(mean(AGE,na.rm = T),2),
            age_sd = round(sd(AGE,na.rm = T),2)) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

age_mean	age_sd
48.45	17.48

Education

df_bsc_elg %>% 
  group_by(edu) %>% 
  summarise(N = n()) %>% 
  ungroup() %>% 
  mutate(Perc = round(100*(N/sum(N)),2)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

edu	N	Perc
less HS	92	5.17
HS	327	18.39
some coll	727	40.89
bachelors	368	20.70
masters	264	14.85

Income

income_med <- median(df_bsc_elg$income_num,na.rm = T)
all_incomes <- c("under $5,000",
                 "$5,000 to $9,999",
                 "$10,000 to $14,999",
                 "$15,000 to $19,999",
                 "$20,000 to $24,999",
                 "$25,000 to $29,999",
                 "$30,000 to $34,999",
                 "$35,000 to $39,999",
                 "$40,000 to $49,999",
                 "$50,000 to $59,999",
                 "$60,000 to $74,999",
                 "$75,000 to $84,999",
                 "$85,000 to $99,999",
                 "$100,000 to $124,999",
                 "$125,000 to $149,999",
                 "$150,000 to $174,999",
                 "$175,000 to $199,999",
                 "$200,000 or more")
income_med_char <- all_incomes[income_med]

df_bsc_elg %>% 
  ggplot(aes(x = income)) +
  geom_bar() +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_blank(),
        axis.title.y = element_blank()) +
  coord_flip()

Median = $60,000 to $74,999

Measures

Value lists

Top 50 most mentioned:

df_bsc_elg %>%
  select(PID,val_1:val_5) %>%
  pivot_longer(-PID,
               names_to = "val_num",
               values_to = "value") %>% 
  filter(value != "98") %>% 
  filter(value != "") %>%
  filter(!is.na(value)) %>% 
  group_by(value) %>% 
  summarise(N = n()) %>% 
  ungroup() %>% 
  mutate(Perc = round(100*(N/sum(N)),2)) %>% 
  ungroup() %>% 
  arrange(desc(N)) %>% 
  slice(1:50) %>% 
  dplyr::select(N,Perc,value) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                position = "left") %>% 
  scroll_box(width = "100%", height = "400px")

N	Perc	value
584	6.61	Freedom
341	3.86	Equality
271	3.07	Liberty
251	2.84	Freedom of speech
189	2.14	Justice
182	2.06	Democracy
145	1.64	Freedom of religion
124	1.40	Independence
117	1.32	freedom
106	1.20	Opportunity
86	0.97	Pursuit of happiness
84	0.95	Honesty
80	0.91	Integrity
79	0.89	equality
67	0.76	Life
63	0.71	Safety
62	0.70	Diversity
61	0.69	liberty
60	0.68	freedom of speech
59	0.67	Family
52	0.59	Respect
51	0.58	Money
50	0.57	Fairness
49	0.55	Right to bear arms
45	0.51	Individualism
43	0.49	Freedom of Speech
43	0.49	Unity
41	0.46	Honor
41	0.46	democracy
40	0.45	Free speech
39	0.44	freedom of religion
38	0.43	Happiness
38	0.43	Trust
36	0.41	Prosperity
35	0.40	justice
34	0.38	Love
33	0.37	Freedom of Religion
33	0.37	Religious freedom
32	0.36	Justice for all
32	0.36	Loyalty
32	0.36	Peace
31	0.35	Truth
30	0.34	God
27	0.31	opportunity
26	0.29	Security
25	0.28	Equal rights
25	0.28	Freedom of choice
24	0.27	Education
23	0.26	Compassion
22	0.25	Capitalism

Top Values

Top 50 most mentioned:

df_bsc_elg %>%
  group_by(top_value_char) %>% 
  summarise(N = n()) %>% 
  ungroup() %>% 
  mutate(Perc = round(100*(N/sum(N)),2)) %>% 
  ungroup() %>% 
  arrange(desc(N)) %>% 
  dplyr::select(N,Perc,top_value_char) %>% 
  slice(1:50) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                position = "left") %>% 
  scroll_box(width = "100%", height = "400px")

N	Perc	top_value_char
356	20.02	Freedom
75	4.22	Democracy
70	3.94	freedom
68	3.82	Liberty
65	3.66	Equality
60	3.37	Freedom of speech
28	1.57	Freedom of religion
24	1.35	Independence
19	1.07	Integrity
19	1.07	Opportunity
18	1.01	freedom of speech
17	0.96	God
17	0.96	Honesty
14	0.79	Freedom of Speech
13	0.73	Money
13	0.73	democracy
13	0.73	liberty
12	0.67	Pursuit of happiness
12	0.67	equality
10	0.56	Respect
9	0.51	Free speech
9	0.51	Freedom of choice
9	0.51	Life
8	0.45	Economy
8	0.45	Family
8	0.45	Justice
8	0.45	Right to bear arms
7	0.39	Equal rights
7	0.39	Equality for all
7	0.39	Freedom from oppression
7	0.39	Freedom of Religion
6	0.34	Love
5	0.28	FREEDOM
5	0.28	Honor
5	0.28	Limited government
5	0.28	Loyalty
5	0.28	Prosperity
5	0.28	Religious freedom
5	0.28	Rights
5	0.28	Safety
5	0.28	Trust
4	0.22	Capitalism
4	0.22	Free Speech
4	0.22	Freedom for all
4	0.22	Honest
4	0.22	Rule of law
4	0.22	equal rights
4	0.22	freedom of religion
4	0.22	life
4	0.22	rule of law

Manipulation responses

Promise Kept Condition

In 2-3 sentences, please describe the ways in which the U.S. is living up to its promise of [TOP-VALUE].

df_bsc_elg %>%
  filter(cond == "kept") %>% 
  mutate(response = iconv(response,from =  "UTF-8", to = "ASCII", sub = "")) %>% 
  rename(top_value = top_value_char) %>% 
  select(PID,top_value,response) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                position = "left") %>% 
  scroll_box(width = "100%", height = "400px")

Promise Broken Condition

In 2-3 sentences, please describe the ways in which the U.S. is NOT living up to its promise of [TOP-VALUE].

df_bsc_elg %>%
  filter(cond == "brkn") %>% 
  mutate(response = iconv(response,from =  "UTF-8", to = "ASCII", sub = "")) %>% 
  rename(top_value = top_value_char) %>% 
  select(top_value,response) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                position = "left") %>% 
  scroll_box(width = "100%", height = "400px")

Control Condition

In 2-3 sentences, please define [TOP-VALUE].

df_bsc_elg %>%
  filter(cond == "ctrl") %>% 
  mutate(response = iconv(response,from =  "UTF-8", to = "ASCII", sub = "")) %>% 
  rename(top_value = top_value_char) %>% 
  select(top_value,response) %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                position = "left") %>% 
  scroll_box(width = "100%", height = "400px")

Anti-Establishment Sentiment

1. The US’s economy is rigged to advantage the rich and powerful
   
2. Traditional politicians and parties don’t care about people like me
   
3. Experts in this country don’t understand the lives of people like me
   
4. Most of the time we can trust people in the government to do what is right [R]
   alpha = 0.7

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = antiest)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,7,1),
                     limits = c(0,8)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$antiest,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

American Pride

How proud are you to be an American?

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = ampride)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,5,1),
                     limits = c(0,6)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$ampride,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

Trust in government

To what extent do you trust the government in Washington, across parties and administrations, to do what is right?

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = trustgov)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,7,1),
                     limits = c(0,8)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$trustgov,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

Satisfaction with American democracy

How satisfied are you with the way democracy is working in the United States?

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = demsatis)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,7,1),
                     limits = c(0,8)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$demsatis,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

Support for radical change

The way this country works needs to be radically changed.

Distribution

df_bsc_elg %>% 
  ggplot(aes(x = radchange)) +
  geom_histogram(fill = "lightblue",
                 color = "black",
                 binwidth = 1) +
  scale_x_continuous(breaks = seq(1,7,1),
                     limits = c(0,8)) +
  ylab("frequency") +
  geom_vline(xintercept = mean(df_bsc_elg$radchange,na.rm = T),
             color = "black",
             linetype = "dashed",
             size = 1.1) +
  theme(panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.ticks = element_blank(),
        axis.line = element_line(color = "grey66"),
        axis.text.y = element_text(color = "black"),
        axis.text.x = element_text(color = "black",
                                   face = "bold"),
        axis.title.x = element_text(color = "black",
                                   face = "bold"))

Analysis

Anti-establishment sentiment

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(antiest,na.rm = T),
            sd = sd(antiest,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	5.184949	1.144444
ctrl	613	4.939868	1.086449
kept	574	4.922291	1.100115

Omnibus Effect:

F(2, 1755) = 10.24, p < .001, \(\eta^2_p\) = .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	-0.2450808	-0.3958080	-0.0943535	0.0004154
kept-brkn	-0.2626577	-0.4162557	-0.1090596	0.0001859
kept-ctrl	-0.0175769	-0.1699946	0.1348407	0.9604668

American pride

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(ampride,na.rm = T),
            sd = sd(ampride,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	3.417377	1.304196
ctrl	613	3.453048	1.269347
kept	574	3.461403	1.300028

Omnibus Effect:

F(2, 1761) = 0.19, p = .826, \(\eta^2_p\) < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	0.0356713	-0.1396235	0.2109661	0.8820131
kept-brkn	0.0440270	-0.1340428	0.2220969	0.8308621
kept-ctrl	0.0083557	-0.1682630	0.1849744	0.9932336

Trust in government

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(trustgov,na.rm = T),
            sd = sd(trustgov,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	2.852041	1.336251
ctrl	613	3.018062	1.333279
kept	574	3.012281	1.355140

Omnibus Effect:

F(2, 1764) = 2.91, p = .055, \(\eta^2_p\) < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	0.1660216	-0.0158910	0.3479342	0.0820727
kept-brkn	0.1602399	-0.0247045	0.3451843	0.1048161
kept-ctrl	-0.0057817	-0.1891498	0.1775864	0.9969887

Satisfaction with American democracy

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(demsatis,na.rm = T),
            sd = sd(demsatis,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	3.254237	1.735412
ctrl	613	3.415435	1.776998
kept	574	3.458988	1.752264

Omnibus Effect:

F(2, 1769) = 2.21, p = .110, \(\eta^2_p\) < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	0.1611979	-0.0766393	0.3990350	0.2502706
kept-brkn	0.2047505	-0.0367351	0.4462361	0.1152253
kept-ctrl	0.0435526	-0.1960698	0.2831751	0.9046851

Support for radical change

df_bsc_elg %>% 
  group_by(cond) %>% 
  summarise(N = n(),
            mean = mean(radchange,na.rm = T),
            sd = sd(radchange,na.rm = T)) %>% 
  ungroup() %>% 
  kbl() %>% 
  kable_styling(bootstrap_options = "hover",
                full_width = F,
                position = "left")

cond	N	mean	sd
brkn	591	5.062925	1.608470
ctrl	613	4.707038	1.659824
kept	574	4.836555	1.677208

Omnibus Effect:

F(2, 1765) = 7.13, p < .001, \(\eta^2_p\) < .01

Tukey-HSD Post-Hoc Comparisons:

	diff	lwr	upr	p adj
ctrl-brkn	-0.3558875	-0.5792878	-0.1324873	0.0005635
kept-brkn	-0.2263698	-0.4537775	0.0010379	0.0513667
kept-ctrl	0.1295177	-0.0957752	0.3548106	0.3685290