LPPL4660-002 Study Lab 4

Research question

Does one’s exposure to alternative political viewpoints alter their level of political engagement for young adults?

Hypothesis

Exposure to alternative political viewpoints that strongly go against their morals will increase political engagement in young adults towards their own political viewpoint and against the alternative viewpoint.

Method

Participants were solicited via Prolific. Study was launched on 10/31 - 5 days before the election. We only recruited participants that live in the US and are young adults (i.e. 18-34 according to the US Census). We distributed two different studies that point to the same Qualtrics: one screened for Conservatives and one screened for Liberals.

Our target N was 265 Conservatives and 265 Liberals.

• Affords 80% power to detect effect size of d >= 0.24 in a two tailed t-test 

• Affords 80% power to detect effect size of d >= 0.35 for simple effects within political party

Procedure

Participants first respond to a moral foundations scale, meant to serve as a moderator.

Moral foundations ¹

1 = Strongly disagree

5 = Strongly agree

1. Compassion for those who are suffering is the most crucial virtue harm/care
2. When the government makes laws, the number one priority should be whether everyone is treated fairly fairness/reciprocity
3. I am proud of my country’s history ingroup loyalty
4. Respect for authority is something all children need to learn authority/respect
5. People should not do things that are disgusting, even if no one is harmed purity/sanctity
6. I think it’s morally wrong that rich children inherit a lot of money while poor children inherit nothing fairness/reciprocity

Moral foundations 2 ²

1 = Not at all relevant (This consideration has nothing to do with my judgments of right and wrong)

5 = Extremely relevant (This is one of the most important factors when I judge right and wrong)

1. Whether or not someone suffered emotionally harm/care
2. Whether or not some people were treated differently than others fairness/reciprocity
3. Whether or not someone’s action showed love for his or her country ingroup loyalty
4. Whether or not someone showed a lack of respect for authority authority/respect
5. Whether or not someone violated standards of purity and decency purity/sanctity
6. Whether or not someone was good at math attention check
7. Whether or not someone cared for someone weak or vulnerable harm/care
8. Whether or not someone acted unfairly fairness/reciprocity
9. Whether or not someone did something to betray his or her group ingroup loyalty
10. Whether or not someone conformed to the traditions of society authority/respect
11. Whether or not someone did something disgusting purity/sanctity

Participants were then randomly assigned to watch a political video in the experimental condition or an apolitical video in the control condition. The experimental video will depend on the participant’s political orientation. Participants could not proceed to the next page until the full length of the video had passed.

Control video ³

Experimental video for Liberals ⁴

Experimental video for Conservatives ⁵

Participants then responded to or interacted with 6 response opportunities to serve as our dependent variables. In the self-report items, the relevant targets were piped in using embedded variables:

• politicalOrientation = participant’s political orientation as indicated on a pretest on Prolific. liberal or conservative

• ownParty_formal = “Democrat” for liberals, or “Republican” for conservatives

Dependent Variables

Likelihood to engage in political activity. ⁶

-2 = Extremely unlikely

2 = Extremely likely

In the following questions, “${e://Field/ownParty_formal} candidates” refers to any of the candidates for your party on your local ballot. This can include the presidential candidates, any candidates for Congress, or any local positions. If you’ve already completed any of the following, indicate the likelihood that you would do it again in the future if given the chance.

1. How likely are you to volunteer for the ${e://Field/ownParty_formal} party?
2. How likely are you to encourage your friends/family to vote for the ${e://Field/ownParty_formal} candidates on the ballot?
3. How likely are you to promote ${e://Field/ownParty_formal} content on social media?
4. How likely are you to attend a rally or other event for any of the ${e://Field/ownParty_formal} candidates?
5. How likely are you to donate money to your ${e://Field/ownParty_formal} candidate(s)?

Likelihood to vote.

• Rate the likelihood that you will vote in the upcoming election. 0 = 0% chance you will vote, 100 = 100% chance you will vote. If you’ve already voted, check the box to indicate that this question doesn’t apply to you.

Strength of preference for political candidate. ⁷

• On a scale of 0 - 100, how strong is your preference for the ${e://Field/ownParty_formal} to win the presidential election?

Time spent looking at vote early info, plus click counts. ⁸

Time spent looking at phonebanking/volunteer opportunities, plus click counts ⁹

Liberals saw:

Conservatives saw:

Donation. ¹⁰

• If you would like us to donate your payment to the ${e://Field/ownParty_formal} National Committee instead of paying you, please indicate your preference below.

  • “please donate my payment”
  • “no thank you, I prefer to be compensated”

Demographic Variables

• Ethnicity
• Hispanic [yes/no]
• Gender
• Age

Results

projectData = read.csv('ProjectData002.csv') %>% 
  filter(Progress==100 & mfq2_6 <3) %>% 
  mutate(#compute moral foundations scores, collapse and recode politically active 
    harm_care = rowMeans(select(.,'mfq_1', 'mfq2_1', 'mfq2_7'), na.rm=TRUE),
    fariness_reciprocity = rowMeans(select(.,'mfq_2', 'mfq_6','mfq2_2', 'mfq2_8'), na.rm=TRUE),
    ingroup_loyalty = rowMeans(select(.,'mfq_3', 'mfq2_3', 'mfq2_9'), na.rm=TRUE),
    authority_respect = rowMeans(select(.,'mfq_4', 'mfq2_4', 'mfq2_10'), na.rm=TRUE),
    purity_sanctity = rowMeans(select(.,'mfq_5', 'mfq2_5', 'mfq2_11'), na.rm=TRUE),
  
    politicalAction = rowMeans(select(.,matches("likelyToEngage")), na.rm=TRUE)-10) %>% 
 select(condition, politicalOrientation, harm_care, fariness_reciprocity, ingroup_loyalty, authority_respect, purity_sanctity, politicalAction,  ethnicity, gender, likelyToVote, preferenceStrength, voteEarlyTiming_PageSubmit, voteEarlyTiming_ClickCount, volunteerTiming_PageSubmit, volunteerTiming_ClickCount, donatePayment) %>% 
  mutate(condition = factor(condition),
         donatePayment= factor(donatePayment))

550 people opened the survey. I excluded anyone who reported higher than a 2 on the attention check item in the moral foundations scale. After dropouts and exclusions, we collected a total of

N = 420

nice_table(projectData %>% group_by(condition) %>%  count() %>% arrange(desc(n)))

condition	n
experimental	218
control	202

nice_table(projectData %>% group_by(condition, politicalOrientation) %>%  count())

condition	politicalOrientation	n
control	conservative	88
control	liberal	114
experimental	conservative	86
experimental	liberal	132

Sample demographics

nice_table(projectData %>% group_by(ethnicity) %>%  count() %>% arrange(desc(n)))
nice_table(projectData %>% group_by(gender) %>%  count() %>% arrange(desc(n)))

ethnicity	n
White	258
Black	66
Asian	57
Not listed	15
Black,White	7
Amer. Indian/Alaskan	5
Asian,White	3
	2
Amer. Indian/Alaskan,Black	2
Amer. Indian/Alaskan,White	2
Amer. Indian/Alaskan,Asian,Black,White	1
Polynesian/Native Hawaiian	1
Polynesian/Native Hawaiian,White	1

gender	n
Woman	202
Man	193
Non-binary	10
Man,Transgender	3
	2
Transgender	2
Non-binary,Gender Queer	1
Not listed	1
Rather not	1
Transgender,Non-binary	1
Woman,Man	1
Woman,Non-binary	1
Woman,Rather not	1
Woman,Transgender	1

Overall responses

Moral foundations by political orientation

projectData_tests = projectData %>% 
  select(c(2:7)) %>% 
  pivot_longer(2:6, 
               names_to = "moralFoundation") 

ggerrorplot(projectData_tests, x = 'moralFoundation', color='politicalOrientation', y = 'value', ylab="", palette = c("#fe2500", "#5bbbd6"), ylim=c(1.5,5)) 

Hypothesis tests

Did people who watched a political video that goes against their views report more intention to engage in politically motivated behaviors?

ggerrorplot(projectData, x = 'condition', color='condition', y = 'politicalAction', ylab="Likelihood of engaging\nin poitical action", xlab="", ylim=c(-1,1), palette = c("#43a38d", "#f18334")) 
nice_table(projectData %>% 
             t_test(politicalAction ~ condition, paired = FALSE, var.equal=TRUE) %>% mutate(p = round(p, 3)) %>% 
             inner_join(cohens_d(projectData, politicalAction~condition, paired=FALSE)) %>%
             select(statistic, df, p, effsize))

statistic	df	p	effsize
-2.11	418	.035*	-0.21

Was this moderated by attitudes about the moral foundations?

ggarrange(
ggscatter(projectData, x = 'harm_care', color='politicalOrientation',y = 'politicalAction', ylab="Likelihood of engaging\nin poitical action", xlab="Harm / Care", ylim=c(-1,1),palette= c("#fe2500", "#5bbbd6"), add="reg.line", add.params=list(linetype='condition'), alpha=0) ,

ggscatter(projectData, x = 'fariness_reciprocity', color='politicalOrientation',y = 'politicalAction', ylab="", xlab="Fairness / Reciprocity", ylim=c(-1,1),palette= c("#fe2500", "#5bbbd6"), add="reg.line", add.params=list(linetype='condition'), alpha=0),

ggscatter(projectData, x = 'ingroup_loyalty', color='politicalOrientation',y = 'politicalAction', ylab="", xlab="Ingroup Loyalty", ylim=c(-1,1),palette= c("#fe2500", "#5bbbd6"), add="reg.line", add.params=list(linetype='condition'), alpha=0),

ggscatter(projectData, x = 'authority_respect', color='politicalOrientation',y = 'politicalAction', ylab="Likelihood of engaging\nin poitical action", xlab="Authority/Respect", ylim=c(-1,1),palette= c("#fe2500", "#5bbbd6"), add="reg.line", add.params=list(linetype='condition'), alpha=0),

ggscatter(projectData, x = 'purity_sanctity', color='politicalOrientation',y = 'politicalAction', ylab="", xlab="Purity/Sanctity", ylim=c(-1,1),palette= c("#fe2500", "#5bbbd6"), add="reg.line", add.params=list(linetype='condition'), alpha=0),
common.legend=TRUE)

model_fairness = summary(lm(politicalAction ~ condition*fariness_reciprocity+politicalOrientation, data=projectData))
nice_table(as.data.frame(model_fairness$coefficients) %>% cbind(data.frame(term=c('intercept','condition','fairness_reciprocity','politicalOrientation','condition*fairness'))))

Estimate	Std. Error	t value	Pr(>|t|)	Term
-0.03	0.46	-0.06	0.95	intercept
-1.47	0.63	-2.35	0.02	condition
0.02	0.12	0.19	0.85	fairness_reciprocity
-0.30	0.13	-2.33	0.02	politicalOrientation
0.41	0.15	2.79	0.01	condition*fairness

Did people who watched a political video that goes against their views report more intention to vote and stronger preference for their own candidate?

note that 137 indicated they already voted and aren’t included in this analysis

projectData_tests = projectData %>% 
  select(c(1,  11, 12)) %>% 
  pivot_longer(2:3, 
               names_to = "variable") 

ggerrorplot(projectData_tests, x = 'variable', color='condition', y = 'value', ylab="", xlab="", ylim=c(50,100), palette = c("#43a38d", "#f18334")) 

Warning: Removed 137 rows containing non-finite outside the scale range
(`stat_summary()`).

nice_table(projectData_tests %>% group_by(variable) %>%
             t_test(value ~ condition, paired = FALSE, var.equal=TRUE) %>% mutate(p = round(p, 3)) %>% 
             inner_join(cohens_d(projectData_tests %>% group_by(variable), value~condition, paired=FALSE)) %>%
             select(variable, statistic, df, p, effsize))

variable	statistic	df	p	effsize
likelyToVote	0.86	281	.389	0.10
preferenceStrength	-0.33	418	.745	-0.03

Did people who watched a political video that goes against their views show more interest in voting early or volunteering for their candidate’s campaign?

projectData_tests = projectData %>% 
  select(c(1,  13:16)) %>% 
  pivot_longer(2:5, 
               names_to = "variable")

ggviolin(subset(projectData_tests, variable=="voteEarlyTiming_PageSubmit" | variable=="volunteerTiming_PageSubmit"), x = 'variable', fill='condition', y = 'value', ylab="", xlab="", add='mean_se', alpha=0.75, ylim=c(0,100), palette = c("#43a38d", "#f18334")) 
ggviolin(subset(projectData_tests, variable=="voteEarlyTiming_ClickCount" | variable=="volunteerTiming_ClickCount"), x = 'variable', fill='condition', y = 'value', ylab="", xlab="", add='mean_se', alpha=0.75, ylim=c(0,5), palette = c("#43a38d", "#f18334"))
nice_table(projectData_tests %>% group_by(variable) %>%
             t_test(value ~ condition, paired = FALSE, var.equal=TRUE) %>% mutate(p = round(p, 3)) %>% 
             inner_join(cohens_d(projectData_tests %>% group_by(variable), value~condition, paired=FALSE)) %>%
             select(variable, statistic, df, p, effsize))

variable	statistic	df	p	effsize
volunteerTiming_ClickCount	0.65	418	.517	0.06
volunteerTiming_PageSubmit	1.57	418	.116	0.15
voteEarlyTiming_ClickCount	0.49	418	.627	0.05
voteEarlyTiming_PageSubmit	1.24	418	.214	0.12

Were people who watched a political video that goes against their views more likely to donate their compensation to their party?

projectData %>% 
  group_by(condition) %>% count(donatePayment) %>%  
  mutate(percent = round(n/sum(n)*100, 2)) %>% 
  ggplot(aes(x=condition,fill=donatePayment, y=percent, label=percent))+
  scale_fill_manual(values=c("grey", "#f18334"))+
  geom_bar(position='stack', stat='identity')+
  theme_classic()+ 
  geom_text(aes(y=percent-1),position='stack', color='white', size=8)+
  theme(legend.title=element_blank())
nice_table(as.data.frame(chisq.test(table(projectData$donatePayment,projectData$condition))[c('statistic','parameter','p.value')]))

statistic	parameter	p
0.09	1	.764

Exploratory Analyses

Did political orientation shape reactions to the political video?

ggerrorplot(projectData, x = 'politicalOrientation', color='condition', y = 'politicalAction', ylim=c(-1, 1), palette = c("#43a38d", "#f18334"))
nice_table(projectData %>% group_by(politicalOrientation) %>% 
  t_test(politicalAction ~ condition, paired = FALSE, var.equal=TRUE) %>% mutate(p = round(p, 3))%>%
  inner_join(cohens_d(projectData %>% group_by(politicalOrientation), politicalAction~condition, paired=FALSE)) %>%
  select(politicalOrientation, statistic, df, p, effsize))

politicalOrientation	statistic	df	p	effsize
conservative	-0.23	172	.816	-0.04
liberal	-2.88	244	.004**	-0.37

3-way interaction: Gender, political orientation, condition

ggerrorplot(projectData %>% filter((gender=='Man' | gender=='Woman') & politicalOrientation=='conservative') , x = 'gender', color='condition', y = 'politicalAction', ylim=c(-1, 1), palette = c("#43a38d", "#f18334"), title='Conservatives', xlab='')
ggerrorplot(projectData %>% filter((gender=='Man' | gender=='Woman') & politicalOrientation=='liberal') , x = 'gender', color='condition', y = 'politicalAction', ylim=c(-1, 1), palette = c("#43a38d", "#f18334"), title='Liberals', xlab='')
nice_table(projectData %>% filter((gender=='Man' | gender=='Woman'))%>% group_by(gender, politicalOrientation) %>% 
  t_test(politicalAction ~ condition, paired = FALSE, var.equal=TRUE) %>% mutate(p = round(p, 3)) %>%
  inner_join(cohens_d(projectData %>% filter((gender=='Man' | gender=='Woman'))%>% group_by(gender, politicalOrientation) , politicalAction~condition, paired=FALSE)) %>%
  select(politicalOrientation, gender, statistic, df, p, effsize))

politicalOrientation	gender	statistic	df	p	effsize
conservative	Man	-0.42	95	.673	-0.09
liberal	Man	-1.48	94	.141	-0.30
conservative	Woman	-0.07	73	.944	-0.02
liberal	Woman	-2.54	125	.012*	-0.46

```

Footnotes

1. (contributions by group 13 & group 8)

2. (contributions by group 7)

3. (contributed by group 10)

4. (contributed by group 3)

5. (contributed by group 3)

6. (contributions by group 11, group 1, group 3, group 8, group 6)

7. (contributed by group 6)

8. (contributed by group 11, group 4)

9. (contributions by group 13, group 2)

10. (contributed by group 5)