LPPL4660-001 Study Lab 4

Research question

Social media’s effects on young people’s political participation broadly

Hypothesis

Greater exposure to charged political content on social media causes an increase in political polarization in young adults.

Method

Participants were solicited via Prolific. Study was launched on 10/31 - 5 days before the election. We only recruited participants that live in the US and are young adults (i.e. 18-34 according to the US Census). We distributed two different studies that point to the same Qualtrics: one screened for Conservatives and one screened for Liberals.

Our target N was 265 Conservatives and 265 Liberals.

• Affords 80% power to detect effect size of d >= 0.24 in a two tailed t-test 

• Affords 80% power to detect effect size of d >= 0.35 for simple effects within political party

Procedure

Participants were randomly assigned to watch a political but neutral video in the control condition or a youTube clip of charged political content in the experimental condition. Participants could not proceed to the next page until the full length of the video had passed.

Control video: ¹

::::::: {style=” display: flex; flex-direction: row; align-items: center;“} ::: {style=”background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 40px; margin-right: 14px; width: 40px;“}

::: {style=” background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; margin-bottom: 6px; width: 100px;“}

::: {style=” background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; width: 60px;“} ::: ::::: :::::::

::: {style=” color:#3897f0; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:550; line-height:18px;“} View this post on Instagram

::::

::: {style=” background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 20px; width: 20px;“}

::: {style=” width: 0; height: 0; border-top: 2px solid transparent; border-left: 6px solid #f4f4f4; border-bottom: 2px solid transparent; transform: translateX(16px) translateY(-4px) rotate(30deg)“}

:::::

::: {style=” width: 0px; border-top: 8px solid #F4F4F4; border-right: 8px solid transparent; transform: translateY(16px);“}

::: {style=” background-color: #F4F4F4; flex-grow: 0; height: 12px; width: 16px; transform: translateY(-4px);“} :::

::: {style=” width: 0; height: 0; border-top: 8px solid #F4F4F4; border-left: 8px solid transparent; transform: translateY(-4px) translateX(8px);“} ::: :::::: ::::::::::::::

::: {style=” background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; margin-bottom: 6px; width: 224px;“}

::: {style=” background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; width: 144px;“} ::: :::::

A post shared by The Wall Street Journal (@wsj)

::::::::::::::::::::::::::::

Treatment video: ²

Participants then responded to four scales that serve as our dependent variables. In the scales, the relevant targets were piped in using embedded variables:

• otherParty = ‘liberals’ for conservatives; or ‘conservatives’ for liberals

• politicalOrientation = participant’s political orientation as indicated on a pretest on Prolific. liberal or conservative

• otherParty_formal = “Democrat” for conservatives, or “Republican” for liberals

Dependent Variables

Feelings about the opposing party. -3 Strongly disagree to 3 Strongly agree ³

1. I believe ${e://Field/otherParty}s are misinformed. 
2. I am frustrated when I see political opinions that oppose mine on social media. 
3. I feel that ${e://Field/otherParty}s are a threat to my country.
4. I feel angry or upset when I see political opinions that differ from my own.
5. I will block or unfollow ${e://Field/otherParty}s on social media because of their political views.

Social distance. -3 Strongly disagree to 3 Strongly agree ⁴

1. I would be upset if my friend married someone who is a ${e://Field/otherParty}
2. I would be upset if a close family member married someone who is a ${e://Field/otherParty}
3. I do not care if my friends are ${e://Field/otherParty}s or ${e://Field/politicalOrientation}s reverse scored
4. I do not follow people from the opposite party on social media
5. I would not be willing to date someone who is a ${e://Field/otherParty}

Aversion. -3 Strongly disagree to 3 Strongly agree ⁵

1. ${e://Field/otherParty_formal}s are untrustworthy
2. I dislike ${e://Field/otherParty_formal}s
3. The ${e://Field/otherParty_formal} party is dishonest
4. The ${e://Field/otherParty_formal} party is immoral
5. The ${e://Field/otherParty_formal} party is cold

Incivility. -3 Strongly disagree to 3 Strongly agree ⁶

1. I do not help ${e://Field/otherParty}s when they need it
2. I am happy when negative things happen to ${e://Field/otherParty}s
3. I try not to support businesses owned by ${e://Field/otherParty}s
4. ${e://Field/otherParty}s deserve sympathy reverse scored
5. It is not appropriate to mock ${e://Field/otherParty}s for their political beliefs reverse scored
6. I try to help ${e://Field/otherParty}s when they need it reverse scored

Demographic Variables

• Ethnicity
• Hispanic [yes/no]
• Gender
• Age

Results

projectData = read.csv('ProjectData.csv') %>% 
  filter(Progress==100) %>% 
  mutate(#reverse score some itenms
    socialDistance_3 = 8 - socialDistance_3,
    incivility_4 = 8 - incivility_4,
    incivility_5 = 8 - incivility_5,
    incivility_6 = 8 - incivility_6) %>% 
  mutate(#collapse & recode all scales to -3 to 3
    feelings = rowMeans(select(.,matches("feelings")), na.rm=TRUE)-4, 
    socialDistance = rowMeans(select(.,matches("socialDistance")), na.rm=TRUE)-4,
    aversion = rowMeans(select(.,matches("aversion")), na.rm=TRUE)-4,
    incivility = rowMeans(select(.,matches("incivility")), na.rm=TRUE)-4) %>% 
 select(condition, politicalOrientation_pre, feelings, socialDistance, aversion, incivility, ethnicity, gender, socialDistance_2) %>% 
  rename(political_orientation = politicalOrientation_pre, social_distance = socialDistance, negative_feelings = feelings)

550 people opened the survey. After dropouts, we collected a total of

N = 535

nice_table(projectData %>% group_by(condition) %>%  count() %>% arrange(desc(n)))

condition	n
control	268
experimental	267

nice_table(projectData %>% group_by(condition, political_orientation) %>%  count())

condition	political_orientation	n
control	conservative	137
control	liberal	131
experimental	conservative	132
experimental	liberal	135

Sample demographics

nice_table(projectData %>% group_by(ethnicity) %>%  count() %>% arrange(desc(n)))
nice_table(projectData %>% group_by(gender) %>%  count() %>% arrange(desc(n)))

ethnicity	n
White	329
Black	100
Asian	56
Not listed	17
Asian,White	9
Black,White	6
Amer. Indian/Alaskan	5
Polynesian/Pacific Islander	3
Amer. Indian/Alaskan,Black	2
Amer. Indian/Alaskan,White	2
Asian,Black	2
	1
Arab	1
Arab,White	1
Black,Not listed	1

gender	n
Man	265
Woman	248
Non-binary	9
Man,Transgender	4
Non-binary,Gender Queer	3
Woman,Transgender	2
Man,Rather not say	1
Transgender	1
Transgender,Gender Queer	1
Woman,Gender Queer	1

Overall responses

Negative feelings

Social distance

I would be upset if a close family member married someone who is a [conservative/liberal]

projectData %>% mutate(upsetMarriage = ifelse(socialDistance_2 >4, 'agree', ifelse(socialDistance_2==4, 'neither', 'disagree'))) %>% 
  count(upsetMarriage) %>% group_by(upsetMarriage) %>% 
  mutate(percent = round(n/nrow(projectData)*100, 2)) %>% 
  ggplot(aes(x=1,fill=upsetMarriage, y=percent, label=percent))+
  geom_bar(position='stack', stat='identity')+
  coord_flip()+ylab('')+xlab('')+
  theme_classic()+ 
  geom_text(aes(y=percent-7),position='stack', color='white')+
  theme(axis.ticks.y=element_blank(), axis.text.y=element_blank())

Aversion

Incivility

Hypothesis tests

Did people who watched a charged political video report stronger negative feelings about the opposite party?

projectData_tests = projectData %>% 
  select(c(1:6)) %>% 
  mutate(ID = 1:nrow(projectData)) %>% 
  pivot_longer(3:6, 
               names_to = "variable")

ggerrorplot(projectData_tests, x = 'variable', color='condition', y = 'value', ylim=c(-1, 2), ylab="", xlab="")
nice_table(projectData_tests %>% group_by(variable) %>% 
  t_test(value ~ condition, paired = FALSE, var.equal=TRUE) %>% mutate(p = round(p, 3)) %>% 
  inner_join(cohens_d(projectData_tests %>% group_by(variable), value~condition, paired=FALSE)) %>%  
  select(variable, statistic, df, p, effsize))

variable	statistic	df	p	effsize
aversion	1.03	533	.303	0.09
incivility	-0.25	533	.804	-0.02
negative_feelings	-0.25	533	.799	-0.02
social_distance	-0.27	533	.785	-0.02

Exploratory Analyses

Did sentiment differ by political orientation?

ggerrorplot(projectData_tests, x = 'variable', color='political_orientation', y = 'value', ylim=c(-1, 2), ylab="", xlab="", palette=c('#cb2212', '#2a7bb7'))
nice_table(projectData_tests %>% group_by(variable) %>% 
  t_test(value ~ political_orientation, paired = FALSE, var.equal=TRUE) %>% mutate(p = round(p, 3)) %>% 
  inner_join(cohens_d(projectData_tests %>% group_by(variable), value~political_orientation, paired=FALSE)) %>%  
  select(variable, statistic, df, p, effsize))

variable	statistic	df	p	effsize
aversion	-7.81	533	< .001***	-0.68
incivility	-3.90	533	< .001***	-0.34
negative_feelings	-8.73	533	< .001***	-0.75
social_distance	-10.52	533	< .001***	-0.91

Did political orientation shape reactions to the charged political video?

ggerrorplot(subset(projectData_tests, political_orientation=='conservative'), x = 'variable', color='condition', y = 'value', ylim=c(-1, 2), ylab="", xlab="", title='Conservatives', palette=c('#fc9d9b', '#cb2212'))
nice_table(subset(projectData_tests, political_orientation=='conservative') %>% group_by(variable) %>% 
  t_test(value ~ condition, paired = FALSE, var.equal=TRUE) %>% mutate(p = round(p, 3)) %>% 
  inner_join(cohens_d(subset(projectData_tests, political_orientation=='conservative') %>% group_by(variable), value~condition, paired=FALSE)) %>%  
  select(variable, statistic, df, p, effsize))
ggerrorplot(subset(projectData_tests, political_orientation=='liberal'), x = 'variable', color='condition', y = 'value', ylim=c(-1, 2), ylab="", xlab="", title='Liberals', palette=c('#a9d0e4', '#2a7bb7'))
nice_table(subset(projectData_tests, political_orientation=='liberal') %>% group_by(variable) %>% 
  t_test(value ~ condition, paired = FALSE, var.equal=TRUE) %>% mutate(p = round(p, 3)) %>% 
  inner_join(cohens_d(subset(projectData_tests, political_orientation=='liberal') %>% group_by(variable), value~condition, paired=FALSE)) %>%  
  select(variable, statistic, df, p, effsize))

variable	statistic	df	p	effsize
aversion	1.24	267	.216	0.15
incivility	-0.18	267	.861	-0.02
negative_feelings	0.04	267	.970	0.00
social_distance	-0.05	267	.964	-0.01

variable	statistic	df	p	effsize
aversion	0.39	264	.695	0.05
incivility	-0.08	264	.933	-0.01
negative_feelings	-0.23	264	.815	-0.03
social_distance	-0.13	264	.898	-0.02

Footnotes

1. contributed by group 3

2. contributed by group 11

3. contributed by group 1

4. contributed by group 8

5. contributed by group 8

6. contributed by group 8