# Loading Packages
library(tidyverse)
library(psych)
library(gridExtra)Replication of The Radical Flank: Curse or Blessing of a Social Movement? by Dasch et al. (2024, Global Environmental Psychology)
Introduction
Within social movement scholarship, a radical flank effect (RFE) is the influence of activists with more extreme ideologies and tactics on a movement, particularly on how the public perceives it. Evidence is mixed as to whether the presence of radical flanks increases or decreases public support. Dasch et al. (2024) used an experimental approach to clarify this relationship and found across two studies that radical flanks can increase support for moderate groups and decrease support for radical ones. I plan to replicate the first study to further test the effect of radical flanks on support for moderate groups and confirm their other findings. This project complements my prior research on the effects of collective action tactics but takes a more psychological rather than sociological approach.
Dasch et al. (2024) provide an online supplement, which contains thorough documentation of their procedures, Qualtrics surveys, data, and code. Additionally, they pre-registered their study. All of these factors increase the likelihood of successful replication.
To replicate experiment 1, I will recruit around 130 participants through Prolific. These participants will be given a Qualtrics survey in which they will read a fictional news story about two activist groups protesting, then answer questions. Participants will be randomly assigned to either a nonviolent or radical flank version of the story. The primary outcome measure will be support for the moderate group.
Links:
Methods
Power Analysis
The size for the main effect of reading about a violent flank group and a non-violent main group compared to two peaceful groups was \(d=0.50\) in the first study. To achieve 80% power to detect this effect, I would need 128 participants in total (172 participants for 90% power and 210 participants for 95% power).
Planned Sample
Given budgetary constraints, I will recruit a sample of 130 participants, with the expectation that I may lose a couple when I exclude invalid responses. In line with the original study, I will restrict my sample to US residents.
Materials
Dasch et al. (2024) provide the following overview of their materials:
After the participants consented to the online survey, they were randomly allocated into one of the three conditions: nonviolent condition, radical flank condition, or violent condition. In each condition, the participants were introduced to the key issue: the concerns of some surrounding the environmental impact of consuming animal products. Next, they each read a fictitious newspaper article which described a social movement protesting against a (fictional) university’s decision to reduce the number of green food products at catered events. After briefly being introduced to the conflict at hand, the participants read short descriptions about the protest actions of two university-based groups, main group (PlanetVeg) and flank group (SuperGreens).
In the nonviolent condition … protesters in the nonviolent main group held a peaceful demonstration and sent an email to staff and students at the university, and protesters in the nonviolent flank group organized a peaceful demonstration and distributed a petition to staff and students. On the other hand, in the violent condition … the violent main group held a forceful demonstration and occupied the University canteen to verbally and physically harass anyone who was eating meat. This caused one person who was in the canteen at the time to take some time away from the university. Further, the violent flank group organized a demonstration and distributed a petition. They verbally and physically harassed anyone who showed a lack of interest in the petition, and the demonstration injured a security guard. Finally, in the radical flank condition … the main group was nonviolent (PlanetVeg) and the flank group was violent (SuperGreens). The description of the nonviolent main group was identical to the text describing their behaviors in the nonviolent condition, and the description of the violent flank group was identical to the text describing their behaviors in the violent condition.
I will use the same materials. However, I will only use two of the conditions—the nonviolent main and flank group condition and the nonviolent main and violent flank group condition—to maximize power.
Procedure
I will follow Dasch et al.’s (2024) procedure:
Following the article and the reading comprehension checks, participants answered several questions about their perceptions of … the two groups separately. Furthermore, participants completed measures about their general attitudes … and a demographic survey.
Analysis Plan
I will use the following measures reported by Dasch et al. (2024):
Manipulation Checks
We measured perceived extremity by asking participants to rate their perception of radicality and violence of the social movements (e.g., “PlanetVeg are radical/violent” … ; “SuperGreens are radical/violent” … ). All measures in this study used a scale from 1 = Strongly Disagree to 7 = Strongly Agree if not noted otherwise
Identification With the Protesters
We adapted three items from Simpson and colleagues (2018) to measure identification with the protesters (e.g., “I feel similar to the protesters”). Participants rated their agreement with these three statements for the main group … and the flank group … separately.
Support for the Movement
We measured the support for … the two subgroups with four items each (e.g., “I would participate in a protest of PlanetVeg”), based on the scale of Feinberg and colleagues (2020) regarding both the main group (PlanetVeg) … and the flank group (SuperGreens).
Sympathy for the Movement’s Cause
We operationalized sympathy for the movement’s cause with four items from the New Environmental Paradigm scale (Dunlap et al., 2000). The identical four items … have already been used in Schmitt and colleagues (2019) (e.g., “Over-consumption is posing a serious risk to humankind and life on planet earth”).
Demographic Questionnaire
Participants completed a demographic survey including items about age, gender, political ideology, and current or achieved level of education.
Like Dasch et al. (2024), I will exclude participants who spend less than 15 seconds on the treatment or who report that they do not consent to the study. I will also drop participants who do not answer the key survey questions.
I will compute Cohen’s d to test if support for the main group systematically varies across conditions. I will similarly test if identification with the main group varies across conditions. As a secondary analysis, I will test whether identification mediates the relationship between condition and support for the main group.
Differences from Original Study
I am dropping the violent main group violent flank condition.
I am dropping the instructional manipulation checks in favor of a comprehension check.
I have adjusted the formatting of the news articles to align with the Study 2 articles because I could not find a picture of Dasch et al.’s (2024) Study 1 articles.
I am including an income demographic variable.
Methods Addendum (Post Data Collection)
I did not perform a mediation analysis because the direct effect of condition on support for the main group was not significant.
Actual Sample
I recruited 130 US participants using Prolific. After the data exclusions described above, I was left with a final sample of 127 with the following demographic makeup:
Mean age: 42
Proportion Female: 0.496
Proportion employed full-time: 0.614
Proportion with a 4-year college degree or more: 0.63
Proportion with a yearly income of 80 thousand or more: 0.48
Proportion liberal or very liberal: 0.378
Proportion conservative or very conservative: 0.268
Proportion moderate: 0.173
Differences from pre-data collection methods plan
While not preregistered, I dropped participants who spent less than 3 seconds on the non-demographic post-treatment questions. This decision decreased the sample size by one participant and did not meaningfully change the results.
Like Dasch et al.’s (2024), I decided to test if sympathy for the movement’s cause moderated the relationship between the condition and support for the main group.
Results
Data preparation
# Importing Data
d <- read_csv("data/anonymized_data- values.csv",
show_col_types = FALSE)
demos <- read_csv("data/anonymized_data- names.csv", show_col_types = FALSE)
# Rename demo variables
demos <- demos |>
select(ResponseId, gender, age, employment, education, income, polit_ideology, reside, contains('comprehend')) |>
rename_with(~ paste0(., "_labeled"), -ResponseId)
# Combine datasets
d <- d |>
left_join(demos, by = 'ResponseId')
# Excluding participants
cat('Total participant count:', NROW(d))Total participant count: 130cat("Didn't consent:", sum(d$consent == 0))Didn't consent: 0d <- d |>
# Remove participants who didn't consent
filter(consent == 1) |>
# Remove participants who spent less than 15 seconds on treatment
filter((condition == 0 & `nv_text_timing_Page Submit` >= 15) |
(condition == 1 & `rad_text_timing_Page Submit` >= 15)) |>
# Remove participants who spent less than 3 seconds on non-demos pages
filter(`manip_time_Page Submit` >= 3,
`ident_time_Page Submit` >= 3,
`dv_time_Page Submit` >= 3,
`sym_time_Page Submit` >= 3) |>
# Remove participants who reside in the UK
filter(reside_labeled == 'No')
cat('Final sample size:', NROW(d))Final sample size: 127d <- d |>
# Reverse coding the manipulation check, identification, and support variables so agreement corresponds to larger values than disagreement
mutate(across(contains('#'), ~ 8 - .x)) |>
# Creating aggregate variables
mutate(
main_manipulation_check = (`manipulation_check#1_1` + `manipulation_check#1_2`) / 2,
rad_manipulation_check = (`manipulation_check#2_1` + `manipulation_check#2_2`) / 2,
main_support = (`dv_support#1_1` + `dv_support#1_2` +
`dv_support#1_3` + `dv_support#1_4`) / 4,
rad_support = (`dv_support#2_1` + `dv_support#2_2` +
`dv_support#2_3` + `dv_support#2_4`) / 4,
# Reverse code the 3rd sympathy question
sympathy_3 = 8 - sympathy_3,
sympathy = (sympathy_1 + sympathy_2 + sympathy_3 + sympathy_4) / 4,
main_identify = (`identification#1_1` + `identification#1_2` + `identification#1_3`) / 3,
rad_identify = (`identification#2_1` + `identification#2_2` + `identification#2_3`) / 3
)Confirmatory analysis
# Plotting Manipulation Checks
d |>
group_by(condition) |>
summarize(mean = mean(rad_manipulation_check, na.rm = T),
CI_lower = t.test(rad_manipulation_check)$conf.int[1],
CI_upper = t.test(rad_manipulation_check)$conf.int[2],
.groups = "drop") |>
mutate(cond_name = ifelse(condition == 1, "Radical Flank", "Non-Violent")) |>
ggplot(aes(x = cond_name, y = mean, fill = cond_name)) +
geom_col(width = 0.5) +
geom_errorbar(aes(ymin = CI_lower, ymax = CI_upper), position = "dodge", width = 0.2) +
labs(y = "Flank Group Perceived as Radical (1-7)", x = 'Condition', title = "Manipulation Check") +
guides(fill="none") +
theme_bw() +
theme(plot.title = element_text(hjust = 0.5)) +
coord_cartesian(ylim = c(NA, 7))
The manipulation worked. Participants in the radical flank condition were significantly more likely to perceive the flank group as violent and radical.
# H1: Support for main group
h1_cohen <- cohen.d(main_support ~ condition, data = d)
cat("Support for main group: Cohen's d =", h1_cohen$cohen.d[, 'effect'],
', p-value = ', h1_cohen$p)Support for main group: Cohen's d = 0.03278967 , p-value = 0.8554554# H2: Identification with main group
h2_cohen <- cohen.d(main_identify ~ condition, data = d)
cat("Identification with main group: Cohen's d =", h2_cohen$cohen.d[, 'effect'],
', p-value = ', h2_cohen$p)Identification with main group: Cohen's d = -0.04003879 , p-value = 0.8232653# Does sympathy for the movement moderate support main group?
lm(main_support ~ condition * sympathy, data = d) |> summary()
Call:
lm(formula = main_support ~ condition * sympathy, data = d)
Residuals:
Min 1Q Median 3Q Max
-2.939 -1.125 0.194 1.213 2.693
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.7984 0.7959 1.003 0.318
condition -0.8629 1.0174 -0.848 0.398
sympathy 0.6282 0.1419 4.427 2.1e-05 ***
condition:sympathy 0.2877 0.1908 1.508 0.134
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 1.471 on 122 degrees of freedom
(1 observation deleted due to missingness)
Multiple R-squared: 0.3688, Adjusted R-squared: 0.3532
F-statistic: 23.76 on 3 and 122 DF, p-value: 3.542e-12# Plotting Replication H1 Effects
replication_h1_plot <- d |>
group_by(condition) |>
summarize(mean = mean(main_support, na.rm = T),
CI_lower = t.test(main_support)$conf.int[1],
CI_upper = t.test(main_support)$conf.int[2],
.groups = "drop") |>
mutate(cond_name = ifelse(condition == 1, "Radical Flank", "Non-Violent")) |>
ggplot(aes(x = cond_name, y = mean, fill = cond_name)) +
geom_col(width = 0.5) +
geom_errorbar(aes(ymin = CI_lower, ymax = CI_upper), position = "dodge", width = 0.2) +
labs(y = "Support for Main Group (1-7)", x = 'Condition', title = "Replication") +
guides(fill="none") +
theme_bw() +
theme(plot.title = element_text(size = 10, hjust = 0.5)) +
coord_cartesian(ylim = c(NA, 7))
# Plotting Original H1 Effects
original_h1_plot <- tibble(
cond_name = c('Non-Violent', 'Radical Flank'),
mean = c(3.36, 4.12),
CI_lower = c(3.36 - 1.96*1.52/sqrt(165), 4.12 - 1.96*1.50/sqrt(154)),
CI_upper = c(3.36 + 1.96*1.52/sqrt(165), 4.12 + 1.96*1.50/sqrt(154))
) |>
ggplot(aes(x = cond_name, y = mean, fill = cond_name)) +
geom_col(width = 0.5) +
geom_errorbar(aes(ymin = CI_lower, ymax = CI_upper), position = "dodge", width = 0.2) +
labs(y = "Support for Main Group (1-7)", x = 'Condition', title = "Original Study") +
guides(fill="none") +
theme_bw() +
theme(plot.title = element_text(size = 10, hjust = 0.5)) +
coord_cartesian(ylim = c(NA, 7))
grid.arrange(original_h1_plot, replication_h1_plot,
ncol = 2,
top = 'Effect of Condition on Support for Main Group')
Unlike in the original study, the peaceful and radical flank conditions were not significantly different in terms of support for or identification with the main group. Sympathy also did not mediate the effect of the condition on support.
Exploratory analyses
# Sample demographics
cat('Mean age:', mean(d$age))Mean age: 42.02362cat('Prop female:', round(mean(d$gender_labeled == 'Female'), 3))Prop female: 0.496cat('Prop employed fulltime:', round(mean(d$employment_labeled == 'Employed full time'), 3))Prop employed fulltime: 0.614cat('Prop 4 yr college+:', round(mean(d$education_labeled %in%
c("4 year degree", "Doctorate",
"Master's degree", "Professional degree")), 3))Prop 4 yr college+: 0.63cat('Prop income 80K+:', round(mean(d$income_labeled %in%
c('$80,000 - $99,999', '$100,000 - $149,999',
"More than $150,000")), 3))Prop income 80K+: 0.48cat('Prop liberal or very liberal:', round(mean(d$polit_ideology_labeled %in%
c('Liberal', "Very Liberal")), 3))Prop liberal or very liberal: 0.378cat('Prop conservative or very conservative:', round(mean(d$polit_ideology_labeled %in%
c('Conservative', "Very Conservative")), 3))Prop conservative or very conservative: 0.268cat('Prop moderate:', round(mean(d$polit_ideology_labeled %in%
c('Moderate')), 3))Prop moderate: 0.173# Check that conditions are demographically balanced
d |>
group_by(condition) |>
summarize(
mean_age = mean(age, na.rm = TRUE),
prop_female = round(mean(gender_labeled == "Female", na.rm = TRUE), 3),
prop_fulltime = round(mean(employment_labeled == "Employed full time", na.rm = TRUE), 3),
prop_4yr_plus = round(mean(education_labeled %in%
c("4 year degree", "Doctorate",
"Master's degree", "Professional degree"), na.rm = TRUE), 3),
prop_income_80k_plus = round(mean(income_labeled %in%
c("$80,000 - $99,999", "$100,000 - $149,999",
"More than $150,000"), na.rm = TRUE), 3),
prop_liberal = round(mean(polit_ideology_labeled %in%
c("Somewhat Liberal", "Liberal",
"Very Liberal"), na.rm = TRUE), 3)
)# A tibble: 2 × 7
condition mean_age prop_female prop_fulltime prop_4yr_plus
<dbl> <dbl> <dbl> <dbl> <dbl>
1 0 43.5 0.469 0.531 0.703
2 1 40.6 0.524 0.698 0.556
# ℹ 2 more variables: prop_income_80k_plus <dbl>, prop_liberal <dbl># Confirm that related items are correlated
d |> select(`manipulation_check#1_1`, `manipulation_check#1_2`) |> cor(use = "complete.obs") manipulation_check#1_1 manipulation_check#1_2
manipulation_check#1_1 1.0000000 0.4032973
manipulation_check#1_2 0.4032973 1.0000000d |> select(`manipulation_check#2_1`, `manipulation_check#2_2`) |> cor(use = "complete.obs") manipulation_check#2_1 manipulation_check#2_2
manipulation_check#2_1 1.0000000 0.7304006
manipulation_check#2_2 0.7304006 1.0000000d |> select(`dv_support#1_1`, `dv_support#1_2`,
`dv_support#1_3`, `dv_support#1_4`) |> cor(use = "complete.obs") dv_support#1_1 dv_support#1_2 dv_support#1_3 dv_support#1_4
dv_support#1_1 1.0000000 0.8222632 0.7225112 0.7172825
dv_support#1_2 0.8222632 1.0000000 0.8143302 0.7954997
dv_support#1_3 0.7225112 0.8143302 1.0000000 0.8027197
dv_support#1_4 0.7172825 0.7954997 0.8027197 1.0000000d |> select(`dv_support#2_1`, `dv_support#2_2`,
`dv_support#2_3`, `dv_support#2_4`) |> cor(use = "complete.obs") dv_support#2_1 dv_support#2_2 dv_support#2_3 dv_support#2_4
dv_support#2_1 1.0000000 0.8818068 0.7380269 0.7579960
dv_support#2_2 0.8818068 1.0000000 0.8278301 0.8310834
dv_support#2_3 0.7380269 0.8278301 1.0000000 0.8270481
dv_support#2_4 0.7579960 0.8310834 0.8270481 1.0000000d |> select(`identification#1_1`, `identification#1_2`,
`identification#1_3`) |> cor(use = "complete.obs") identification#1_1 identification#1_2 identification#1_3
identification#1_1 1.0000000 0.8933712 0.9062206
identification#1_2 0.8933712 1.0000000 0.8950362
identification#1_3 0.9062206 0.8950362 1.0000000d |> select(`identification#2_1`, `identification#2_2`,
`identification#2_3`) |> cor(use = "complete.obs") identification#2_1 identification#2_2 identification#2_3
identification#2_1 1.0000000 0.9334886 0.9293937
identification#2_2 0.9334886 1.0000000 0.9195071
identification#2_3 0.9293937 0.9195071 1.0000000d |> select(sympathy_1, sympathy_2, sympathy_3, sympathy_4) |> cor(use = "complete.obs") sympathy_1 sympathy_2 sympathy_3 sympathy_4
sympathy_1 1.0000000 0.7311595 0.4466335 0.7707607
sympathy_2 0.7311595 1.0000000 0.5450924 0.7385207
sympathy_3 0.4466335 0.5450924 1.0000000 0.4689126
sympathy_4 0.7707607 0.7385207 0.4689126 1.0000000# Comprehension check
d |>
mutate(
main_comprehend_only_1_correct =
((condition == 0) & (nv_comprehend_main_labeled %in% c("PlanetVeg held a peaceful demonstration outside the Academy building.", "PlanetVeg sent an email to staff and students asking them to reflect on the effect of the University’s catering on climate change."))) |
((condition == 1) & (rad_comprehend_main_labeled %in% c("PlanetVeg held a peaceful demonstration outside the Academy building.", "PlanetVeg sent an email to staff and students asking them to reflect on the effect of the University’s catering on climate change."))),
flank_comprehend_only_1_correct =
((condition == 0) & (nv_comprehend_flank_labeled %in% c("SuperGreens held a peaceful demonstration at the Engineering Faculty building.", "SuperGreens distributed a petition aimed at staff and students."))) |
((condition == 1) & (rad_comprehend_flank_labeled %in% c("SuperGreens organised a demonstration that escalated into violence and caused injury to a security guard.", "SuperGreens distributed a petition and if people refused to sign it, they harassed them."))),
) |>
select(main_comprehend_only_1_correct, flank_comprehend_only_1_correct) |>
lapply(summary)$main_comprehend_only_1_correct
Mode FALSE TRUE
logical 11 116
$flank_comprehend_only_1_correct
Mode FALSE TRUE
logical 17 110 # Time spend on treatments
d |>
mutate(treatment_time = if_else(condition == 1,
`rad_text_timing_Page Submit`,
`nv_text_timing_Page Submit`)) |>
group_by(condition) |>
summarize(deciles = list(quantile(treatment_time, probs = seq(0, 1, 0.1)))) |>
unnest_wider(deciles)# A tibble: 2 × 12
condition `0%` `10%` `20%` `30%` `40%` `50%` `60%` `70%` `80%` `90%` `100%`
<dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 0 110. 135. 147. 171. 203. 230. 255. 287. 375. 459. 1417.
2 1 38.4 115. 146. 209. 250. 267. 299. 344. 472. 546. 937.Discussion
Summary of Replication Attempt
The primary result of interest—the positive effect of reading about a radical flank activist group on support for a moderate main group—failed to replicate. I found no significant difference in support between participants who read a news article about a peaceful main and flank group and those who read about a peaceful main group and a violent flank group. I similarly found that the treatment groups did not differ in their level of identification with the moderate main group of activists.
Commentary
There are several factors that could plausibly have contributed to the failure to replicate. First, my sample differed from Dasch et al.’s (2024) across several dimensions. Their sample was 68% female, while mine was 50%; theirs was also 48% conservative, while mine was 27%. It may be that their results do not generalize to the people in my sample and vice versa. Second, my sample size was relatively small (127 participants), so the probability that my treatment groups differed in important ways after randomization is higher. For example, by chance, the non-violent condition had 15% more people with at least a 4-year college degree. Perhaps these differences obscured the treatment effect. Third, most participants only selected one option on the comprehension checks, even though the question invited them to “select all actions that apply.” While the single selection was correct most of the time, the failure to select the two correct answers for each comprehension check may indicate an issue with participant attention or understanding. Ultimately, there is significant heterogeneity in the direction and strength of the radical flank effect across the literature. Even in Dasch et al.’s (2024) paper, the second study did not replicate the effect on support for the main group. More work is needed to identify when and how radical flank effects emerge.