# power analysis for an ANCOVA (key test), including for 3 conditions and covariates
# install.packages("pwrss")
# library(pwrss)
#
# f <- 0.30 # The effect size the authors incorporated in their power analysis (effect size for Study 2 not reported in paper)
# f2 <- f^2 # Cohen's f-squared
# f2
# # [1] 0.09
#
# power <- pwrss.f.ancova(
# f2 = 0.30^2, # Cohen's f^2
# n.levels = 3, # 3 groups
# n.covariates = 3, # account for potential covariates
# alpha = 0.05,
# power = 0.80
# )
# For 0-3 covariates - 80%: 111, 90%: 144, 95%: 177Replication of Facing Discomfort: Avoided Negative Affect Shapes the Acknowledgment of Systemic Racism by Murray & Koopmann-Holm (2024, Emotion)
Introduction
Murray & Koopmann-Holm (2024) https://doi.org/10.1037/emo0001364 contributed a compelling finding to the broader literature on how cultural norms and emotional processes can impact how Americans make sense of racial inequality. The paper cites previous findings on how culture can influence the types of emotions that are considered socially acceptable or valuable, and in turn, individuals’ emotional processes. In Study 2 of the paper, the authors found that such culturally-driven emotional processes, specifically, the desire to avoid negative emotions (i.e., Avoided Negative Affect, ANA) in the United States, inhibited people’s perception of racism as embedded throughout America’s historical and current policies and practices. The results of this study identified a key barrier toward the development of a systemic understanding of racism in the United States, a type of understanding that has been shown to carry implications for support for racially progressive policies and other social change efforts. As my research focuses on these sorts of barriers, namely, the psychological factors that deter or compel Americans to address racial inequality, including the role of affect and American cultural values, this finding is particularly relevant. Ultimately, I aim to design interventions that incentivize White Americans to participate in racial equity efforts, or at the very least, to see racial inequality as a pressing, enduring issue, and this finding provides some of the foundation for that work, with its emphasis on the power of cultural context and the suppression of particular emotions.
To replicate this study, I will administer an online survey involving 3 conditions, an increased Avoided Negative Affect (ANA) condition, a decreased ANA condition, and a control condition. A copy of the full survey was made available online, and all 3 conditions are brief, written manipulations which did not require any advanced formatting or tools. I will randomly assign participants to one of the three conditions. After they review their assigned manipulation, they will respond to a series of measures, many of which are multi-item scales. They will answer questions about the emotions they feel in the current moment, including what they actually feel, ideally would like to feel, and would like to avoid feeling, the extent to which a series of incidents (some interpersonal, some systemic or institutional) reflect racism, how frequently they engage in behaviors or habits that reflect social desirability, their demographics (e.g., political ideology, gender, age, race, major, etc.), and what they thought the study was about. Participants will be debriefed slightly differently based on the condition they were assigned to since the first sentence of the debrief explains the manipulation from the beginning of the survey. The GitHub repository for this replication project can be found here: https://github.com/psych251/murray2024, and the preregistration can be found here: https://osf.io/kp3j4/overview.
I predict a few challenges with this replication. First, for the original study, the authors recruited undergraduate students from one university, which could indicate that effects are constrained to a niche population, and certain aspects of the survey may become inapplicable (e.g., demographic questions having to do with participants’ majors and school years). Also, the sample was racially diverse, and in an online sample, it might be more difficult to obtain the same racial composition. In the original analysis, the authors had to control for many factors. One challenge that could stem from this is that the some of the statistical models could be complicated to interpret, depending on whether it becomes necessary to incorporate demographic covariates. Finally, given the current political climate and perceptions of race-focused conversations, some participants in this replication study might be more hostile or less attentive to measures focused on these topics, which could lead to missing data and the need for exclusions. Further, the sample size is already quite small, so there is a possibility that too many exclusions could leave the replication study under-powered.
Methods
Paradigm link: https://stanforduniversity.qualtrics.com/jfe/form/SV_4IMpjoGrnAre61M
Power Analysis
Planned Sample
Based on power analyses at 80% and 90% power, the planned sample size is 124 participants, which is around the mean of the two samples stemming from each of those analyses (M = 127.5). This particular number was selected as the initial planned sample size was 130 participants; 130 participants were budgeted for the 13-minute replication study, but 6 of those participants were used for a pilot study (2 participants per condition). The pilot study data is not being counted toward the final dataset as it was used for troubleshooting the paradigm rather than confirmatory results. Power analyses were based on the effect size the original authors included in their power analysis, which was the effect size of a similar effect in their Study 1. They were based on an ANCOVA as that was the statistical test used in Study 1, which will also be used in Study 2 (the current study being replicated). I ran multiple power analyses to account for one simple main effect ANCOVA, with 1 covariate, versus an ANCOVA with 3 covariates. I incorporated these covariate numbers in the analyses given that, depending on the sample demographics across conditions, one of the primary statistical tests may include 2 to 3 covariates. However, samples did not change when 0 versus 3 covariates were incorporated into the analyses.
The sample will be a standard sample from Prolific, where the only criteria will be that participants are 18 years or older and based in the United States.
Materials
Conditions “ANA Manipulation. After completing the consent form and seeing a reminder to please answer the following questions thoughtfully and honestly, participants in the ‘increase ANA’ condition were shown the following instructions: ‘Research indicates that feeling negative emotions is counterproductive for information processing. Therefore, while you complete this survey, we ask you to try to avoid feeling negative emotions. For example, if you are starting to feel frustrated, stressed, anxious, sad, bored, or any other negative emotion while completing the survey, please try to push these feelings away. Please try your best to follow this instruction as we are studying the effects of you trying to avoid feeling negative emotions.’ In the ‘decrease ANA’ condition, participants were shown the following instructions: ‘Research indicates that feeling negative emotions is productive for information processing. Therefore, while you complete this survey, we ask you to try to accept feeling negative emotions. For example, if you are starting to feel frustrated, stressed, anxious, sad, bored, or any other negative emotion while completing the survey, please try to embrace these feelings. Please try your best to follow this instruction as we are studying the effects of you trying to accept feeling negative emotions.’ Lastly, participants in the control condition were shown the following instructions: ‘Research indicates that feeling emotions is sometimes productive and sometimes counterproductive for information processing. Therefore, while you complete this survey, we ask you to try to select the option that best describes you. For example, if you are unsure about a question in the survey, please try to select the response that is closest to representing your views. Please try your best to follow this instruction as we are studying the effects of information processing.’ Before completing a new portion of the survey, participants were reminded of their instructions. In the ‘increase ANA’ condition, participants saw, ‘Friendly reminder: Please try to avoid feeling negative emotions while completing this portion of the survey.’ In the ‘decrease ANA’ condition, participants saw, ‘Friendly reminder: Please try to accept feeling negative emotions while completing this portion of the survey.’ Finally, in the control condition, participants saw, ‘Friendly reminder: Please select the option that best describes you while completing this portion of the survey.’”
Dependent Measures “Momentary Affect Valuation Index (Momentary AVI). As in Study 1, we used the extended version (as described in Koopmann-Holm & Tsai, 2014) of the [Affect Valuation Index] (Tsai et al., 2006). However, to assess participants’ affective goals as a result of our manipulations, instead of using global ratings (average ratings over the course of a typical week), participants in this study rated their actual, avoided, and ideal affect (in that order) at that moment (i.e., ‘rate how you actually feel/want to avoid feeling/would ideally like to feel right now’). They rated the same 37 different affective states as in Study 1, but this time, they used a 5-point scale ranging from 1 (not at all) to 5 (extremely). We created the same mean-deviated aggregate scores for avoided negative (Cronbach’s alpha = .87, M = 1.26, SD = 0.42), actual negative (Cronbach’s alpha = .81, M = 0.01, SD = 0.52), ideal positive (Cronbach’s alpha = .88, M = 1.58, SD = 0.53), and actual positive affect (Cronbach’s alpha = .86, M = 0.24, SD = 0.62) as in Study 1.”
“Perceptions of Racism. Like in Study 1, participants rated the extent to which they perceived systemic and isolated racism in 14 scenarios on a 7-point scale from 1 (not at all) to 7 (certainly) (Bonam et al., 2019). However, because we were interested in acknowledging racism toward all people of color, we made a few adjustments to the wording of some of the isolated racism items, most of which were specifically about Black people. We also made these changes hoping that the internal consistency of the isolated racism subscale would increase. The updated items read ‘A person of color was pulled over for speeding by a White highway patrol officer. Unknown to the man, his registration had expired earlier that month. Rather than give him a ticket and let him continue, the officer impounded the vehicle at the man’s expense’ and ’A person of color made reservations for a rental car over the phone, but when she arrived in person to collect the car, the agent informed her that no cars were available.’Because most of the original systemic racism items referred to people of color in general, we made no changes to those items. Cronbach’s alphas (means and standard deviations in parentheses) were .85 for systemic racism (M = 4.70, SD = 1.23) and .62 for isolated racism (M = 5.09, SD = 1.10). Hence, the internal consistency for isolated racism was slightly higher compared to in Study 1, but so was the internal consistency for systemic racism.”
[Removed: Moral Foundations Questionnaire - measure not part of primary result being replicated]
“Social Desirability Scale. Because we instructed participants to try to avoid or accept negative emotions and then asked them about their ANA, we could have induced demand characteristics.Therefore, to control for participants’ tendency to complete the survey in a socially desirable manner, participants completed 14 items from the Marlowe–Crowne Social Desirability Scale (Crowne & Marlowe, 1960), which assesses participants’ desire for social approval using a 5-point scale ranging from 1 (not at all true) to 5 (extremely true). Items include statements like ‘Before voting I thoroughly investigate the qualifications of all the candidates.’ Cronbach’s alpha (mean and standard deviation in parentheses) was .66 for the scale (M = 3.08, SD = 0.46).”
“Demographics Questionnaire. Participants completed the same demographics questionnaire as in Study 1 assessing their gender, age, ethnicity, and political ideology.”
Procedure
Can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.
“All participants completed the measures listed below [in the order under ”Measures” in the Materials section] at a place and time convenient for them. Because this study was conducted online, we added seven attention check items throughout the survey (see the online supplemental materials [sample: ‘Please select ’2’ as your response for this item.’]). As in Study 1, we excluded participants who did not pass all attention checks. The study was approved by [Stanford University’s IRB as part of the PSYCH 251 course projects], and we obtained informed consent from all participants. At the end of the study, all participants were debriefed.”
Participants were assigned to one of the three conditions (discussed in the Materials section). See also the Introduction section, second paragraph for the full procedure.
Analysis Plan
Per the original study, to test for condition effects, I will run an ANCOVA; first, I will fit a linear model predicting systemic racism perceptions from experimental condition, while controlling for isolated racism perceptions. I will then run a Type II Sum of Squares ANCOVA on this model. If there are variations in demographic distributions across the three conditions (e.g., political ideology, age), I will run another model where I control for those demographic factors (i.e., add covariates). These covariates may differ from the exact covariates in the model from the original paper as they are sample-specific. I then will conduct planned pairwise contrasts comparing (1) the Increase ANA condition vs. the Control condition, and (2) the Increase ANA condition vs. the Decrease ANA condition. I will calculate reliability via Cronbach’s Alpha for systemic racism perceptions and isolated racism perceptions as well.
I will abide by the original paper’s exclusion rules of removing participants who do not pass all of the attention check questions (e.g., “Please select ‘2’ for this question”). To account for bot detection, I will also plan to remove participants with a reCAPTCHA score of below .5.
One exploratory analysis I will run will be the manipulation check analysis the authors ran: an ANCOVA with avoided negative affect as the outcome, condition as the independent variable, and social desirability, actual negative affect, and any necessary demographics (based on differences across conditions) as covariates. First, I will calculate the Cronbach’s Alpha of avoided negative affect, actual negative affect, and social desirability to check for reliability.
Differences from Original Study
The key differences in this replication from the original study include the sample, which will now come from Prolific instead of the Santa Clara University campus (where undergraduates were recruited, and there was a specific racial composition of the sample). Given the logistical and budgetary constraints of data collection for this course project, I will be using a standard Prolific convenience sample with no tailored screeners, apart from location in the United States. I also removed the Moral Foundations Questionnaire from my replication study to shorten the length, due to the aforementioned constraints but also because the specific results I aim to replicate do not include this measure in their respective statistical models. Finally, I tweaked the racial demographic questions; there is now only one question that asks about race/ethnicity, and the format is multiple choice rather than free response for the purpose of simpler data analysis. To shorten the paradigm, I removed some demographic questions: Place of Birth, the city grown up in and lived in for most of life, mother and father’s cultural backgrounds, marital status,record of travelling or living outside home country, time spent abroad in total, and current religion. I added one demographic question asking which country participants are currently residing in. For the affective measures (Momentary AVI), I reduced the number of items per matrix questions and randomized the items.
Methods Addendum (Post Data Collection)
Actual Sample
Since the paradigm took longer than estimated, I had to increase payment per participant and in turn, recruited a smaller sample size than the planned sample size at 112 participants. All recruited participants passed the reCAPTCHA check and fit the demographic criteria; I only had to remove one participant who failed one of the attention checks, so the final sample size was 111.
Differences from pre-data collection methods plan
None other than the sample change. Despite finding no significant differences across conditions in my primary and exploratory analyses (i.e., ANCOVAs), I still ran the planned contrasts.
Results
When running the confirmatory analysis model, where I only controlled for isolated racism acknowledgment, I found that there were no significant differences in systemic racism acknowledgment across conditions (F(2, 107) = .50, p = .61). Systemic racism acknowledgment and isolated racism acknowledgment composites were sufficiently reliable with alphas at .88 and .74 respectively. Descriptively, I observed some variation in condition samples based on race, gender, and family socioeconomic status, so I ran a follow up model incorporating these demographics as covariates. None of these covariates emerged as significant predictors, although race had a marginal effect (F(2, 98) = 2.2, p = .06). My exploratory analysis found that there were no significant differences in avoided negative affect across conditions when controlling for actual negative affect, social desirability, race, gender, and family socioeconomic status (F(2, 97) = .77, p = .47). This was still the case when running a simpler model where I only controlled for actual negative affect (F(2, 107) = .83, p = .44). Actual negative affect, avoided negative affect, and social desirability composites were highly reliable with alphas at .92, .95, and .83 respectively.
Data preparation
Confirmatory analysis
# primary result: does acknowledgment of systemic racism vary by condition (when controlling for isolated racism)?
conf_model <- lm(
Systemic_Racism_composite ~ Condition + Isolated_Racism_composite, data = data)
Anova(conf_model)Anova Table (Type II tests)
Response: Systemic_Racism_composite
Sum Sq Df F value Pr(>F)
Condition 1.086 2 0.4977 0.6093
Isolated_Racism_composite 82.618 1 75.7492 4.313e-14 ***
Residuals 116.703 107
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# contrasts
# get adjusted (estimated marginal) means for condition
emm <- emmeans(conf_model, ~ Condition)
emm <- emmeans(
conf_model, ~ Condition,
at = list(
Isolated_Racism_composite = mean(data$Isolated_Racism_composite, na.rm = TRUE))
)
contrast_list <- list(
increase_vs_control = c( 1, 0, -1),
increase_vs_decrease = c( 1, -1, 0)
)
pairs(emm) contrast estimate SE df t.ratio
Control Condition - Decrease ANA Condition -0.0175 0.243 107 -0.072
Control Condition - Increase ANA Condition -0.2186 0.245 107 -0.893
Decrease ANA Condition - Increase ANA Condition -0.2011 0.242 107 -0.832
p.value
0.9971
0.6457
0.6839
P value adjustment: tukey method for comparing a family of 3 estimates # check for demographic variation across conditions (race, gender, politics, age, SES, religious background/level)
data %>% count(Condition, Race, sort = TRUE) # decrease ANA more racially diverse # A tibble: 13 × 3
Condition Race n
<chr> <chr> <int>
1 Increase ANA Condition White 31
2 Control Condition White 30
3 Decrease ANA Condition White 20
4 Decrease ANA Condition Asian 8
5 Decrease ANA Condition Black or African American 8
6 Control Condition Black or African American 3
7 Increase ANA Condition Black or African American 3
8 Control Condition Asian 2
9 Increase ANA Condition Asian 2
10 Control Condition Hispanic or Latina/e/o/x 1
11 Decrease ANA Condition Hispanic or Latina/e/o/x 1
12 Decrease ANA Condition Other (please specify): 1
13 Increase ANA Condition American Indian or Alaskan Native 1data %>% count(Condition, Gender, sort = TRUE) # decrease and increase ANA majority female, control more gender balanced # A tibble: 6 × 3
Condition Gender n
<chr> <chr> <int>
1 Decrease ANA Condition Female 23
2 Increase ANA Condition Female 22
3 Control Condition Male 19
4 Control Condition Female 17
5 Decrease ANA Condition Male 15
6 Increase ANA Condition Male 15data %>% count(Condition, Political_Party, sort = TRUE)# A tibble: 12 × 3
Condition Political_Party n
<chr> <chr> <int>
1 Control Condition Democratic Party 18
2 Decrease ANA Condition Democratic Party 18
3 Increase ANA Condition Democratic Party 18
4 Control Condition Republican Party 14
5 Decrease ANA Condition Republican Party 14
6 Increase ANA Condition Republican Party 12
7 Decrease ANA Condition Other (please specify below) 6
8 Increase ANA Condition Other (please specify below) 4
9 Control Condition Other (please specify below) 3
10 Increase ANA Condition Libertarian Party 2
11 Control Condition Libertarian Party 1
12 Increase ANA Condition Green Party 1data %>% count(Condition, Family_SES, sort = TRUE) # variation across all 3 conditions # A tibble: 12 × 3
Condition Family_SES n
<chr> <chr> <int>
1 Increase ANA Condition Middle Income 18
2 Decrease ANA Condition Lower Middle Income 17
3 Decrease ANA Condition Middle Income 15
4 Control Condition Lower Income 12
5 Control Condition Lower Middle Income 10
6 Control Condition Middle Income 10
7 Increase ANA Condition Lower Middle Income 10
8 Increase ANA Condition Lower Income 7
9 Control Condition Upper Middle Income 4
10 Decrease ANA Condition Lower Income 4
11 Decrease ANA Condition Upper Middle Income 2
12 Increase ANA Condition Upper Middle Income 2# mean age by condition
data %>%
mutate(Age_num = parse_number(as.character(Age))) %>%
group_by(Condition) %>%
summarise(
mean_age = mean(Age_num, na.rm = TRUE),
n_with_age = sum(!is.na(Age_num)),
.groups = "drop"
)# A tibble: 3 × 3
Condition mean_age n_with_age
<chr> <dbl> <int>
1 Control Condition 47.8 36
2 Decrease ANA Condition 44.2 38
3 Increase ANA Condition 49.8 37# mean religious level by condition
data %>%
group_by(Condition) %>%
summarise(
mean_relig = mean(relig_level_num, na.rm = TRUE),
n_with_relig = sum(!is.na(relig_level_num)),
.groups = "drop"
) # A tibble: 3 × 3
Condition mean_relig n_with_relig
<chr> <dbl> <int>
1 Control Condition 4.94 36
2 Decrease ANA Condition 5.37 38
3 Increase ANA Condition 5.16 37# follow up analyses with demographic covariates
conf_model_cov <- lm(
Systemic_Racism_composite ~ Condition + Isolated_Racism_composite + Race + Gender + Family_SES, data = data)
Anova(conf_model_cov) Anova Table (Type II tests)
Response: Systemic_Racism_composite
Sum Sq Df F value Pr(>F)
Condition 4.239 2 2.0119 0.13923
Isolated_Racism_composite 56.765 1 53.8772 6.304e-11 ***
Race 11.510 5 2.1848 0.06193 .
Gender 0.528 1 0.5007 0.48086
Family_SES 1.308 3 0.4137 0.74351
Residuals 103.254 98
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# contrasts for covariate models
# ensure covariates are factors
data <- within(data, {
Race = as.factor(Race)
Gender = as.factor(Gender)
Family_SES = as.factor(Family_SES)
})
emm <- emmeans(
conf_model_cov, ~ Condition,
at = list(
Isolated_Racism_composite = mean(data$Isolated_Racism_composite, na.rm = TRUE)
),
weights = "proportional" # averages over factor covariates using sample proportions
)
contrast_list <- list(
increase_vs_control = c( 1, 0, -1),
increase_vs_decrease = c( 1, -1, 0)
)
contrast(emm, contrast_list) contrast estimate SE df t.ratio p.value
increase_vs_control -0.288 0.249 98 -1.156 0.2503
increase_vs_decrease 0.223 0.257 98 0.867 0.3879
Results are averaged over the levels of: Race, Gender, Family_SES Plot of primary model (without demographics)
# convert to a data frame for plotting
emm_df <- as.data.frame(emm)
# plot
ggplot(emm_df, aes(x = Condition, y = emmean, fill = Condition)) +
geom_col(color = "black", width = 0.6) +
geom_errorbar(aes(ymin = lower.CL, ymax = upper.CL), width = 0.15) +
scale_x_discrete(labels = c(
"Control Condition" = "Control",
"Decrease ANA Condition" = "Decrease ANA",
"Increase ANA Condition" = "Increase ANA"
)) +
labs(x = "Condition", y = "Acknowledgment of Systemic Racism")
Exploratory analyses
# manipulation check - does condition change levels of avoided negative affect?
exp_model <- lm(Avoided_Neg_composite ~ Condition + SD_composite + Actual_Neg_composite + Race + Gender + Family_SES, data = data)
summary(exp_model)
Call:
lm(formula = Avoided_Neg_composite ~ Condition + SD_composite +
Actual_Neg_composite + Race + Gender + Family_SES, data = data)
Residuals:
Min 1Q Median 3Q Max
-3.1096 -0.2868 0.2793 0.6315 1.2489
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.580560 1.281299 3.575 0.000548 ***
ConditionDecrease ANA Condition 0.263327 0.246430 1.069 0.287916
ConditionIncrease ANA Condition 0.259088 0.245759 1.054 0.294394
SD_composite -0.128475 0.139026 -0.924 0.357724
Actual_Neg_composite -0.047341 0.174820 -0.271 0.787123
RaceAsian -0.370913 1.069210 -0.347 0.729415
RaceBlack or African American 0.021055 1.052453 0.020 0.984080
RaceHispanic or Latina/e/o/x -0.857922 1.268328 -0.676 0.500384
RaceOther (please specify): -0.006236 1.450997 -0.004 0.996580
RaceWhite 0.273901 1.016581 0.269 0.788168
GenderMale -0.188444 0.200781 -0.939 0.350290
Family_SESLower Middle Income -0.108007 0.275319 -0.392 0.695699
Family_SESMiddle Income -0.275749 0.267486 -1.031 0.305155
Family_SESUpper Middle Income -0.819590 0.408425 -2.007 0.047564 *
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.9806 on 97 degrees of freedom
Multiple R-squared: 0.1163, Adjusted R-squared: -0.002153
F-statistic: 0.9818 on 13 and 97 DF, p-value: 0.4748Anova(exp_model) Anova Table (Type II tests)
Response: Avoided_Neg_composite
Sum Sq Df F value Pr(>F)
Condition 1.479 2 0.7692 0.4662
SD_composite 0.821 1 0.8540 0.3577
Actual_Neg_composite 0.071 1 0.0733 0.7871
Race 5.708 5 1.1872 0.3211
Gender 0.847 1 0.8809 0.3503
Family_SES 4.386 3 1.5203 0.2141
Residuals 93.273 97 exp_model_simple <- lm(Avoided_Neg_composite ~ Condition + Actual_Neg_composite, data = data)
summary(exp_model_simple)
Call:
lm(formula = Avoided_Neg_composite ~ Condition + Actual_Neg_composite,
data = data)
Residuals:
Min 1Q Median 3Q Max
-2.9992 -0.3161 0.3149 0.7084 1.0130
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 3.997401 0.300739 13.292 <2e-16 ***
ConditionDecrease ANA Condition 0.127315 0.229328 0.555 0.580
ConditionIncrease ANA Condition 0.302794 0.236042 1.283 0.202
Actual_Neg_composite -0.008555 0.162398 -0.053 0.958
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.985 on 107 degrees of freedom
Multiple R-squared: 0.01634, Adjusted R-squared: -0.01124
F-statistic: 0.5925 on 3 and 107 DF, p-value: 0.6212Anova(exp_model_simple) Anova Table (Type II tests)
Response: Avoided_Neg_composite
Sum Sq Df F value Pr(>F)
Condition 1.603 2 0.8261 0.4405
Actual_Neg_composite 0.003 1 0.0028 0.9581
Residuals 103.821 107 Discussion
Summary of Replication Attempt
Based on the primary result (ANCOVA), which showed no significant difference (F(2, 107) = .50, p = .61), the original result was not replicated. In other words, the current replication study, unlike the original study, did not find that avoided negative affect reduces acknowledgment of systemic racism. This was still the case when controlling for demographic variation across conditions (race, gender, and family socioeconomic status).
Commentary
To interpret this outcome, it is important to consider the results of the my exploratory analysis; these results indicated that the manipulations in this replication study did not shift avoided negative affect to begin with, as there was no significant difference in avoided negative affect across conditions (F(2, 97) = .77, p = .47). It is possible that the difference in participant population (i.e., Prolific, rather than undergraduates), could have led to this, as undergraduates were taking the study as part of coursework and therefore could have been more focused on the study to benefit their own learning. On Prolific, the incentives are different, so participants may have been more focused on finishing the survey to get their compensation rather than paying truly close attention to the manipulations. The majority of participants passing the attention checks perhaps is not an indicator of whether participants were immersed in the manipulation since the attention check questions simply ensure broader attention to the study (e.g., that participants are reading the questions). Additionally, perhaps for the purposes of this online study, there were small formatting changes I could have made to ensure strong attention. For example, I did reduce the number of emotions presented per page (in comparison to the original study), but to ensure participants slowed down and fully considered their different emotional responses, I could have presented one emotion at a time.