Replication of The moral roots of environmental attitudes by Feinberg and Willer (2012, Psychological Science)
My focus (in my masters program and generally) is on sustainability. From a psychological lens, I’m particularly interested in how people can be persuaded to care more about the climate crisis and to take action as a result. When focusing specifically on potential climate-psychology interventions in the US, I’ve reflected on political polarization as being one of the driving forces that diminishes climate care and action; a result like the one shown in Study 3 could provide a really exciting basis for interventions, and honestly I’d just love to replicate it and see for myself if it works. I actually worked in Robb Willer’s lab for a few months, some years ago, and remember always being excited about the idea of moral reframing, and although it’s been a while since I’ve worked in the lab and indeed since I’ve worked in the field of psychology in general, I’m still really intrigued and excited by these ideas.
The stimuli used for this survey-based study are all passages of texts and accompanying pictures in the form of newspaper articles, easy enough to embed in a survey once one actually has those articles. I wasn’t able to find the stimuli online, but I have a friend who works closely with Robb Willer’s lab, and he already assured me that he can put me in contact with somebody from whom I can procure the stimuli. This may also be helpful if I have trouble tracking down some of the particular scales they used (including the disgust measure and measure of pro-environmental attitudes).
I’m more worried about data analysis, but this is because I’m pretty rusty with statistical methods, and so I think a lot of the process will just be reminding myself of the basics and then potentially learning some new methods, especially the mediation analysis. However, these are skills I’ve been wanting to brush up on for a while, so I’m excited for the opportunity.
Introduction
[No abstract is needed.] Each replication project will have a straightforward, no frills report of the study and results. These reports will be publicly available as supplementary material for the aggregate report(s) of the project as a whole. Also, to maximize project integrity, the intro and methods will be written and critiqued in advance of data collection. Introductions can be just 1-2 paragraphs clarifying the main idea of the original study, the target finding for replication, and any other essential information. It will NOT have a literature review – that is in the original publication. You can write both the introduction and the methods in past tense.
Methods
Power Analysis
Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.
Planned Sample
Planned sample size and/or termination rule, sampling frame, known demographics if any, preselection rules if any.
Materials
All materials - can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.
Procedure
Can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.
Analysis Plan
Can also quote directly, though it is less often spelled out effectively for an analysis strategy section. The key is to report an analysis strategy that is as close to the original - data cleaning rules, data exclusion rules, covariates, etc. - as possible.
Clarify key analysis of interest here You can also pre-specify additional analyses you plan to do.
Differences from Original Study
Explicitly describe known differences in sample, setting, procedure, and analysis plan from original study. The goal, of course, is to minimize those differences, but differences will inevitably occur. Also, note whether such differences are anticipated to make a difference based on claims in the original article or subsequent published research on the conditions for obtaining the effect.
Methods Addendum (Post Data Collection)
You can comment this section out prior to final report with data collection.
Actual Sample
Sample size, demographics, data exclusions based on rules spelled out in analysis plan
Differences from pre-data collection methods plan
Any differences from what was described as the original plan, or “none”.
Results
Data preparation
Data preparation following the analysis plan.
Confirmatory analysis
The analyses as specified in the analysis plan.
Side-by-side graph with original graph is ideal here
Exploratory analyses
Any follow-up analyses desired (not required).
Discussion
Summary of Replication Attempt
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Commentary
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.