### Data Preparation
#### Load Relevant Libraries and Functions
library(tidyverse)
library(qualtRics)
library(emmeans)
#### Import data
test <- read_survey('~/Desktop/Test_data.csv')
#### Data exclusion / filtering
test$fail <- (rowSums(test %>% select(fail_ck_1, fail_ck_2)))
#### Prepare data for analysis - create columns etc.
test$harm <- ifelse(test$condition=="lib_exp", 1, 0)
test$orig <- ifelse(test$condition=="orig_con", 1, 0)
test$newcon <- ifelse(test$condition=="env_con", 1, 0)
test$ideology <- scale(test$Pol_ideology)[,1]
test$attitude <- (rowSums(test %>%
select(starts_with("GW"), GHG_reduc, Protect_env, Env_rank)) / 8)Replication of The moral roots of environmental attitudes by Feinberg and Willer (2012, Psychological Science)
[link to preview the survey]https://stanforduniversity.qualtrics.com/jfe/preview/previewId/c7925870-fb66-4dcb-bcac-aaa3cfa95694/SV_bK1wMQAbLJPkcwS?Q_CHL=preview&Q_SurveyVersionID=current
Note to TAs: this replication will be done in collaboration with a friend of mine, using his sources of funding, as he is also interested in whether this study will replicate. We will be adding in an additional control condition, but the nature of the analyses used in the original study will mean that this work will both demonstrate whether the original study replicates and also show any results from the additional control conditions. We will still just be focusing on Study 3, and primary attention will be paid to the “Environmental attitudes and policy preferences” section of Study 3, but disgust and its potential mediating role will also be measured.
Introduction
This project aims to both determine whether the findings of Study 3 replicate. It additionally aims to assess whether the nature of the control condition used in the original study affected these findings, by adding in a control condition that focuses on environmental topics without using moralized language.
Methods
Power Analysis
There’s no option for a multiple regression with multiple regression coefficients in G*Power.
Planned Sample
Sample size - 1000.
Materials
The experimental articles:
“The harm/care message described the harm and destruction humans are causing to the environment and emphasized how important it is for people to care about and protect the environment. Participants in the purity/sanctity condition read about how polluted and contaminated the environment has become and how important it is for people to clean and purify the environment. Additionally, three small pictures were presented with the persuasive messages. The purity/sanctity pictures showed a cloud of pollution looming over a city, a person drinking contaminated water, and a forest covered in garbage. The harm/care pictures showed a destroyed forest of tree stumps, a barren coral reef, and cracked land suffering from drought. Importantly, both messages ended positively, providing information regarding what people can do to improve the environment”
The control article:
“Participants in the neutral condition read an apolitical message on the history of neckties.”
Two additional control articles, created by my friend and myself, will be added. One will be an article that talks about the environment/climate change but uses morally neutral language, and the other will be a political but non-partisan article (talking about the branches of government or something along those lines).
Procedure
The original study was described as follows:
“Participants completed a demographic questionnaire…Participants in the harm/care and purity/sanctity conditions read a persuasive message modeled after a newspaper op-ed. These messages utilized words typi- cal of their respective moral domains…Participants in the neutral condition read an apolitical message on the history of neckties…After reading the message, participants indicated how much they felt five emotions, including disgust…Participants then completed a 3-item measure of proenvironmental attitudes, a 5-item mea- sure of support for proenvironmental legislation, and a 3-item measure of belief in global warming.”
My replication will differ from this in the following way: there will one additional control condition with a message about the environment and climate change that uses morally neutral language. This will allow for a deeper understanding of whether it was specifically the morally valenced language that had an effect on conservatives’ pro-environmental attitudes.
Additionally, we will be including some attention checks, but not filtering on them, because the original study did not filter on them.
Analysis Plan
Multiple regression analysis that compares the interactions between each of the five conditions and political ideology. Simple slopes analyses comparing both conservatives across conditions, and comparing liberals across conditions, as well as comparing conservatives and liberals within each condition.
Differences from Original Study
The addition of the control conditions is an obvious difference, but the multiple regression analysis will still allow for the same results to be calculated.
Methods Addendum (Post Data Collection)
Actual Sample
Differences from pre-data collection methods plan
Results
Data preparation
For repeat survey-takers, include the data from the first time they took the survey but exclude all other data.
Confirmatory analysis
Multiple regression analysis that compares the interactions between each of the four conditions and political ideology.
Simple slopes analyses comparing both conservatives across conditions, and comparing liberals across conditions, as well as comparing conservatives and liberals within each condition. (haven’t added this in yet because I’m trying to figure out whether there’s a better way to represent this than simple slopes. it’s very unintuitive in the original paper)
NOTE: the test analysis here used randomly generated data from Qualtrics, as I would have needed a lot of friends to take this survey if I wanted to properly conduct this analysis! Thus, it is unsurprising that there is no effect found :).
test_model = lm(scale(attitude)~harm*ideology+orig*ideology+newcon*ideology, data = test)
summary(test_model)
Call:
lm(formula = scale(attitude) ~ harm * ideology + orig * ideology +
newcon * ideology, data = test)
Residuals:
Min 1Q Median 3Q Max
-1.5446 -0.5429 -0.1511 0.5606 1.8542
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.24299 0.47082 -0.516 0.612
harm 0.41435 0.68225 0.607 0.552
ideology -0.12988 0.43220 -0.301 0.767
orig 0.13793 0.65813 0.210 0.836
newcon 0.34641 0.73382 0.472 0.643
harm:ideology 0.05194 0.68872 0.075 0.941
ideology:orig -0.24071 0.66319 -0.363 0.721
ideology:newcon 0.03940 0.71182 0.055 0.956
Residual standard error: 1.15 on 17 degrees of freedom
Multiple R-squared: 0.06369, Adjusted R-squared: -0.3219
F-statistic: 0.1652 on 7 and 17 DF, p-value: 0.9891Side-by-side graph with original graph is ideal here
Exploratory analyses
Discussion
Summary of Replication Attempt
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Commentary
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.