Replication of How Quick Decisions Illuminate Moral Character by Clayton R. Critcher, Yoel Inbar, & David A. Pizarro (2013, Social Psychological and Personality Science)
Introduction
In the article How Quick Decisions Illuminate Moral Character, authors Clayton R. Critcher, Yoel Inbar and David A. Pizarro examined how decision speed affects judgment of moral character. The hypothesis was that onlookers interpret quick decision speed as a signal of certainty that then aligns with a moral or immoral category. Participants in the study were asked to interpret a scenario where actors found a wallet and decided whether to return it or not. The actors decided either quickly or slowly and whether the actor acted morally or immorally was randomly assigned to the participant. Results showed that actors who made quick decisions to act morally were evaluated more positively, whereas quick decisions to act immorally led to more negative evaluations. Overall, the study found that the speed of moral decision-making can influence how others perceive moral character. After reading the passage participants ranked quickness, moral character evaluation, certainty, and emotional impulsivity on a scale from 1-7. A two-way ANOVA was used to analyze the results and reach a conclusion.
Design Overview
How many factors were manipulated?
- One factor will be manipulated, whether or not they get the moral or immoral scenario.
How many measures were taken?
- The original study had nine measures, a 1-7 ranking of quickness, moral character evaluation, certainty, emotional impulsivity for both Nate and Justin, and the experimental condition (moral or immoral). Our replication will include an extra measure from the attention check.
Did it use a within-participants or between-participants design?
- It is a between-participants design because the are assigned either moral or immoral and are never exposed to the condition they are not assigned.
Were measures repeated?
- No measures were repeated.
What would have been the consequence of changing one of these decisions? (e.g. moving from within to between participants). Could it have been either way?
- By changing from between to within participant design the consequences would be that the participants would become aware of the purpose of the study (moral or immoral condition).
Were steps taken to reduce demand characteristics?
- The structure of the study reduced demand characteristics by randomly assigning conditions.
How would you critique the design?
Can you think of any potential confounds to the study?
- Asking participants to rank certainty reduces confounds.
Do you see any limitations to generalizability?
- The original study participants were only UC Berkely undergraduate students or community members.
Methods
Power Analysis
The original study did not report the mean difference between decision speeds and its corresponding standard error. They also did not report the relevant mean and standard error for the interaction between decision and speed. Additionally, no error bars were present in their figure for visualizing experiment 1, so that information was not available to calculate required sample size that way either. Therefore, we are calculating our required sample size by doing the standard procedure of multiplying the original sample size (n = 119) by 2.5, to results in a desired sample size of N = 289 participants.
The information we needed to calculate was not available so we are doing the standard 2.5 times the original sample size.
Planned Sample
Planned sample size is 289 participants.
Materials
To collect data jsPsych will be used to create a publicly accessible online experiment. Below is the scenario and questions the participants will be asked. This follows the original article exactly.
“Participants read about both Justin and Nate, two men who each independently came upon two separate cash-filled wallets in the parking lot of a local grocery store. Justin ‘‘was able to decide quickly’’ what to do. Nate ‘‘was only able to decide after long and careful deliberation.’’ Participants assigned to the moral condition learned both men ‘‘did not steal the money but instead left the wallet with customer service.’’ Those in the immoral condition learned instead that both men ‘‘pocketed the money and drove off.’’
Quickness. As a manipulation check, participants indicated how quickly (vs. slowly) the decision was made.
Moral character evaluation. The three moral evaluation items had participants assess the agents’ underlying moral principles and standards by asking whether the agent: ‘‘has entirely good (vs. entirely bad) moral principles,’’ ‘‘has good (vs. bad) moral standards,’’ and ‘‘deep down has the moral principles and knowledge to do the right thing.’’
Certainty. We included 4 items to assess each actor’s perceived decision certainty. Participants indicated ‘‘how conflicted [each] felt when making his decision’’ (reverse-scored), ‘‘how many reservations [each] had’’ (reverse-scored), whether the target ‘‘was quite certain in his decision’’ (vs. had considerable reservations), and ‘‘how far [each] was from choosing the alternate course of action.’’
Emotional impulsivity. In order to ensure that decision speed was not simply taken as a proxy for emotional impulsivity (a feature previously shown to affect moral judgments; Pizarro, Uhlmann, & Salovey, 2003), we assessed perceptions of the emotionally impulsive nature of the decision with 2 items. Participants indicated to what extent the person remained ‘‘calm and emotionally contained’’ (reverse-scored) and ‘‘became upset and acted without thinking.’’”
Link to experimental paradigm here.
Procedure
The replication project will be replicating experiment 1. The following passages are quoted directly from the original article:
- “Participants read about both Justin and Nate, two men who each independently came upon two separate cash-filled wallets in the parking lot of a local grocery store.”
- “Participants assigned to the moral condition learned both men ‘did not steal the money but instead left the wallet with customer service.’ Those in the immoral condition learned instead that both men ‘pocketed the money and drove off.’”
- “Immediately following the description of Justin and Nate’s actions, we asked participants the following sets of items (all on 1–7 scales): quickness, moral character evaluation, certainty, and emotional impulsivity.”
Analysis Plan
The key analysis of interest is how the speed of decision-making influences moral character evaluations. The analysis plan will follow the steps outlined in the original article. The analysis plan will cover a manipulation check, moral character evaluation, and perceived emotional impulsivity. The statistical test performed will be a two-way ANOVA, simple effects analysis, and mediation analysis. Data will be collected for the following variables: subject ID, immoral or moral condition, and a 1-7 ranking of quickness, moral character evaluation, certainty, and emotional impulsivity.
Data that is incomplete or from participants who do not pass the attention check will be excluded.
Differences from Original Study
The three main differences between replication and the original study will be that the sample population will be random, an attention check will be built-in, and demographic information will be collected. The original article surveyed 119 undergraduate students at the University of California, Berkeley, or members of the surrounding community. The replication trials will have a random sample population because the trials will be conducted online and open to the public. The difference in data collection should not affect the findings but may allow generalization to a wider population.
An attention check will be added to the end of the study where participants will be asked to type the name of either of the characters from the scenarios they were presented. The addition of the attention check will help eliminate misleading or incomplete data that could come from participants who did not truly engage with the material.
The last difference is that two demographic questions will be added to the study about age and gender. This may allow for greater generalizability of the findings.
Methods Addendum (Post Data Collection)
You can comment this section out prior to final report with data collection.
Actual Sample
Sample size, demographics, data exclusions based on rules spelled out in analysis plan
Differences from pre-data collection methods plan
Any differences from what was described as the original plan, or “none”.
Results
Data preparation
For the statistical analysis, our main question of interest involves testing the effect of two factors (speed and decision) and their interaction on the response variable (moral character judgement); thus a type III, two-way analysis of variance (ANOVA) test is appropriate. A type III ANOVA tests each main effect and interaction, adjusting for all other factors and interactions in the model.
Below is pseudo-code for the planned analysis:
{r} # Fit a model with the interaction of interest intmod <- aov(Judg ~ speed * decision + uncertainty + impluse, data = dat)
Use a Type III, 2x2 Two-way ANOVA test the results will tell us if the interaction term in needed in the model
Anova(intmod, type = “III”)
The results of the type III two-way ANOVA will tell us whether a model with the interaction term is better than just a model with all the predictors additively. After the best model is found, the ANOVA assumptions and model diagnostics, such as normality, homogeneity of variances, and independence will be assessed.
Assessing Independence of Observations is a thought exercise
No participants responses should impact another participants responses, therefore there is no reason to suspect violations of this assumption.
Next, generate the standard suite of diagnostic plots
par(mfrow=c(2,2)) plot(intmod)
Assess Homogeneity of Variance with a Residuals vs Fitted plot
Assess Normality of Residuals with a Normal Q-Q Plot
Extra residual plots
eij = residuals(intmod) hist(eij,main = “Histogram of residuals”) plot(density(eij), main = “Density plot of residuals”, ylab = “Density”, xlab = “Residuals”)
Determine whether we had a balanced design with the tally function
This will give us the sample size of each randomized condition group
library(mosaic) tally(~ decision, data = dat)
Confirmatory analysis
The confirmatory analysis will be the Two-way Analysis of Variance. This text will examine whether perceived certainty influenced moral judgments by using the decision speed data. From the original article, “it will be a two-way 2 (decision) x 2 (speed) ANOVA on the relative motive composite returned a main effect of decision as well as the predicted Decision Speed interaction.”
Exploratory analyses
Any follow-up analyses desired (not required).
Discussion
Summary of Replication Attempt
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Commentary
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.