Replication of Study ‘How Quick Decisions Illuminate Moral Character’ by Clayton R. Critcher, Yoel Inbar and David A. Pizarro (2012, Psychological Science)

Author

Caroline Porche (cporche@ucsd.edu)

Published

Invalid Date

Introduction

Critcher, Inbar, and Pizarro (2012) found that the speed of decision-making plays a major role in shaping how we evaluate someone’s moral character. Quick decisions often signal certainty, which leads to stronger judgments—quick moral decisions get positive evaluations, while quick immoral ones result in harsher criticism. In contrast, slow decisions suggest indecision or conflict, leading to more moderate assessments. In this replication, I’ll be testing these findings again, but with a stronger emphasis on randomization to reduce any potential biases that may have influenced the original study. This could pose challenges, especially if additional variables or confounding factors emerge that were not fully accounted for before. By refining the experimental design, I aim to provide a clearer understanding of how decision speed impacts moral evaluations and address any limitations in the original methodology. To conduct this experiment, participants will be presented with scenarios where individuals make either moral or immoral decisions, with the critical variable being the speed of the decision (quick vs. slow). After each scenario, participants will rate the decision-maker’s moral character, perceived certainty, and impulsivity. The challenge will be creating scenarios that clearly manipulate decision speed without introducing unintended biases—ensuring quick decisions don’t seem impulsive by default and slow decisions aren’t perceived as overly reflective. Pretesting will be necessary to refine the stimuli and ensure participants interpret them consistently and as intended.

[Repository] (https://github.com/carolineporche1/critcher2012)

[Original Paper] (https://github.com/carolineporche1/critcher2012/blob/main/original_paper/How_quick_decisions_illuminate_moral_cha.pdf)

[Paradigm] (https://ucsd-psych201a.github.io/critcher2013_2/quick_decisions.html)

Methods

Power Analysis

Moral Character Evaluations

A significant main effect of decision type (F(1, 117) = 541.52, p < .001) showed that Justin, who made a quick decision (M = 6.44, SE = .08), was perceived differently than Nate (M = 2.15, SE = .12). Additionally, the moral nature of the decision significantly influenced character evaluations (F(1, 117) = 127.07, p < .001), with returning the wallet regarded as more morally favorable.

Decision Speed and Emotional Impulsivity

Justin was perceived as less emotionally impulsive (M = 2.40, SE = .11) compared to Nate (M = 3.79, SE = .12), (F(1, 117) = 95.26, p < .001). However, this difference in emotional impulsivity did not influence moral evaluations (t < 1).

Decision Speed and Moral Evaluation Polarization

A significant interaction between decision type and speed was observed (F(1, 117) = 127.07, p < .001). Quick immoral decisions led to more negative evaluations (t(54) = 8.28, p < .001), while quick moral decisions resulted in more positive evaluations (t(63) = 7.71, p < .001).

Certainty as a Mediator

Quick decisions were associated with higher certainty (F(1, 117) = 706.6, p < .001). Certainty mediated the relationship between decision speed and moral evaluation, with greater certainty explaining more negative evaluations for immoral decisions and more positive evaluations for moral decisions.

Power Analysis and Sample Size

We planned to perform power analyses to determine the sample sizes needed to achieve 80%, 90%, and 95% power for detecting effect sizes from the original study. However, the original study did not report exact effect sizes, mean differences, or error bars. As a result, effect sizes could not be calculated, and sample sizes could not be determined using this methodology. Given the original study’s sample size of 119 participants, we followed the standard practice of multiplying the sample size by 2.5, resulting in a target sample size of 298 participants.

Planned Sample

The planned sample size is 298.

Materials

“Immediately following the description of Justin and Nate’s actions, we asked participants the following sets of items (all on 1–7 scales):

  • Quickness: “As a manipulation check, participants indicated how quickly (vs. slowly) the decision was made”

  • Moral character evaluation: “The three moral evaluation items had participants assess the agents’ underlying moral principles and standards… by asking whether the agent: “has entirely good (vs. entirely bad) moral principles,” “has good (vs. bad) moral standards,” and “deep down has the moral principles and knowledge to do the right thing”.”

  • Certainty: “Participants indicated “how conflicted she felt when making her decision” (reverse-scored), “how many reservations she had” (reverse-scored), whether the target “was quite certain in her decision” (vs. had considerable reservations), and “how far she was from choosing the alternate course of action”.”

  • Emotional Impulsivity: “Participants indicated to what extent the person remained “calm and emotionally contained” (reverse-scored) and “became upset and acted without thinking.””

  • Perceived Motives: “…participants rated their motives to: “get more money” and ’protect her children”.”

Procedure

Participants will read about two men, Justin and Nate, who come upon separate cash-filled wallets in a grocery store parking lot. Justin “was able to decide quickly” what to do, while Nate “was only able to decide after long and careful deliberation.”

Participants will then be assigned to one of two conditions:

  1. Moral decision: Justin and Nate both decide not to steal the money and return the wallets.
  2. Immoral decision: Justin and Nate both decide to keep the money and drive off.

Participants will be asked to rate how quickly each decision was made and evaluate the moral character, certainty, and impulsivity of both Justin and Nate. Randomization will be used to control for order effects.

  • Note:

    • This procedure was followed precisely as outlined in the original article without deviations.

Analysis Plan

  • Primary Analysis:
    I will conduct a two-way ANOVA to examine how decision speed (quick vs. slow) interacts with their decision (accept vs. reject the offer) in influencing participants’ moral character evaluations. Based on the original study, I expect that quick decisions to sell her son will result in more negative judgments, while quick refusals will lead to more positive (though marginal) evaluations.

  • Additional Analyses:
    I plan to explore whether these effects generalize to different types of moral dilemmas. Another potential avenue is to examine whether participant demographics (e.g., gender or age) moderate the observed effects.

  • Data Cleaning and Exclusion Rules:
    I will follow the same data cleaning procedures outlined in the original study. I’ll ensure that participants with incomplete responses are excluded and covariates like emotional impulsivity are properly accounted for.

  • Note:
    This analysis plan closely follows the approach described in the original article, with the same data exclusions, control variables, and covariate adjustments. Any additional analyses I conduct will build on the original methodology to enhance our understanding of decision speed and moral judgment.

Design Overview

  • The two factors that are manipulated throughout this study are ‘decision type’ and ‘decision speed’

  • Throughout the study there are five measures taken and they were not repeated

  • This study uses a between-participants design which tests the robustness of the effect rather than a within subjects design which could have had the consequence of an order effect

  • There is no mention of steps taken to reduce demand characteristics within the study

  • Participant’s previous exposures to relevant situations of the study may pose as potential confounds

Differences from Original Study

Explicitly describe known differences in sample, setting, procedure, and analysis plan from original study. The goal, of course, is to minimize those differences, but differences will inevitably occur. Also, note whether such differences are anticipated to make a difference based on claims in the original article or subsequent published research on the conditions for obtaining the effect.

Different from the original study, we will only be conducting Experiment One. In addition, our participants will not be from UC Berkeley and our experiment will be conducted online rather than in person. We can’t control the exact environment as the original study but that is not predicted to make a difference. In the end of our study, we are planning to add an attention check.

Methods Addendum (Post Data Collection)

You can comment this section out prior to final report with data collection.

Actual Sample

Sample size, demographics, data exclusions based on rules spelled out in analysis plan

Differences from pre-data collection methods plan

Any differences from what was described as the original plan, or “none”.

Results

Data preparation

In the initial stages of our data preparation, we will begin by loading our data set and uploading the necessary libraries and functions. Then we will begin cleaning our data, which may include dropping any Na values or special characters that we do not need for our analysis. After this, we will filter the data and variables needed for the analysis and create the necessary rows and columns.

```{r include=F} ### Data Preparation

Load Required Libraries

library(dplyr) library(tidyr) library(purrr) library(readr) library(stringr) library(jsonlite) library(magrittr)

Load Data

Define the directory path

data_directory <- “/Users/carolineporche/Desktop/CSS/CSS204/critcher2013_2/data/osfstorage-archive”

Retrieve a list of all CSV files within the directory

file_list <- list.files(path = data_directory, pattern = “*.csv”, full.names = TRUE, recursive = TRUE)

Read and merge data from all CSV files, adding a ‘subject_id’ column

merged_data <- file_list %>% map_dfr(~ { # Load the data from each file temp_data <- read_csv(.x)

# Extract and format the subject ID from the filename
id <- tools::file_path_sans_ext(basename(.x))

# Insert subject ID as the initial column
temp_data %>% mutate(subject_id = id, .before = 1)

})

Data Cleaning and Filtering

Make sure ‘processed_data’ is correctly defined

Keep essential columns and create ‘processed_data’

processed_data <- combined_data %>% select(subject_id, rt, stimulus, response) %>%

# Add ‘condition’ column based on stimulus content mutate(condition = case_when( str_detect(stimulus, “steal the money”) ~ “moral”, str_detect(stimulus, “pocketed the money”) ~ “immoral”, TRUE ~ NA_character_ )) %>%

# Propagate ‘condition’ values downward to fill in any gaps fill(condition, .direction = “down”) %>%

# Organize data by ‘subject_id’ group_by(subject_id) %>%

# Add attention check based on the last row’s response mutate(attention_check = if_else( row_number() == n(), str_extract(response, ‘“Q0”:\d+’) %>% str_extract(“\d+$”), NA_character_ )) %>%

ungroup() %>% mutate(attention_check = as.numeric(attention_check)) %>% fill(attention_check, .direction = “downup”)

Parse ‘response’ column and annotate target information

Convert JSON-formatted response data into structured format

expanded_data <- processed_data %>% mutate(parsed_response = map(response, ~ fromJSON(.) %>% as.data.frame())) %>% unnest(cols = parsed_response) %>%

# Add a ‘target’ column to label data for Justin and Nate mutate(target = case_when( row_number() %% 2 == 1 ~ “Justin”, TRUE ~ “Nate” ))

Transform data to a long format and generate a combined identifier

reshaped_data <- expanded_data %>% pivot_longer( cols = starts_with(“Q”), names_to = “question”, values_to = “response_value” ) %>% mutate(question_value = paste(target, question, sep = “_“)) %>% select(-target, -question, -response, -stimulus)

Include only participants who successfully passed the attention check

valid_data <- reshaped_data %>% filter(attention_check == 0)

eligible_subjects <- valid_data %>% filter( (question_value == “Justin_Q0” & response_value == 1) | (question_value == “Nate_Q0” & response_value == 0) ) %>% distinct(subject_id) %>% pull(subject_id)

Visualize the results

ggplot(reshaped_data, aes(x = decision_speed, y = moral_character, fill = decision)) + geom_bar(stat = “identity”, position = “dodge”) + labs(title = “Interaction Between Decision Speed and Moral Character”, x = “Decision Speed”, y = “Moral Character Evaluation”)

Confirmatory analysis

model <- aov(moral_character ~ decision_speed * decision, data = reshaped_data) summary(model)

“Replicating Experiment 1, a two-way ANOVA on the moral character evaluation composite confirmed that the influence of decision speed depended on their decision, F(1, 549) = 16.08, p < .001 (see Figure 1).”

Exploratory analyses

Any follow-up analyses desired (not required).

Discussion

Summary of Replication Attempt

Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.

Commentary

Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.