Replication of Study “How Quick Decisions Illuminate Moral Character” by Critcher et al (2013, Social Psychological and Personality Science)

Author

Prosperity Land

Published

October 31, 2024

Introduction

The original study examined how the speed of decision-making influences perception of moral character. It found that decision speed helps classify whether a choice is moral or immoral depending on the perceived certainty of the decision. Quick decisions perceived as moral led to more positive character evaluations, while quick immoral decisions resulted in harsher judgments. In contrast, slower decisions were viewed as less certain, leading to more moderate evaluations.

Replicating Experiment 1 from “How Quick Decisions Illuminate Moral Character” offers an opportunity to explore how decision speed affects character judgments. In this experiment, participants will read a scenario about two characters who find separate cash-filled wallets in a grocery store parking lot. One character decides quickly, while the other takes longer. Participants will be randomly assigned to either a moral or immoral condition. Those in the moral condition will learn that both characters returned the wallets, while those in the immoral condition will read that both characters kept the money. Participants will then rate each character based on perceived decision speed, moral principles, decision certainty, and impulsiveness. A two-way ANOVA will then be used to analyze whether decision speed consistently affects moral judgments. 

Design Overview

Manipulated Variables

  • Decision speed (quick or slow) and moral conditions (immoral or moral act) were the two manipulated variables.

Measurements

  • Four primary measures were taken in Experiment 1

    • Moral character evaluation

    • Quickness

    • Certainty

    • Emotional impulsivity

Study Design

  • The experiment is a between participants design because each participant was only exposed to one condition as they were randomly assigned to either the moral or immoral scenario.

Repetition in Measures

  • Although the same set of measures were used to evaluate Justin and Nate, it does not count as repetition because the scenarios were not identical (decision speed was different).

Consequences of Design Alterations

  • If the study was modified to a within participant design, where participants were exposed to both moral and immoral scenarios, it may introduce bias as participants might become more aware of the study’s purpose. This awareness could potentially influence the way they respond to survey questions.

Reducing Demand characteristics

  • Randonly assigning conditions and using a within participant design helps reduce demand characteristics.

Methods

Power Analysis

Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.

Planned Sample

Planned sample size and/or termination rule, sampling frame, known demographics if any, pre-selection rules if any.

Materials

To replicate this experiment, we will use JSPsych to create an online study involving written scenarios and participant surveys. Participants will read scenarios involving Justin and Nate, and then rate them on quickness, moral character, decision certainty, and impulsivity.

From the article:

  • “Participants read about both Justin and Nate”

  • “Participants read about Pamela”

  • “Participants were asked to rate the quickness, moral character evaluation, decision certainty, and emotional Impulsivity of the agents”

Procedure

We plan to follow the procedure outlined by the authors.

Experiment 1:

  • “Participants read about both Justin and Nate, two men who each independently came upon two separate cash-filled wallets in the parking lot of a local grocery store. Justin ‘was able to decide quickly’ what to do. Nate ‘was only able to decide after long and careful deliberation.’”

  • “Participants assigned to the moral condition learned both men ‘did not steal the money but instead left the wallet with customer service.’ Those in the immoral condition learned instead that both men ‘pocketed the money and drove off.’”

  • “Participants were asked to rate the quickness, moral character evaluation, decision certainty, and emotional impulsivity of the agents on 1-7 scales.”

  • Quickness: “participants indicated how quickly (vs. slowly) the decision was made”

  • Moral Character Evaluation: “participants assess the agents’ underlying moral principles and standards…by asking whether the agent: “has entirely good (vs. entirely bad) moral principles,” “has good (vs. bad) moral standards,” and “deep down has the moral principles and knowledge to do the right thing.”

  • Certainty: “Participants indicated ‘how conflicted [each] felt when making his decision’, ‘how many reservations [each] had’, whether the target ‘was quite certain in his decision’, and ‘how far [each] was from choosing the alternate course of action.’”

  • Emotional Impulsivity: “Participants indicated to what extent the person remained “calm and emotionally contained” (reverse-scored) and ‘became upset and acted without thinking.’”

Analysis Plan

Our primary focus in this study is to evaluate moral character based on decision morality and decision speed. To minimize error and confounding, we plan to include several covariates, including randomization decision category (moral or immoral condition), decision speed (quickness), moral character evaluation, decision certainty, and emotional impulsivity. Once our data is collected, we will exclude responses with non-answers and participants who fail our attention check at the end of the survey, which we will implement to assess attentiveness during the experiment. With these measures, we hope to ensure that the data is good quality and ready for analysis. We also intent to increase the generalizability of our findings by collecting demographic data, such as participants’ age and gender.

Differences from Original Study

Explicitly describe known differences in sample, setting, procedure, and analysis plan from original study. The goal, of course, is to minimize those differences, but differences will inevitably occur. Also, note whether such differences are anticipated to make a difference based on claims in the original article or subsequent published research on the conditions for obtaining the effect.

In our replication project, we will introduce several features that will differ from the original experiment. One difference is the addition of an attention check at the end of the experiment, where we will ask participants to name one of the characters from the scenarios. This attention check is designed to ensure participants are engaged throughout the study and will be used to assist in the data cleaning process. Another difference is that our participants will not be limited to undergraduate students since we are conducting an online study. While our goal is to replicate the original findings, we anticipate slight variance due to the diverse backgrounds of our participants.

Methods Addendum (Post Data Collection)

You can comment this section out prior to final report with data collection.

Actual Sample

Sample size, demographics, data exclusions based on rules spelled out in analysis plan

Differences from pre-data collection methods plan

Any differences from what was described as the original plan, or “none”.

Results

Data preparation

For our preparation plan, we will remove observation that meet our exclusion criteria. We will also pivot our dataset to a long format to make graph creation easier. Lastly, we will investigate each outlier to determine whether it is suspicious or problematic before deciding to remove it.

Data preparation following the analysis plan.

Confirmatory analysis

The analyses as specified in the analysis plan.

Side-by-side graph with original graph is ideal here

The statistical test we will use to justify the main inference of the paper is a two-way ANOVA. This test will allow us to analyze the interaction between decision speed and decision type to determine how they influence moral character evaluations. It will also use it to assess the individual and combined effects of both independent variables.

Exploratory analyses

Any follow-up analyses desired (not required).

Discussion

Summary of Replication Attempt

Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.

Commentary

Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.