Replication of How Quick Decisions Illuminate Moral Character by Critcher et al (2013, Social Psychological and Personality Science)

Original Authors: Clayton R. Critcher, Yoel Inbar, & David A. Pizarro

Introduction

Morality is one of the many complex facets and areas of research within social psychology. Being able to study relationships between a multitude of different justifications for certain positive or negative actions can provide valuable insight into how people reach these perceptions. Performing a replication study of the first experiment in “How Quick Decisions Illuminate Moral Character” is a key step in building a foundation for these insights. In replicating this experiment there will be two fictional characters who will be part of four different scenarios: one in which an individual comes to a “moral” decision quickly, one where the same individual comes to the same decision after a longer time of consideration, another where another individual comes to an “immoral” decision quickly, and lastly, one where the same individual comes to the same “immoral” decision after some consideration. Following the disclosure of these four scenarios, the participants will be surveyed on their opinions regarding the following: quickness, moral character, certainty, emotional impulsivity. To analyze the results, we will be running a two-way 2x2 ANOVA.

Some potential challenges in this replication involve the possibility of introducing confounding variables when surveying for emotional impulsivity or certainty. This may lead to multicollinearity and some difficulties in determining whether the moral judgements were solely based upon speed.

Design Overview

•How many factors were manipulated?

The only factor being manipulated is the scenario given to each participant: moral or immoral.

•How many measures were taken?

There are 9 measures: a 1-7 ranking of quickness, moral character, certainty, emotional impulsivity, experimental condition (moral/immoral). In addition to these, we will implement an attention check.

•Did it use a within-participants or between-participants design?

Between-participants because they are only exposed to one condition (moral or immoral scenario), and not both.

•Were measures repeated?

No, measures were not repeated.

•What would have been the consequence of changing one of these decisions? (e.g. moving from within to between participants). Could it have been either way?

Moving to a within-participants study would ultimately reveal the intention of the study to participants.

•Were steps taken to reduce demand characteristics?

The study aims to reduce demand characteristics by implementing a between-participants design in addition to randomization of scenarios given to participants.

How would you critique the design?

• Can you think of any potential confounds to the study?

  • By asking participants to rate their certainty, we aim to reduce any potential confounds to the study.

• Do you see any limitations to generalizability?

  • In the original study, there are some limitations due to the sample used which only included UC Berkeley undergraduate students and some community members. This limits the generalizability but we aim to reduce this issue through the use of an online format that reaches a wider range of participants.

Methods

Materials

We will be using JSPsych to develop the majority of this online experiment. We believe that due to the nature of this experiment and its reliance on written scenarios and survey responses, the online format through an interactive interface is ideal.

In terms of the instructions for participants, we will present three passages: one that introduces the two men (Justin and Nate), who have found the cash-filled wallets. Justin is the character that came to a quick decision and Nate is the one who took longer. Next, we will randomly assign each participant to one of two morality conditions: one that describes both men returning the wallet(acting morally), and the other where they keep the money(acting immorally).

Procedure

Experiment 1:

• “Participants read about both Justin and Nate, two men who each independently came upon two separate cash-filled wallets in the parking lot of a local grocery store. Justin was able to decide quickly what to do. Nate was only able to decide after long and careful deliberation.”
• “Participants assigned to the moral condition learned both men ‘did not steal the money but instead left the wallet with customer service.’ Those in the immoral condition learned instead that both men ‘pocketed the money and drove off.’”
• “Participants were asked to rate the quickness, moral character evaluation, decision certainty, and emotional impulsivity of the agents on 1-7 scales.”

Analysis Plan

Our main analysis will be to understand the impact of decision morality and speed on moral character judgement. In efforts to achieve this goal, we will be collecting subject IDs and the randomized decision category of the participants. Each participant will then be surveyed on the following for each character (Justin and Nate): quickness (decision speed), moral character evaluation, decision certainty, and emotional impulsivity. We may also be helpful to collect demographics such as age and gender to make more informed generalizability decisions post-experiment.

Furthermore, we plan on excluding any surveys that contain non-answers from our data analysis. In addition, we will be utilizing an attention check and any participants who fail will be removed as well.

Differences from Original Study

One potential difference from the original experiment is the setting in which it will be conducted. While it is never explicitly revealed whether the experiment was run electronically or in person, we have assumed that running our replication online, will introduce variability in terms of where each participant will be contributing from. However, this should not have any significant impact on our data or results.

Additionally, the online nature of our replication enables us to have a greater chance of diversifying our sample in terms of demographics such as age, race, region etc. This will inevitably increase our generalizability, unlike the original study that only had participants from UC Berkeley and its community members.

Lastly, we will be incorporating an attention check in our experiment, which was not part of the original. Because of the online nature of our experiment, it is difficult to identify any potential bots who may skew our survey results. Furthermore, there may be participants who are simply selecting answers without actually reading the instructions or scenarios. We have introduced an attention check at the end of the experiment that will ask the partipant to name a character from the scenarios. We are hopeful that this will mitigate the possibility of biases in our data.

Results

Data preparation

We plan to conduct data cleaning and wrangling prior to our analysis by removing any of the exclusion criteria as listed in the previous section. Additionally, we will be closely monitoring any potential outliers that may prove to be problematic in our final analysis. Lastly, we expect to pivot data into a long format for better data visualizations and easier statistical testing.

Confirmatory analysis

Our main analysis method is a two-way 2 (decision) x 2 (speed) Analysis of Variance (ANOVA) on moral evaluations. We will also be exloring the relationship between decision speed (fast vs. slow) and decision type (moral vs. immoral). Other possible analysis methods include a manipulation check, exploring the possible interactions between decision speed/emotional impulsivity and decision speed/certainty.