title:Replication of Study ‘How Quick Decisions Illuminate Moral Character’ by Critcher et al. (2013, Social Psychological and Personality Science)

https://ucsd-psych201a.github.io/critcher2013_2/quick_decisions.html
Project Check 1 link above. The responses help us understand how people perceive decision making in different moral scenarios. We have the scale from 1 to 7 and other scales form slowly or quickly for the people to look through after reading the prompt. [Repository] (https://github.com/carolineporche1/critcher2012)

[Original Paper] (https://github.com/carolineporche1/critcher2012/blob/main/original_paper/How_quick_decisions_illuminate_moral_cha.pdf)

[Paradigm] (https://ucsd-psych201a.github.io/critcher2013_2/quick_decisions.html)>

Pre Registration

Critcher2013

Introduction

I chose to replicate this project because of how it relates to my research interest in misinformation in how people develop their beliefs in ideas from social media such as

the far right. This theory is based on the idea of how people make quick judgments based on others’ quick decisions. The idea of people reacting to misinformation quickly based on assumptions is similar to this research done in this article. I am excited to hear more about how important reaction ime is going to affect people’s judgments on others.

I would like to conduct this by doing one question similar to the one about Pamela who gives her children a man for adoption. They have this situation and have participants react to it and start to compare in which the two people makes the decision quickly or longer and based on her response. This project is about people making quick judgments based on others’ quick decisions such as in the action of selling one’s child and many other examples. This could be in a matter of seconds in a certain group of experiments to test the reactant time and difference whether it be positive or negative based on an individual’s action.

Methods

Power Analysis
- The original paper did not provide us with enough information for this sample. We have looked through the paper to find them saying they did a medium sample size but did not provide an amount of people. This therefore made us choose a sample size of 298 as a default.
Planned Sample

Our sample size would be 298 people we surveyed.
Materials

“Quickness. As a manipulation check, participants indicated how quickly (vs. slowly) the decision was made”.

Moral character evaluation. The three moral evaluation items had participants assess the agents’ underlying moral principles and standards by asking whether the agent: ‘‘has entirely good (vs. entirely bad) moral principles,’’ ‘‘has good (vs. bad) moral standards,’’ and ‘‘deep down has the moral principles and knowledge to do the right thing.’’

Certainty. We included 4 items to assess each actor’s perceived decision certainty. Participants indicated ‘‘how conflicted [each] felt when making his decision’’ (reverse-scored), ‘‘how many reservations [each] had’’ (reverse-scored), whether the target ‘‘was quite certain in his decision’’ (vs. had considerable reservations), and ‘‘how far [each] was from choosing the alternate course of action.’’ The items had high internal reliability for both Justin (a 1⁄4 .89) and Nate (a 1⁄4 .81).

Emotional impulsivity. In order to ensure that decision speed was not simply taken as a proxy for emotional impulsivity (a feature previously shown to affect moral judgments; Pizarro, Uhlmann, & Salovey, 2003), we assessed perceptions of the emotionally impulsive nature of the decision with 2 items. Participants indicated to what extent the person remained‘‘calm and emotionally contained’’ (reverse-scored) and ‘‘became upset and acted without thinking.’’ The items were significantly correlated (Justin: r 1⁄4 .51, p < .001; Nate:r 1⁄4 .33, p < .001).”(Perceived Motives: “…participants rated Pamela’s motives to: “get more money” and”protect her children”.”

“Participants read about Pamela, who struggled to earn enough to provide for her two children. Pamela worked for Mr. Muir, a wealthy bachelor who took a special interest in Pamela’s son Alan, taking him on outings and buying him expensive presents. Although Pamela had no direct evidence that Mr. Muir’s intentions were unsavory, his obsession with Alan made Pamela feel suspicious of Muir’s true intentions. One day, Mr. Muir approached Pamela with a proposition. He told Pamela that he cared for Alan very much and would like to adopt him. If Pamela agreed, Muir would triple her sal- ary. We assumed that people would find it morally abhorrent to, in effect, sell one’s son to a man who might abuse him. Pamela was described as taking 3 seconds (quick) or 3 days (slow) to make her decision. We independently varied whether Pamela accepted or rejected Mr. Muir’s offer to sell her son. Participants answered the same quickness, moral character eva- luation (a 1⁄4 .83), certainty (a 1⁄4 .86), and emotional impulsivity (r 1⁄4 .43) items as in Experiment 2. We added 2 items that assessed Pamela’s perceived motives. On scales from 1 (not at all) to 7 (very strong), participants rated Pamela’s motives to: ‘‘get more money’’ and ‘‘protect her children’’ (r 1⁄4 .64, p < .001).”
Procedure

In Experiment 1, participants looked at two characters (Justin and Nate) who either quickly or slowly had a decision to either keep or return a found wallet. The study found that quick moral actions led to positive moral character evaluations, while quick immoral actions led to negative evaluations. The speed of the decision signaled the actor’s certainty, and certainty then changed the judgments of their character.

Analysis Plan

We got rid of participants who they can’t find data on in the missing data section. We used a two-way ANOVA in order to find the main effect of decision type and how fast the decision is based on the how good the person is We used a two-way ANOVA in order to analyze the interaction effects from how fast the decision is done and the type of decision We used T-tests in order to have a comparison of how decision speed on both types of decision impacted moral character evaluation.T-tests for perceived certainty of decisions and its relationship to decision speed and perceived impulsivity. We used a 2 (decision) x 2 (speed) ANOVA was performed to analyze the perceived motives of Pamela.

#Design overview

The factors that were manipulated was decision type and decision speed. There was measures of perceived motives being 4 of them which were quickness, moral character evaluation, emotional impulsivity, and certainty. The between because they changed the expirment each particitciant only ahd one group and one condition. The measures were not repeated. The order effect could have been one of the consequences. Their was not mention of tackling the reduction of demand characteristics. They couldn’t take into account people with similar situations in their lives to account for these potential cofounds.

Differences from original study

We are only doing experiment 1 which we will replicate in order to analyze the situation . The people who are participating in the study are not students from a university but rather from a website. This study will be conducted through the internet causing an impact in the way the people can give a response. The environment in which it is conducted will be different as well since we are having two different ways in which people answer the questions.

Results

Data Preparation

In order to begin to get have thorw away of missing values we need to get rid of data with failed attention checks. This has importance because of the way the values impact the study.We need to lLook towards assessing outliers to see how much of an impact they will have and prevent any issues.. We need to make sure we have the correct data formatting in order to not get the data lost.

To prepare the dataset for analysis, we start with cleaning the data. The entry that has a missing values were rid of to keep data integrity. Additionally, any towrads responses which weere not able to pass attention checks were not included to make sure it had high data quality. Outliers were found and looked through; if they proved to be problematic they were also gottenr id of. Lastly we made sure the dataset had the correct format.

```{r echo = TRUE, message = FALSE, warning = FALSE}

Data Preparation

Load Relevant Libraries and Functions

library(dplyr) library(tidyr) library(purrr) library(readr) library(stringr) library(jsonlite) library(magrittr)

Import data

Set the path to the folder

folder_path <- “/Users/anthonyzavala/Desktop/CSS/CSS204/critcher2013_2/data/osfstorage-archive”

List all CSV files in the folder

csv_files <- list.files(path = folder_path, pattern = “*.csv”, full.names = TRUE, recursive = TRUE)

Update combined_data by reading and combining all CSV files with subject_id column at the front

combined_data <- csv_files %>% map_dfr(~ { # Read each file data <- read_csv(.x)

Extract the subject ID from the file name (remove directory path and ".csv" extension)
subject_id <- tools::file_path_sans_ext(basename(.x))

# Add subject_id as the first column
data %>% mutate(subject_id = subject_id, .before = 1)

})

Data exclusion / filtering

Select relevant columns (adjusted based on column names)

processed_data <- combined_data %>% # Retain subject_id and condition columns select(subject_id, condition, starts_with(“Justin”), starts_with(“Nate”), group) %>%

# Propagate ‘condition’ values downward to fill in gaps fill(condition, .direction = “down”) %>%

# Organize data by ‘subject_id’ group_by(subject_id) %>%

# Add an attention check column based on group or response patterns mutate(attention_check = if_else( group == “control”, 0, # Example check for control group (adjust logic as needed) NA_real_ )) %>%

ungroup() %>%

# Ensure attention check is numeric mutate(attention_check = as.numeric(attention_check)) %>%

# Fill attention check values up and down fill(attention_check, .direction = “downup”)

Print a preview of the cleaned data

head(processed_data)

```{r} # Transform data to a long format and retain ‘condition’ reshaped_data <- processed_data %>% pivot_longer(

cols = starts_with(“Justin”):starts_with(“Nate”), names_to = “question”, values_to = “response_value” ) %>% mutate(question_value = paste(condition, question, sep = “_”)) %>% select(subject_id, condition, question_value, response_value, group, attention_check)



# Check unique values in 'condition'
unique(combined_data$condition)

# Check summary of response columns
summary(combined_data %>% select(starts_with("Justin"), starts_with("Nate")))

# Check for rows with non-NA responses
valid_responses <- combined_data %>%
  filter_at(vars(starts_with("Justin"), starts_with("Nate")), any_vars(!is.na(.)))

# Count rows with valid responses
nrow(valid_responses)

} reshaped_data \<- combined_data %\>% pivot_longer( cols = starts_with("Justin"):starts_with("Nate"), names_to = "question", values_to = "response_value" ) %\>% select(subject_id, condition, group, question, response_value)

head(filtered_long_data) nrow(filtered_long_data)

Ensure necessary columns exist

names(filtered_long_data)





library(ggplot2)

# Ensure group is a factor with appropriate levels
filtered_long_data <- filtered_long_data %>%
  mutate(group = factor(group, levels = c("control", "experimental")))

# Plot the grouped bar chart
ggplot(filtered_long_data, aes(x = condition, y = response_value, fill = group)) +
  geom_bar(stat = "identity", position = position_dodge(width = 0.8), width = 0.7) +
  labs(
    title = "Response Values by Condition and Group",
    x = "Condition",
    y = "Response Value"
  ) +
  scale_fill_manual(values = c("control" = "#8B008B", "experimental" = "#FF00FF")) +  # Dark Purple and Magenta
  theme_minimal() +
  theme(
    text = element_text(size = 14),
    legend.title = element_blank()
  )

#Analysis code

<!DOCTYPE html>

Moral Decision Experiment

Results Confirmatory analysis

r echo = TRUE, message = FALSE, warning = FALSE} # ANOVA for just my group data # Perform Two-Way ANOVA anova_results <- aov(moral_score ~ condition * decision_speed, data = final_dataset)

Display the ANOVA table

summary(anova_results)

#Condition has a strong and significant main effect on moral_score. #Decision_Speed alone does not significantly affect moral_score. #There is a significant interaction effect between condition and decision_speed, meaning the influence of condition on moral_score changes depending on the level of decision_speed.


Above are the ANOVA results for my individual sample, n=96.

 echo = TRUE, message = FALSE, warning = FALSE}
# Bar Plot for my groups data

plot <- ggplot(final_dataset, aes(x = condition, y = moral_score, fill = decision_speed)) +
  stat_summary(
    fun = "mean",
    geom = "bar",
    position = position_dodge(0.7),
    color = "black",
    width = 0.6
  ) +
  stat_summary(
    fun.data = "mean_cl_boot",
    geom = "linerange",
    position = position_dodge(0.7),
    linewidth = 0.8
  ) +
  scale_fill_manual(values = c("gray", "blue")) +
  labs(
    title = "Moral Character Evaluation - Individual Sample",
    x = "Condition",
    y = "Mean Moral Score",
    fill = "Decision Speed"
  ) +
  theme_minimal() +
  theme(
    panel.background = element_rect(fill = "white", color = NA), # Ensure white background
    plot.background = element_rect(fill = "white", color = NA)  # Ensure white plot background
  )

# Save the plot
desktop_path <- "~/Desktop/moral_character_evaluation_plot.png"
ggsave(desktop_path, plot = plot, bg = "white", width = 8, height = 6, dpi = 300)

# Display
print(plot)

Above are the plotted results for my individual sample, n=96.

echo = TRUE, message = FALSE, warning = FALSE}
# ANOVA for just combined groups data
# Run the two-way ANOVA
anova_result_combined <- aov(moral_score ~ condition * decision_speed, data = final_combined)

# Display the summary of the ANOVA
summary(anova_result_combined)

#Condition has a strong and significant main effect on moral_score.
#Decision_Speed alone does not significantly affect moral_score.
#There is a significant interaction effect between condition and decision_speed, meaning the influence of condition on moral_score changes depending on the level of decision_speed.


Above are the ANOVA results for the combined sample, n = 207.

{r echo = TRUE, message = FALSE, warning = FALSE}
#Bar plot for combined groups data
library(ggplot2)

# Bar Plot for the groups data
plot <- ggplot(final_combined, aes(x = condition, y = moral_score, fill = decision_speed)) +
  stat_summary(
    fun = "mean",
    geom = "bar",
    position = position_dodge(0.7),
    color = "black",
    width = 0.6
  ) +
  stat_summary(
    fun.data = "mean_cl_boot",
    geom = "linerange",
    position = position_dodge(0.7),
    linewidth = 0.8
  ) +
  scale_fill_manual(values = c("gray", "blue")) +
  labs(
    title = "Moral Character Evaluation - Combined Sample",
    x = "Condition",
    y = "Mean Moral Score",
    fill = "Decision Speed"
  ) +
  theme_minimal() +
  theme(
    panel.background = element_rect(fill = "white", color = NA), 
    plot.background = element_rect(fill = "white", color = NA) 
  )

# Save the plot
desktop_path <- "~/Desktop/combined_plot.png"
ggsave(desktop_path, plot = plot, bg = "white", width = 8, height = 6, dpi = 300)

# Display the plot
print(plot)

Above are the plotted results for the combined sample, n=207.

Comparison of replicated and original graph.

Exploratory analyses

echo = TRUE, message = FALSE, warning = FALSE} #Exploring the effect size for my groups data effect_sizes <- eta_squared(anova_results, partial = TRUE) print(effect_sizes)

#Condition has the strongest independent effect, explaining 50% of the variance in moral_score. #Decision Speed has no meaningful effect on its own. #The interaction effect is also strong, explaining 39% of the variance. This supports the idea that the effect of condition changes depending on the level of decision_speed (or vice versa).

Above are the effect size results for my individual sample, n=96.

{r echo = TRUE, message = FALSE, warning = FALSE} Exploring the effect size for combined data effect_sizes_combined <- eta_squared(anova_result_combined, partial = TRUE) print(effect_sizes_combined)

Condition has the strongest independent effect, explaining 43% of the variance in moral_score. #Decision Speed has no meaningful effect on its own. The interaction effect is also strong, explaining 32% of the variance. This supports the idea that the effect of condition changes depending on the level of decision_speed (or vice versa).

Above are the effect size results for the combined sample, n=207.

{r echo = TRUE, message = FALSE, warning = FALSE} # Interaction plot interaction_plot <- ggplot(final_combined, aes(x = condition, y = moral_score, color = decision_speed, group = decision_speed)) + geom_point(size = 4, shape = 21, fill = “white”) + geom_line(linewidth = 1.2) + scale_color_manual(values = c(“darkgrey”, “blue”)) + labs( title = “Effect of Condition on Moral Evaluation by Decision Speed”, x = “Condition”, y = “Moral Score”, color = “Decision Speed” ) + theme_minimal(base_size = 14) + theme( plot.title = element_text(hjust = 0.5, face = “bold”, size = 16), axis.title.x = element_text(size = 14, face = “italic”), axis.title.y = element_text(size = 14, face = “italic”), axis.text = element_text(size = 12), legend.position = “top”, legend.title = element_text(face = “bold”), panel.grid.major = element_line(color = “gray85”, size = 0.5) # Subtle gridlines )

interaction_plot

Above is an interaction plot for the combined sample, n=207

{r echo = TRUE, message = FALSE, warning = FALSE} ggplot(final_combined, aes(x = condition, y = moral_score, fill = decision_speed)) + geom_boxplot() + scale_fill_manual(values = c(“grey”, “blue”)) + labs( title = “Moral Scores by Condition and Character”, x = “Condition”, y = “Moral Score”, fill = “Speed” ) + theme_minimal(base_size = 14) + theme( plot.title = element_text(hjust = 0.5, face = “bold”, size = 16), axis.title.x = element_text(size = 14, face = “italic”), axis.title.y = element_text(size = 14, face = “italic”), axis.text = element_text(size = 12), legend.position = “top” )

Above is a box plot for the combined sample, n=207

The statistical test we decided on was replicating experiment 1 which utilizes a two way ANOVA based on what the perceived morals of someone character composite confirmed through the influence of how fast the decision was T-tests . We did this in order to find a comparison of how fast the decision is on both types of decision impacting what the moral character is like T-tests.

A two-way ANOVA was conducted to examine the main effects of decision type (whether participants decided to return or keep the wallet) and decision speed (quick or slow) on the moral character evaluation. This analysis allowed us to assess how these factors independently had an influence on moral judgments.{r} #Two way ANOVA

Convert factors if necessary

final_dataset <- final_dataset %>% mutate( condition = factor(condition, levels = c(“moral”, “immoral”), labels = c(“Returned Wallet”, “Kept Wallet”)), decision_speed = factor(decision_speed, levels = c(“quick”, “slow”), labels = c(“Quick”, “Slow”)) )

Perform two-way ANOVA

anova_results <- aov(moral_score ~ condition * decision_speed, data = final_dataset)

Display the ANOVA table

summary(anova_results)

Check interaction plot

interaction.plot( x.factor = final_dataset$condition, trace.factor = final_dataset$decision_speed, response = final_dataset$moral_score, xlab = “Decision Type (Condition)”, ylab = “Moral Character Evaluation”, trace.label = “Decision Speed”, col = c(“blue”, “red”), type = “b”, pch = c(19, 17),

lwd = 2 ) ```

Side-by-side graph with original graph is ideal here

Original

Our experiment

Post data collection addendum

The whole on partiicpants we got were 300 whne puttogehtew ithteh the other two replication groups. This went overthe original sample size of 119, growing the statistical powerThe number of participants we got for the our data was 120 participants. We didn’t expect people to finish our experiment faster than what we posted how long our experiment would take. This became a problem when we had to edit our experiment again because of some missing things and someone was taking it the same time we posted it.

#Unexpected parts

We had a problme with the wording for decions speed measurse when they came up during our times of collecting the data. We addressed this by clarifying the descriptions in the trials that came up and having some attention check to make sure we have quality data

Exploratory analyses

To follow up in this study we can improve the scenarios to create an impact for the decision. This is in order to have the speed to be more explicit. We can experiment with a number of scenarios in order to look for broader applicability and nuances.

Discussion

Summary of Replication Attempt

Our experiment was able to replicate since we both saw a lot of change of condition on moral scores. They both demostrated an a significant interaction with speed and condition. We expected our experiment to replicate which it did.

Commentary

The exploratory inghts we found are that the exploratory analsyis ranging with what the original study’s findings but showed a big amount of the lower indpendent result of how quick the decision is. We had the definition of replication with somehwat of a contrast in an mephasis on how fast the decisions is and its effects suggest possible moderators, like thw way we words the scneario orput some diversity in partiicpants, which areable to influence replication results. Differences we saw in the experiment are the role in how fast the decision was. The exact size of the effect were not reported. The original replication graph used different name for their variables like steal and return money rather than ours which moral and immoral condition. Our replication was different also in how it was conducted online rather than in person in the original which was done with the student population from UC Berkeley.

Caroline Porche: Methodology (lead); software (support); writing - original draft (equal); writing - review and editing (equal); writing - review and editing (equal). Mila Dutton: Software (lead); formal analysis (lead); writing - original draft (equal); writing - review and editing (equal). Anthony Zavala: Methodology (support); Writing - original draft (equal); writing - review and editing (equal).