Replication of Study 3a by García-Castro, Rodríguez-Bailón, & Willis (2020, Journal of Experimental Social Psychology)
Cedric Lim Chun Hui (limch@stanford.edu)
December 03, 2022
Introduction
This paper contributes importantly to the understanding that any behavioral effect of economic inequality is contingent on people’s perceptions. In addition, the study tests the novel prediction that perceptions of high inequality will lead to greater intolerance for inequality and increased support for redistributive policies. Perceived economic inequality is first manipulated, following which measures intolerance of inequality and support for economic redistribution are asked. There are two hypotheses for this study: (1) participants in the percevied economic inequality condition should show a higher intolerance of inequality than participants in the control condition, and (2) Intolerance of inequality would mediate the relationship between perceived economic inequality and support for economic redistribution.
This replication will take the form of a questionnaire. Both the manipulation and outcome measures are administered in the form of open-ended and multiple-choice questions. It is expected to take no more than 10 minutes to complete. Qualtrics will be the platform of choice and the study will be conducted on Prolific. Challenges include an unfamiliarity with using Prolific as a data collection platform, and whether we can avoid or filter out potential bots to ensure a real participant sample. Attention checks will be used to verify this as well. Another expected challenge will involve sourcing out the exact materials used in the original study as the article only provides broad details on some measures. It may be possible to obtain the exact materials from any form of supplemental materials, the preregistration document, or by contacting the authors directly.
Methods
Power Analysis
An average effect size of d=0.32 was found across the studies (2,3a,3b) using the thought manipulation. G*Power suggests that to detect this effect size (with one tail), a sample size of 244 is required to achieve 80% power with an alpha of .05. However, due to financial limitations, we aim to collect data from at least 140 participants.
Planned Sample
Participants are expected to take approximately 10 minutes to complete the study, to which they will be reimbursed $1.34 (based on the average U.S. minimum wage). With a minimum funding of $50, it guarantees a sample of at least more than 40 participants. The only inclusion criteria is that all participants must be from the U.S.
Materials
Thought Manipulation - To manipulate perceived economic inequality, participants will be asked to think about the wealthiest and the least wealthy persons they know. Then, they are asked to write a paragraph about how economic resources influence the lives of the people they considered. In the control condition, participants are instead asked to think about the tallest and shortest persons they knew and how height influences their lives.
Experimental condition: Please describe the person you know personally who has the most economic resources and the person you know personally who has the least economic resources. Think about what they can do and what not with the resources they have. Tell us how the economic resources influence their lives by writing at least one paragraph for each of these two people.
Control condition: Please describe the person of the smallest height you know personally and the tallest person you know personally. Think about what they can do and what not with their height. Tell us how their height influence their lives by writing at least one paragraph for each of these two people.
Manipulation Check - A single-item measure, “For the two people you have just thought about, how different do you think they are in terms of wealth?” Response options will range from 1 (not at all different) to 5 (extremely different). The higher the score, the greater the perceived economic inequality.
Intolerance of Inequality - A single item measure, “To what extent do you agree that the income differences in the U.S. are too large?” will be used (Gonthier, 2017; Larsen, 2016; Schröder, 2017). Response options range from 1 (totally disagree) to 7 (totally agree). The higher the score, the less tolerance to inequality.
Support for Economic Redistribution - Participants will respond to three items, (1) “The government has the responsibility to reduce the income differences between those who have high incomes and those who have low incomes”, (2) “The government should provide a decent standard of living for the unemployed”, and (3) “The government should spend less on benefits for the poor” (ISSP, 2017). Response options range from 1 (completely disagree) to 7 (completely agree). The higher the score, the greater the support for redistributive policies.
Demographic Variables - Participants will be asked to indicate their age, gender, nationality, state, subjective socioeconomic status, annual household income, highest educational qualification, and political orientation.
Procedure
Participants will be randomly assigned to one of the two thought manipulation conditions, following which they will complete the manipulation check and the measures of intolerance of inequality and support for redistribution.
Analysis Plan
Participants who spend less than a minute or write fewer than 250 characters on the thought manipulation will be excluded from the analysis. In addition, participants whose responses are clearly and explicitly unrelated to the manipulation prompt will be excluded as well.
Key Hypothesis: An independent samples t-test will be conducted to see if there are significant differences between the two conditions on the manipulation check and the measure of intolerance of inequality.
Secondary Hypothesis: A mediation analysis will be conducted to see if intolerance of inequality mediates the relationship between perceived economic inequality and support for redistributive policies.
Differences from Original Study
Unlike the original study which sampled from Spanish university students, participants for this replication effort will be recruited through the Prolific platform.
Methods Addendum (Post Data Collection)
You can comment this section out prior to final report with data collection.
Actual Sample
A total of 140 participants (Mage = 38, 54 males, 60 females, 4 others) were recruited from the Prolific platform for this study. Twenty-two participants were excluded from the sample for writing fewer than 250 characters on the thought manipulation, leaving a final sample of 118 participants that were included in the analysis.
Differences from pre-data collection methods plan
None.
Results (Pilot A)
Data preparation
Participants who spend less than a minute or write fewer than 250 characters on the thought manipulation will be excluded from the analysis. A mean will be computed for the 3-item measure of support for economic redistribution.
# Change scientific notation
options(scipen = 999)# Loading libraries and reading data
library(mediation)
library(tidyverse)
library(car)
library(ggplot2)
library(effects)
repstudytest <- read.csv("repstudytest.csv", header=TRUE)# Reverse-code negatively worded items and create a composite mean of support for economic redistribution.
repstudytest = repstudytest %>%
mutate(EcoRed3R = 8-EcoRed3R) %>%
mutate(EcoRedM = (EcoRed1+EcoRed2+EcoRed3R/3))Confirmatory analysis
Key Hypothesis: An independent samples t-test will be conducted to see if there are significant differences between the two conditions on the manipulation check and the measure of intolerance of inequality.
# Manipulation Check
t.test(ManiChk ~ Condition, var.equal=TRUE, data = repstudytest)##
## Two Sample t-test
##
## data: ManiChk by Condition
## t = -2.7111, df = 3, p-value = 0.0731
## alternative hypothesis: true difference in means between group Control and group Experimental is not equal to 0
## 95 percent confidence interval:
## -5.0723470 0.4056803
## sample estimates:
## mean in group Control mean in group Experimental
## 2.000000 4.333333# Intolerance of Inequality
t.test(Intol ~ Condition, var.equal=TRUE, data = repstudytest)##
## Two Sample t-test
##
## data: Intol by Condition
## t = -7.746, df = 3, p-value = 0.004475
## alternative hypothesis: true difference in means between group Control and group Experimental is not equal to 0
## 95 percent confidence interval:
## -4.702840 -1.963826
## sample estimates:
## mean in group Control mean in group Experimental
## 3.000000 6.333333plot(allEffects(lm(Intol~Condition, data=repstudytest)))
plot(allEffects(lm(ManiChk~Condition, data=repstudytest)))
Secondary Hypothesis: A mediation analysis will be conducted to see if intolerance of inequality mediates the relationship between perceived economic inequality and support for redistributive policies.
med.fit <- lm(Intol ~ ConditionNum, data = repstudytest)
out.fit <- lm(EcoRedM ~ ConditionNum + Intol, data = repstudytest)
summary(med.fit)##
## Call:
## lm(formula = Intol ~ ConditionNum, data = repstudytest)
##
## Residuals:
## 1 2 3
## -0.33333333333333470261 -0.00000000000000001388 -0.00000000000000002776
## 4 5
## -0.33333333333333270421 0.66666666666666740682
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.6667 0.2152 21.689 0.000215 ***
## ConditionNum -1.6667 0.2152 -7.746 0.004475 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.4714 on 3 degrees of freedom
## Multiple R-squared: 0.9524, Adjusted R-squared: 0.9365
## F-statistic: 60 on 1 and 3 DF, p-value: 0.004475summary(out.fit)##
## Call:
## lm(formula = EcoRedM ~ ConditionNum + Intol, data = repstudytest)
##
## Residuals:
## 1 2 3
## -1.166666666666666519 0.833333333333332926 -0.833333333333332704
## 4 5
## 1.166666666666668739 -0.000000000000002109
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 13.583 8.220 1.652 0.240
## ConditionNum -2.250 2.999 -0.750 0.531
## Intol -0.500 1.756 -0.285 0.803
##
## Residual standard error: 1.434 on 2 degrees of freedom
## Multiple R-squared: 0.7045, Adjusted R-squared: 0.4089
## F-statistic: 2.384 on 2 and 2 DF, p-value: 0.2955med.out <- mediate(med.fit, out.fit, treat = "ConditionNum", mediator = "Intol",
robustSE = F, sims = 100)
summary(med.out)##
## Causal Mediation Analysis
##
## Quasi-Bayesian Confidence Intervals
##
## Estimate 95% CI Lower 95% CI Upper p-value
## ACME 0.404 -4.452 5.57 0.84
## ADE -1.863 -6.736 2.56 0.50
## Total Effect -1.459 -2.629 -0.32 <0.0000000000000002 ***
## Prop. Mediated -0.440 -9.584 3.89 0.84
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Sample Size Used: 5
##
##
## Simulations: 100Results (Pilot B)
Data preparation
Participants who spend less than a minute or write fewer than 250 characters on the thought manipulation will be excluded from the analysis. A mean will be computed for the 3-item measure of support for economic redistribution.
repstudytest <- read.csv("pilotb.csv", header=TRUE)
library(mediation)
library(tidyverse)
library(car)
library(ggplot2)
library(effects)# Reverse-code negatively worded items and create a composite mean of support for economic redistribution.
repstudytest = repstudytest %>%
mutate(EcoRed3R = 8-EcoRed3R) %>%
mutate(EcoRedM = (EcoRed1+EcoRed2+EcoRed3R/3))Confirmatory analysis
Key Hypothesis: An independent samples t-test will be conducted to see if there are significant differences between the two conditions on the manipulation check and the measure of intolerance of inequality.
# Manipulation Check
t.test(ManiChk ~ Condition, var.equal=TRUE, data = repstudytest)##
## Two Sample t-test
##
## data: ManiChk by Condition
## t = 0.33333, df = 2, p-value = 0.7706
## alternative hypothesis: true difference in means between group Control and group Experimenter is not equal to 0
## 95 percent confidence interval:
## -5.953979 6.953979
## sample estimates:
## mean in group Control mean in group Experimenter
## 3.5 3.0# Intolerance of Inequality
t.test(Intol ~ Condition, var.equal=TRUE, data = repstudytest)##
## Two Sample t-test
##
## data: Intol by Condition
## t = 1, df = 2, p-value = 0.4226
## alternative hypothesis: true difference in means between group Control and group Experimenter is not equal to 0
## 95 percent confidence interval:
## -4.953979 7.953979
## sample estimates:
## mean in group Control mean in group Experimenter
## 7.0 5.5plot(allEffects(lm(Intol~Condition, data=repstudytest)))
plot(allEffects(lm(ManiChk~Condition, data=repstudytest)))
Secondary Hypothesis: A mediation analysis will be conducted to see if intolerance of inequality mediates the relationship between perceived economic inequality and support for redistributive policies.
med.fit <- lm(Intol ~ ConditionNum, data = repstudytest)
out.fit <- lm(EcoRedM ~ ConditionNum + Intol, data = repstudytest)
summary(med.fit)##
## Call:
## lm(formula = Intol ~ ConditionNum, data = repstudytest)
##
## Residuals:
## 1 2 3
## 0.00000000000000008327 1.50000000000000000000 -1.50000000000000000000
## 4
## -0.00000000000000013878
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.25 0.75 8.333 0.0141 *
## ConditionNum 0.75 0.75 1.000 0.4226
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.5 on 2 degrees of freedom
## Multiple R-squared: 0.3333, Adjusted R-squared: 0
## F-statistic: 1 on 1 and 2 DF, p-value: 0.4226summary(out.fit)##
## Call:
## lm(formula = EcoRedM ~ ConditionNum + Intol, data = repstudytest)
##
## Residuals:
## 1 2 3
## -1.49999999999999955591 0.00000000000000016653 -0.00000000000000005551
## 4
## 1.49999999999999933387
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.028 6.339 0.951 0.516
## ConditionNum 0.250 1.299 0.192 0.879
## Intol 1.222 1.000 1.222 0.437
##
## Residual standard error: 2.121 on 1 degrees of freedom
## Multiple R-squared: 0.73, Adjusted R-squared: 0.19
## F-statistic: 1.352 on 2 and 1 DF, p-value: 0.5196med.out <- mediate(med.fit, out.fit, treat = "ConditionNum", mediator = "Intol",
robustSE = F, sims = 100)
summary(med.out)##
## Causal Mediation Analysis
##
## Quasi-Bayesian Confidence Intervals
##
## Estimate 95% CI Lower 95% CI Upper p-value
## ACME 0.955 -0.914 5.17 0.54
## ADE 0.225 -1.936 2.83 0.80
## Total Effect 1.180 -1.786 4.30 0.38
## Prop. Mediated 0.393 -1.835 5.77 0.40
##
## Sample Size Used: 4
##
##
## Simulations: 100Final Results (Data after exclusions, N=118)
Data preparation
Participants who spend less than a minute or write fewer than 250 characters on the thought manipulation will be excluded from the analysis. A mean will be computed for the 3-item measure of support for economic redistribution.
# Loading libraries and reading the data
library(mediation)
library(tidyverse)
library(car)
library(ggplot2)
library(effects)
library(jtools) # Gives theme_apa
fulldata <- read.csv("fulldata.csv", header=TRUE, na.strings=c(""))# Tidying the dataset
fulldata <- fulldata %>%
mutate(subjID = row_number()) %>%
filter(subjID != 1) %>%
mutate(ConditionNum = ifelse(is.na(Q262), -1, 1)) %>%
mutate(Condition = ifelse(is.na(Q262), "Control", "Experimental")) %>%
unite('Reflect1', c(Q262,Q265), remove=FALSE, na.rm=T) %>%
unite('Reflect2', c(Q264,Q266), remove=FALSE, na.rm=T) %>%
rename(Intol = Q251,
EcoRed1 = Q252,
EcoRed2 = Q253,
EcoRed3R = Q254,
AttChk = Q271,
PolOri = Q134,
subjSES = Q196_1,
age = Q106,
gender = Q108,
citizen = Q268,
state = Q121,
objSES = Q359,
education = Q358,
ManiChk = Q250)# Excluding participants based on <250 characters written exclusion criteria
fulldata <- fulldata %>%
mutate(numchars1 = nchar(Reflect1),
numchars2 = nchar(Reflect2),
totalchars = numchars1 + numchars2) %>%
filter(totalchars > 249)# Reverse-code negatively worded items and create a composite mean of support for economic redistribution.
fulldata = fulldata %>%
mutate(EcoRed3R = 8-EcoRed3R) %>%
mutate(EcoRedM = (EcoRed1+EcoRed2+EcoRed3R/3))Confirmatory analysis
Key Hypothesis: An independent samples t-test will be conducted to see if there are significant differences between the two conditions on the manipulation check (replicated) and the measure of intolerance of inequality (failed to replicate).
# Manipulation check
t.test(ManiChk ~ Condition, var.equal=TRUE, data = fulldata)##
## Two Sample t-test
##
## data: ManiChk by Condition
## t = -13.799, df = 116, p-value < 0.00000000000000022
## alternative hypothesis: true difference in means between group Control and group Experimental is not equal to 0
## 95 percent confidence interval:
## -2.370717 -1.775596
## sample estimates:
## mean in group Control mean in group Experimental
## 2.410714 4.483871# Intolerance of inequality
t.test(Intol ~ Condition, var.equal=TRUE, data = fulldata)##
## Two Sample t-test
##
## data: Intol by Condition
## t = -0.55088, df = 116, p-value = 0.5828
## alternative hypothesis: true difference in means between group Control and group Experimental is not equal to 0
## 95 percent confidence interval:
## -0.7597187 0.4290736
## sample estimates:
## mean in group Control mean in group Experimental
## 5.625000 5.790323# Preparing the values for plotting confidence intervals bars
fulldata_summary <- fulldata %>%
group_by(Condition) %>%
summarise(n = n(),
mean_intol = mean(Intol),
ci_intol = 1.96 * sd(Intol) / sqrt(n),
mean_manichk = mean(ManiChk),
ci_manichk = 1.96 * sd(ManiChk) / sqrt(n))
# Plotting results for the manipulation check
ggplot(fulldata_summary, aes(x=Condition, y=mean_manichk, ymin=mean_manichk-ci_manichk, ymax=mean_manichk+ci_manichk, fill=Condition)) +
geom_bar(stat="identity", width=.3, show.legend = F) +
coord_cartesian(ylim=c(1,7)) +
geom_errorbar(width=.1) +
scale_y_continuous(breaks = seq(0,7,by=1), limits=c(0,7)) +
ylab("Manipulation Check") +
theme_classic()
# Plotting results for intolerance to inequality
ggplot(fulldata_summary, aes(x=Condition, y=mean_intol, ymin=mean_intol-ci_intol, ymax=mean_intol+ci_intol, fill=Condition)) +
geom_bar(stat="identity", width=.3, show.legend = F) +
coord_cartesian(ylim=c(1,7)) +
geom_errorbar(width=.1) +
scale_y_continuous(breaks = seq(0,7,by=1), limits=c(0,7)) +
ylab("Intolerance to Inequality") +
theme_classic()
# Side-by-side graph comparison of main result between original and replication
df3 <- data.frame(mean=c(5.18,5.50,5.625,5.790),
ci=c(0.18,0.18,0.434,0.398),
Study=c("Original","Original","Replication","Replication"),
Condition=c("Control","Experimental","Control","Experimental"))
ggplot(df3, aes(x=Study, y=mean, fill=Condition, ymin=mean-ci, ymax=mean+ci)) +
geom_bar(stat="identity", position=position_dodge(),width=.5) +
coord_cartesian(ylim=c(4,7)) +
scale_y_continuous(breaks = seq(0,7,by=1), limits=c(0,7)) +
ylab("Intolerance to Inequality") +
geom_errorbar(position=position_dodge(.5),width=.1) +
theme_classic()
Secondary Hypothesis: A mediation analysis will be conducted to see if intolerance of inequality mediates the relationship between perceived economic inequality and support for redistributive policies. (Failed to replicate)
# Testing mediation (Unstandardized coefficients)
med.fit <- lm(Intol ~ ConditionNum, data = fulldata)
out.fit <- lm(EcoRedM ~ ConditionNum + Intol, data = fulldata)
summary(med.fit)##
## Call:
## lm(formula = Intol ~ ConditionNum, data = fulldata)
##
## Residuals:
## Min 1Q Median 3Q Max
## -4.790 -0.625 0.375 1.210 1.375
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 5.70766 0.15005 38.038 <0.0000000000000002 ***
## ConditionNum 0.08266 0.15005 0.551 0.583
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.628 on 116 degrees of freedom
## Multiple R-squared: 0.002609, Adjusted R-squared: -0.005989
## F-statistic: 0.3035 on 1 and 116 DF, p-value: 0.5828summary(out.fit)##
## Call:
## lm(formula = EcoRedM ~ ConditionNum + Intol, data = fulldata)
##
## Residuals:
## Min 1Q Median 3Q Max
## -11.3529 -1.4532 0.3651 1.8431 7.7720
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.1244 0.8724 1.289 0.2001
## ConditionNum -0.4002 0.2380 -1.681 0.0954 .
## Intol 1.8517 0.1471 12.591 <0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.579 on 115 degrees of freedom
## Multiple R-squared: 0.5812, Adjusted R-squared: 0.5739
## F-statistic: 79.8 on 2 and 115 DF, p-value: < 0.00000000000000022# Standardizing variables
fulldata_s <- fulldata %>%
mutate(Intol_s = scale(Intol),
EcoRedM_s = scale(EcoRedM),
ManiChk_s = scale(ManiChk))
# Testing mediation (Standardized coefficients)
med.fit_s <- lm(Intol_s ~ ConditionNum, data = fulldata_s)
out.fit_s <- lm(EcoRedM_s ~ ConditionNum + Intol_s, data = fulldata_s)
summary(med.fit_s)##
## Call:
## lm(formula = Intol_s ~ ConditionNum, data = fulldata_s)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.9515 -0.3851 0.2311 0.7453 0.8472
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.00259 0.09245 -0.028 0.978
## ConditionNum 0.05093 0.09245 0.551 0.583
##
## Residual standard error: 1.003 on 116 degrees of freedom
## Multiple R-squared: 0.002609, Adjusted R-squared: -0.005989
## F-statistic: 0.3035 on 1 and 116 DF, p-value: 0.5828summary(out.fit_s)##
## Call:
## lm(formula = EcoRedM_s ~ ConditionNum + Intol_s, data = fulldata_s)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.87386 -0.36785 0.09243 0.46656 1.96740
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.005151 0.060168 0.086 0.9319
## ConditionNum -0.101302 0.060246 -1.681 0.0954 .
## Intol_s 0.760782 0.060425 12.591 <0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.6527 on 115 degrees of freedom
## Multiple R-squared: 0.5812, Adjusted R-squared: 0.5739
## F-statistic: 79.8 on 2 and 115 DF, p-value: < 0.00000000000000022# Testing indirect effect
med.out <- mediate(med.fit, out.fit, treat = "ConditionNum", mediator = "Intol",
robustSE = F, sims = 100)
summary(med.out)##
## Causal Mediation Analysis
##
## Quasi-Bayesian Confidence Intervals
##
## Estimate 95% CI Lower 95% CI Upper p-value
## ACME 0.146641 -0.401287 0.65 0.52
## ADE -0.404158 -0.908113 0.06 0.14
## Total Effect -0.257518 -1.114828 0.40 0.52
## Prop. Mediated -0.000256 -31.649663 7.50 1.00
##
## Sample Size Used: 118
##
##
## Simulations: 100# Mediation Diagram (Standardized coefficients)
library(diagram)## Loading required package: shapedata <- c(0, "'.05'", 0,
0, 0, 0,
"'.76***'", "'-.06 (-.10)'", 0)
M <- matrix (nrow=3, ncol=3, byrow = TRUE, data=data)
plot <- plotmat (M, pos=c(1,2), shadow.size = 0,
name= c( "Intolerance to \nInequality","Condition", "Support for \nEconomic Redistribution"),
box.type = "rect", box.size = 0.16, box.prop=0.3, curve=0)
Exploratory analyses
# Does the manipulation check predict intolerance to inequality?
summary(lm(Intol_s ~ ManiChk_s, data=fulldata_s))##
## Call:
## lm(formula = Intol_s ~ ManiChk_s, data = fulldata_s)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.1075 -0.3705 0.3819 0.5893 0.9980
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.00000000000000009618 0.09094865799672272044 0.000 1.0000
## ManiChk_s 0.17968490793776628611 0.09133650051526003011 1.967 0.0515 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.988 on 116 degrees of freedom
## Multiple R-squared: 0.03229, Adjusted R-squared: 0.02394
## F-statistic: 3.87 on 1 and 116 DF, p-value: 0.05154# Does SES moderate the relationship between condition and intolerance to inequality?
summary(lm(Intol ~ Condition*objSES, data=fulldata))##
## Call:
## lm(formula = Intol ~ Condition * objSES, data = fulldata)
##
## Residuals:
## Min 1Q Median 3Q Max
## -4.9326 -0.6564 0.3949 1.2318 1.4395
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 5.69427 0.50859 11.196 <0.0000000000000002
## ConditionExperimental 0.29313 0.69879 0.419 0.676
## objSES -0.02229 0.14770 -0.151 0.880
## ConditionExperimental:objSES -0.03250 0.19029 -0.171 0.865
##
## (Intercept) ***
## ConditionExperimental
## objSES
## ConditionExperimental:objSES
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.64 on 114 degrees of freedom
## Multiple R-squared: 0.004629, Adjusted R-squared: -0.02156
## F-statistic: 0.1767 on 3 and 114 DF, p-value: 0.912Discussion
Summary of Replication Attempt
The original effect failed to replicate.
There was no significant difference in intolerance to inequality between participants in the economic inequality (M = 5.79, 95% CI [5.39,6.19]) and height inequality condition (M = 5.63, 95% CI [5.19,6.06], t(116) = -0.55, p = .58, d = 0.10, 95% CI [-0.76,0.43]).
Interestingly however, the failure to replicate was not because the thought manipulation failed to induce a perception of economic inequality. It did work. There was a significant difference on the manipulation check between participants in the economic inequality (M = 4.48, 95% CI [4.33,4.64]) and height inequality condition (M = 2.41, 95% CI [2.15,2.67], t(116) = -13.80, p < .001, d = -2.56, 95% CI [-2.37,-1.78]).
Commentary
One possibility for the failure to replicate could certainly be an insufficient sample size. Given the small effect size of the original paper, dmean = 0.32, a sample size of 244 was required to achieve 80% power. The sample we collected was less than half of that. However, even with a sample of 118 participants, we observe that the effect size in this replication was three times smaller than the original study at d = 0.10, a difference that is difficult to attribute to a lack of sample size. I believe there is another likelier possibility for the failure to replicate - cultural difference. While the original study was conducted in Spain, this replication study was conducted on U.S. participants. Because of the strong belief in hard work and the American Dream, people in the U.S. may perceive inequality as a natural feature of society. Those who succeed must have worked hard and those who failed must have been lazy. Therefore, regardless of how high people perceive inequality in the U.S., they won’t necessarily see it as a problem that needs fixing. Instead, what might predict intolerance to inequality could be a perception of corruption or financial fraud instead.