Replication of Study 1 by Smiley & Fisher (2022, Psychological Science)
Author
Sarah H. Wu (sarwu@stanford.edu)
Published
December 14, 2025
Introduction
Despite the idiosyncracies of individual technological developments, there is a fundamental human panic about new technologies. In the 1800s, novels were accused of corrupting the minds of youth and housewives, in the 1940s “television addiction” was coined, and in current discourse smartphones and social media are the supposed culprits behind adolescents’ declining mental health. (See Pessimists Archive for more examples.) Because my research investigates people’s perceptions of digital media, I found Smiley & Fisher (2022) relevant because it probes a potential mechanism underlying cycles of tech panic: status quo bias, which has been documented across domains and developmental stages. In their Study 1, Smiley and Fisher (2022) experimentally manipulated participants’ age at the time of an obscure technology’s invention (15 years before vs. after the invention of the technology) and measured participants’ self-reported perceptions of the technology’s societal impact. Smiley and Fisher (2022) found that participants in the born-after condition evaluated the technology more positively than those in the born-before condition.
The stimuli for this experiment are descriptions of an obscure technology called aerogel, “a synthetic porous ultralight material derived from a gel, in which the liquid component for the gel has been replaced with a gas without significant collapse of the gel structure.” The authors selected this technology because it was possible to manipulate its perceived age without too many participants knowing its actual date of invention. In the born-before condition, participants were told that aerogel was invented 15 years after they had been born, with the exact year dynamically populated based on the participant’s birth year. (For example, a participant born in 2000 would read that aerogel was invented in 2015.) In the born-after condition, participants were told that aerogel had been invented 15 years prior to their birth. (So a participant born in 2000 would see that aerogel was invented in 1985.) After reading about aerogel, participants were asked, “How would you describe the impact of aerogel on society?” and reported their impression on a 7-point Likert scale from -3 (very negative) to 3 (very positive). Then, they answered a manipulation check question and a previous knowledge question.
The experiment seems relatively straightforward to replicate. However, as with any online study, it is possible that participants might look up details about the stimuli during the study, which could render the manipulation ineffective. Because aerogel is a real invention, people could easily find its actual date of invention.
The original effect size from Smiley and Fisher (2022) was d = .25. As in the original paper, I aimed for 80% power with a medium effect size of d = .3. G*Power confirmed that this would take at least 176 participants in each condition.
Planned Sample
The planned sample size is 400 participants, using a representative sample based on US census data.
Materials
The materials and procedure were followed precisely.
“To begin, participants provided demographic information: age, gender, and level of education. Next, participants were told, “Aerogel (pictured above) [see Section S1 in the Supplemental Material at https://osf.io/bxjnr] is a synthetic porous ultralight material derived from a gel, in which the liquid component for the gel has been replaced with a gas without significant collapse of the gel structure.” This technology was selected because, though real, it is largely unknown, making it possible to manipulate its perceived age.”
Procedure
From the original paper: “Participants were randomly assigned to the born-before or born-after condition. In the born-before condition, participants were next told that aerogel was invented 15 years after they had been born. We dynamically presented the invention date based on each participant’s age. For example, a participant born in 1980 would see, “It was invented in 1995 and is now widely used.” In the born-after condition, participants were told that aerogel had been invented 15 years before they were born. For example, a participant born in 1980 would see, “It was invented in 1965 and is now widely used.” After viewing this information, all participants were asked, “How would you describe the impact of aerogel on society?” and responded on a 7-point Likert scale ranging from −3 (very negative) to 3 (very positive). Finally, as a manipulation check, participants were asked, “Was aerogel invented before or after the year you were born?” (before/after), and a previous knowledge question asked, “Did you know about aerogel prior to completing this survey?” (yes/no).”
Analysis Plan
The key analysis of interest is an unpaired t-test comparing evaluations of aerogel in the born-before and born after conditions. Like the original paper, we will compare the size of the effect with and without exclusions based on the manipulation check and prior knowledge about aerogel.
In line with the original paper, we will also conduct a zero-order correlation to test a competing hypothesis: that people simply prefer older technology. By this account, older people would tend to have more positive evaluations because aerogel was described as especially old (either 15 years before or after their birth).
Differences from Original Study
The main difference between this replication study and the original study is that the original study used an MTurk sample, whereas we used a Prolific sample. Prolific is generally recognized as having higher-quality participants (Douglas et al., 2023), but these differences are probably negligible given the ease of the task.
Methods Addendum (Post Data Collection)
Actual Sample
My actual sample size was 394 after excluding participants who failed the initial attention check. Like Smiley and Fisher (2022), I also tracked who failed the manipulation check (N = 37) and who had prior knowledge of aerogel (N = 69), analyzing the data both with and without these exclusions.
Differences from pre-data collection methods plan
None.
Results
Pilot B: Data preparation
Data preparation following the analysis plan.
Loading required package: psych
Loading required package: tidyverse
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.1.4 ✔ readr 2.1.5
✔ forcats 1.0.0 ✔ stringr 1.5.1
✔ ggplot2 3.5.1 ✔ tibble 3.2.1
✔ lubridate 1.9.4 ✔ tidyr 1.3.1
✔ purrr 1.0.2
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ ggplot2::%+%() masks psych::%+%()
✖ ggplot2::alpha() masks psych::alpha()
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
Loading required package: effsize
Attaching package: 'effsize'
The following object is masked from 'package:psych':
cohen.d
Loading required package: ggpubr
Loading required package: grid
Loading required package: futile.logger
Attaching package: 'VennDiagram'
The following object is masked from 'package:ggpubr':
rotate
Attaching package: 'rstatix'
The following object is masked from 'package:stats':
filter
Loading required package: tinylabels
Warning: package 'gtsummary' was built under R version 4.4.3
#### Impgrid#### Import datapilot_b <-read_csv("../data/pilot_b.csv") %>%drop_na(id) %>%# Only include Prolific participants, not Pilot A participantsfilter(DistributionChannel=="anonymous")
Rows: 24 Columns: 36
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (36): StartDate, EndDate, Status, Progress, Duration (in seconds), Finis...
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
#### Data exclusion / filteringpilot_b_recoded <- pilot_b %>%rename(Condition=FL_9_DO,Eval = attitude)### DATA WRANGLING & CLEANING ##### Participant Infopilot_b_recoded$gender <-factor(pilot_b_recoded$gender, levels =c("1", "2"), labels=c("Male","Female"))pilot_b_recoded$age <-as.numeric(pilot_b_recoded$age)pilot_b_recoded$Eval <-as.numeric(pilot_b_recoded$Eval)### END DATA WRANGLING & CLEANING ##### Demographic informationtable(pilot_b_recoded$gender)
## T-test and Cohen's d## Condition refers to born after/before invention (experimentally manipulated)t.test(Eval ~ Condition, var.equal=TRUE, data=pilot_b_recoded)
Two Sample t-test
data: Eval by Condition
t = -0.44721, df = 2, p-value = 0.6985
alternative hypothesis: true difference in means between group AerogelBornAfter and group AerogelBornBefore is not equal to 0
95 percent confidence interval:
-5.310512 4.310512
sample estimates:
mean in group AerogelBornAfter mean in group AerogelBornBefore
1.0 1.5
effsize::cohen.d(Eval ~ Condition, data = pilot_b_recoded)
Cohen's d
d estimate: -0.4472136 (small)
95 percent confidence interval:
lower upper
-4.803317 3.908890
Pearson's product-moment correlation
data: pilot_b_recoded$age and pilot_b_recoded$Eval
t = 0.3658, df = 2, p-value = 0.7496
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
-0.9359204 0.9764898
sample estimates:
cor
0.2504171
### END Analysis WITHOUT exclusions### Analysis WITH exclusions ##### Exclusiontable(pilot_b_recoded$attn_check_1) #
3
4
table(pilot_b_recoded$attn_check_2)
2
4
table(pilot_b_recoded$prior_knowledge) #
0 1
3 1
# Exclude participants who failed manipulation check or knew what aerogel wass1.df.f <- pilot_b_recoded %>%filter(attn_check_1 =="3"& attn_check_2 =="2") %>%filter(prior_knowledge !="1")## Descriptive statisticss1.cond.difs <- s1.df.f %>%group_by(Condition) %>%summarise(EVAL =mean(Eval), EVAL_SD =sd(Eval))s1.cond.difs
# A tibble: 2 × 3
Condition EVAL EVAL_SD
<chr> <dbl> <dbl>
1 AerogelBornAfter 1 1.41
2 AerogelBornBefore 2 NA
## T-test and Cohen's d t.test(Eval ~ Condition, var.equal=TRUE, data=s1.df.f)
Two Sample t-test
data: Eval by Condition
t = -0.57735, df = 1, p-value = 0.6667
alternative hypothesis: true difference in means between group AerogelBornAfter and group AerogelBornBefore is not equal to 0
95 percent confidence interval:
-23.00779 21.00779
sample estimates:
mean in group AerogelBornAfter mean in group AerogelBornBefore
1 2
effsize::cohen.d(Eval ~ Condition, data = s1.df.f)
Cohen's d
d estimate: NA (NA)
95 percent confidence interval:
lower upper
NA NA
## Age and Evaluation correlationcor.test(s1.df.f$age, s1.df.f$Eval)
Pearson's product-moment correlation
data: s1.df.f$age and s1.df.f$Eval
t = 1.7321, df = 1, p-value = 0.3333
alternative hypothesis: true correlation is not equal to 0
sample estimates:
cor
0.8660254
## Make Condition.Num numeric so it can be scaled in lms1.df.f$Condition.Num <-rep(NA, length(s1.df.f$Condition))s1.df.f$Condition.Num[s1.df.f$Condition=="AerogelBornAfter"] <-0s1.df.f$Condition.Num[s1.df.f$Condition=="AerogelBornBefore"] <-1## Age and Condition multilevel models1.lm.age <-lm(scale(Eval) ~scale(age) +scale(Condition.Num), data = s1.df.f)summary(s1.lm.age)
Call:
lm(formula = scale(Eval) ~ scale(age) + scale(Condition.Num),
data = s1.df.f)
Residuals:
ALL 3 residuals are 0: no residual degrees of freedom!
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -1.355e-17 NaN NaN NaN
scale(age) 1.732e+00 NaN NaN NaN
scale(Condition.Num) -1.000e+00 NaN NaN NaN
Residual standard error: NaN on 0 degrees of freedom
Multiple R-squared: 1, Adjusted R-squared: NaN
F-statistic: NaN on 2 and 0 DF, p-value: NA
── Column specification ────────────────────────────────────────────────────────
cols(
.default = col_double(),
StartDate = col_datetime(format = ""),
EndDate = col_datetime(format = ""),
Status = col_character(),
Finished = col_logical(),
RecordedDate = col_datetime(format = ""),
ResponseId = col_character(),
DistributionChannel = col_character(),
UserLanguage = col_character(),
attn_check_2 = col_character(),
id = col_character(),
gender = col_character(),
gender_4_TEXT = col_logical(),
education = col_character(),
attitude = col_character(),
manip_check = col_character(),
prior_knowledge = col_character(),
PROLIFIC_PID = col_character(),
STUDY_ID = col_character(),
SESSION_ID = col_character(),
`Create New Field or Choose From Dropdown...` = col_logical()
)
ℹ Use `spec()` for the full column specifications.
full_sample_conditions <- full_sample %>%mutate(Condition =if_else(is.na(FL_9_DO_AerogelBornBefore), "Born-After", "Born-Before")) %>%# Create condition column filter(attn_check_1 =="50"& attn_check_2 =="Her mother") %>%# Filter out those who failed attention check and didn't get to finishfilter(Finished==1) %>%mutate(attitude_num =as.numeric(str_extract(attitude, "^[^a-zA-Z]+")) ) %>%relocate(attitude_num, .after = attitude)
Demographics
demographics_table <- full_sample_conditions %>%tbl_summary(include =c(age, gender), # Variables to includelabel =list( age ~"Age (Years)", # Custom Label for Age gender ~"Gender Identity"# Custom Label for Gender ),statistic =list(all_continuous() ~"{mean} ({sd})", # Default is Median (IQR); change to Mean (SD)all_categorical() ~"{n} ({p}%)"# Count and Percentage ),digits =all_continuous() ~1# Round continuous stats to 1 decimal place ) %>%add_n() %>%# Add column with total N gtsummary::modify_header(label ="**Variable**") # Rename the stub header# 3. Print the tabledemographics_table
Pearson's product-moment correlation
data: full_sample_conditions$age and full_sample_conditions$attitude_num
t = -0.64001, df = 394, p-value = 0.5225
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
-0.13035872 0.06653103
sample estimates:
cor
-0.03222648
# Distribution of agesggplot(full_sample_conditions, aes(x=age)) +geom_histogram(binwidth=5, color="white") +ylim(c(0,60)) +labs(x="Age") + jtools::theme_apa()
Participants in the born-before condition evaluated aerogel no more favorably than participants in the born-after condition, \(\Delta M = 0.11\), 95% CI \([-0.11, 0.32]\), \(t(394) = 0.98\), \(p = .327\)
# Exclude participants who failed manipulation check or knew what aerogel wasfull_sample_exclusions <- full_sample_conditions %>%filter(manip_check == Condition) %>%# Their answer should match the condition to which they were assignedfilter(prior_knowledge !="Yes") # They should not know what aerogel is## Descriptive statisticscondition_diffs_exclusions <- full_sample_exclusions %>%group_by(Condition) %>%get_summary_stats(attitude_num, type="mean_ci")condition_diffs_exclusions
# A tibble: 2 × 5
Condition variable n mean ci
<chr> <fct> <dbl> <dbl> <dbl>
1 Born-After attitude_num 153 1.05 0.169
2 Born-Before attitude_num 145 0.924 0.183
## T-test and Cohen's d with_exclusions <-t.test(attitude_num ~ Condition, var.equal=TRUE, data=full_sample_exclusions)with_exclusions_apa <- papaja::apa_print(with_exclusions)effsize::cohen.d(attitude_num ~ Condition, data = full_sample_exclusions)
Cohen's d
d estimate: 0.1120242 (negligible)
95 percent confidence interval:
lower upper
-0.1162443 0.3402927
## Age and Evaluation correlationcor.test(full_sample_exclusions$age, full_sample_exclusions$attitude_num)
Pearson's product-moment correlation
data: full_sample_exclusions$age and full_sample_exclusions$attitude_num
t = -0.065693, df = 296, p-value = 0.9477
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
-0.1173881 0.1098501
sample estimates:
cor
-0.003818276
In line with Smiley and Fisher (2022), I also analyzed the data after excluding participants who failed the manipulation check or had prior knowledge of aerogel.
Still, participants in the born-before condition evaluated aerogel no more favorably than those in the born-after condition, \(\Delta M = 0.12\), 95% CI \([-0.13, 0.37]\), \(t(296) = 0.97\), \(p = .335\)
# Visualizations for original studyoriginal_data <-read_csv("../data/TechAgeS1.csv")
Rows: 401 Columns: 9
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (2): Check, Condition
dbl (7): Subject, Gender, Age, Education, Eval, Knowledge, Year
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
# Visualizations for my replication studyreplication_plot <-ggplot(condition_diffs_no_exclusion, aes(x=Condition, y=mean, color=Condition)) +geom_point(size=3) +geom_errorbar(aes(ymin = mean - ci, ymax = mean + ci), width =0.2) +geom_text(aes(label =paste0("italic(N) == ", n), y = mean + ci +0.3), parse=TRUE,color ="black", size =4) +scale_y_continuous(limits =c(-3, 3), breaks =seq(-3, 3, 1)) +scale_color_manual(values=c("steelblue", "orange")) +labs(y="Evaluation of Aerogel",title="Replication Attempt") + jtools::theme_apa() +theme(legend.position ="none",axis.text.x =element_text(size =12),plot.title =element_text(hjust=.5)) ggplot(condition_diffs_exclusions, aes(x=Condition, y=mean, color=Condition)) +geom_point(size=3) +geom_errorbar(aes(ymin = mean - ci, ymax = mean + ci), width =0.2) +geom_text(aes(label =paste0("italic(N) == ", n), y = mean + ci +0.3), parse=TRUE,color ="black", size =4) +scale_y_continuous(limits =c(-3, 3), breaks =seq(-3, 3, 1)) +scale_color_manual(values=c("steelblue", "orange")) +labs(y="Evaluation of Aerogel") + jtools::theme_apa() +theme(legend.position ="none",axis.text.x =element_text(size =12))
## Make Condition.Num numeric so it can be scaled in lmfull_sample_exclusions$Condition.Num <-rep(NA, length(full_sample_exclusions$Condition))full_sample_exclusions$Condition.Num[full_sample_exclusions$Condition=="Born-After"] <-0full_sample_exclusions$Condition.Num[full_sample_exclusions$Condition=="Born-Before"] <-1## Age and Condition multilevel modelfull_sample_exclusions.lm.age <-lm(scale(attitude_num) ~scale(age) +scale(Condition.Num), data = full_sample_exclusions)summary(full_sample_exclusions.lm.age)
Call:
lm(formula = scale(attitude_num) ~ scale(age) + scale(Condition.Num),
data = full_sample_exclusions)
Residuals:
Min 1Q Median 3Q Max
-2.80766 -0.85415 0.06535 0.88168 1.91441
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -5.992e-17 5.803e-02 0.000 1.000
scale(age) -2.829e-03 5.814e-02 -0.049 0.961
scale(Condition.Num) -5.604e-02 5.814e-02 -0.964 0.336
Residual standard error: 1.002 on 295 degrees of freedom
Multiple R-squared: 0.003154, Adjusted R-squared: -0.003604
F-statistic: 0.4667 on 2 and 295 DF, p-value: 0.6275
Neither condition nor age predicted attitudes toward aerogel.
Side-by-side plots
smiley2022_plot + replication_plot
Exploratory analyses
What are the characteristics of those who knew aerogel vs. those who did not?
# A tibble: 13 × 4
# Groups: education [7]
education prior_knowledge n prop
<chr> <chr> <int> <dbl>
1 Associates or technical degree No 36 0.857
2 Associates or technical degree Yes 6 0.143
3 Bachelor’s degree No 125 0.850
4 Bachelor’s degree Yes 22 0.150
5 Graduate or professional degree (MA, MS, MBA, Ph… No 48 0.774
6 Graduate or professional degree (MA, MS, MBA, Ph… Yes 14 0.226
7 High school diploma or GED No 48 0.814
8 High school diploma or GED Yes 11 0.186
9 Prefer not to say No 1 0.5
10 Prefer not to say Yes 1 0.5
11 Some college, but no degree No 67 0.807
12 Some college, but no degree Yes 16 0.193
13 Some high school or less No 1 1
# A tibble: 2 × 5
prior_knowledge variable n mean sd
<chr> <fct> <dbl> <dbl> <dbl>
1 No age 326 46.7 15.3
2 Yes age 70 39.6 15.3
full_sample_conditions %>%group_by(prior_knowledge) %>%get_summary_stats(`Duration (in seconds)`, type="mean_sd")
# A tibble: 2 × 5
prior_knowledge variable n mean sd
<chr> <fct> <dbl> <dbl> <dbl>
1 No Duration (in seconds) 326 201. 161.
2 Yes Duration (in seconds) 70 178. 121.
# Examine potential overlap in the people who failed the manipulation check and those who knew aerogelfailed_manip <- full_sample_conditions %>%filter(manip_check != Condition) knew_aerogel <- full_sample_conditions %>%filter(prior_knowledge =="Yes")ggvenn(list("Failed Manipulation Check"= failed_manip$ResponseId, "Knew About Aerogel"= knew_aerogel$ResponseId),set_name_size =4,fill_color =c("darkred", "turquoise"))
# Examine potential interaction between condition and ageggplot(original_data, aes(x = Age, y = Eval, color = Condition)) +geom_point(alpha =0.3) +# Raw data pointsgeom_smooth(method ="lm", se =TRUE) +# Trend linesscale_color_manual(values=c("steelblue", "orange")) +labs(x="Age",y="Evaluation of Aerogel") + jtools::theme_apa()
`geom_smooth()` using formula = 'y ~ x'
# Test the interaction formallymodel_interaction <-lm(Eval ~ Condition * Age, data = original_data)# Check the summary for the "Condition:Age" termsummary(model_interaction)
Call:
lm(formula = Eval ~ Condition * Age, data = original_data)
Residuals:
Min 1Q Median 3Q Max
-3.1428 -0.8372 0.0875 0.8987 2.2413
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.505718 0.236520 6.366 5.36e-10 ***
ConditionBorn Before -0.809781 0.330189 -2.452 0.0146 *
Age -0.010370 0.005477 -1.893 0.0591 .
ConditionBorn Before:Age 0.013508 0.007544 1.791 0.0741 .
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 1.017 on 397 degrees of freedom
Multiple R-squared: 0.02464, Adjusted R-squared: 0.01727
F-statistic: 3.343 on 3 and 397 DF, p-value: 0.01927
ggplot(full_sample_conditions, aes(x = age, y = attitude_num, color = Condition)) +geom_point(alpha =0.3) +# Raw data pointsgeom_smooth(method ="lm", se =TRUE) +# Trend linesscale_color_manual(values=c("steelblue", "orange")) +labs(x ="Age",y="Evaluation of Aerogel") + jtools::theme_apa()
`geom_smooth()` using formula = 'y ~ x'
model_interaction_replication <-lm(attitude_num ~ Condition * age, data = full_sample_conditions)# Check the summary for the "Condition:Age" termsummary(model_interaction_replication)
Call:
lm(formula = attitude_num ~ Condition * age, data = full_sample_conditions)
Residuals:
Min 1Q Median 3Q Max
-3.16197 -1.04004 -0.08453 0.86464 2.03242
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.1137703 0.2374661 4.690 3.77e-06 ***
ConditionBorn-Before 0.1589030 0.3374084 0.471 0.638
age 0.0007416 0.0049784 0.149 0.882
ConditionBorn-Before:age -0.0058264 0.0070308 -0.829 0.408
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 1.087 on 392 degrees of freedom
Multiple R-squared: 0.005154, Adjusted R-squared: -0.00246
F-statistic: 0.6769 on 3 and 392 DF, p-value: 0.5666
Discussion
Summary of Replication Attempt
I failed to replicate the main finding from the original Study 1 from Smiley & Fisher (2022), which was that participants in the born-after condition had more favorable attitudes toward aerogel than participants in the born-before condition. Smiley and Fisher (2022) documented this effect both when they included participants who failed the manipulation check and had prior knowledge of aerogel AND when they excluded those participants. In my case, I did not find the effect in either case.
In my study, I also had to exclude a greater number of participants because 38 participants failed the manipulation check and an additional 60 participants knew about aerogel before the study (vs. only 33 participants in the original study).
For those participants who knew about aerogel, they may not have believed the information I presented to them about its year of invention.
Commentary
The fact that I failed to replicate the effect could have been due to a few reasons: (1) the effect does not exist, (2) my manipulation of year of aerogel invention was not as obvious as the original authors’, (3) my study was underpowered to detect an effect. My final sample size was 297, which is substantially smaller than the planned sample size.
On point (3), I noticed that the original authors were also underpowered in their post-exclusion analysis of the effect of year of invention. They determined they needed 176 participants per condition (352 total) to detect a medium effect with 80% power, but after their exclusions they had a total sample size of 328.
On a broader note, this (non) replication has reinforced the notion of “playing 20 questions with nature” (Almaatouq et al., 2024). When an effect cannot be replicated, it’s difficult to tell if it’s because the replication involved a slightly different population/paradigm or because the effect doesn’t actually exist. Are we all just touching different parts of the elephant? (And what is enough to constitute “different” in the context of any experiment?)
