Replication of Study 2 by van Prooijen et al. (2018, European Journal of Social Psychology)

Author

Aya Salim (ayasalim@stanford.edu)

Published

December 4, 2025

Introduction

Justification

My research interests lie in how people process and trust information in complex media environments. One phenomenon I am especially interested in is why some individuals find misleading claims compelling. Work demonstrates that illusory pattern perception - the tendency to perceive meaning or structure in randomness - may help explain people’s openness to pseudo-profound statements (Walker et al., 2019). van Prooijen et al. (2018) provide one of the few studies that experimentally manipulate pattern perception itself. In Study 2, the authors induced participants to rely either on intuition and pattern search or on analytic reasoning while evaluating random sequences of coin tosses. Those encouraged to rely on intuition perceived more structure in randomness and, in turn, reported stronger irrational beliefs. I am interested in understanding why some individuals perceive hidden structure or meaning in randomness and how this tendency relates to belief in conspiracy theories and misinformation.

I selected Study 2 particularly because it offers a direct causal test of the relationship between intuitive pattern search and belief formation - an ideal foundation for later work examining how heightened pattern perception might make people more receptive to misinformation or “bullshit” statements. Whereas other paradigms manipulate control or uncertainty to elicit pattern detection indirectly, this study isolates the pattern-perception process itself.

Given that the literature connecting illusory pattern perception to conspiracy beliefs has already produced mixed results, a careful replication of this paradigm can provide valuable clarity about the reliability and boundary conditions in this sphere. This replication contributes not only to my own program of research but also to broader efforts to assess the reproducibility of psychological findings about belief formation and illusory pattern perception.

Stimuli and Procedures

Participants completed a short task involving random coin-toss sequences, followed by ratings of perceived structure and measures of irrational belief.

The key result I hope to replicate is that participants instructed to use intuitive pattern search perceive more structure in random sequences than those instructed to analyze logically, and that higher pattern perception is correlated with stronger endorsement of irrational beliefs (e.g., conspiracy/supernatural).

Pattern-Search Instructions (Manipulation). Participants were randomly assigned to one of two instruction sets: a high pattern-search (intuitive) condition that encouraged relying on “gut feelings” and actively looking for meaningful patterns, or a low pattern-search (analytic) condition that emphasized logical, non-intuitive analysis.

Random Sequence Task (Pattern-Perception Measure). All participants viewed a series of randomly generated coin-toss sequences. After each sequence, they rated how structured or meaningful it appeared on a 1-7 scale (1 = completely random; 7 = clear pattern).

Belief Measures. Participants then completed short conspiracy and supernatural belief scales (Likert ratings of agreement with statements).

For this replication, I will recreate the same structure using Qualtrics and a Prolific participant pool. Primary challenges will be ensuring that the stimuli work reliably online and that participants stay attentive throughout the task.

Methods

Power Analysis

A power analysis to achieve 80% power was conducted using G*Power (using an original effect size of η2 = .04 that was converted and rounded to an effect size f = 0.2). This gives a sample size of 200 participants, but the planned sample size is n = 223 to fully replicate the size of the original study.

Planned Sample

223 U.S. adults recruited from Prolific.

Materials

Link to experiment: https://stanforduniversity.qualtrics.com/jfe/form/SV_6DytqtdUq1o5Tgi

Preview/demo: https://stanforduniversity.qualtrics.com/jfe/preview/previewId/e0dea73e-1583-41ec-abb0-0b6fd379b06e/SV_6DytqtdUq1o5Tgi?Q_CHL=preview&Q_SurveyVersionID=current

Procedure

Procedure from van Prooijen et al. (2018) was followed exactly:

“The study was presented as consisting of two parts. In the first part, participants were asked to play a “coin tossing game”. They saw the outcomes of 10 sequences of 10 coin tosses—these sequences were identical to those used in the pattern perception measure of Study 1. The first of these sequences was presented as an example; the “real” coin tossing game consisted of the remaining nine sequences.

For each sequence, participants’ task was to guess what the next coin outcome would be (Heads or Tails). Within this context, we manipulated intuitive pattern search. In the high pattern search condition, participants received the following instruction before starting the game: “Try to see if you can find a pattern in each sequence. Do NOT try to calculate this—use your intuition. Ask yourself: ‘Do I see a pattern here—and based on that, what next coin outcome would make most sense?’” In the low pattern search condition, participants received the following instruction: “These are random sequences, generated by the website http://random.org. In a particular sequence there may be more Heads or Tails; this is to be expected when a sequence is random. Each coin toss is independent and has an exact probability of 50% of being a Head or a Tail.” After completing the game, we assessed participants’ pattern perception with the following item: “To what extent were the coin flip sequences random, or showed a pattern?” (1 = they were totally random, 7 = they totally showed a pattern). Furthermore, we assessed participants’ current mood on a slider ranging from 1 (very negative) to 100 (very positive) as a filler task and also as a means to test whether the effects of the intuitive pattern search manipulation are attributable to mood effects.

Participants then started the second part of the study, in which they responded to a series of statements. Here, we measured belief in existing conspiracy theories (α = .84), belief in fictitious conspiracy theories (α = .86), and supernatural beliefs (α = .94) with the same scales as in Study 1. Upon completion of the questionnaire, participants were thanked and debriefed.”

Analysis Plan

The key analysis found in Study 2 is as follows:

“An ANOVA on the pattern perception measure revealed a significant effect of the intuitive pattern search manipulation, F(1, 179) = 8.26, p = .005; η2 = .04. Participants in the high pattern search condition detected clearer patterns in the random sequences (M = 4.03, SD = 1.64) than participants in the low pattern search condition (M = 3.32, SD = 1.71). These findings indicate that the manipulation successfully influenced the extent to which participants perceived patterns in the coin toss sequences.”

Additional analyses from Study 2 will be analyzed as well:

“A MANOVA on the three dependent variables yielded no significant multivariate or univariate effects, all Fs < 1. Contrary to predictions, the intuitive pattern search manipulation did not exert a direct effect on the dependent variables.

As noted above, however, the manipulation did influence the extent to which participants perceived patterns in the coin toss outcomes. Furthermore, consistent with Study 1, we found that the pattern perception measure was significantly correlated with belief in existing conspiracy theories (r = .23, p = .002), belief in fictitious conspiracy theories (r = .29, p < .001), and magical ideation (r = .32, p < .001). Given that we predicted the manipulation to influence irrational beliefs because of its effects on people’s tendency to see patterns in the sequences, we tested the indirect effect of the intuitive pattern search manipulation (effect‐coded: 1 high pattern search, −1 low pattern search) on irrational beliefs through pattern perception. As indicated by the fact that 0 was not in the 95% confidence interval, bootstrapping analyses (5000 samples) utilizing the “MEDIATE” macro by Hayes and Preacher (2014) revealed a significant indirect effect on all three dependent variables: for belief in existing conspiracy theories, (B = 0.04, SE = 0.02) CI95%[0.01; 0.09], for belief in fictitious conspiracy theories (B = 0.05, SE = 0.02) CI95%[0.02; 0.11], and for supernatural beliefs (B = 0.05, SE = 0.02) CI95%[0.01; 0.10]. These findings suggest that whereas the intuitive pattern search manipulation did not exert a direct effect on irrational beliefs, it did exert an indirect effect on all three dependent variables through pattern perception.

Although the lack of a direct effect precludes conclusions about causality, the findings of the present study suggest that intuitively searching for patterns in the coin toss sequences increases pattern perception, which in turn predicts irrational beliefs. As such, the indirect effect that we observed in Study 2 further supports a role for pattern perception in belief in conspiracy theories and supernatural beliefs.”

“The intuitive pattern search manipulation did not influence participants’ mood, F < 1. Hence, the results relating to irrational beliefs are not attributable to variations in participants’ mood.”

Differences from Original Study

The original paper uses a sample run online through the Crowdflower forum on a US sample, while this study recruits participants from Prolific.

Explicitly describe known differences in sample, setting, procedure, and analysis plan from original study. The goal, of course, is to minimize those differences, but differences will inevitably occur. Also, note whether such differences are anticipated to make a difference based on claims in the original article or subsequent published research on the conditions for obtaining the effect.

Methods Addendum (Post Data Collection)

You can comment this section out prior to final report with data collection.

Actual Sample

Sample size, demographics, data exclusions based on rules spelled out in analysis plan

Differences from pre-data collection methods plan

Any differences from what was described as the original plan, or “none”.

Results

Data preparation

Data preparation following the analysis plan.

### Data Preparation

library(tidyverse)
Warning: package 'tibble' was built under R version 4.3.3
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.4     ✔ readr     2.1.5
✔ forcats   1.0.0     ✔ stringr   1.5.1
✔ ggplot2   3.5.0     ✔ tibble    3.3.0
✔ lubridate 1.9.2     ✔ tidyr     1.3.0
✔ purrr     1.0.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(mediation)
Loading required package: MASS

Attaching package: 'MASS'

The following object is masked from 'package:dplyr':

    select

Loading required package: Matrix

Attaching package: 'Matrix'

The following objects are masked from 'package:tidyr':

    expand, pack, unpack

Loading required package: mvtnorm
Loading required package: sandwich
mediation: Causal Mediation Analysis
Version: 4.5.0
df <- read.csv('/Users/as4508/Downloads/PSYCH 251 Replication_December 4, 2025_15.18.csv')

#### Data exclusion / filtering
names(df) <- make.names(names(df))

stopifnot(all(c("Finished","QID1185","Q11","Q32","Q52_1") %in% names(df)))

# Existing conspiracy belief items: Q147_1–Q147_9
exist_cols <- paste0("Q147_", 1:9)

# Fictitious conspiracy belief items: Q61_1–Q61_9
fict_cols  <- paste0("Q61_", 1:9)

# Supernatural beliefs: Q62_1–Q62_30
super_cols <- paste0("Q62_", 1:30)

d <- df %>%
  filter(as.logical(Finished) == TRUE) %>%
  filter(tolower(trimws(QID1185)) %in% c("yes","y","true","1"))

# drop the data from friends / pilots
drop_ids <- c("R_1liTJPYAoikbwhU", "R_1Qmr1xstM6JpSYb", "R_64w2157Kz9bsjqa",
              "R_6FR4OAFpcZdyD3k", "R_3SeQxLORbYzjQN4","R_7kL1ACZPfz9rbcs",
              "R_3AFX4LwXrljWHIZ", "R_7mOfv2ExW0w7sCB", "R_6JL50stmvNPPtuy",
              "R_7JpQKNTqNFrnklj", "R_5FfHfOV3nyFznaB", "R_3xvtrU0slmNfzkl",
              "R_6oHJj6XoQTL8tjI", "R_3DkUBNSwPjJYXCx", "R_6sT1jlx2gMUSAsZ",
              "R_3xQdnhEeqHBgurg", "R_5EMFymcxI6cP46X", "R_6pSYCAZ6T51Ew0x",
              "R_1nVDozRCmfZm7r7", "R_1kCaOpobkDyCjk4", "R_7ypvG8EQ4odXSfc",
              "R_1ekQ49CnQzZoEZh", "R_5tbN9M9sPl8C6Fr", "R_12okXl4lCtnUlUD")

d <- d %>%
  filter(!ResponseId %in% drop_ids)

d <- d %>%
  mutate(Condition = case_when(tolower(trimws(condition)) == "high" ~ "High pattern search",
      tolower(trimws(condition)) == "low"  ~ "Low pattern search",
      TRUE ~ NA_character_),
    Condition = factor(Condition,
                       levels = c("Low pattern search", "High pattern search")),
    # 0/1 coding for mediation
    HPS_num = ifelse(Condition == "High pattern search", 1, 0)) %>%
  filter(!is.na(Condition))

#### Pattern perception
d <- d %>%
  mutate(PatternPerception = (as.numeric(Q52_1)))

#### Irrational belief scales

lkrt_map <- c("definitely not true" = 1,
              "probably not true" = 2,
              "unsure" = 3,
              "probably true" = 4,
              "definitely true" = 5)

recode_belief <- function(x) {
  x_chr <- tolower(trimws(as.character(x)))
  as.numeric(unname(lkrt_map[x_chr]))}

d <- d %>%
  mutate(across(all_of(c(exist_cols, fict_cols, super_cols)), recode_belief),
    # Existing conspiracy beliefs (Q147_* scale)
    ExistingConspiracyBeliefs =
      rowMeans(across(all_of(exist_cols)), na.rm = TRUE),
    # Fictitious conspiracy beliefs (Q61_* scale)
    FictitiousConspiracyBeliefs =
      rowMeans(across(all_of(fict_cols)), na.rm = TRUE),
    # Supernatural beliefs (Q62_* scale)
    SupernaturalBeliefs =
      rowMeans(across(all_of(super_cols)), na.rm = TRUE))
# analyses

# condition on pattern perception

print(summary(aov(PatternPerception ~ Condition, data = d)))
            Df Sum Sq Mean Sq F value Pr(>F)
Condition    1      1       1     0.2  0.698
Residuals    2     10       5               
print(t.test(PatternPerception ~ Condition, data = d, var.equal = TRUE))

    Two Sample t-test

data:  PatternPerception by Condition
t = -0.44721, df = 2, p-value = 0.6985
alternative hypothesis: true difference in means between group Low pattern search and group High pattern search is not equal to 0
95 percent confidence interval:
 -10.621024   8.621024
sample estimates:
 mean in group Low pattern search mean in group High pattern search 
                                3                                 4 
# correlation

print(cor.test(d$PatternPerception, d$ExistingConspiracyBeliefs,
               use = "pairwise.complete.obs"))

    Pearson's product-moment correlation

data:  d$PatternPerception and d$ExistingConspiracyBeliefs
t = 5.0999, df = 2, p-value = 0.03636
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 0.03451432 0.99926536
sample estimates:
      cor 
0.9636364 
print(summary(lm(ExistingConspiracyBeliefs ~ PatternPerception, data = d)))

Call:
lm(formula = ExistingConspiracyBeliefs ~ PatternPerception, data = d)

Residuals:
       1        2        3        4 
 0.07071 -0.15152  0.16162 -0.08081 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)  
(Intercept)        1.81313    0.20328   8.919   0.0123 *
PatternPerception  0.26768    0.05249   5.100   0.0364 *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1741 on 2 degrees of freedom
Multiple R-squared:  0.9286,    Adjusted R-squared:  0.8929 
F-statistic: 26.01 on 1 and 2 DF,  p-value: 0.03636
print(cor.test(d$PatternPerception, d$FictitiousConspiracyBeliefs,
               use = "pairwise.complete.obs"))

    Pearson's product-moment correlation

data:  d$PatternPerception and d$FictitiousConspiracyBeliefs
t = 0.60613, df = 2, p-value = 0.6061
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 -0.9127082  0.9828932
sample estimates:
      cor 
0.3939394 
print(summary(lm(FictitiousConspiracyBeliefs ~ PatternPerception, data = d)))

Call:
lm(formula = FictitiousConspiracyBeliefs ~ PatternPerception, 
    data = d)

Residuals:
       1        2        3        4 
-0.29293  0.37374 -0.16162  0.08081 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)  
(Intercept)        2.96465    0.41953   7.067   0.0194 *
PatternPerception  0.06566    0.10832   0.606   0.6061  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.3593 on 2 degrees of freedom
Multiple R-squared:  0.1552,    Adjusted R-squared:  -0.2672 
F-statistic: 0.3674 on 1 and 2 DF,  p-value: 0.6061
print(cor.test(d$PatternPerception, d$SupernaturalBeliefs,
               use = "pairwise.complete.obs"))

    Pearson's product-moment correlation

data:  d$PatternPerception and d$SupernaturalBeliefs
t = -0.18351, df = 2, p-value = 0.8713
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 -0.9698263  0.9498810
sample estimates:
       cor 
-0.1286843 
print(summary(lm(SupernaturalBeliefs ~ PatternPerception, data = d)))

Call:
lm(formula = SupernaturalBeliefs ~ PatternPerception, data = d)

Residuals:
       1        2        3        4 
 0.08182 -0.28485  0.40606 -0.20303 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)  
(Intercept)        2.52424    0.44768   5.639    0.030 *
PatternPerception -0.02121    0.11559  -0.184    0.871  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.3834 on 2 degrees of freedom
Multiple R-squared:  0.01656,   Adjusted R-squared:  -0.4752 
F-statistic: 0.03368 on 1 and 2 DF,  p-value: 0.8713
# mediations

# a-path (same mediator model for all three)
m.mod <- lm(PatternPerception ~ HPS_num, data = d)

set.seed(251)

## (a) Existing conspiracy beliefs
y.exist <- lm(ExistingConspiracyBeliefs ~ HPS_num + PatternPerception, data = d)

#  med.exist <- mediate(
#   m.mod, y.exist,
#   treat = "HPS_num",
#   mediator = "PatternPerception",
#   boot = TRUE, sims = 5000
# )

# print(summary(med.exist))

## (b) Fictitious conspiracy beliefs
y.fict <- lm(FictitiousConspiracyBeliefs ~ HPS_num + PatternPerception, data = d)

# # med.fict <- mediate(
#   m.mod, y.fict,
#   treat = "HPS_num",
#   mediator = "PatternPerception",
#   boot = TRUE, sims = 5000
# )

# print(summary(med.fict))

## (c) Supernatural beliefs
y.super <- lm(SupernaturalBeliefs ~ HPS_num + PatternPerception, data = d)

# # med.super <- mediate(
#   m.mod, y.super,
#   treat = "HPS_num",
#   mediator = "PatternPerception",
#   boot = TRUE, sims = 5000
# )

# print(summary(med.super))

# MANOVA

mv.mod <- manova(
  cbind(
    ExistingConspiracyBeliefs,
    FictitiousConspiracyBeliefs,
    SupernaturalBeliefs
  ) ~ Condition,
  data = d
)

# print(summary(mv.mod, test = "Wilks"))
print(summary.aov(mv.mod))
 Response ExistingConspiracyBeliefs :
            Df  Sum Sq Mean Sq F value Pr(>F)
Condition    1 0.25000 0.25000  0.8351 0.4573
Residuals    2 0.59877 0.29938               

 Response FictitiousConspiracyBeliefs :
            Df  Sum Sq Mean Sq F value Pr(>F)
Condition    1 0.15124 0.15124    1.96 0.2965
Residuals    2 0.15432 0.07716               

 Response SupernaturalBeliefs :
            Df   Sum Sq  Mean Sq F value Pr(>F)
Condition    1 0.217778 0.217778  5.3699 0.1464
Residuals    2 0.081111 0.040556               
ggplot(d, aes(PatternPerception, ExistingConspiracyBeliefs, color = Condition)) +
  geom_point(alpha = .7) +
  geom_smooth(method = "lm", se = TRUE) +
  labs(title = "Relation between pattern perception and existing conspiracy beliefs",
       x = "Pattern perception (1–7)",
       y = "Existing conspiracy beliefs (1–5)", 
       color = "Condition") +
  theme_minimal()
`geom_smooth()` using formula = 'y ~ x'
Warning in qt((1 - level)/2, df): NaNs produced

Warning in qt((1 - level)/2, df): NaNs produced
Warning in max(ids, na.rm = TRUE): no non-missing arguments to max; returning
-Inf

Warning in max(ids, na.rm = TRUE): no non-missing arguments to max; returning
-Inf

# main plot

ggplot(d, aes(Condition, PatternPerception, fill = Condition)) +
  geom_boxplot(alpha = .65) +
  labs(
    title = "Pattern Perception by Condition",
    x = "Instruction Condition",
    y = "Pattern Perception (1–7)"
  ) +
  guides(fill = "none") +
  theme_minimal()

Confirmatory analysis

The analyses as specified in the analysis plan.

Side-by-side graph with original graph is ideal here

Exploratory analyses

Any follow-up analyses desired (not required).

Discussion

Summary of Replication Attempt

Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.

Commentary

Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.