Replication of Study 2 by van Prooijen et al. (2018, European Journal of Social Psychology)
Introduction
Justification
My research interests lie in how people process and trust information in complex media environments. One phenomenon I am especially interested in is why some individuals find misleading claims compelling. Work demonstrates that illusory pattern perception - the tendency to perceive meaning or structure in randomness - may help explain people’s openness to pseudo-profound statements (Walker et al., 2019). van Prooijen et al. (2018) provide one of the few studies that experimentally manipulate pattern perception itself. In Study 2, the authors induced participants to rely either on intuition and pattern search or on analytic reasoning while evaluating random sequences of coin tosses. Those encouraged to rely on intuition perceived more structure in randomness and, in turn, reported stronger irrational beliefs. I am interested in understanding why some individuals perceive hidden structure or meaning in randomness and how this tendency relates to belief in conspiracy theories and misinformation.
I selected Study 2 particularly because it offers a direct causal test of the relationship between intuitive pattern search and belief formation - an ideal foundation for later work examining how heightened pattern perception might make people more receptive to misinformation or “bullshit” statements. Whereas other paradigms manipulate control or uncertainty to elicit pattern detection indirectly, this study isolates the pattern-perception process itself.
Given that the literature connecting illusory pattern perception to conspiracy beliefs has already produced mixed results, a careful replication of this paradigm can provide valuable clarity about the reliability and boundary conditions in this sphere. This replication contributes not only to my own program of research but also to broader efforts to assess the reproducibility of psychological findings about belief formation and illusory pattern perception.
Stimuli and Procedures
Participants completed a short task involving random coin-toss sequences, followed by ratings of perceived structure and measures of irrational belief.
The key result I hope to replicate is that participants instructed to use intuitive pattern search perceive more structure in random sequences than those instructed to analyze logically, and that higher pattern perception is correlated with stronger endorsement of irrational beliefs (e.g., conspiracy/supernatural).
Pattern-Search Instructions (Manipulation). Participants were randomly assigned to one of two instruction sets: a high pattern-search (intuitive) condition that encouraged relying on “gut feelings” and actively looking for meaningful patterns, or a low pattern-search (analytic) condition that emphasized logical, non-intuitive analysis and discouraged intuition.
Random Sequence Task (Pattern-Perception Measure). All participants viewed a series of randomly generated coin-toss sequences. After each sequence, they rated how structured or meaningful it appeared on a 1–7 scale (1 = completely random; 7 = clear pattern).
Belief Measures. Participants then completed short conspiracy and supernatural belief scales (Likert ratings of agreement with statements).
For this replication, I will recreate the same structure using Qualtrics and a Prolific participant pool. Primary challenges will be ensuring that the stimuli work reliably online and that participants stay attentive throughout the task.
Links
Methods
Power Analysis
Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.
Planned Sample
Planned sample size and/or termination rule, sampling frame, known demographics if any, preselection rules if any.
Materials
All materials - can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.
Procedure
Can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.
Analysis Plan
Can also quote directly, though it is less often spelled out effectively for an analysis strategy section. The key is to report an analysis strategy that is as close to the original - data cleaning rules, data exclusion rules, covariates, etc. - as possible.
Clarify key analysis of interest here You can also pre-specify additional analyses you plan to do.
Differences from Original Study
Explicitly describe known differences in sample, setting, procedure, and analysis plan from original study. The goal, of course, is to minimize those differences, but differences will inevitably occur. Also, note whether such differences are anticipated to make a difference based on claims in the original article or subsequent published research on the conditions for obtaining the effect.
Methods Addendum (Post Data Collection)
You can comment this section out prior to final report with data collection.
Actual Sample
Sample size, demographics, data exclusions based on rules spelled out in analysis plan
Differences from pre-data collection methods plan
Any differences from what was described as the original plan, or “none”.
Results
Data preparation
Data preparation following the analysis plan.
Confirmatory analysis
The analyses as specified in the analysis plan.
Side-by-side graph with original graph is ideal here
Exploratory analyses
Any follow-up analyses desired (not required).
Discussion
Summary of Replication Attempt
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Commentary
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.