Replication of Study ‘Intentional looks facilitate faster responding in observers’ (2024, Psychological Science)

Introduction

I chose this experiment because I believe we can better serve children with autism (and others with similar conditions) by enhancing our understanding of human perception and communication. Many children with autism struggle to interpret social cues like eye gaze and the intentions behind subtle changes in eye movement. By recognizing the cues that people naturally focus on to infer intention, we can contribute to the development or improvement of training tools, therapies, and educational curriculum to help children with autism navigate their social environment. This research, combined with findings from other communication psychology studies, have the potential to empower parents of autistic children with the knowledge to confidently support and contribute to their child’s development.

The researchers divided the experiment into three segments. The stimuli for all three experiments consists of video clips of two female gazers who were instructed to either make intentional or computer-instructed eye movements to the left or right. For experiments 2 and 3, asterisks were added to the clips as ‘peripheral targets’ and positioned to either match the direction of the gaze or be opposite to it. In terms of procedure, participants watched the clips and predicted the direction the gazer would look by clicking key ‘b’ or ‘h’. In experiment 2, participants were shown similar clips but were asked to track the location of the peripheral targets instead of gaze direction. The routine for experiment 3 was identical to experiment 2, except the targets appeared at different times for each trial. In each experiment, participants completed between 320 and 640 test trials that were divided into four blocks.

From a replication standpoint, a foreseeable challenge is the time-consuming nature of the data collection process. The experiment needs to be conducted in three stages, with each stage requiring an average of 75 unique participants. While this experiment is feasible, it may be taxing to complete within a single quarter.

Methods

Power Analysis

Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.

Planned Sample

Planned sample size and/or termination rule, sampling frame, known demographics if any, preselection rules if any.

Materials

All materials - can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.

Procedure

Can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.

Analysis Plan

Can also quote directly, though it is less often spelled out effectively for an analysis strategy section. The key is to report an analysis strategy that is as close to the original - data cleaning rules, data exclusion rules, covariates, etc. - as possible.

Clarify key analysis of interest here You can also pre-specify additional analyses you plan to do.

Differences from Original Study

Explicitly describe known differences in sample, setting, procedure, and analysis plan from original study. The goal, of course, is to minimize those differences, but differences will inevitably occur. Also, note whether such differences are anticipated to make a difference based on claims in the original article or subsequent published research on the conditions for obtaining the effect.

Methods Addendum (Post Data Collection)

You can comment this section out prior to final report with data collection.

Actual Sample

Sample size, demographics, data exclusions based on rules spelled out in analysis plan

Differences from pre-data collection methods plan

Any differences from what was described as the original plan, or “none”.

Results

Data preparation

Data preparation following the analysis plan.

Confirmatory analysis

The analyses as specified in the analysis plan.

Side-by-side graph with original graph is ideal here

Exploratory analyses

Any follow-up analyses desired (not required).

Discussion

Summary of Replication Attempt

Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.

Commentary

Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.