Replication of Study What makes words special? Words as unmotivated cues (2015, Cognition)
Introduction
This study aimed to explore why verbal labels, such as the words “dog” or “guitar,” activate conceptual knowledge more effectively than environmental sounds associated with these objects, such as the bark of a dog or the strum of a guitar. I chose this topic because it intersects with my interests in language learning and auditory perception. This study finds that verbal labels (or words) are more effective than sounds in activating abstract category concepts because labels act as “unmotivated cues,” broadly representing a category without specific reference to particular instances. In contrast, sounds are “motivated cues” that link directly to specific sources or instances, limiting their effectiveness in promoting conceptual abstraction. This difference is highlighted by experiments showing that words activate category-level knowledge more selectively than environmental sounds.
In this experiment, participants will be presented with either a verbal representation or environmental sound for the following categories: bird, dog, drum, guitar, motorcycle, and phone. Participants are presented with an auditory cue (either a word or sound) and a picture presented 1 second after the auditory input is made. Participants are tested on how quickly and accurately they can determine if the picture presented matches the auditory cue they received. They will use a yes or no button on the computer screen. Potential challenges of this study could be sound quality of the environmental sounds to ensure they are clearly recognizable. Additionally, finding a diverse group of participants for this study could be a challenge.
Methods
Power Analysis
Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.
Planned Sample
We plan to have a sample size of n = 43 to be as close as possible to the original study. For pre-screening, participants must speak English fluently to ensure comprehension in the task. Participants are recruited on Prolific online platform.
Materials
The materials were followed precisely as follows. “The auditory cues comprised basic-level category labels and environmental sounds for six categories: bird, dog, drum, guitar, motorcycle, and phone. For each category, we obtained two distinct environmental sound cues, e.g.,
Procedure
The procedure was followed precisely as in the original study for Experiment 1A. “On each trial participants heard a cue and saw a picture. We instructed participants to decide as quickly and accurately as possible if the picture they saw came from the same basic-level cate- gory as the word or sound they heard Participants were tested in individual rooms sitting approximately 2400 from a monitor such that images subtended 10 10°. Trials began with a 250 ms. fixation cross followed immediately by the auditory cue, delivered via headphones. The target image appeared centrally 1 s after the off- set of the auditory cue and remained visible until a response was made. Each participant completed 6 practice and 384 test trials. If the picture matched the auditory cue (50% of trials) participants were instructed to respond ‘Yes’ on a gaming controller (e.g.,
Analysis Plan
Clarify key analysis of interest here
A linear mixed regression model will be used to analyze response times for correct responses including “random intercepts and random slopes for within-subject factors and random intercepts for repeated items (unique trial types)”(Edmiston & Lupyan, 2015).
In addition, significance tests using chi-square tests will be used to compare nested models, as was done in the original study, to assess the impact of factors with and without interest on response times.
Differences from Original Study
The original study used a remote controller. The replication study will use the computer for responses and implement key responses to simulate buttons on controller. This should allow for accurate and quick responses.
Methods Addendum (Post Data Collection)
You can comment this section out prior to final report with data collection.
Actual Sample
Sample size, demographics, data exclusions based on rules spelled out in analysis plan
Differences from pre-data collection methods plan
Any differences from what was described as the original plan, or “none”.
Design Overview
How many factors were manipulated? In this experiment one factor, the auditory cue, was manipulated. Within this factor there are conditions - congruent sound, incongruent sound, and label.
How many measures were taken? Reaction time was the only measure taken.
Did it use a within-participants or between-participants design? It uses within-participants design where everyone does each condition.
Were measures repeated? Yes measures were repeated for 384 experimental trials.
What would have been the consequence of changing one of these decisions? (e.g. moving from within to between participants). Could it have been either way? There would be more confounds if it was between participants because people have different reaction times to stimuli, therefore the measures wouldn’t be as accurate.
Were steps taken to reduce demand characteristics? They didn’t take any measures to reduce demand characteristics.
How would you critique the design? To improve the experiment attention checks could be added. Given that incorrect responses could be labeled incongruent, incorrect responses could have also been included in the data.
Can you think of any potential confounds to the study?
Do you see any limitations to generalizability? This sample could not be representative of all populations because it only studied undergraduates and WEIRD populations. ## Results
Data preparation
We are replicating Experiment 1A from the paper. To clean the data we will remove incorrect responses or participants who failed attention checks. As for data exclusion rules, we will follow the parameters set by the original study in which “Response times (RTs) shorter than 250 ms. or longer than 1500 ms. [will be] removed”. This accounts for outliers that may skew results. There aren’t any covariates mentioned in this experiment.
Confirmatory analysis
A linear mixed regression model will be used to analyze response times for correct responses including “random intercepts and random slopes for within-subject factors and random intercepts for repeated items (unique trial types)”(Edmiston & Lupyan, 2015).
In addition, significance tests using chi-square tests will be used to compare nested models, as was done in the original study, to assess the impact of factors with and without interest on response times.
Side-by-side graph with original graph is ideal here
Exploratory analyses
Any follow-up analyses desired (not required).
Discussion
Summary of Replication Attempt
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Commentary
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.