Replication of Study 1 by Yeshurun & Levy (2003). Transient spatial attention degrades temporal resolution. Psychological Science
Author
Irmak Ergin (irmak.ergin@stanford.edu)
Published
November 12, 2023
Introduction
Justification: My research program focuses on speech comprehension, in which considering temporal factors is essential. I am planning to work with degraded/noisy speech signals, adn one way to introduce noise is by adding multiple sound sources positioned in various locations. Considering this, learning more about spatial attention, as well as temporal attention, might provide me with useful insights. I was working on attention and auditory object recognition during my master’s, and paradigms in the field were heavily influenced by the literature on visual object recognition. Therefore, I hope to find inspiration through this rescue project.
Required Stimuli & Procedures:“Two flashes of light were presented successively to the same location, and we determined the minimal interval between the flashes at which they were still perceived as two separate flashes, rather than as a single continuous flash. The shorter this critical interval is, the higher the temporal resolution. This measurement of temporal resolution was combined with a direct manipulation of transient spatial attention. In half the trials, the cued trials, a peripheral cue indicated the target’s onset and location. This exogenous cue, a small horizontal bar, allowed observers to focus their attention, in advance, on the target location, and was assumed to capture attention in a stimulus-driven, automatic manner. In the rest of the trials, the neutral trials, two long horizontal lines indicated the target’s onset but not its location (i.e., the target was equally likely to appear at any location).” (Yeshurun & Levy, 2003, p.226)
Link to Repo: https://github.com/psych251/yeshurun2003_rescue/tree/main Link to RPubs: https://rpubs.com/irmak_ergin/1102430 Link to Original Paper: https://github.com/psych251/yeshurun2003_rescue/blob/main/original_paper/Yeshurun%20%26%20Levy(2003).pdf
Summary of prior replication attempt
#PSYCH251 1st Replication Project General summary of the prior replication attempt/original experiment: As in the original study, each observer’s threshold of temporal resolution were measured in two different conditions (cued and neutral). In cued trials, a periphery cue were presented before the target onset, which indicates the target’s location. While in neutral trials, neutral cue with no indication of the target’s location were presented.
The results from the original study was not replicated: * No main effect of cueing was observed in the prior replication attempt,unlike the original study. * The original study reported no interaction effect between cueing and ISI. Whereas, the first replication study found a significant interaction effect between cueing and ISI: In longer ISI conditions, spatial attention enhanced performance in the temporal resolution task, while in shorter ISI, spatial attention degraded the performance.
The original study had 18 participants while the 1st replication study had 10. The author of the 1st replication notes that this was due to not being able to reach out to enough Psych1 students as they were collecting data in-person.This might be the reason behind not being able to replicate the original study.
The original study did not report any exclusion from the sample. The 1st replication report states that no participant’s data was excluded from the study.
The original experiment and the 1st replication study have the same analysis methods but the replication study did not analyze the Reaction Time (RT) data. The original experiment did not find significant effects on RT.
#Forward Literature Search for Other Replication Attempts A forward literature search was conducted to find possible replication studies of the original paper, other than the 1st replication project done by Insub Kim for the PSYCH251 class. Chica & Christine (2009) Experiment 1: Using 12 participants, they replicated the finding that temporal resolution is impaired on spatially cued trials as compared with neutral trials, even when controlled for Reaction Time.
Methods
Power Analysis
The effect size is not reported in the original paper.The first replication study had 10 participants although it aimed to have 18 as in the original study. Therefore, I aim to get 18 participants following the number of participant reported in the original paper and see if I can replicate the findings.
Planned Sample
There will be no pre-selection rules except for normal or corrected-to-normal vision. I will recruit data from 18 PSYCH1 participants via a 2023 MacBook Pro. Participants will receive a course credit for participation.
Materials
Two-flash fusion threshold: Measure of temporal resolution. “Two flashes of light were presented successively to the same location, and we determined the minimal interval between the flashes at which they were still perceived as two separate flashes, rather than as a single continuous flash. The shorter this critical interval is, the higher the temporal resolution (e.g., Artieda, Pastor, Lacruz, & Obeso, 1992;Reeves, 1996).”
Target: The target (37-cd/m2 disk with a diameter of 3 degrees) was composed of two flashes of light separated by a brief interval or a single continuous flash. “It appeared on a black background at one of 11 possible locations along the horizontal meridian, with eccentricity (distance from the center) ranging from 0 to 16.5 degrees. The cue in the cued trials was a 1 X 0.3 degrees green horizontal bar (43 cd/m2) appearing 0.5 degree above the target’s location. In the neutral trials, two 17 x 0.3 degree green horizontal lines appeared above and below the entire display.”
(taken from the original paper & the first replication attempt report)
Procedure
I am planning to follow the exact procedure of the original study, used by the 1st replication attempt as well. Quoting from the original article:
“Each trial began with a fixation point followed by the attentional cue. The cue was present until the target’s offset, to prevent confusion between the flickering of the cue and that of the target. The target was presented 94 ms after onset of the cue: On 50% of the trials, two disks appeared, each for 47 ms, separated by a variable interval (inter stimulus interval, ISI, of 11–34 ms). On the rest of the trials, a single disk was presented for a duration ranging from 105 to 130 ms. These brief durations ensured that eye movements could not occur between cue onset and target offset (Mayfrank, Kimmig, & Fischer,1987). Each observer viewed 864 trials presented in a randomized order. Both accuracy and reaction time (RT) were recorded.”
Link to experiment code: https://github.com/psych251/yeshurun2003_rescue/blob/main/data/spaital_temporal.m
Controls
What attention checks, positive or negative controls, or other quality control measures are you adding so that a (positive or negative) result will be more interpretable?
Analysis Plan
The original study do not mention any participant or data exclusion. Participants were required to have normal or corrected to normal vision.
“Dependent Variables:
Accuracy(d’) was calculated following equation: d’ = z(hit) - z(false alarm), (Macmillan & Creelman, 1991).
Criterion(c, measure of response bias) was calculated using: c = -0.5(z(hit) + z(false alarm)), (Macmillan & Creelman, 1991).
Reaction Time (RT)“*
Clarify key analysis of interest here
“- Differences between cued and neutral trials - The effect of inter-stimulus interval (ISI)”*
Key analysis: A within-subject three-way analysis of variance (ANOVA; Cuing x Eccentricity x ISI) will be performed on the accuracy (d’) and RT data collected on trials with correct responses. The original study does not state a directional hypothesis in the Introduction section, but in Experiment 1 it is written that ” …we reasoned that attending to the location of this temporal target might decrease temporal resolution. Alternatively, the effects of attention on temporal resolution might not be constrained by its effects on spatial resolution.” (p.226)
*Quoted sections are taken from the 1st replication report.
Differences from Original Study and 1st replication
Data collection and Participants: The only difference I can see is that the first replication study had 10 participants although it aimed to have 18 as in the original study. This was dues to collecting data in-person and not being able to find enough participants in time. I will be collecting data from PPSYCH1 students.
The 1st replication attempt collected but not analyzed RT data, unlike the original experiment. I will be analyzing the effects of cueing, ISI, and eccentricity on RT as well as on accuracy (d prime).
Methods Addendum (Post Data Collection)
You can comment this section out prior to final report with data collection.
Actual Sample
Sample size, demographics, data exclusions based on rules spelled out in analysis plan
Differences from pre-data collection methods plan
Any differences from what was described as the original plan, or “none”.
Results
Data preparation
Link to plot A data: https://github.com/psych251/yeshurun2003_rescue/tree/main/data/_DATA/behavior/pilot_A
Link to data organization code (to be able to take transfer the data from MATLAB to R): https://github.com/psych251/yeshurun2003_rescue/blob/main/data/Mat2R.m
Data preparation plan: - Load the data - Cued and neural trials will be determined. (changing variable names to more “intuitive” ones) - Compute accuracy (d prime) value based on the equation stated in the analysis plan section. - Calculate the mean d primes per subgroups for analysis and plotting. E.g. for cuing conditions (cued vs neural), eccentricity x cue, ISI x cue, eccentricity x ISI. - Calculate the mean RTs per subgroups for analysis and plotting. E.g. for cuing conditions (cued vs neural), eccentricity x cue, ISI x cue, eccentricity x ISI.
The data preparation code below has taken from the first replication report with changes.
####Load Relevant Libraries and Functionslibrary(tidyverse)
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.1.3 ✔ readr 2.1.4
✔ forcats 1.0.0 ✔ stringr 1.5.0
✔ ggplot2 3.4.4 ✔ tibble 3.2.1
✔ lubridate 1.9.3 ✔ tidyr 1.3.0
✔ purrr 1.0.2
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
Attaching package: 'gridExtra'
The following object is masked from 'package:dplyr':
combine
library(cowplot)
Attaching package: 'cowplot'
The following object is masked from 'package:ggpubr':
get_legend
The following object is masked from 'package:ggthemes':
theme_map
The following object is masked from 'package:lubridate':
stamp
####Import datad <-read_csv("sptioTemporal.csv")
Rows: 8640 Columns: 9
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (2): sub, SDT
dbl (7): cue, numFlash, targetLoc, ISI, response, RT, accuracy
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
#### Prepare data for analysis - create columns and change variable names#Create a new column 'cue' in the dataset d. If the value in the 'cue' column is 1, it assigns 'cued', otherwise 'neutral' to the new 'cue' column.d$cue <-ifelse(d$cue ==1, 'cued', 'neutral')#Generate new subject names with the format "s0XX" (where XX is a two-digit number) for each unique subject in the 'sub' column and assign these new names to the 'sub' column in the dataset.x=1:length(unique(d$sub))new_subNames=sprintf("s0%02d",x)subNames =unique(d$sub)for(i in1:length(subNames)) { d$sub[d$sub == subNames[i]] = new_subNames[i]}end
function (x, ...)
UseMethod("end")
<bytecode: 0x13bd0d7b0>
<environment: namespace:stats>
#TargetLoc is the column with eccentricity information that are arbitraryly labeled from 1 to 6 in the stimulus creation code. The code below assigns back eccentricity values to 'targetLoc' labels. kk =sort(unique(d$targetLoc))mask1=d$targetLoc ==kk[1]mask2=d$targetLoc ==kk[2]mask3=d$targetLoc ==kk[3]mask4=d$targetLoc ==kk[4]mask5=d$targetLoc ==kk[5]mask6=d$targetLoc ==kk[6]d$targetLoc[mask1] =0d$targetLoc[mask2] =3d$targetLoc[mask3] =6d$targetLoc[mask4] =9d$targetLoc[mask5] =13d$targetLoc[mask6] =16#Convert 'cue,' 'targetLoc,' and 'ISI' columns in the dataset to factorsd <- d %>%mutate( cue =factor(cue),targetLoc =factor(targetLoc), ISI =factor(ISI))
#Make sub-selection of the data- RTdplot_cue_rt <- d %>%group_by(sub, cue) %>%summarize(mean_RT =mean(RT))
`summarise()` has grouped output by 'sub'. You can override using the `.groups`
argument.
dplot_ecc_rt <- d %>%group_by(cue, targetLoc) %>%summarize(mean_RT =mean(RT))
`summarise()` has grouped output by 'cue'. You can override using the `.groups`
argument.
dplot_ecc2_rt <- d %>%group_by(targetLoc, ISI) %>%summarize(mean_RT =mean(RT))
`summarise()` has grouped output by 'targetLoc'. You can override using the
`.groups` argument.
dplot_ISI_rt <- d %>%group_by(cue, ISI) %>%summarize(mean_RT =mean(RT))
`summarise()` has grouped output by 'cue'. You can override using the `.groups`
argument.
dplot_stat_ISI_rt <- d %>%group_by(sub, cue, ISI) %>%summarize(mean_RT =mean(RT))
`summarise()` has grouped output by 'sub', 'cue'. You can override using the
`.groups` argument.
dplot_ecc_sd_rt <- d %>%group_by(sub, targetLoc, ISI) %>%summarize(mean_RT =mean(RT))
`summarise()` has grouped output by 'sub', 'targetLoc'. You can override using
the `.groups` argument.
Results of control measures
There are no control measures planned for this study.
Confirmatory analysis
Following the original paper, and as it was done in the 1st replication attempt, a within-subject three-way analysis of variance (ANOVA; Cuing x Eccentricity x ISI) will be performed on the accuracy (d’) and RT data collected on trials with correct responses.
The key statistic of 3 way ANOVA anova was performed. The code below is taken by the first replication report.
###Accuracy# within-subject 3-way Anova on Accuracy (dPrime), testing the interaction of cue,eccentricity,and ISIResult_accuracy <-aov(dPrime ~ (cue*targetLoc*ISI) +Error(sub/cue*targetLoc*ISI), data=dp)summary(Result_accuracy)
Error: sub
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 9 145.6 16.18
Error: targetLoc
Df Sum Sq Mean Sq
targetLoc 5 1.926 0.3853
Error: ISI
Df Sum Sq Mean Sq
ISI 2 175.2 87.6
Error: sub:cue
Df Sum Sq Mean Sq F value Pr(>F)
cue 1 1.534 1.534 1.308 0.282
Residuals 9 10.557 1.173
Error: sub:targetLoc
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 45 23.69 0.5264
Error: sub:ISI
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 18 14.59 0.8107
Error: targetLoc:ISI
Df Sum Sq Mean Sq
targetLoc:ISI 10 5.62 0.562
Error: sub:cue:targetLoc
Df Sum Sq Mean Sq F value Pr(>F)
cue:targetLoc 5 2.118 0.4235 1.457 0.223
Residuals 45 13.080 0.2907
Error: sub:cue:ISI
Df Sum Sq Mean Sq F value Pr(>F)
cue:ISI 2 8.451 4.226 10.1 0.00115 **
Residuals 18 7.531 0.418
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Error: sub:targetLoc:ISI
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 90 24.86 0.2762
Error: sub:cue:targetLoc:ISI
Df Sum Sq Mean Sq F value Pr(>F)
cue:targetLoc:ISI 10 3.953 0.3953 1.92 0.0524 .
Residuals 90 18.534 0.2059
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
# main effect: Cuingsummary(Result_accuracy[5])
Error: sub:cue
Df Sum Sq Mean Sq F value Pr(>F)
cue 1 1.534 1.534 1.308 0.282
Residuals 9 10.557 1.173
# interaction effect: Cuing X ISIsummary(Result_accuracy[10])
Error: sub:cue:ISI
Df Sum Sq Mean Sq F value Pr(>F)
cue:ISI 2 8.451 4.226 10.1 0.00115 **
Residuals 18 7.531 0.418
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Error: sub
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 9 665.2 73.91
Error: targetLoc
Df Sum Sq Mean Sq
targetLoc 5 0.9979 0.1996
Error: ISI
Df Sum Sq Mean Sq
ISI 2 8.256 4.128
Error: sub:cue
Df Sum Sq Mean Sq F value Pr(>F)
cue 1 1.423 1.4230 3.179 0.108
Residuals 9 4.029 0.4476
Error: sub:targetLoc
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 45 11.47 0.255
Error: sub:ISI
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 18 10.79 0.5995
Error: targetLoc:ISI
Df Sum Sq Mean Sq
targetLoc:ISI 10 1.945 0.1945
Error: sub:cue:targetLoc
Df Sum Sq Mean Sq F value Pr(>F)
cue:targetLoc 5 0.812 0.1625 1.316 0.274
Residuals 45 5.555 0.1235
Error: sub:cue:ISI
Df Sum Sq Mean Sq F value Pr(>F)
cue:ISI 2 0.1317 0.06584 1.08 0.36
Residuals 18 1.0970 0.06094
Error: sub:targetLoc:ISI
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 90 18.22 0.2025
Error: sub:cue:targetLoc:ISI
Df Sum Sq Mean Sq F value Pr(>F)
cue:targetLoc:ISI 10 0.712 0.07116 0.763 0.663
Residuals 90 8.390 0.09322
Error: Within
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 8280 1939 0.2341
# Main effect: Cuingsummary(Result_rt[5])
Error: sub:cue
Df Sum Sq Mean Sq F value Pr(>F)
cue 1 1.423 1.4230 3.179 0.108
Residuals 9 4.029 0.4476
# Interaction effect: Cuing X ISIsummary(Result_rt[10])
Error: sub:cue:ISI
Df Sum Sq Mean Sq F value Pr(>F)
cue:ISI 2 0.1317 0.06584 1.08 0.36
Residuals 18 1.0970 0.06094
Three-panel graph with original, 1st replication, and your replication is ideal here
Exploratory analyses
Any follow-up analyses desired (not required).
Discussion
Mini meta analysis
Combining across the original paper, 1st replication, and 2nd replication, what is the aggregate effect size?
Summary of Replication Attempt
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Commentary
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.