Replication of Study 1 by Yeshurun & Levy (2003). Transient spatial attention degrades temporal resolution. Psychological Science

Author

Irmak Ergin (irmak.ergin@stanford.edu)

Published

November 12, 2023

Introduction

Justification: My research program focuses on speech comprehension, in which considering temporal factors is essential. I am planning to work with degraded/noisy speech signals, adn one way to introduce noise is by adding multiple sound sources positioned in various locations. Considering this, learning more about spatial attention, as well as temporal attention, might provide me with useful insights. I was working on attention and auditory object recognition during my master’s, and paradigms in the field were heavily influenced by the literature on visual object recognition. Therefore, I hope to find inspiration through this rescue project.

Required Stimuli & Procedures:“Two flashes of light were presented successively to the same location, and we determined the minimal interval between the flashes at which they were still perceived as two separate flashes, rather than as a single continuous flash. The shorter this critical interval is, the higher the temporal resolution. This measurement of temporal resolution was combined with a direct manipulation of transient spatial attention. In half the trials, the cued trials, a peripheral cue indicated the target’s onset and location. This exogenous cue, a small horizontal bar, allowed observers to focus their attention, in advance, on the target location, and was assumed to capture attention in a stimulus-driven, automatic manner. In the rest of the trials, the neutral trials, two long horizontal lines indicated the target’s onset but not its location (i.e., the target was equally likely to appear at any location).” (Yeshurun & Levy, 2003, p.226)

Link to Repo: https://github.com/psych251/yeshurun2003_rescue/tree/main Link to RPubs: https://rpubs.com/irmak_ergin/1102430 Link to Original Paper: https://github.com/psych251/yeshurun2003_rescue/blob/main/original_paper/Yeshurun%20%26%20Levy(2003).pdf

Summary of prior replication attempt

#PSYCH251 1st Replication Project General summary of the prior replication attempt/original experiment: As in the original study, each observer’s threshold of temporal resolution were measured in two different conditions (cued and neutral). In cued trials, a periphery cue were presented before the target onset, which indicates the target’s location. While in neutral trials, neutral cue with no indication of the target’s location were presented.

The results from the original study was not replicated: * No main effect of cueing was observed in the prior replication attempt,unlike the original study. * The original study reported no interaction effect between cueing and ISI. Whereas, the first replication study found a significant interaction effect between cueing and ISI: In longer ISI conditions, spatial attention enhanced performance in the temporal resolution task, while in shorter ISI, spatial attention degraded the performance.

The original study had 18 participants while the 1st replication study had 10. The author of the 1st replication notes that this was due to not being able to reach out to enough Psych1 students as they were collecting data in-person.This might be the reason behind not being able to replicate the original study.

The original study did not report any exclusion from the sample. The 1st replication report states that no participant’s data was excluded from the study.

The original experiment and the 1st replication study have the same analysis methods but the replication study did not analyze the Reaction Time (RT) data. The original experiment did not find significant effects on RT.

#Forward Literature Search for Other Replication Attempts A forward literature search was conducted to find possible replication studies of the original paper, other than the 1st replication project done by Insub Kim for the PSYCH251 class. Chica & Christine (2009) Experiment 1: Using 12 participants, they replicated the finding that temporal resolution is impaired on spatially cued trials as compared with neutral trials, even when controlled for Reaction Time.

Methods

Power Analysis

The effect size is not reported in the original paper.The first replication study had 10 participants although it aimed to have 18 as in the original study. Therefore, I aim to get 18 participants following the number of participant reported in the original paper and see if I can replicate the findings.

Planned Sample

There will be no pre-selection rules except for normal or corrected-to-normal vision. I will recruit data from 18 PSYCH1 participants via a 2023 MacBook Pro. Participants will receive a course credit for participation.

Materials

Two-flash fusion threshold: Measure of temporal resolution. “Two flashes of light were presented successively to the same location, and we determined the minimal interval between the flashes at which they were still perceived as two separate flashes, rather than as a single continuous flash. The shorter this critical interval is, the higher the temporal resolution (e.g., Artieda, Pastor, Lacruz, & Obeso, 1992;Reeves, 1996).”

Target: The target (37-cd/m2 disk with a diameter of 3 degrees) was composed of two flashes of light separated by a brief interval or a single continuous flash. “It appeared on a black background at one of 11 possible locations along the horizontal meridian, with eccentricity (distance from the center) ranging from 0 to 16.5 degrees. The cue in the cued trials was a 1 X 0.3 degrees green horizontal bar (43 cd/m2) appearing 0.5 degree above the target’s location. In the neutral trials, two 17 x 0.3 degree green horizontal lines appeared above and below the entire display.”

(taken from the original paper & the first replication attempt report)

Procedure

I am planning to follow the exact procedure of the original study, used by the 1st replication attempt as well. Quoting from the original article:

“Each trial began with a fixation point followed by the attentional cue. The cue was present until the target’s offset, to prevent confusion between the flickering of the cue and that of the target. The target was presented 94 ms after onset of the cue: On 50% of the trials, two disks appeared, each for 47 ms, separated by a variable interval (inter stimulus interval, ISI, of 11–34 ms). On the rest of the trials, a single disk was presented for a duration ranging from 105 to 130 ms. These brief durations ensured that eye movements could not occur between cue onset and target offset (Mayfrank, Kimmig, & Fischer,1987). Each observer viewed 864 trials presented in a randomized order. Both accuracy and reaction time (RT) were recorded.”

Fig.1. adapted from the original paper by the author of the 1st replication attempt, Insub Kim Link to experiment code: https://github.com/psych251/yeshurun2003_rescue/blob/main/data/spaital_temporal.m

Controls

What attention checks, positive or negative controls, or other quality control measures are you adding so that a (positive or negative) result will be more interpretable?

Analysis Plan

The original study do not mention any participant or data exclusion. Participants were required to have normal or corrected to normal vision.

“Dependent Variables:

  • Accuracy(d’) was calculated following equation: d’ = z(hit) - z(false alarm), (Macmillan & Creelman, 1991).
  • Criterion(c, measure of response bias) was calculated using: c = -0.5(z(hit) + z(false alarm)), (Macmillan & Creelman, 1991).
  • Reaction Time (RT)“*

Clarify key analysis of interest here

“- Differences between cued and neutral trials - The effect of inter-stimulus interval (ISI)”*

Key analysis: A within-subject three-way analysis of variance (ANOVA; Cuing x Eccentricity x ISI) will be performed on the accuracy (d’) and RT data collected on trials with correct responses. The original study does not state a directional hypothesis in the Introduction section, but in Experiment 1 it is written that ” …we reasoned that attending to the location of this temporal target might decrease temporal resolution. Alternatively, the effects of attention on temporal resolution might not be constrained by its effects on spatial resolution.” (p.226)

*Quoted sections are taken from the 1st replication report.

Differences from Original Study and 1st replication

Data collection and Participants: The only difference I can see is that the first replication study had 10 participants although it aimed to have 18 as in the original study. This was dues to collecting data in-person and not being able to find enough participants in time. I will be collecting data from PPSYCH1 students.

The 1st replication attempt collected but not analyzed RT data, unlike the original experiment. I will be analyzing the effects of cueing, ISI, and eccentricity on RT as well as on accuracy (d prime).

Methods Addendum (Post Data Collection)

You can comment this section out prior to final report with data collection.

Actual Sample

Sample size, demographics, data exclusions based on rules spelled out in analysis plan

Differences from pre-data collection methods plan

Any differences from what was described as the original plan, or “none”.

Results

Data preparation

Link to plot A data: https://github.com/psych251/yeshurun2003_rescue/tree/main/data/_DATA/behavior/pilot_A

Link to data organization code (to be able to take transfer the data from MATLAB to R): https://github.com/psych251/yeshurun2003_rescue/blob/main/data/Mat2R.m

Data preparation plan: - Load the data - Cued and neural trials will be determined. (changing variable names to more “intuitive” ones) - Compute accuracy (d prime) value based on the equation stated in the analysis plan section. - Calculate the mean d primes per subgroups for analysis and plotting. E.g. for cuing conditions (cued vs neural), eccentricity x cue, ISI x cue, eccentricity x ISI. - Calculate the mean RTs per subgroups for analysis and plotting. E.g. for cuing conditions (cued vs neural), eccentricity x cue, ISI x cue, eccentricity x ISI.

The data preparation code below has taken from the first replication report with changes.

####Load Relevant Libraries and Functions

library(tidyverse)
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.3     ✔ readr     2.1.4
✔ forcats   1.0.0     ✔ stringr   1.5.0
✔ ggplot2   3.4.4     ✔ tibble    3.2.1
✔ lubridate 1.9.3     ✔ tidyr     1.3.0
✔ purrr     1.0.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(ggplot2)
library(ggthemes)
library(psycho)
library(ggpubr)
library(gridExtra)

Attaching package: 'gridExtra'

The following object is masked from 'package:dplyr':

    combine
library(cowplot)

Attaching package: 'cowplot'

The following object is masked from 'package:ggpubr':

    get_legend

The following object is masked from 'package:ggthemes':

    theme_map

The following object is masked from 'package:lubridate':

    stamp
####Import data

d <- read_csv("sptioTemporal.csv")
Rows: 8640 Columns: 9
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (2): sub, SDT
dbl (7): cue, numFlash, targetLoc, ISI, response, RT, accuracy

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
#### Prepare data for analysis - create columns and change variable names
#Create a new column 'cue' in the dataset d. If the value in the 'cue' column is 1, it assigns 'cued', otherwise 'neutral' to the new 'cue' column.

d$cue <- ifelse(d$cue == 1, 'cued', 'neutral')

#Generate new subject names with the format "s0XX" (where XX is a two-digit number) for each unique subject in the 'sub' column and assign these new names to the 'sub' column in the dataset.

x=1:length(unique(d$sub))
new_subNames=sprintf("s0%02d",x)
subNames = unique(d$sub)
for(i in 1:length(subNames)) {
  d$sub[d$sub == subNames[i]] = new_subNames[i]
}
end
function (x, ...) 
UseMethod("end")
<bytecode: 0x13bd0d7b0>
<environment: namespace:stats>
head(d)
# A tibble: 6 × 9
  cue     numFlash targetLoc   ISI response    RT accuracy sub   SDT  
  <chr>      <dbl>     <dbl> <dbl>    <dbl> <dbl>    <dbl> <chr> <chr>
1 neutral        1         1    35        1 0.464        1 s001  CR   
2 neutral        2         6    35        2 0.447        1 s001  HI   
3 cued           1         4    23        1 0.413        1 s001  CR   
4 cued           2         3    23        2 0.411        1 s001  HI   
5 neutral        1         4    23        1 0.298        1 s001  CR   
6 cued           1         6    35        1 0.370        1 s001  CR   
#TargetLoc is the column with eccentricity information that are arbitraryly labeled from 1 to 6 in the stimulus creation code. The code below assigns back eccentricity values to 'targetLoc' labels. 

kk = sort(unique(d$targetLoc))
mask1=d$targetLoc ==kk[1]
mask2=d$targetLoc ==kk[2]
mask3=d$targetLoc ==kk[3]
mask4=d$targetLoc ==kk[4]
mask5=d$targetLoc ==kk[5]
mask6=d$targetLoc ==kk[6]

d$targetLoc[mask1] = 0
d$targetLoc[mask2] = 3
d$targetLoc[mask3] = 6
d$targetLoc[mask4] = 9
d$targetLoc[mask5] = 13
d$targetLoc[mask6] = 16

#Convert  'cue,' 'targetLoc,' and 'ISI' columns in the dataset to factors
d <- d %>% mutate( 
  cue = factor(cue),targetLoc = factor(targetLoc), ISI = factor(ISI)
)
#### Compute dPrime

dp <- d %>% group_by(sub,cue,targetLoc,ISI) %>% summarize(nHit = sum(SDT == 'HI' ), nFA = sum(SDT == 'FA' ), nMiss = sum(SDT == 'MI' ), nCR = sum(SDT == 'CR' ), dPrime= dprime(nHit, nFA, nMiss, nCR)$dprime, criterion=dprime(nHit, nFA, nMiss, nCR)$c) 
`summarise()` has grouped output by 'sub', 'cue', 'targetLoc'. You can override
using the `.groups` argument.
#### Make sub-selection of the data for plotting later on -accuracy

dplot_cue <- dp %>% group_by(sub,cue) %>% summarize(dPrime = mean(dPrime))
`summarise()` has grouped output by 'sub'. You can override using the `.groups`
argument.
dplot_ecc <- dp %>% group_by(cue,targetLoc) %>% summarize(dPrime = mean(dPrime))
`summarise()` has grouped output by 'cue'. You can override using the `.groups`
argument.
dplot_ecc2 <- dp %>% group_by(targetLoc,ISI) %>% summarize(dPrime = mean(dPrime))
`summarise()` has grouped output by 'targetLoc'. You can override using the
`.groups` argument.
dplot_ISI <- dp %>% group_by(cue,ISI) %>% summarize(dPrime = mean(dPrime))
`summarise()` has grouped output by 'cue'. You can override using the `.groups`
argument.
dplot_stat_ISI <- dp %>% group_by(sub,cue,ISI) %>% summarize(dPrime = mean(dPrime))
`summarise()` has grouped output by 'sub', 'cue'. You can override using the
`.groups` argument.
dplot_ecc_sd <- dp %>% group_by(sub,targetLoc,ISI) %>% summarize(dPrime = mean(dPrime))
`summarise()` has grouped output by 'sub', 'targetLoc'. You can override using
the `.groups` argument.
dplot_ecc_sd
# A tibble: 180 × 4
# Groups:   sub, targetLoc [60]
   sub   targetLoc ISI   dPrime
   <chr> <fct>     <fct>  <dbl>
 1 s001  0         11     1.77 
 2 s001  0         23     2.52 
 3 s001  0         35     2.97 
 4 s001  3         11     1.27 
 5 s001  3         23     2.52 
 6 s001  3         35     3.25 
 7 s001  6         11     0.612
 8 s001  6         23     2.97 
 9 s001  6         35     3.54 
10 s001  9         11     1.98 
# ℹ 170 more rows
#Make sub-selection of the data- RT

dplot_cue_rt <- d %>% group_by(sub, cue) %>% summarize(mean_RT = mean(RT))
`summarise()` has grouped output by 'sub'. You can override using the `.groups`
argument.
dplot_ecc_rt <- d %>% group_by(cue, targetLoc) %>% summarize(mean_RT = mean(RT))
`summarise()` has grouped output by 'cue'. You can override using the `.groups`
argument.
dplot_ecc2_rt <- d %>% group_by(targetLoc, ISI) %>% summarize(mean_RT = mean(RT))
`summarise()` has grouped output by 'targetLoc'. You can override using the
`.groups` argument.
dplot_ISI_rt <- d %>% group_by(cue, ISI) %>% summarize(mean_RT = mean(RT))
`summarise()` has grouped output by 'cue'. You can override using the `.groups`
argument.
dplot_stat_ISI_rt <- d %>% group_by(sub, cue, ISI) %>% summarize(mean_RT = mean(RT))
`summarise()` has grouped output by 'sub', 'cue'. You can override using the
`.groups` argument.
dplot_ecc_sd_rt <- d %>% group_by(sub, targetLoc, ISI) %>% summarize(mean_RT = mean(RT))
`summarise()` has grouped output by 'sub', 'targetLoc'. You can override using
the `.groups` argument.

Results of control measures

There are no control measures planned for this study.

Confirmatory analysis

Following the original paper, and as it was done in the 1st replication attempt, a within-subject three-way analysis of variance (ANOVA; Cuing x Eccentricity x ISI) will be performed on the accuracy (d’) and RT data collected on trials with correct responses.

The key statistic of 3 way ANOVA anova was performed. The code below is taken by the first replication report.

###Accuracy

# within-subject 3-way Anova on Accuracy (dPrime), testing the interaction of cue,eccentricity,and ISI
Result_accuracy <- aov(dPrime ~ (cue*targetLoc*ISI) + Error(sub/cue*targetLoc*ISI), data=dp)

summary(Result_accuracy)

Error: sub
          Df Sum Sq Mean Sq F value Pr(>F)
Residuals  9  145.6   16.18               

Error: targetLoc
          Df Sum Sq Mean Sq
targetLoc  5  1.926  0.3853

Error: ISI
    Df Sum Sq Mean Sq
ISI  2  175.2    87.6

Error: sub:cue
          Df Sum Sq Mean Sq F value Pr(>F)
cue        1  1.534   1.534   1.308  0.282
Residuals  9 10.557   1.173               

Error: sub:targetLoc
          Df Sum Sq Mean Sq F value Pr(>F)
Residuals 45  23.69  0.5264               

Error: sub:ISI
          Df Sum Sq Mean Sq F value Pr(>F)
Residuals 18  14.59  0.8107               

Error: targetLoc:ISI
              Df Sum Sq Mean Sq
targetLoc:ISI 10   5.62   0.562

Error: sub:cue:targetLoc
              Df Sum Sq Mean Sq F value Pr(>F)
cue:targetLoc  5  2.118  0.4235   1.457  0.223
Residuals     45 13.080  0.2907               

Error: sub:cue:ISI
          Df Sum Sq Mean Sq F value  Pr(>F)   
cue:ISI    2  8.451   4.226    10.1 0.00115 **
Residuals 18  7.531   0.418                   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Error: sub:targetLoc:ISI
          Df Sum Sq Mean Sq F value Pr(>F)
Residuals 90  24.86  0.2762               

Error: sub:cue:targetLoc:ISI
                  Df Sum Sq Mean Sq F value Pr(>F)  
cue:targetLoc:ISI 10  3.953  0.3953    1.92 0.0524 .
Residuals         90 18.534  0.2059                 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
# main effect: Cuing
summary(Result_accuracy[5])

Error: sub:cue
          Df Sum Sq Mean Sq F value Pr(>F)
cue        1  1.534   1.534   1.308  0.282
Residuals  9 10.557   1.173               
# interaction effect: Cuing X ISI
summary(Result_accuracy[10])

Error: sub:cue:ISI
          Df Sum Sq Mean Sq F value  Pr(>F)   
cue:ISI    2  8.451   4.226    10.1 0.00115 **
Residuals 18  7.531   0.418                   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
### RT

# Within-subject 3-way ANOVA - Reaction Time
Result_rt <- aov(RT ~ (cue * targetLoc * ISI) + Error(sub / cue * targetLoc * ISI), data = d)

# Summary of RT ANOVA
summary(Result_rt)

Error: sub
          Df Sum Sq Mean Sq F value Pr(>F)
Residuals  9  665.2   73.91               

Error: targetLoc
          Df Sum Sq Mean Sq
targetLoc  5 0.9979  0.1996

Error: ISI
    Df Sum Sq Mean Sq
ISI  2  8.256   4.128

Error: sub:cue
          Df Sum Sq Mean Sq F value Pr(>F)
cue        1  1.423  1.4230   3.179  0.108
Residuals  9  4.029  0.4476               

Error: sub:targetLoc
          Df Sum Sq Mean Sq F value Pr(>F)
Residuals 45  11.47   0.255               

Error: sub:ISI
          Df Sum Sq Mean Sq F value Pr(>F)
Residuals 18  10.79  0.5995               

Error: targetLoc:ISI
              Df Sum Sq Mean Sq
targetLoc:ISI 10  1.945  0.1945

Error: sub:cue:targetLoc
              Df Sum Sq Mean Sq F value Pr(>F)
cue:targetLoc  5  0.812  0.1625   1.316  0.274
Residuals     45  5.555  0.1235               

Error: sub:cue:ISI
          Df Sum Sq Mean Sq F value Pr(>F)
cue:ISI    2 0.1317 0.06584    1.08   0.36
Residuals 18 1.0970 0.06094               

Error: sub:targetLoc:ISI
          Df Sum Sq Mean Sq F value Pr(>F)
Residuals 90  18.22  0.2025               

Error: sub:cue:targetLoc:ISI
                  Df Sum Sq Mean Sq F value Pr(>F)
cue:targetLoc:ISI 10  0.712 0.07116   0.763  0.663
Residuals         90  8.390 0.09322               

Error: Within
            Df Sum Sq Mean Sq F value Pr(>F)
Residuals 8280   1939  0.2341               
# Main effect: Cuing
summary(Result_rt[5])

Error: sub:cue
          Df Sum Sq Mean Sq F value Pr(>F)
cue        1  1.423  1.4230   3.179  0.108
Residuals  9  4.029  0.4476               
# Interaction effect: Cuing X ISI
summary(Result_rt[10])

Error: sub:cue:ISI
          Df Sum Sq Mean Sq F value Pr(>F)
cue:ISI    2 0.1317 0.06584    1.08   0.36
Residuals 18 1.0970 0.06094               
### Plotting

# Cueing effect
p1 <- ggplot(dplot_cue, aes(x=cue, y=dPrime))+
  geom_boxplot() +
  #geom_jitter((aes(colour = sub)), size=3, alpha=0.9) +
  scale_y_continuous(breaks = seq(0,3.75,0.5),limits = c(0,3.75))+
  ggtitle("Cueing Condition") +
  theme_classic()

#or a bar plot?
#p1 <- ggplot(dplot_cue, aes(x=cue, y=dPrime))+
 # geom_bar(stat = "identity") +
 # scale_y_continuous(breaks = seq(0,3.75,0.5),limits = c(0,3.75))+
 # ggtitle("Cueing Condition") +
 # theme_classic()


# Cueing X Ecc
p2 <- ggplot(dplot_ecc, aes(x=targetLoc, y=`dPrime`, group=cue)) +
  geom_line(aes(linetype=cue))+
  geom_point(aes(shape=cue))+
  scale_y_continuous(breaks = seq(0,4,0.5),limits = c(0,4))+
  scale_x_discrete(name="Eccentricity (deg)")+
  ggtitle("Eccentricity") +
  theme_classic()

# Cueing X ISI
p3 <- ggplot(dplot_ISI, aes(x=ISI, y=dPrime, group=cue)) +
  geom_line(aes(linetype=cue))+
  geom_point(aes(shape=cue))+
  scale_y_continuous(breaks = seq(0,4,0.5),limits = c(0,4))+
  scale_x_discrete(name="ISI (ms)")+
  theme_classic()

# ECC X ISI
p4 <- ggplot(dplot_ecc2, aes(x=targetLoc, y=`dPrime`, group=ISI)) +
  geom_line(aes(linetype=ISI))+
  geom_point(aes(shape=ISI))+
  scale_y_continuous(breaks = seq(0,4,0.5),limits = c(0,4))+
  scale_x_discrete(name="Eccentricity (deg)")+
  theme_classic()

plot_grid(p1, p2, p3, p4,ncol = 2, nrow = 2)

Three-panel graph with original, 1st replication, and your replication is ideal here

Exploratory analyses

Any follow-up analyses desired (not required).

Discussion

Mini meta analysis

Combining across the original paper, 1st replication, and 2nd replication, what is the aggregate effect size?

Summary of Replication Attempt

Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.

Commentary

Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.