For this exercise, please try to reproduce the results from Experiment 1 of the associated paper (Farooqui & Manly, 2015). The PDF of the paper is included in the same folder as this Rmd file.
Participants (N=21) completed a series of trials that required them to switch or stay from one task to the other. One task was to choose the larger value of the two values if surrounded by a green box. The other task was to choose the value with the larger font if surrounded by a blue box. Subliminal cues followed by a mask were presented before each trial. Cues included “O” (non-predictive cue), “M” (switch predictive cue), and “T” (repeat predictive cue). Reaction times and performance accuracy were measured.
Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 1):
Performance on switch trials, relative to repeat trials, incurred a switch cost that was evident in longer RTs (836 vs. 689 ms) and lower accuracy rates (79% vs. 92%). If participants were able to learn the predictive value of the cue that preceded only switch trials and could instantiate relevant anticipatory control in response to it, the performance on switch trials preceded by this cue would be better than on switch trials preceded by the nonpredictive cue. This was indeed the case (mean RT-predictive cue: 819 ms; nonpredictive cue: 871 ms; mean difference = 52 ms, 95% confidence interval, or CI = [19.5, 84.4]), two-tailed paired t(20) = 3.34, p < .01. However, error rates did not differ across these two groups of switch trials (predictive cue: 78.9%; nonpredictive cue: 78.8%), p = .8.
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
# #optional packages:
# library(broom)# This reads all the participants data (each is in a seperate xls file) in and combines them into one dataframe
# Each xls has 250 rows, the rest is their calculations using excel, which we don't want in the data
files <- dir('data/Experiment 1')
data <- data.frame()
id <- 1
for (file in files){
if(file != 'Codebook.xls'){
temp_data <- read_xls(file.path('data/Experiment 1', file))
temp_data$id <- id
id <- id + 1
temp_data <- temp_data[1:250, ]
data <- rbind(data, temp_data)
}
}Each row is an observation. The data is already in tidy format.
Performance on switch trials, relative to repeat trials, incurred a switch cost that was evident in longer RTs (836 vs. 689 ms)
# reproduce the above results here
data %>%
group_by(TrialType) %>%
summarize(meanRT = mean(RT, na.rm=T))## # A tibble: 3 × 2
## TrialType meanRT
## <dbl> <dbl>
## 1 0 2021.
## 2 1 731.
## 3 2 896.These results do not reproduce the exact values however I computed a mean RT of 896 ms for switch trials, compared to a RT of 731 for repeat trials. This does not change the finding, but the values are different.
Performance on switch trials, relative to repeat trials, incurred a switch cost that was evident in […] lower accuracy rates (79% vs. 92%)
data## # A tibble: 5,250 × 23
## Block_Number Event_Number Prime PrimeVisible TaskType TrialType CorrResp
## <dbl> <dbl> <chr> <dbl> <dbl> <dbl> <dbl>
## 1 3 1 O 1 2 0 1
## 2 3 2 T 1 2 1 1
## 3 3 3 M 1 1 2 1
## 4 3 4 O 1 1 1 0
## 5 3 5 T 1 1 1 1
## 6 3 6 T 1 1 1 1
## 7 3 7 M 1 2 2 1
## 8 3 8 T 1 2 1 1
## 9 3 9 O 1 2 1 0
## 10 3 10 T 1 2 1 0
## # … with 5,240 more rows, and 16 more variables: RT <dbl>, RespCorr <lgl>,
## # lnum <dbl>, rnum <dbl>, lFont <dbl>, swt <dbl>, stay <dbl>,
## # stay_2...15 <dbl>, stay_4...16 <dbl>, swt_2...17 <dbl>, swt_8...18 <dbl>,
## # swt_2...19 <dbl>, swt_8...20 <dbl>, stay_2...21 <dbl>, stay_4...22 <dbl>,
## # id <dbl># reproduce the above results here
data %>%
group_by(TrialType) %>%
summarize(Accuracy = mean(RespCorr, na.rm=T))## # A tibble: 3 × 2
## TrialType Accuracy
## <dbl> <dbl>
## 1 0 0.833
## 2 1 0.914
## 3 2 0.792Accuracy rates seem to be alot closer to values from the paper with 79.2% for switch trials and 91.4% for repeated trials.
Now you will analyze Predictive Switch Cues vs Non-predictive Switch Cues. Let’s start with reaction time.
This was indeed the case (mean RT-predictive cue: 819 ms; nonpredictive cue: 871 ms; … )
# reproduce the above results here
data %>%
filter(Prime=="O" | Prime=="M") %>%
mutate(Cue = factor(ifelse(Prime=="O", FALSE, TRUE), levels = c(FALSE, TRUE), labels = c("NonpredictiveSwitchCue", "PredictiveSwitchCue"))) %>%
group_by(Cue) %>%
summarize(meanRT = mean(RT, na.rm=T))## # A tibble: 2 × 2
## Cue meanRT
## <fct> <dbl>
## 1 NonpredictiveSwitchCue 812.
## 2 PredictiveSwitchCue 937.Reproducing the values of predictive switch cues (TRUE in table) and non-predictive cues (False in table) failed. Mean RT-predictive cue: 937 ms and non-predictive cue 812 ms, thus we get an opposite effect.
Next you will try to reproduce error rates for Switch Predictive Cues vs Switch Non-predictive Cues.
However, error rates did not differ across these two groups of switch trials (predictive cue: 78.9%; nonpredictive cue: 78.8%)
# reproduce the above results here
data %>%
filter(Prime=="O" | Prime=="M") %>%
mutate(Cue = factor(ifelse(Prime=="O", FALSE, TRUE), levels = c(FALSE, TRUE), labels = c("NonpredictiveSwitchCue", "PredictiveSwitchCue"))) %>%
group_by(Cue) %>%
summarize(Accuracy = mean(CorrResp, na.rm=T))## # A tibble: 2 × 2
## Cue Accuracy
## <fct> <dbl>
## 1 NonpredictiveSwitchCue 0.520
## 2 PredictiveSwitchCue 0.466Again this failed to reproduce, resulting in slightly better performance rate for the Non-predictive Switch Cues.
The first claim is that in switch trials, predictive cues lead to statistically significant faster reaction times than nonpredictive cues.
… the performance on switch trials preceded by this cue would be better than on switch trials preceded by the nonpredictive cue. This was indeed the case (mean RT-predictive cue: 819 ms; nonpredictive cue: 871 ms; mean difference = 52 ms, 95% confidence interval, or CI = [19.5, 84.4]), two-tailed paired t(20) = 3.34, p < .01.
data %>%
filter(Prime=="O" | Prime=="M") %>%
mutate(Cue = factor(ifelse(Prime=="O", FALSE, TRUE), levels = c(FALSE, TRUE), labels = c("NonpredictiveSwitchCue", "PredictiveSwitchCue"))) %>%
group_by(Cue) %>%
summarise(meanRT = mean(RT, na.rm = TRUE),
ssd = sd(RT, na.rm = TRUE),
count = n()) %>%
mutate(se = ssd / sqrt(count),
lower_ci_RT = meanRT - qt(1 - (0.05 / 2), count - 1) * se,
upper_ci_RT = meanRT + qt(1 - (0.05 / 2), count - 1) * se)## # A tibble: 2 × 7
## Cue meanRT ssd count se lower_ci_RT upper_ci_RT
## <fct> <dbl> <dbl> <int> <dbl> <dbl> <dbl>
## 1 NonpredictiveSwitchCue 812. 377. 1814 8.85 794. 829.
## 2 PredictiveSwitchCue 937. 465. 384 23.7 891. 984.# reproduce the above results hereNext, test the second claim.
However, error rates did not differ across these two groups of switch trials (predictive cue: 78.9%; nonpredictive cue: 78.8%), p = .8.
# reproduce the above results heres
Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?
I had a difficult time reproducing several parts. In fact I only reproduced one of the descriptive statistics for accuracy rates.
How difficult was it to reproduce your results?
There were uninterpretable values in the data, for instance for Prime values, numerical values 1 and others came up that were not part of the O, M, T set.
What aspects made it difficult? What aspects made it easy?
Difficult because I am not familiar with R but I think with more time I would have at least completed the assingment. I spent three hours on it and needed to move on to the other one in order to submit the assignment in time.