Reanalysis of Müller & Rothermund

library(BayesFactor)
options(BFprogress=FALSE) # Turn off progress bars, since we're compiling in HTML.
library(xtable)
library(reshape2)
library(dplyr)

First, we read in the data.

Now we read in the subject information.

Filter out invalid participants, with missing data.

missing.any = c(missing.sem, missing.res, missing.info)
all.data = all.data %>% filter( !(ID %in% missing.any) )
kept.subs = !(info.data$ID %in% missing.any)

The number of participants matches the replication report, which states N=294:

length(unique(all.data$ID))

## [1] 294

The average, standard deviation, and range of age matches replication report:

mean(info.data$Alter[kept.subs])

## [1] 23.84014

sd(info.data$Alter[kept.subs])

## [1] 5.857082

range(info.data$Alter[kept.subs])

## [1] 18 55

The proportion women matches the replication report:

table(info.data$Geschlecht[kept.subs])/sum(kept.subs)

## 
## maennlich  weiblich 
## 0.4897959 0.5102041

Next, we clean the data according to the replication report.

cond.data = all.data %>%
  filter( RequiredResponse != "town" | primetype == "response")

acc.data = cond.data %>%
  filter( Accuracy == 1 )

filtered.data = acc.data %>%
  group_by(ID) %>% 
  filter( RT < (quantile(RT, p=.75) + 1.5*IQR(RT)) )

The proportion of valid condition removed for inaccuracy matches replication report of 4.8%:

(nrow(cond.data) - nrow(acc.data)) / nrow(cond.data)

## [1] 0.04824995

There is a mismatch with the replication report, which reports that 4.3% of trials were were moved, but it isn’t clear 4.3% of what. I contacted Florian Müller, and he confirmed that the outlier filter was performed after the accuracy filter. Here I calculate the number of trials removed as outliers, as a proportion of:

Total trials
Trials after removing irrelevent conditions
Trials left after removing incorrect trials

## Trials removed for being outliers
nOutliers = nrow(acc.data) - nrow(filtered.data)

## Where did the 4.3% outliers come from?

## Not as a percentage of all data
nOutliers / nrow(all.data)

## [1] 0.03201119

## Not as a percentage of the condition filtered data
nOutliers / nrow(cond.data)

## [1] 0.04801679

## Not as a percentage of the accuracy filtered data
nOutliers / nrow(acc.data)

## [1] 0.05045105

None of these matches the 4.3% listed in the replication report.

Reproduce Figure 2

agg.data = aggregate(RT ~ ID + PrimeGender + TargetGender + primetype, data = filtered.data, mean)

These means do not seem to match the replication report, Figure 2.

mns = with(agg.data,tapply(RT, list(PrimeGender,TargetGender,primetype), mean))[,-3,]
stderrs = with(agg.data,tapply(RT, list(PrimeGender,TargetGender,primetype), function(v) sd(v)/sqrt(length(v))))[,-3,]

par(mfrow = c(1,2))
barplot(t(mns[,,2]), beside = TRUE, ylim = c(540,620), xpd=FALSE, xlab="Prime Gender", ylab="Reaction Time",main="Response priming",col=c("green","blue"))
abline(h=seq(550,630,20), col="grey",lty=3)
legend(1,620,legend=c("Female target", "Male target"),pt.bg=c("green","blue"), pch=22,col="black")

barplot(t(mns[,,1]), beside = TRUE, ylim = c(540,620), xpd=FALSE, xlab="Prime Gender", ylab="Reaction Time",main="Semantic priming",col=c("green","blue"))
abline(h=seq(550,630,20), col="grey",lty=3)
legend(1,620,legend=c("Female target", "Male target"),pt.bg=c("green","blue"), pch=22,col="black")

ANOVA

Does not match replication report (may be down to the outliers).

aov.obj = aov(RT ~ PrimeGender*TargetGender*primetype + Error(ID/(PrimeGender*TargetGender*primetype)), data = agg.data)

summary(aov.obj)

## 
## Error: ID
##            Df   Sum Sq Mean Sq F value Pr(>F)
## Residuals 293 13242715   45197               
## 
## Error: ID:PrimeGender
##              Df Sum Sq Mean Sq F value Pr(>F)  
## PrimeGender   1    904   904.1   3.511  0.062 .
## Residuals   293  75456   257.5                 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error: ID:TargetGender
##               Df Sum Sq Mean Sq F value Pr(>F)    
## TargetGender   1 129909  129909   157.4 <2e-16 ***
## Residuals    293 241875     826                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error: ID:primetype
##            Df Sum Sq Mean Sq F value Pr(>F)    
## primetype   1 250636  250636   90.24 <2e-16 ***
## Residuals 293 813809    2778                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error: ID:PrimeGender:TargetGender
##                           Df Sum Sq Mean Sq F value   Pr(>F)    
## PrimeGender:TargetGender   1  22938   22938   57.53 4.45e-13 ***
## Residuals                293 116830     399                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error: ID:PrimeGender:primetype
##                        Df Sum Sq Mean Sq F value Pr(>F)  
## PrimeGender:primetype   1    947   947.4   3.713 0.0549 .
## Residuals             293  74752   255.1                 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error: ID:TargetGender:primetype
##                         Df Sum Sq Mean Sq F value Pr(>F)    
## TargetGender:primetype   1  74327   74327   98.57 <2e-16 ***
## Residuals              293 220944     754                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error: ID:PrimeGender:TargetGender:primetype
##                                     Df Sum Sq Mean Sq F value   Pr(>F)    
## PrimeGender:TargetGender:primetype   1  11622   11622   35.42 7.56e-09 ***
## Residuals                          293  96126     328                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Reanalysis of Müller & Rothermund

Richard D. Morey

26 February 2015

Reproduce Figure 2

ANOVA