PS#2: Diving into real data using Sklar et al. (2012)
Mike Frank
February 9, 2016
Intro
This is problem set #2, in which we hope you will practice the visualization package ggplot2, as well as hone your knowledge of the packages tidyr and dplyr.
Sklar et al. (2012) claims evidence for unconscious arithmetic processing. We’re going to do a reanalysis of their Experiment 6, which is the primary piece of evidence for that claim. The data are generously contributed by Asael Sklar.
First let’s set up a few preliminaries.
Data Prep
First read in two data files and subject info. A and B refer to different trial order counterbalances.
subinfo <- read_csv("http://langcog.stanford.edu/sklar_expt6_subinfo_corrected.csv")
d.a <- read_csv("http://langcog.stanford.edu/sklar_expt6a_corrected.csv")
d.b <- read_csv("http://langcog.stanford.edu/sklar_expt6b_corrected.csv")Gather these datasets into long form and get rid of the Xs in the headers.
d.a <-
d.a %>%
gather(key = "subject", value = "time", matches("[0-9]+"), convert = TRUE)
d.b <-
d.b %>%
gather(key = "subject", value = "time", matches("[0-9]+"), convert = TRUE) Bind these together. Check out bind_rows.
d_bound <- bind_rows(d.a, d.b)Merge these with subject info. You will need to look into merge and its relatives, left_ and right_join. Call this dataframe d, by convention.
d <-
d_bound %>%
select(subject, everything()) %>% #makes subject the first variable
left_join(subinfo, by = c("subject" = "subid"))Clean up the factor structure.
d$presentation.time <- factor(d$presentation.time)
levels(d$operand) <- c("addition","subtraction")Data Analysis Preliminaries
Examine the basic properties of the dataset. First, take a histogram.
d %>%
ggplot(aes(time)) +
geom_histogram(binwidth = 30) ## Warning: Removed 237 rows containing non-finite values (stat_bin).
The response times look pretty normally distributed.
summary(d)## subject prime prime.result target
## Min. : 1.0 Length:6468 Min. : 0.000 Min. : 0.000
## 1st Qu.:11.0 Class :character 1st Qu.: 1.000 1st Qu.: 1.000
## Median :21.5 Mode :character Median : 8.000 Median : 8.000
## Mean :21.5 Mean : 6.494 Mean : 6.494
## 3rd Qu.:32.0 3rd Qu.:11.000 3rd Qu.:11.000
## Max. :42.0 Max. :14.000 Max. :14.000
##
## congruent operand distance counterbalance
## Length:6468 Length:6468 Min. :-5 Min. :1.0
## Class :character Class :character 1st Qu.:-1 1st Qu.:1.0
## Mode :character Mode :character Median : 0 Median :1.5
## Mean : 0 Mean :1.5
## 3rd Qu.: 0 3rd Qu.:2.0
## Max. : 5 Max. :2.0
##
## time presentation.time subjective.test objective.test
## Min. : 101.0 1700:3080 Min. :0.0 Min. :0.3594
## 1st Qu.: 595.0 2000:3388 1st Qu.:0.0 1st Qu.:0.5238
## Median : 664.0 Median :0.5 Median :0.5992
## Mean : 667.7 Mean :0.5 Mean :0.6482
## 3rd Qu.: 736.0 3rd Qu.:1.0 3rd Qu.:0.7969
## Max. :2336.0 Max. :1.0 Max. :1.0000
## NA's :237From the summary and looking at the data, it looks like everything makes sense and there are no weird values.
d %>%
ggplot(aes(time)) +
geom_histogram(binwidth = 30) +
facet_wrap(congruent~operand)## Warning: Removed 237 rows containing non-finite values (stat_bin).
The distributions for each condition and operand combination look similar.
d %>%
ggplot(aes(time)) +
geom_histogram(binwidth = 30) +
facet_wrap(congruent~presentation.time)## Warning: Removed 237 rows containing non-finite values (stat_bin).
The distributions for each condition and presentation time combination look similar.
d %>%
count(congruent) ## # A tibble: 2 × 2
## congruent n
## <chr> <int>
## 1 no 3234
## 2 yes 3234Exactly half the subjects were in the “no” condition, and half were in the “yes” condition.
d %>%
group_by(congruent) %>%
summarise(mean_time = mean(time, na.rm = TRUE)) ## # A tibble: 2 × 2
## congruent mean_time
## <chr> <dbl>
## 1 no 669.5445
## 2 yes 665.8964The mean reaction time for the incongruent condition is longer than the mean time for the congruent condition, but not by much.
d %>%
group_by(congruent, presentation.time) %>%
summarise(mean_time = mean(time, na.rm = TRUE))## Source: local data frame [4 x 3]
## Groups: congruent [?]
##
## congruent presentation.time mean_time
## <chr> <fctr> <dbl>
## 1 no 1700 695.2922
## 2 no 2000 646.2774
## 3 yes 1700 696.9419
## 4 yes 2000 637.6906There are two presentation times–1700 ms and 2000 ms. The mean reaction times for the shorter presentation times are longer than the mean reaction times for the longer presentation times.
d %>%
unite(col = "pres_time_operand", presentation.time, operand) %>%
ggplot(aes(pres_time_operand, time)) +
geom_boxplot() +
facet_wrap(~congruent)## Warning: Removed 237 rows containing non-finite values (stat_boxplot).
Again, you can see that the reaction times for the 1700 ms presentation times are longer than those for the 2000 ms presention times, regardless of condition. The difference between the two conditions appears small. There’s not really a noticeable difference between operands in the incongruent condition. In the congruent condition, subtraction looks like it has slightly lower reaction times.
d %>%
group_by(prime.result) %>%
summarise(mean_time = mean(time, na.rm = TRUE)) %>%
ggplot(aes(prime.result, mean_time)) +
geom_col()
There doesn’t appear to be a relationship between the result of the prime and the reaction time.
d %>%
group_by(target) %>%
summarise(mean_time = mean(time, na.rm = TRUE)) %>%
ggplot(aes(target, mean_time)) +
geom_col()
There also doesn’t appear to be a relationship between the target and the reaction time.
Challenge question: what is the sample rate of the input device they are using to gather RTs?
d %>%
ggplot(aes(time)) +
geom_histogram(binwidth = 1)## Warning: Removed 237 rows containing non-finite values (stat_bin).
There are gaps of around 35.7 ms between all the measurements, so the sample rate of the device is probably around 36 ms/measurement.
Sklar et al. did two manipulation checks. Subjective - asking participants whether they saw the primes - and objective - asking them to report the parity of the primes (even or odd) to find out if they could actually read the primes when they tried. Examine both the unconscious and conscious manipulation checks. What do you see? Are they related to one another?
d %>%
ggplot(aes(subjective.test, objective.test, group = subjective.test)) +
geom_boxplot()
It looks like the median score on the objective test is higher for those who said they could see the primes.
OK, let’s turn back to the measure and implement Sklar et al.’s exclusion criterion. You need to have said you couldn’t see (subjective test) and also be not significantly above chance on the objective test (< .6 correct). Call your new data frame ds.
ds <-
d %>%
filter(subjective.test == 0,
objective.test < .6) Sklar et al.’s analysis
Sklar et al. show a plot of a “facilitation effect” - the amount faster you are for prime-congruent naming compared with prime-incongruent naming. They then show plot this difference score for the subtraction condition and for the two prime times they tested. Try to reproduce this analysis.
HINT: first take averages within subjects, then compute your error bars across participants, using the sem function (defined above).
ds_fe <-
ds %>%
group_by(operand, presentation.time, congruent, subject) %>%
summarise(mean_time = mean(time, na.rm = TRUE)) %>%
spread(congruent, mean_time) %>%
mutate(fe = no - yes) %>%
group_by(operand, presentation.time) %>%
summarise(mean_fe = mean(fe, na.rm = TRUE),
sem = sem(fe))Now plot this summary, giving more or less the bar plot that Sklar et al. gave (though I would keep operation as a variable here. Make sure you get some error bars on there (e.g. geom_errorbar or geom_linerange).
ds_fe %>%
ggplot(aes(presentation.time, mean_fe, fill = operand)) +
geom_col(position = "dodge") +
geom_errorbar(aes(ymin = mean_fe - sem, ymax = mean_fe + sem), width = .1,
position = position_dodge(width = .9))
What do you see here? How close is it to what Sklar et al. report? Do the error bars match? How do you interpret these data?
The Sklar et al. plot only shows the subtraction condition. The subtraction portion of my plot appears to match the Sklar et al. plot, disregarding the error bars. The error bars on the Sklar et al. plot are shorter than my error bars, even though they say that their error bars denote SEM. Maybe they calculated SEM with a portion of the standard deviation, misreported what their error bars represent, or miscalculated.
It also looks like there is a negative facilitation effect for addition in the 1700 ms presentation time condition.
Challenge problem: verify Sklar et al.’s claim about the relationship between RT and the objective manipulation check.
ds2 <-
ds %>%
group_by(presentation.time, congruent, subject, objective.test) %>%
summarise(mean_time = mean(time, na.rm = TRUE)) %>%
spread(congruent, mean_time) %>%
mutate(fe = no - yes)
cor(ds2$fe, ds2$objective.test)## [1] 0.1749418fit <- lm(fe~objective.test, data = ds2)
summary(fit)##
## Call:
## lm(formula = fe ~ objective.test, data = ds2)
##
## Residuals:
## Min 1Q Median 3Q Max
## -35.287 -3.778 -0.364 6.440 19.030
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -11.78 24.73 -0.476 0.641
## objective.test 33.47 48.64 0.688 0.502
##
## Residual standard error: 12.81 on 15 degrees of freedom
## Multiple R-squared: 0.0306, Adjusted R-squared: -0.03402
## F-statistic: 0.4736 on 1 and 15 DF, p-value: 0.5019There seems to be a weak positive correlation between facilitation effect and objective block scores, not a somewhat strong negative correlation as they say.
Your own analysis
Show us what you would do with these data, operating from first principles. What’s the fairest plot showing a test of Sklar et al.’s original hypothesis?
The exclusion criteria were a bit confusing. They tested awareness of the stimuli by making subjects repeatedly choose between 2 alternatives about the primes. This seems kind of strange, however, as the claim that the paper makes is that we do some type of processing on the numbers even if they don’t make it into conscious awareness. Couldn’t this also be happening with the objective block test? It also is unclear why being aware of the stimuli would interfere with the facilitation effect, and thus why these participants had to be excluded. The facilitation effects are now much smaller and, in the case of addition at 1700 ms, strongly negative. I think the following is a fairer plot:
d_fe <-
d %>%
group_by(operand, presentation.time, congruent, subject) %>%
summarise(mean_time = mean(time, na.rm = TRUE)) %>%
spread(congruent, mean_time) %>%
mutate(fe = no - yes) %>%
group_by(operand, presentation.time) %>%
summarise(mean_fe = mean(fe, na.rm = TRUE),
sem = sem(fe))
d_fe %>%
ggplot(aes(presentation.time, mean_fe, fill = operand)) +
geom_col(position = "dodge") +
geom_errorbar(aes(ymin = mean_fe - sem, ymax = mean_fe + sem), width = .1,
position = position_dodge(width = .9))
I also find the choice to only look at the operands separately, the choice to only plot subtraction, and the explanations about why it makes sense for there to be less of a facilitation effect/ a negative facilitation effect in the addition condition somewhat questionable. If you look at the facilitation effects for subtraction and addition combined, you get the following plot:
ds_all <-
ds %>%
group_by(presentation.time, congruent, subject) %>%
summarise(mean_time = mean(time, na.rm = TRUE)) %>%
spread(congruent, mean_time) %>%
mutate(fe = no - yes) %>%
group_by(presentation.time) %>%
summarise(mean_fe = mean(fe, na.rm = TRUE),
sem = sem(fe))
ds_all %>%
ggplot(aes(presentation.time, mean_fe)) +
geom_col() +
geom_errorbar(aes(ymin = mean_fe - sem, ymax = mean_fe + sem), width = .1,
position = position_dodge(width = .9))
You can see that the facilitation effects are much smaller. And the error bars on 1700 ms covers 0.
If you include all participants, you get the following:
d_all <-
d %>%
group_by(presentation.time, congruent, subject) %>%
summarise(mean_time = mean(time, na.rm = TRUE)) %>%
spread(congruent, mean_time) %>%
mutate(fe = no - yes) %>%
group_by(presentation.time) %>%
summarise(mean_fe = mean(fe, na.rm = TRUE),
sem = sem(fe))
d_all %>%
ggplot(aes(presentation.time, mean_fe)) +
geom_col() +
geom_errorbar(aes(ymin = mean_fe - sem, ymax = mean_fe + sem), width = .1,
position = position_dodge(width = .9))
The facilitation effect for 1700 ms is negative, and again, the error bar covers 0.
Challenge problem: Do you find any statistical support for Sklar et al.’s findings?