Reproducibility Report: Group B Choice 3
Submitted by: Radhika Kapoor
Group work with: Madi Bunderson, Catie Connolly, Jamie Mitchell
For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.
Methods summary:
Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.
Target outcomes:
Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):
According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjectS factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).
Step 1: Load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
library(afex) # anova functions
# #optional packages/functions:
# library(ez) # anova functions 2
#library(scales) # for plotting
#std.err <- function(x) sd(x)/sqrt(length(x)) # standard errorStep 2: Load data
# Just Experiment 2
data_path <- '~/Documents/Stanford readings/251 Experimental methods/Problem sets/problem_sets/ps3/Group B/Choice 3/data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)
view(d)
summary(d)## group participant subscale item
## Length:1680 Min. : 1.00 Length:1680 Length:1680
## Class :character 1st Qu.:10.75 Class :character Class :character
## Mode :character Median :20.50 Mode :character Mode :character
## Mean :20.88
## 3rd Qu.:31.25
## Max. :40.00
## Agreement (0=complete disagreement; 5=complete agreement)
## Min. :1.000
## 1st Qu.:2.000
## Median :3.000
## Mean :3.138
## 3rd Qu.:4.000
## Max. :5.000colnames(d)## [1] "group"
## [2] "participant"
## [3] "subscale"
## [4] "item"
## [5] "Agreement (0=complete disagreement; 5=complete agreement)"unique(d$Agreement)## NULLStep 3: Tidy data
#The data is tidy - each observation is in a row, and values are in columns
Step 4: Run analysis
Pre-processing
summary(d)## group participant subscale item
## Length:1680 Min. : 1.00 Length:1680 Length:1680
## Class :character 1st Qu.:10.75 Class :character Class :character
## Mode :character Median :20.50 Mode :character Mode :character
## Mean :20.88
## 3rd Qu.:31.25
## Max. :40.00
## Agreement (0=complete disagreement; 5=complete agreement)
## Min. :1.000
## 1st Qu.:2.000
## Median :3.000
## Mean :3.138
## 3rd Qu.:4.000
## Max. :5.000#identify duplicates
duplicates <- d %>%
arrange(group, participant, subscale, item) %>%
group_by(group, participant, subscale, item) %>%
filter(n()>1)
#d add id column and rename group
d <- d %>%
mutate(id=ifelse(group=="Moroccan",participant,participant+40),
group=ifelse(group=="Moroccan","Moroccan","Spaniards"))
#drop duplicated rows
d_drop <- d %>%
arrange(group, participant,item) %>%
rename(rating= "Agreement (0=complete disagreement; 5=complete agreement)") %>%
distinct(id, group, participant, subscale, item, .keep_all=T)
#average ratings for duplicate rows
##wrote this code but didnt use it, used the version above
d_average <- d %>%
arrange(group, participant,item) %>%
rename(rating= "Agreement (0=complete disagreement; 5=complete agreement)") %>%
group_by(id, group, participant, subscale, item) %>%
summarise(rating=mean(rating)) #mean for all combinations of participant and itemDescriptive statistics
Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):
datasummary <- d_drop %>%
group_by(group, subscale) %>%
summarise(meanRating = mean(rating),
n= n(),
seRating=sd(rating, na.rm=T)/sqrt(n))
datasummary$group <- factor(datasummary$group, levels=c("Spaniards", "Moroccan"))
view(datasummary)
ggplot(datasummary, aes(x=group, y=meanRating, fill=subscale)) +
geom_bar(position="dodge", stat="identity") +
geom_errorbar(aes(ymin=meanRating-seRating,ymax=meanRating+seRating),width=0.2,position=position_dodge(.9), stat="identity") +
scale_fill_brewer(palette="Set1") +
coord_cartesian(ylim=c(2,4)) 
Inferential statistics
According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).
# reproduce the above results here
d3 <- d_drop %>%
group_by(id, group, participant, subscale) %>%
summarise(rating=mean(rating)) #mean rating by participant for past/future subscale
d3$id <- factor(d3$id)
Within.aov.1 <- aov_car(rating ~ group*subscale + Error(id/subscale), data=d3, anova_table=list(es="pes"), na.rm = TRUE)
summary(Within.aov.1)##
## Univariate Type III Repeated-Measures ANOVA Assuming Sphericity
##
## Sum Sq num Df Error SS den Df F value Pr(>F)
## (Intercept) 1536.97 1 15.388 76 7591.0606 < 2.2e-16 ***
## group 0.35 1 15.388 76 1.7195 0.1937007
## subscale 3.76 1 37.591 76 7.5937 0.0073282 **
## group:subscale 8.26 1 37.591 76 16.7057 0.0001073 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1knitr::kable(nice(Within.aov.1))| Effect | df | MSE | F | pes | p.value |
|---|---|---|---|---|---|
| group | 1, 76 | 0.20 | 1.72 | .022 | .194 |
| subscale | 1, 76 | 0.49 | 7.59 ** | .091 | .007 |
| group:subscale | 1, 76 | 0.49 | 16.71 *** | .180 | <.001 |
Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,
# reproduce the above results here
d4 <- d3 %>%
filter(subscale=="PAST")
t.test(rating ~ group, data = d4)##
## Welch Two Sample t-test
##
## data: rating by group
## t = 3.6595, df = 74.738, p-value = 0.0004684
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## 0.2527584 0.8567987
## sample estimates:
## mean in group Moroccan mean in group Spaniards
## 3.261072 2.706294and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)
# reproduce the above results here
d4 <- d3 %>%
filter(subscale=="FUTURE")
t.test(rating ~ group, data = d4)##
## Welch Two Sample t-test
##
## data: rating by group
## t = -3.0635, df = 71.23, p-value = 0.003088
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -0.5681658 -0.1201675
## sample estimates:
## mean in group Moroccan mean in group Spaniards
## 3.133333 3.477500Step 5: Reflection
Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?
I could replicate the figures, but the results are a little different for ANOVA and t-test
How difficult was it to reproduce your results?
It was pretty difficult to replicate the inferential statistics
What aspects made it difficult? What aspects made it easy?
- Directions on using ANOVA were not clear (2) Not clear how they handled the duplicate participants (3) The degree of freedom for t-tests I have are not the same as the paper, I am not sure why