Reproducibility Report: Group B Choice 3
For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.
Methods summary:
Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.
Target outcomes:
Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):
According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjectS factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).
Step 1: Load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
#install.packages("expss")
library(expss)
#install.packages(effectsize)
library(effectsize)
library(rstatix)
library(car)
#install.packages("psych")
library(psych)
# #optional packages/functions:
# library(afex) # anova functions
library(ez) # anova functions 2
library(ggplot2)
# library(scales) # for plotting
std.err <- function(x) sd(x)/sqrt(length(x)) # standard errorStep 2: Load data
# Just Experiment 2
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)Step 3: Tidy data
d <- d %>%
mutate( group = ifelse(grepl("young", group), "Spaniard", "Morrocan"),
id = ifelse(group == "Morrocan", participant, participant + 40)) %>%
select(-c(participant)) %>%
rename( condition = subscale,
agreement = `Agreement (0=complete disagreement; 5=complete agreement)`) %>%
apply_labels (agreement = "0=complete disagreement; 5=complete agreement")Step 4: Run analysis
Pre-processing
d_plot <- d %>%
group_by(group, condition) %>%
summarise( mean_agree = mean(agreement),
se = std.err(agreement))
View(d_plot)Descriptive statistics
Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):
bar_plot <- d_plot %>%
ggplot(mapping = aes( x = group, y = mean_agree, fill = condition,
ymin = mean_agree-se,ymax = mean_agree+se)) +
geom_bar(position="dodge", stat = "identity") +
scale_fill_brewer(palette="Set1") +
geom_errorbar(position = position_dodge(0.9), width = 0.3) +
coord_cartesian(ylim = c(2, 4))
bar_plot
Inferential statistics
According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).
# mixed ANOVA
mixed_anova <- aov(agreement ~ group * condition + Error(id/condition), data = d)
# Summarize the ANOVA results
summary(mixed_anova)##
## Error: id
## Df Sum Sq Mean Sq
## group 1 15.84 15.84
##
## Error: id:condition
## Df Sum Sq Mean Sq
## group 1 96.29 96.29
##
## Error: Within
## Df Sum Sq Mean Sq F value Pr(>F)
## group 1 0.7 0.67 0.384 0.53580
## condition 1 15.2 15.22 8.697 0.00323 **
## group:condition 1 35.6 35.63 20.359 6.86e-06 ***
## Residuals 1674 2929.6 1.75
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1#eta_squared(mixed_anova)
# Extracting the eta squared did not work. I am not sure whether this mixed anova is correct,
# but i exceeded the 3 hours trying to make it work. ezANOVA also did not work, unfortunately.
# Anyhow, the ANOVA did not seem to reproduce perfectlyMoroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,
# Not sure if they grouped or just did raw means by item
t.test(agreement ~ group, subset = condition == "PAST", data = d)##
## Welch Two Sample t-test
##
## data: agreement by group
## t = 6.8228, df = 848.31, p-value = 1.698e-11
## alternative hypothesis: true difference in means between group Morrocan and group Spaniard is not equal to 0
## 95 percent confidence interval:
## 0.4403454 0.7960182
## sample estimates:
## mean in group Morrocan mean in group Spaniard
## 3.293182 2.675000## OR
t_test_data <- d %>%
group_by(id, condition, group) %>%
summarise( agreement = mean(agreement))
t.test(agreement ~ group, subset = condition == "PAST", data = t_test_data)##
## Welch Two Sample t-test
##
## data: agreement by group
## t = 3.8562, df = 74.91, p-value = 0.0002416
## alternative hypothesis: true difference in means between group Morrocan and group Spaniard is not equal to 0
## 95 percent confidence interval:
## 0.2850812 0.8944060
## sample estimates:
## mean in group Morrocan mean in group Spaniard
## 3.280886 2.691142# Again, not perfect reproducibilityand Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)
# Not sure if they grouped or just did raw means by item
t.test(agreement ~ group, subset = condition == "FUTURE", data = d)##
## Welch Two Sample t-test
##
## data: agreement by group
## t = -4.0469, df = 780.2, p-value = 5.708e-05
## alternative hypothesis: true difference in means between group Morrocan and group Spaniard is not equal to 0
## 95 percent confidence interval:
## -0.5531879 -0.1918121
## sample estimates:
## mean in group Morrocan mean in group Spaniard
## 3.1200 3.4925## OR
t_test2_data <- d %>%
group_by(id, condition, group) %>%
summarise( agreement = mean(agreement))
t.test(agreement ~ group, subset = condition == "FUTURE", data = t_test2_data)##
## Welch Two Sample t-test
##
## data: agreement by group
## t = -3.2098, df = 70.047, p-value = 0.002005
## alternative hypothesis: true difference in means between group Morrocan and group Spaniard is not equal to 0
## 95 percent confidence interval:
## -0.5762537 -0.1345797
## sample estimates:
## mean in group Morrocan mean in group Spaniard
## 3.138333 3.493750# Again, not perfect reproducibilityStep 5: Reflection
Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?
It was not perfect reproducibility. Due to the nature of the data, some parts were unclear of how they went about performing specific inferential tests and I was unsure of how to structure the data for a certain test or model. The mixed ANOVA and the t-tests had problems with.
How difficult was it to reproduce your results?
6 out of 10. The data was there, easy to deal with. But lack of specificit in procedures.
What aspects made it difficult? What aspects made it easy?
See above. Data structure was easy to deal with, but lack of specificity in tests was a barrier.