Reproducibility Report: Group B Choice 3
For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.
Methods summary:
Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.
Target outcomes:
Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):
According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjectS factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).
Step 1: Load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
# #optional packages/functions:
library(afex) # anova functions
# library(ez) # anova functions 2
# library(scales) # for plotting
# std.err <- function(x) sd(x)/sqrt(length(x)) # standard errorStep 2: Load data
# Just Experiment 2
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)Step 3: Tidy data
# data is already tidy in that each row is one observation and one participant
# rename agreement variable & remove unnecessary columns
d <- d %>%
dplyr::rename(
agreement = "Agreement (0=complete disagreement; 5=complete agreement)") %>%
dplyr::select(-item)
head(d)## # A tibble: 6 × 4
## group participant subscale agreement
## <chr> <dbl> <chr> <dbl>
## 1 Moroccan 1 PAST 4
## 2 Moroccan 1 PAST 4
## 3 Moroccan 1 PAST 5
## 4 Moroccan 1 PAST 2
## 5 Moroccan 1 PAST 4
## 6 Moroccan 1 PAST 3Step 4: Run analysis
Pre-processing
# rename group from 'young Spaniard' to 'Spaniard'
d <- d %>%
mutate(group = ifelse(group == "young Spaniard", "Spaniard", group))Descriptive statistics
Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):
# calculate mean and standard error, and arrange subscale and group factors
plot_descriptives <- d %>%
group_by(group, subscale) %>%
summarise(
mean = mean(agreement, na.rm = TRUE),
sd = sd(agreement, na.rm = TRUE),
se = sd / sqrt(n())
) %>%
ungroup() %>%
dplyr::mutate(subscale = factor(subscale,
levels = c('PAST','FUTURE'),
labels = c('PAST','FUTURE')),
group = factor(group,
levels = c('Spaniard','Moroccan'),
labels = c('Spaniard','Moroccan')))
# reproduce the plot
ggplot(plot_descriptives, aes(x = group, y = mean, fill = subscale)) +
geom_bar(stat = "identity", position = position_dodge(width = 0.7), width = 0.6) +
geom_errorbar(aes(ymin = mean-se, ymax = mean+se), width = 0.1, position = position_dodge(0.7)) +
scale_y_continuous(limits = c(2, 4), oob = scales::squish) +
labs(x = "", y = "Rating") +
scale_fill_grey(start = 0.8, end = 0.5) +
theme_minimal() +
scale_fill_manual(values = c("PAST" = "darkgrey", "FUTURE" = "lightgrey")) +
theme(legend.title = element_blank(),
legend.position = "top",
axis.text.x = element_text(angle = 45, hjust = 1),
plot.title = element_text(hjust = 0.5)) +
ggtitle("Figure 2. Results of Experiment 2") 
Inferential statistics
According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).
# reproduce the above results here
d$unique_id <- if_else(d$group == "Moroccan",
d$participant, d$participant + 50)
# Run a mixed ANOVA
anova_results <- aov_ez("unique_id", "agreement", d, between = "group", within = "subscale", detailed = TRUE, fun_aggregate = mean) #there are 2 participants who only had data for future but not past
# Get the summary of the ANOVA
summary(anova_results)##
## Univariate Type III Repeated-Measures ANOVA Assuming Sphericity
##
## Sum Sq num Df Error SS den Df F value Pr(>F)
## (Intercept) 1543.46 1 15.250 76 7692.0904 < 2.2e-16 ***
## group 0.44 1 15.250 76 2.1920 0.14287
## subscale 3.97 1 37.777 76 7.9793 0.00604 **
## group:subscale 9.12 1 37.777 76 18.3456 5.328e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,
# reproduce the above results here
mean_agreement <- d %>%
group_by(group, subscale, unique_id) %>%
summarize(mean_agreement = mean(agreement, na.rm = TRUE), .groups = 'drop')
t_test_results <- t.test(mean_agreement ~ group, data = subset(mean_agreement, subscale == "PAST"))
# Output the results
print(t_test_results)##
## Welch Two Sample t-test
##
## data: mean_agreement by group
## t = 3.8562, df = 74.91, p-value = 0.0002416
## alternative hypothesis: true difference in means between group Moroccan and group Spaniard is not equal to 0
## 95 percent confidence interval:
## 0.2850812 0.8944060
## sample estimates:
## mean in group Moroccan mean in group Spaniard
## 3.280886 2.691142and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)
# reproduce the above results here
t_test_results <- t.test(mean_agreement ~ group, data = subset(mean_agreement, subscale == "FUTURE"))
# Output the results
print(t_test_results)##
## Welch Two Sample t-test
##
## data: mean_agreement by group
## t = -3.2098, df = 70.047, p-value = 0.002005
## alternative hypothesis: true difference in means between group Moroccan and group Spaniard is not equal to 0
## 95 percent confidence interval:
## -0.5762537 -0.1345797
## sample estimates:
## mean in group Moroccan mean in group Spaniard
## 3.138333 3.493750Step 5: Reflection
Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?
No…. The statistical values for the inferential statistics don’t seem to match for he
How difficult was it to reproduce your results?
I didn’t manage to…
What aspects made it difficult? What aspects made it easy?
I wasn’t sure if I used a wrong package, or misunderstood the statistical test…