Reproducibility Report: Group B Choice 3
For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.
Methods summary:
Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.
Target outcomes:
Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):
According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjectS factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).
Step 1: Load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
# #optional packages/functions:
# library(afex) # anova functions
# library(ez) # anova functions 2
# library(scales) # for plotting
# std.err <- function(x) sd(x)/sqrt(length(x)) # standard errorStep 2: Load data
# Just Experiment 2
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)Step 3: Tidy data
head(d)## # A tibble: 6 × 5
## group participant subscale item Agreement (0=complet…¹
## <chr> <dbl> <chr> <chr> <dbl>
## 1 Moroccan 1 PAST 1. Para mí son muy impor… 4
## 2 Moroccan 1 PAST 2. Los jóvenes deben con… 4
## 3 Moroccan 1 PAST 3. Creo que las personas… 5
## 4 Moroccan 1 PAST 4. La juventud de hoy en… 2
## 5 Moroccan 1 PAST 5. Los ancianos saben má… 4
## 6 Moroccan 1 PAST 6. El modo correcto de h… 3
## # ℹ abbreviated name:
## # ¹`Agreement (0=complete disagreement; 5=complete agreement)`colnames(d)## [1] "group"
## [2] "participant"
## [3] "subscale"
## [4] "item"
## [5] "Agreement (0=complete disagreement; 5=complete agreement)"colnames(d) <- c("group", "participant", "subscale", "item", "agreement")
head(d)## # A tibble: 6 × 5
## group participant subscale item agreement
## <chr> <dbl> <chr> <chr> <dbl>
## 1 Moroccan 1 PAST 1. Para mí son muy importantes las tr… 4
## 2 Moroccan 1 PAST 2. Los jóvenes deben conservar las tr… 4
## 3 Moroccan 1 PAST 3. Creo que las personas eran más fel… 5
## 4 Moroccan 1 PAST 4. La juventud de hoy en día necesita… 2
## 5 Moroccan 1 PAST 5. Los ancianos saben más que los jóv… 4
## 6 Moroccan 1 PAST 6. El modo correcto de hacer las cosa… 3Step 4: Run analysis
Pre-processing
#no pre-processing requiredDescriptive statistics
Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):
DescriptiveStats <- d %>%
group_by(group, subscale) %>%
summarize(mean_agreement = mean(agreement, na.rm = TRUE),
sd_agreement = sd(agreement, na.rm = TRUE),
se_agreement = sd_agreement / sqrt(n())) %>%
ungroup()
ggplot(DescriptiveStats, aes(x = group, y = mean_agreement, fill = subscale)) +
geom_bar(stat = "identity", position = position_dodge(), width = 0.7) +
geom_errorbar(aes(ymin = mean_agreement - se_agreement, ymax = mean_agreement + se_agreement),
position = position_dodge(0.7), width = 0.25) +
scale_fill_grey(start = 0.8, end = 0.2, labels = c("Past-Focused Statements", "Future-Focused Statements")) +
labs(x = "", y = "Rating") +
theme_minimal() +
theme(legend.position = "bottom",
legend.title = element_blank()) +
scale_x_discrete(labels=c("Spaniards", "Moroccans")) +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
scale_y_continuous(breaks = seq(2, 4, by = 0.25)) +
coord_cartesian(ylim = c(2, 4))
Inferential statistics
According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).
AnovaResults <- aov(agreement ~ group*subscale + Error(participant/subscale), data = d)
summary(AnovaResults)##
## Error: participant
## Df Sum Sq Mean Sq
## subscale 1 7.499 7.499
##
## Error: participant:subscale
## Df Sum Sq Mean Sq
## subscale 1 27.87 27.87
##
## Error: Within
## Df Sum Sq Mean Sq F value Pr(>F)
## group 1 9.0 9.01 5.146 0.0234 *
## subscale 1 16.5 16.45 9.401 0.0022 **
## group:subscale 1 102.8 102.82 58.752 3.01e-14 ***
## Residuals 1674 2929.6 1.75
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,
PastData <- d %>% filter(subscale == "PAST")
TtestResult <- t.test(agreement ~ group, data = PastData)
print(TtestResult)##
## Welch Two Sample t-test
##
## data: agreement by group
## t = 6.8228, df = 848.31, p-value = 1.698e-11
## alternative hypothesis: true difference in means between group Moroccan and group young Spaniard is not equal to 0
## 95 percent confidence interval:
## 0.4403454 0.7960182
## sample estimates:
## mean in group Moroccan mean in group young Spaniard
## 3.293182 2.675000and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)
FutureData <- d %>% filter(subscale == "FUTURE")
TtestFuture <- t.test(agreement ~ group, data = FutureData)
print(TtestFuture)##
## Welch Two Sample t-test
##
## data: agreement by group
## t = -4.0469, df = 780.2, p-value = 5.708e-05
## alternative hypothesis: true difference in means between group Moroccan and group young Spaniard is not equal to 0
## 95 percent confidence interval:
## -0.5531879 -0.1918121
## sample estimates:
## mean in group Moroccan mean in group young Spaniard
## 3.1200 3.4925Step 5: Reflection
Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?
Not exactly. Direction of effect and significance was the same as original study but values are higher than reported in the original.
How difficult was it to reproduce your results?
I was not able to successfully reproduce the results so it was difficult. I tried to troubleshoot for a little more than 3 hours.
What aspects made it difficult? What aspects made it easy?
I think there were data handling discrepancies between what I did and what the original study did, which is why my values are higher. For ex, I didn’t do any preprocessing because I don’t think it’s needed for this specific analysis. I tried different things for preprocessing but it did not change the results, so I think I’m missing something big in the data that I can’t see.