For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.

Methods summary:

Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.

Target outcomes:

Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjectS factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).

Step 1: Load packages

library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files

# #optional packages/functions:
library(afex) # anova functions
# library(ez) # anova functions 2
# library(scales) # for plotting
# std.err <- function(x) sd(x)/sqrt(length(x)) # standard error

Step 2: Load data

# Just Experiment 2
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)

Step 3: Tidy data

d_tidy <- d %>% 
  rename(rating = `Agreement (0=complete disagreement; 5=complete agreement)`) %>% 
  mutate(item = as.numeric(gsub("\\D", "", item))) %>% 
  mutate(group = case_when(group == "young Spaniard" ~ "Spanish", TRUE ~ group)) %>% 
  mutate(group = as.factor(group), subscale = as.factor(subscale))

Step 4: Run analysis

Pre-processing

d_summary <- d_tidy %>%
  group_by(group, subscale) %>%
  summarize(average_rating = mean(rating, na.rm = TRUE), std_dev = sd(rating, na.rm = TRUE),
  n_count = n(), SEM = std_dev / sqrt(n_count), .groups = 'drop') %>% 
  mutate(group = fct_relevel(group, "Spanish", "Moroccan"), subscale = fct_relevel(subscale, "PAST", "FUTURE"))

Descriptive statistics

Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):

ggplot(d_summary, aes(group, average_rating, fill = subscale)) +
  geom_col(position = "dodge") + 
   scale_fill_discrete(labels = c("PAST" = "Past-Focused Statements", "FUTURE" = "Future-Focused Statements")) +
  geom_errorbar(aes(ymin = average_rating - SEM, ymax = average_rating + SEM),
                position = position_dodge(width = 0.9), width = 0) +
  labs(y = "Rating") +
  coord_cartesian(ylim = c(2, 4)) +
  theme_classic() +
  theme(axis.title.x = element_blank(), legend.title = element_blank(), legend.position = "top") +
  guides(fill = guide_legend(nrow = 2, byrow = TRUE))

Inferential statistics

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).

# reproduce the above results here

d_tidy_anova <- d_tidy %>% 
  mutate(participant = as.character(participant)) %>% 
  mutate(participant = if_else(group == "Spanish", paste0(participant, "_S"), participant))

mixed_anova_results <- afex::aov_ez(id = "participant", dv = "rating", between = "group", within = "subscale", 
                                    data = d_tidy_anova)

print(mixed_anova_results)
## Anova Table (Type 3 tests)
## 
## Response: rating
##           Effect    df  MSE         F  ges p.value
## 1          group 1, 76 0.20      2.19 .008    .143
## 2       subscale 1, 76 0.50   7.98 ** .070    .006
## 3 group:subscale 1, 76 0.50 18.35 *** .147   <.001
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1

Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,

# reproduce the above results here

past <- d_tidy_anova %>% 
  filter(subscale == "PAST") %>% 
  group_by(group, participant) %>% 
  summarise(mean_rating = mean(rating))

t.test(mean_rating ~ group, alternative = c("greater"), var.equal = TRUE, data = past)
## 
##  Two Sample t-test
## 
## data:  mean_rating by group
## t = 3.8562, df = 76, p-value = 0.0001197
## alternative hypothesis: true difference in means between group Moroccan and group Spanish is greater than 0
## 95 percent confidence interval:
##  0.3350884       Inf
## sample estimates:
## mean in group Moroccan  mean in group Spanish 
##               3.280886               2.691142

and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)

# reproduce the above results here

future <- d_tidy_anova %>% 
  filter(subscale == "FUTURE") %>% 
  group_by(group, participant) %>% 
  summarise(mean_rating = mean(rating))

t.test(mean_rating ~ group, alternative = c("less"), var.equal = TRUE, data = future)
## 
##  Two Sample t-test
## 
## data:  mean_rating by group
## t = -3.2098, df = 78, p-value = 0.0009645
## alternative hypothesis: true difference in means between group Moroccan and group Spanish is less than 0
## 95 percent confidence interval:
##        -Inf -0.1710965
## sample estimates:
## mean in group Moroccan  mean in group Spanish 
##               3.138333               3.493750

Step 5: Reflection

Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?

I was unable to reproduce the F statistic and DFs for the ANOVA and t statistic and DFs for both t-tests.

How difficult was it to reproduce your results?

I did not reproduce my rests.

What aspects made it difficult? What aspects made it easy?

Difficult: the authors reused their participant IDs for participants in different conditions (e.g., condition A subjects had same IDs as condition B). Booo! Easy: data was pretty tidy already, but I had to clean up their column and condition labels (kinda tedius, but not hard)