For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.

Methods summary:

Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.

Target outcomes:

Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjectS factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).

Step 1: Load packages

library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
library(janitor)
# #optional packages/functions:
library(afex) # anova functions
# library(ez) # anova functions 2
# library(scales) # for plotting
# std.err <- function(x) sd(x)/sqrt(length(x)) # standard error

Step 2: Load data

# Just Experiment 2
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)

Step 3: Tidy data

tidy_d <- d %>% 
  janitor::clean_names() %>% 
  mutate(
    agreement_score = agreement_0_complete_disagreement_5_complete_agreement,
    group = case_when(
      group == "young Spaniard" ~ "Spanish", 
      TRUE ~ group
    )
  ) %>% 
  select(-agreement_0_complete_disagreement_5_complete_agreement)

Step 4: Run analysis

Pre-processing

m_d <- tidy_d %>%
  filter(group == "Moroccan") %>% 
  mutate(
    new_participant_id = participant
  )

s_d <- tidy_d %>% 
  filter(group == "Spanish") %>% 
  mutate(
    new_participant_id = participant + nrow(m_d %>% distinct(participant))
  )

tidy_data <- bind_rows(m_d, s_d) %>% 
  select(-participant) %>% 
  mutate(
    participant = new_participant_id
  ) %>% 
  select(-new_participant_id)


summarize_df <- tidy_data %>% 
  group_by(subscale, group) %>% 
  summarize(
    mean = mean(agreement_score), 
    sd = sd(agreement_score),
    n = n(), 
    ci_range_95 = 1.96 * (sd / sqrt(n)), 
    ci_lower = mean - ci_range_95, 
    ci_upper = mean + ci_range_95
  )

Descriptive statistics

Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):

summarize_df %>% 
  mutate (
    group_print = case_when(
      group == "Spanish" ~ "Spaniards", 
      group == "Moroccan" ~ "Moroccans"
    ), 
    subscale_print = case_when(
      subscale == "PAST" ~ "Past-Focused Statements", 
      subscale == "FUTURE" ~ "Future-Focused Statements"
    )
  ) %>% 
  ggplot(aes(x = fct_rev(group_print), y = mean, fill = fct_rev(subscale_print))) + 
  scale_fill_manual(values = c("gray52", "gray89")) + 
  guides(fill = guide_legend(title = "")) +
  geom_col(position = "dodge", width = 0.5, color = "black") + 
  geom_errorbar(aes(ymin = ci_lower, ymax = ci_upper), width = 0.2, position = position_dodge(0.5)) + 
  ylab("Rating") + 
  xlab("") + 
  coord_cartesian(ylim = c(2,4)) + 
  scale_y_continuous(breaks = seq(2, 4, by = 0.25)) +
  theme_classic()

Inferential statistics

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).

# reproduce the above results here
tidy_data$participant <- as.factor(tidy_data$participant)
tidy_data$subscale <- as.factor(tidy_data$subscale)
tidy_data$group <- as.factor(tidy_data$group)
tidy_data <- tidy_data %>% ungroup()

res.aov <- aov_car(agreement_score ~ subscale*group + Error(participant/subscale),
        dv = c("agreement_score"),
        between = c("group"),
        within = c("subscale"),
        fun_aggregate = mean,
         data = tidy_data, 
        na.rm = TRUE) 
res.aov
## Anova Table (Type 3 tests)
## 
## Response: agreement_score
##           Effect    df  MSE         F  ges p.value
## 1          group 1, 76 0.20      2.19 .008    .143
## 2       subscale 1, 76 0.50   7.98 ** .070    .006
## 3 group:subscale 1, 76 0.50 18.35 *** .147   <.001
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1

Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,

# reproduce the above results here
grouped_data <- tidy_data %>% 
  group_by(participant, group, subscale) %>% 
  summarize(
    agreement_score = mean(agreement_score)
  ) %>% 
  ungroup()

moroccan_past_focused <- grouped_data %>% 
  filter(group == "Moroccan", subscale == "PAST") %>% 
  select(agreement_score) %>% 
  pull()

spanish_past_focused <- grouped_data %>% 
  filter(group == "Spanish", subscale == "PAST") %>% 
  select(agreement_score) %>% 
  pull()

t.test(moroccan_past_focused, 
       spanish_past_focused, 
       alternative = "two.sided",
       paired = FALSE,
       conf.level = 0.95)
## 
##  Welch Two Sample t-test
## 
## data:  moroccan_past_focused and spanish_past_focused
## t = 3.8562, df = 74.91, p-value = 0.0002416
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.2850812 0.8944060
## sample estimates:
## mean of x mean of y 
##  3.280886  2.691142

and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)

# reproduce the above results here

moroccan_future_focused <- grouped_data %>% 
  filter(group == "Moroccan", subscale == "FUTURE") %>% 
  select(agreement_score) %>% 
  pull()

spanish_future_focused <- grouped_data %>% 
  filter(group == "Spanish", subscale == "FUTURE") %>% 
  select(agreement_score) %>% 
  pull()

t.test(moroccan_future_focused, 
       spanish_future_focused, 
       alternative = "two.sided",
       paired = FALSE,
       conf.level = 0.95)
## 
##  Welch Two Sample t-test
## 
## data:  moroccan_future_focused and spanish_future_focused
## t = -3.2098, df = 70.047, p-value = 0.002005
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.5762537 -0.1345797
## sample estimates:
## mean of x mean of y 
##  3.138333  3.493750

Step 5: Reflection

Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?

ANSWER HERE： No, the statistics test do not reproduce.

How difficult was it to reproduce your results?

ANSWER HERE: Not easy. First the participants are not uniquely identified by their participant ID! Then Spent quite some time trying to figure out why the mismatch, gave up after the time-limit.

What aspects made it difficult? What aspects made it easy?

ANSWER HERE: The annoying duplicate participant ID issue definitely makes it more difficult than necessary. Also the group naming is not consistent with the ones reported in the paper. Also the fact the number just doesn’t match…might be because I’m missing something about the stats test they ran? This would not be a problem if the reporting is more detailed. Easy aspect is that other than the issue stated above the dataset is relavively tidy.