For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.

Methods summary:

Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.

Target outcomes:

Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjectS factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).

Step 1: Load packages

library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
library(rstatix)


# #optional packages/functions:
# library(afex) # anova functions
# library(ez) # anova functions 2
# library(scales) # for plotting
# std.err <- function(x) sd(x)/sqrt(length(x)) # standard error

Step 2: Load data

# Just Experiment 2
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)

Step 3: Tidy data

d_tidy <- d %>%
  select(-item) %>%
  rename(agreement = "Agreement (0=complete disagreement; 5=complete agreement)")

Step 4: Run analysis

Pre-processing

d_tidy <- d_tidy %>% 
  mutate(group = factor(group,
                        levels = c("young Spaniard", "Moroccan"))) %>% 
  mutate(subscale = factor(subscale,
                        levels = c("PAST", "FUTURE")))

#prepping stats
d_averages <- d_tidy %>% 
  group_by(group,
           participant,
           subscale) %>% 
  summarize(avg_agree = mean(agreement)) %>% 
  ungroup()

Descriptive statistics

Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):

d_tidy %>% 
  group_by(group,
           subscale) %>% 
  ggplot(mapping = aes(x = group,
                       y = agreement,
                       fill = subscale)) +
  stat_summary(fun = "mean",
               geom = "bar",
               position = position_dodge(width = 0.9)) +
  stat_summary(fun.data = mean_se,
               geom = "linerange",
               position = position_dodge(width = 0.9)) +
  coord_cartesian(ylim = c(2.00, 4.00)) +
  xlab("") + 
  ylab("Rating") +
  scale_x_discrete(labels = c("Spaniards", "Moroccans")) +
  scale_fill_brewer(palette = "Set1",
                    name = "",
                    labels = c("Past-Focused Statements", "Future-Focused Statements"))

Inferential statistics

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).

# reproduce the above results here
rating.aov <- d_averages %>% 
  anova_test(wid = participant,
           dv = avg_agree,
           between = group,
           within = subscale,
           type = 3,
           effect.size = 'pes')
get_anova_table(rating.aov)
## ANOVA Table (type III tests)
## 
##           Effect DFn DFd      F        p p<.05   pes
## 1          group   1  76  2.192 1.43e-01       0.028
## 2       subscale   1  76  7.979 6.00e-03     * 0.095
## 3 group:subscale   1  76 18.346 5.33e-05     * 0.194

Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,

# reproduce the above results here
d_past <- d_averages %>% 
  filter(subscale == 'PAST')

t.test(d_past$avg_agree[d_past$group == 'Moroccan'],
       d_past$avg_agree[d_past$group == 'young Spaniard'])
## 
##  Welch Two Sample t-test
## 
## data:  d_past$avg_agree[d_past$group == "Moroccan"] and d_past$avg_agree[d_past$group == "young Spaniard"]
## t = 3.8562, df = 74.91, p-value = 0.0002416
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.2850812 0.8944060
## sample estimates:
## mean of x mean of y 
##  3.280886  2.691142

and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)

# reproduce the above results here
d_future<- d_averages %>% 
  filter(subscale == 'FUTURE')

t.test(d_past$avg_agree[d_future$group == 'young Spaniard'],
       d_past$avg_agree[d_future$group == 'Moroccan'])
## 
##  Welch Two Sample t-test
## 
## data:  d_past$avg_agree[d_future$group == "young Spaniard"] and d_past$avg_agree[d_future$group == "Moroccan"]
## t = -3.695, df = 73.517, p-value = 0.0004204
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.8785695 -0.2629377
## sample estimates:
## mean of x mean of y 
##  2.707955  3.278708

Step 5: Reflection

Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?

I was able to reproduce all of the major outcomes reported in the paper. While my ANOVA and t-test stats did not match the published numbers exactly, the patterns of results and significance levels were consistent with the original findings. Because the conclusions remained the same, I consider the reproduction successful.

How difficult was it to reproduce your results?

I felt like the graph was not too hard, but I found it hard to get the same anova and t-test as were in the paper.

What aspects made it difficult? What aspects made it easy?

The hardest part was figuring out how the original authors structured their stats. I realized I needed to do the average scores for each subject so within- and between-subject factors would be represented properly. This helped to get closer results. It helped that the dataset was clear and variable names were straightforward. There was not much data cleaning/preprocessing needed.