For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.

Methods summary:

Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.

Target outcomes:

Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjectS factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).

Step 1: Load packages

library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files

library(rstatix) # For anova tests

# #optional packages/functions:
# library(afex) # anova functions
# library(ez) # anova functions 2
# library(scales) # for plotting
# std.err <- function(x) sd(x)/sqrt(length(x)) # standard error

Step 2: Load data

# Just Experiment 2
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)

Step 3: Tidy data

data  = d %>%
  mutate(agreement = `Agreement (0=complete disagreement; 5=complete agreement)`)

Step 4: Run analysis

Pre-processing

Descriptive statistics

Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):

#Defind Graphing Functions 

se_plus <- function(x) mean(x) + sqrt(var(x)/length(x))
se_minus <- function(x) mean(x) - sqrt(var(x)/length(x))

limits <- aes(ymax = se_u, ymin= se_l)
dodge <- position_dodge(width=0.9)


# GRaphs

summary_ratings <- d %>%
  mutate(agreement = `Agreement (0=complete disagreement; 5=complete agreement)`, 
         population = ifelse(group=="young Spaniard", "Spaniards", group)) %>% 
  dplyr::group_by(population, subscale) %>%
  dplyr::summarise("Rating" = mean(agreement, na.rm=TRUE), "se_u"=se_plus(agreement), "se_l"=se_minus(agreement))

barplot <- ggplot(data=summary_ratings, aes(x=population, y=Rating, 
                                            group=subscale, fill=subscale)) +
  geom_bar(stat = "identity", position = "dodge", width =0.75) +
  geom_errorbar(limits, position=dodge, width=0.25) +
  theme_bw() + 
  coord_cartesian(ylim = c(2, 4))

  

barplot

Inferential statistics

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).

# reproduce the above results here

res.aov <- anova_test(
  data = data, dv = agreement, wid = participant,
  between = group, within = subscale
  )
get_anova_table(res.aov)
# We get an error: it seems that ID  40 is used for two different participants. 

# Let's try to rename one of them to an arbitrary ID: 

data_summarized = data %>% 
  mutate(participant=ifelse(participant==40&group=="Moroccan",
                            999, 
                            participant)) %>% 
  group_by(participant, subscale, group) %>% 
  summarise(mean_agreenment=mean(agreement)) %>% ungroup()
         
         

res.aov <- anova_test(
  data = data_summarized, dv = mean_agreenment, wid = participant,
  between = group, within = subscale
  )
get_anova_table(res.aov)
## ANOVA Table (type II tests)
## 
##           Effect DFn DFd      F        p p<.05   ges
## 1          group   1  76  2.192 1.43e-01       0.008
## 2       subscale   1  76  7.979 6.00e-03     * 0.070
## 3 group:subscale   1  76 18.346 5.33e-05     * 0.147

Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,

data_test1 = data_summarized %>% filter(subscale=="PAST")
t.test(data_test1$mean_agreenment ~ data_test1$group)
## 
##  Welch Two Sample t-test
## 
## data:  data_test1$mean_agreenment by data_test1$group
## t = 3.8562, df = 74.91, p-value = 0.0002416
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.2850812 0.8944060
## sample estimates:
##       mean in group Moroccan mean in group young Spaniard 
##                     3.280886                     2.691142

and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)

# reproduce the above results here
data_test2 = data_summarized %>% filter(subscale=="FUTURE")
t.test(data_test2$mean_agreenment ~ data_test2$group)
## 
##  Welch Two Sample t-test
## 
## data:  data_test2$mean_agreenment by data_test2$group
## t = -3.2098, df = 70.047, p-value = 0.002005
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.5762537 -0.1345797
## sample estimates:
##       mean in group Moroccan mean in group young Spaniard 
##                     3.138333                     3.493750

Step 5: Reflection

Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?

Not quite. For the mixed anova, I got different degrees of freedom (78 vs. 76), F-values (9.12 vs. 18.346) and p-values (<.02 vs. <.001) from what the authors reported.

For the t-tests, I also got different t-values: t = 3.8562 vs. t =4.04 and -3.2 vs. ???3.32. Again, the degrees of freedom were also different.

How difficult was it to reproduce your results?

It was somewhat difficult.

What aspects made it difficult? What aspects made it easy?

I was not sure how to handle the participant with the same ID and I am also not sure why there’s a difference in the degrees of freedoms–Did the authors drop participants? Is it an error?