For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.

Methods summary:

Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.

Target outcomes:

Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subject factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).

Step 1: Load packages

library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files

# #optional packages/functions:
library(afex) # anova functions
library(ez) # anova functions 2
library(scales) # for plotting
std.err <- function(x) sd(x)/sqrt(length(x)) # standard error

Step 2: Load data

# Just Experiment 2
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)

Step 3: Tidy data

Step 4: Run analysis

Pre-processing

# Change "young Spaniard" to "Spaniard"
d <- d%>% mutate (group = replace(group, group=="young Spaniard",  "Spaniard"))

# remove the item column
d <- select(d, -item)

# assigning new names to the columns of the data frame 
colnames(d) <- c('group', 'participant','subscale', 'agreement')

Descriptive statistics

Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):

summary_df <- d %>%
  group_by(group, subscale) %>%
  summarise(response_mean = mean(agreement), 
            response_se = std.err(agreement))

# reorder the sequence to generate identical plot
summary_df$group <- factor(summary_df$group, levels = c("Spaniard", "Moroccan"))
summary_df$subscale <- factor(summary_df$subscale, levels = c("PAST", "FUTURE"))

ggplot(summary_df, aes(x=group, y=response_mean, fill=subscale)) +
  geom_col(position=position_dodge()) +
  geom_errorbar(aes(ymin=response_mean - response_se, ymax=response_mean + response_se), width=0.2, position=position_dodge(0.9)) +
  xlab("Group") + 
  ylab("Rating") +
  theme_classic() +
  coord_cartesian(ylim = c(2, 4))

Inferential statistics

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).

# reproduce the above results here
# average across items
d_avg <- d%>%
  group_by(group, subscale, participant) %>%
  summarise(mean_agreement = mean(agreement, na.rm = TRUE)) %>%
  ungroup() 
  
mixed(mean_agreement ~ subscale*group + (1|participant), data=d_avg)
## Mixed Model Anova Table (Type 3 tests, S-method)
## 
## Model: mean_agreement ~ subscale * group + (1 | participant)
## Data: d_avg
##           Effect     df         F p.value
## 1       subscale 1, 154 12.32 ***   <.001
## 2          group 1, 154      1.55    .215
## 3 subscale:group 1, 154 25.25 ***   <.001
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1

Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,

# reproduce the above results here
# Subset the data for Moroccans and Spaniards focusing on the past
moroccans_past <- d_avg[ d_avg$group == "Moroccan" & d_avg$subscale == "PAST", "mean_agreement"]
spaniards_past <- d_avg[ d_avg$group == "Spaniard" & d_avg$subscale == "PAST", "mean_agreement"]

# Perform the t-test
t_test_results <- t.test(moroccans_past, spaniards_past, var.equal = TRUE)

# Output the results
print(t_test_results)
## 
##  Two Sample t-test
## 
## data:  moroccans_past and spaniards_past
## t = 3.8562, df = 76, p-value = 0.0002394
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.2851528 0.8943343
## sample estimates:
## mean of x mean of y 
##  3.280886  2.691142

and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)

# reproduce the above results here
# Subset the data for Moroccans and Spaniards focusing on the past
moroccans_future <- d_avg[ d_avg$group == "Moroccan" & d_avg$subscale == "FUTURE", "mean_agreement"]
spaniards_future <- d_avg[ d_avg$group == "Spaniard" & d_avg$subscale == "FUTURE", "mean_agreement"]

# Perform the t-test
t_test_results <- t.test(moroccans_future, spaniards_future, var.equal = TRUE)

# Output the results
print(t_test_results)
## 
##  Two Sample t-test
## 
## data:  moroccans_future and spaniards_future
## t = -3.2098, df = 78, p-value = 0.001929
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.5758588 -0.1349746
## sample estimates:
## mean of x mean of y 
##  3.138333  3.493750

Step 5: Reflection

Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce? I was able to reproduce the most of the results. All statistical significant results are reproduced. However, the specific statistics (e.g., F score, T score, DF) anare not reproduced.

How difficult was it to reproduce your results? Difficult.

What aspects made it difficult? What aspects made it easy?

There is a missing data problem. Participant 25 did not finish the entire survey, it does not have any information in PAST section. But this is not mentioned in the paper at all.