For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.

https://rpubs.com/kmwhitman/1366630

Methods summary:

Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.

Target outcomes:

Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).

Step 1: Load packages

library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files

# #optional packages/functions:
library(afex) # anova functions
library(ez) # anova functions 2
library(scales)
library(readr)
std.err <- function(x) sd(x)/sqrt(length(x)) # standard error

Step 2: Load data

# Just Experiment 2
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)

d %>% distinct(participant, group) %>% count(group)
## # A tibble: 2 × 2
##   group              n
##   <chr>          <int>
## 1 Moroccan          40
## 2 young Spaniard    40

Step 3: Tidy data

#Data is already long, woo! 
d_tidy <- d %>%
  rename(rating = `Agreement (0=complete disagreement; 5=complete agreement)`) %>%
  mutate(
    item   = parse_number(item),               
    group  = recode(group, "young Spaniard" = "Spanish"),
    group  = factor(group),
    subscale = factor(subscale))

Step 4: Run analysis

Pre-processing

d_summary <- d_tidy %>%
  group_by(group, subscale) %>%
  summarize(
            n_obs = sum(!is.na(rating)),
            average_rating = mean(rating, na.rm = TRUE),
            std_dev = sd(rating, na.rm = TRUE),
            n_count = n(), 
            SEM = std_dev / sqrt(n_count), 
            .groups = 'drop') %>% 
  mutate(group = fct_relevel(group, "Spanish", "Moroccan"),
         subscale = fct_relevel(subscale, "PAST", "FUTURE"))

#where I tried to find the missing from any value participants 
d_anova_1 <- d_tidy %>%
  mutate(
    group = recode(group,
                   "young Spaniard" = "Spanish"),
    group = factor(group, levels = c("Spanish", "Moroccan")),
    subscale = factor(subscale, levels = c("PAST", "FUTURE")),
    participant = as.character(participant),
    # make IDs unique across groups (suffix both sides)
    participant = paste(participant, group, sep = "_"))

missing_within <- d_anova_1 %>%
  distinct(participant, subscale) %>%
  count(participant, name = "n_levels") %>%
  filter(n_levels < 2)

#if (nrow(missing_within) > 0) {
  #stop("Some participants are missing a subscale level:\n",
       #paste(missing_within$participant, collapse = ", "))}
#Error: Some participants are missing a subscale level: 25_Moroccan, 25_Spanish

print(d_anova_1 %>% distinct(participant, group) %>% count(group))
## # A tibble: 2 × 2
##   group        n
##   <fct>    <int>
## 1 Spanish     40
## 2 Moroccan    40

Descriptive statistics

Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):

pd <- position_dodge(width = .9)

ggplot(d_summary, aes(group, average_rating, fill = subscale)) +
  geom_col(position = pd, witdth = 0.25, coloe = "black") + 
  geom_errorbar(aes(ymin = average_rating - SEM, ymax = average_rating + SEM),
                position = pd, width = .15) +
  labs(y = "Rating") +
  scale_fill_discrete(labels = c("PAST" = "Past-Focused Statements", "FUTURE" = "Future-Focused Statements")) +
  coord_cartesian(ylim = c(2, 4)) +
  theme_classic() +
  theme(axis.title.x = element_blank(), legend.title = element_blank(), legend.position = "top") +
  guides(fill = guide_legend(nrow = 2, byrow = TRUE))

Inferential statistics

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).

# reproduce the above results here

d_tidy_anova <- d_tidy %>% 
  mutate(participant = as.character(participant)) %>% 
  mutate(participant = if_else(group == "Spanish", paste0(participant, "_S"), participant))

mixed_anova_results <- afex::aov_ez(id = "participant", dv = "rating", between = "group", within = "subscale", 
                                    data = d_tidy_anova)
#Participant 25_moroccan and participant 25_spanish are missing past, but not future, values 

print(mixed_anova_results)
## Anova Table (Type 3 tests)
## 
## Response: rating
##           Effect    df  MSE         F  ges p.value
## 1          group 1, 76 0.20      2.19 .008    .143
## 2       subscale 1, 76 0.50   7.98 ** .070    .006
## 3 group:subscale 1, 76 0.50 18.35 *** .147   <.001
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1

Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,

# reproduce the above results here

past_focused <- d_tidy_anova %>% 
  filter(subscale == "PAST") %>% 
  group_by(group, participant) %>% 
  summarise(mean_rating = mean(rating))

past_comp <- t.test(mean_rating ~ group, alternative = c("greater"), var.equal = TRUE, data = past_focused)
print(past_comp)
## 
##  Two Sample t-test
## 
## data:  mean_rating by group
## t = 3.8562, df = 76, p-value = 0.0001197
## alternative hypothesis: true difference in means between group Moroccan and group Spanish is greater than 0
## 95 percent confidence interval:
##  0.3350884       Inf
## sample estimates:
## mean in group Moroccan  mean in group Spanish 
##               3.280886               2.691142

and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)

# reproduce the above results here

future_focused <- d_tidy_anova %>% 
  filter(subscale == "FUTURE") %>% 
  group_by(group, participant) %>% 
  summarise(mean_rating = mean(rating))

future_comp <- t.test(mean_rating ~ group, alternative = c("less"), var.equal = TRUE, data = future_focused)
print(future_comp)
## 
##  Two Sample t-test
## 
## data:  mean_rating by group
## t = -3.2098, df = 78, p-value = 0.0009645
## alternative hypothesis: true difference in means between group Moroccan and group Spanish is less than 0
## 95 percent confidence interval:
##        -Inf -0.1710965
## sample estimates:
## mean in group Moroccan  mean in group Spanish 
##               3.138333               3.493750

Step 5: Reflection

Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?

Trying to reproduce the graph (Figure 2) mostly went well, but I couldn’t get my columns to be narrow. I left them in color for readibility. I struggled in the statistics reproduction but am not sure if it only my own limited knowledge and coding that is causing me to make some mistake. I know my n’s are off (by getting different df (at 76 instead of the paper’s 78) but still could not reproduce even when trying to include subjects with NA values.

How difficult was it to reproduce your results?

I found myself doubting which statistics I was doing and if I was missing something obvious, and became more confused when I tried to get coding feedback online to see where the error might be. I hope an answer key can be posted in the future to see where I was going wrong; I suspect it is with the 76 or 78 person difference.

What aspects made it difficult? What aspects made it easy?

Starting in long form was nice! But this also made it easier not to notice the same ID numbers being used for different groups (so we had to change the label on one group to reflect this - the first Spanish participant is labeled 1_S, etc.). There were small things to watch for, like renaming young Spainard to Spainard.