For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.

Methods summary:

Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.

Target outcomes:

Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjectS factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).

Step 1: Load packages

library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files

# #optional packages/functions:
library(afex) # anova functions
library(ez) # anova functions 2
library(scales) # for plotting
std.err <- function(x) sd(x)/sqrt(length(x)) # standard error

Step 2: Load data

# Just Experiment 2
setwd("~/Desktop/problem_sets/ps3/GroupB/Choice 3")
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)

Step 3: Tidy data

colnames(d)[5] <- "agreement_rating" # change column name
# remember: (0=complete disagreement; 5=complete agreement)

Step 4: Run analysis

Pre-processing

# Participant #25 is excluded to run ANOVA 
excluded <- "25"

d <- d %>%
  filter(!participant %in% excluded) #participant exclusions

Descriptive statistics

Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):

d_analyzed <- d %>%
  group_by(group, subscale) %>%
  summarize(AvgRating = mean(agreement_rating),
            sd_rating = sd(agreement_rating),
            n_obs = length(agreement_rating),
            sem = sd_rating / sqrt(n_obs),
            ci = sem * 1.96,
            ci_lower = AvgRating - ci,
            ci_upper = AvgRating + ci) %>%
  ungroup()

d_analyzed$group <- factor(d_analyzed$group, levels = c('young Spaniard', 'Moroccan'))

d_analyzed$subscale <- factor(d_analyzed$subscale, levels = c('PAST', 'FUTURE'))

ggplot(d_analyzed, aes(x = group, y = AvgRating, fill = subscale)) +
  geom_bar(stat='identity', position='dodge', width = .5) +
  geom_errorbar(aes(ymin=AvgRating - sem, ymax=AvgRating + sem), width = .1, position=position_dodge(.5)) +
  coord_cartesian(ylim=c(2,4)) + 
    scale_fill_manual(values=c("#999999", "#D3D3D3"))

Inferential statistics

According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).

# reproduce the above results here

d_anova <- d %>%
  group_by(participant, group, subscale) %>%
  summarize(AvgRating = mean(agreement_rating),
            sd_rating = sd(agreement_rating),
            n_obs = length(agreement_rating),
            sem = sd_rating / sqrt(n_obs),
            ci = sem * 1.96,
            ci_lower = AvgRating - ci,
            ci_upper = AvgRating + ci)

ezDesign(data=d_anova,y=participant,x=group,col=subscale) # are we missing data from PAST? Participant 25?

ezANOVA(data = d_anova,
        dv = AvgRating,
        within = subscale,
        between = group,
        wid = participant,
        type = 3)
## $ANOVA
##           Effect DFn DFd         F            p p<.05         ges
## 2          group   1  76  2.191977 1.428650e-01       0.008226326
## 3       subscale   1  76  7.979308 6.040164e-03     * 0.069591536
## 4 group:subscale   1  76 18.345608 5.327735e-05     * 0.146734962

Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,

# reproduce the above results here

d_past_data <- d %>%
  filter(subscale == "PAST") %>%
  group_by(participant, group) %>%
  summarise(MeanRating.Past = mean(agreement_rating),
            sd.Past = sd(agreement_rating))

t.test(MeanRating.Past ~ group, data = d_past_data, var.equal = TRUE)
## 
##  Two Sample t-test
## 
## data:  MeanRating.Past by group
## t = 3.8562, df = 76, p-value = 0.0002394
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.2851528 0.8943343
## sample estimates:
##       mean in group Moroccan mean in group young Spaniard 
##                     3.280886                     2.691142

and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)

# reproduce the above results here

d_future_data <- d %>%
  filter(subscale == "FUTURE") %>%
  group_by(participant, group) %>%
  summarise(MeanRating.Future = mean(agreement_rating),
            sd.Future = sd(agreement_rating))

t.test(MeanRating.Future ~ group, data = d_future_data, var.equal = TRUE)
## 
##  Two Sample t-test
## 
## data:  MeanRating.Future by group
## t = -3.3898, df = 76, p-value = 0.001112
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.5990628 -0.1556380
## sample estimates:
##       mean in group Moroccan mean in group young Spaniard 
##                     3.116239                     3.493590

Step 5: Reflection

Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?

I… don’t know! It looks like we were missing some data from participant 25 from the past-focused condition which reduced df for all our statistical inferences… I also was not so sure how to rectify this problem (or if I caused it??)… so rather than trying to identify a solution to that, or try to find that data somehow (could I have done that?) I instead just excluded them — at least for now! — and compared my analysis with theirs. That being said, for each statisitcal inference, I got t statistics and p values that were extremely close despite losing about 2 degrees of freedom. So I want to assume that if we did find this data, that it would produce the same results (assuming they are not an egregious outlier, is it possible for a single participant to have such a dramatic effect on the statistical analysis?).

How difficult was it to reproduce your results?

I… wanted to cry… I thought I was/still think I am doing something wrong and spent so much time trying to debug my code! Once I realized I was missing a participant, I was not sure if I should use imputations to replace their data, but I figured it would be easier to just exclude them from the analysis and then compare my answers with what they reported.

What aspects made it difficult? What aspects made it easy?

I had such a tough time figuring out why the ANOVA wouldn’t run (and I tried many times to do ezDesign() from their warning message but it only worked after randomly trying something? Was very weird…). That said, like my analysis in Group A, it was not the most complicated data to clean up, which meant that running a t-test or an ANOVA (once I finally got it to work) was fine. I appreciated that the data was formatted perfectly to just plug and chug away.