Reproducibility Report: Group B Choice 3
For this exercise, please try to reproduce the results from Experiment 2 of the associated paper (de la Fuente, Santiago, Roman, Dumitrache, & Casasanto, 2014). The PDF of the paper is included in the same folder as this Rmd file.
Methods summary:
Researchers tested the question of whether temporal focus differs between Moroccan and Spanish cultures, hypothesizing that Moroccans are more past-focused, whereas Spaniards are more future-focused. Two groups of participants (\(N = 40\) Moroccan and \(N=40\) Spanish) completed a temporal-focus questionnaire that contained questions about past-focused (“PAST”) and future-focused (“FUTURE”) topics. In response to each question, participants provided a rating on a 5-point Likert scale on which lower scores indicated less agreement and higher scores indicated greater agreement. The authors then performed a mixed-design ANOVA with agreement score as the dependent variable, group (Moroccan or Spanish, between-subjects) as the fixed-effects factor, and temporal focus (past or future, within-subjects) as the random effects factor. In addition, the authors performed unpaired two-sample t-tests to determine whether there was a significant difference between the two groups in agreement scores for PAST questions, and whether there was a significant difference in scores for FUTURE questions.
Target outcomes:
Below is the specific result you will attempt to reproduce (quoted directly from the results section of Experiment 2):
According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjectS factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2). Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001, and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001. (de la Fuente et al., 2014, p. 1685).
Step 1: Load packages
library(tidyverse) # for data munging
library(knitr) # for kable table formating
library(haven) # import and export 'SPSS', 'Stata' and 'SAS' Files
library(readxl) # import excel files
# #optional packages/functions:
library(afex) # anova functions
library(ez) # anova functions 2
library(scales) # for plotting
library(bruceR)##
## bruceR (version 0.8.9)
## BRoadly Useful Convenient and Efficient R functions
##
## Packages also loaded:
## √ dplyr √ emmeans √ ggplot2
## √ tidyr √ effectsize √ ggtext
## √ stringr √ performance √ cowplot
## √ forcats √ lmerTest √ see
## √ data.table
##
## Main functions of `bruceR`:
## cc() Describe() TTEST()
## add() Freq() MANOVA()
## .mean() Corr() EMMEANS()
## set.wd() Alpha() PROCESS()
## import() EFA() model_summary()
## print_table() CFA() lavaan_summary()
##
## https://psychbruce.github.io/bruceR/
##
## These R packages are dependencies of `bruceR` but not installed:
## MuMIn
## ***** Please Install All Dependencies *****
## install.packages("bruceR", dep=TRUE)# std.err <- function(x) sd(x)/sqrt(length(x)) # standard errorStep 2: Load data
# Just Experiment 2
data_path <- 'data/DeLaFuenteEtAl_2014_RawData.xls'
d <- read_excel(data_path, sheet=3)Step 3: Tidy data
# check missing
sum(is.na(d))## [1] 0# sum score
sum_d = d |>
group_by(participant, group, subscale) |>
summarize(Rating = mean(`Agreement (0=complete disagreement; 5=complete agreement)`), .groups = "drop")
sum_d = sum_d |>
mutate(ID = ifelse(group=='young Spaniard', participant+40, participant))Step 4: Run analysis
Pre-processing
# delete #25 and #65 participant
table(sum_d$ID)##
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
## 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2
## 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52
## 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
## 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78
## 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2 2 2
## 79 80
## 2 2with(sum_d,table(ID,group,subscale))## , , subscale = FUTURE
##
## group
## ID Moroccan young Spaniard
## 1 1 0
## 2 1 0
## 3 1 0
## 4 1 0
## 5 1 0
## 6 1 0
## 7 1 0
## 8 1 0
## 9 1 0
## 10 1 0
## 11 1 0
## 12 1 0
## 13 1 0
## 14 1 0
## 15 1 0
## 16 1 0
## 17 1 0
## 18 1 0
## 19 1 0
## 20 1 0
## 21 1 0
## 22 1 0
## 23 1 0
## 24 1 0
## 25 1 0
## 26 1 0
## 27 1 0
## 28 1 0
## 29 1 0
## 30 1 0
## 31 1 0
## 32 1 0
## 33 1 0
## 34 1 0
## 35 1 0
## 36 1 0
## 37 1 0
## 38 1 0
## 39 1 0
## 40 1 0
## 41 0 1
## 42 0 1
## 43 0 1
## 44 0 1
## 45 0 1
## 46 0 1
## 47 0 1
## 48 0 1
## 49 0 1
## 50 0 1
## 51 0 1
## 52 0 1
## 53 0 1
## 54 0 1
## 55 0 1
## 56 0 1
## 57 0 1
## 58 0 1
## 59 0 1
## 60 0 1
## 61 0 1
## 62 0 1
## 63 0 1
## 64 0 1
## 65 0 1
## 66 0 1
## 67 0 1
## 68 0 1
## 69 0 1
## 70 0 1
## 71 0 1
## 72 0 1
## 73 0 1
## 74 0 1
## 75 0 1
## 76 0 1
## 77 0 1
## 78 0 1
## 79 0 1
## 80 0 1
##
## , , subscale = PAST
##
## group
## ID Moroccan young Spaniard
## 1 1 0
## 2 1 0
## 3 1 0
## 4 1 0
## 5 1 0
## 6 1 0
## 7 1 0
## 8 1 0
## 9 1 0
## 10 1 0
## 11 1 0
## 12 1 0
## 13 1 0
## 14 1 0
## 15 1 0
## 16 1 0
## 17 1 0
## 18 1 0
## 19 1 0
## 20 1 0
## 21 1 0
## 22 1 0
## 23 1 0
## 24 1 0
## 25 0 0
## 26 1 0
## 27 1 0
## 28 1 0
## 29 1 0
## 30 1 0
## 31 1 0
## 32 1 0
## 33 1 0
## 34 1 0
## 35 1 0
## 36 1 0
## 37 1 0
## 38 1 0
## 39 1 0
## 40 1 0
## 41 0 1
## 42 0 1
## 43 0 1
## 44 0 1
## 45 0 1
## 46 0 1
## 47 0 1
## 48 0 1
## 49 0 1
## 50 0 1
## 51 0 1
## 52 0 1
## 53 0 1
## 54 0 1
## 55 0 1
## 56 0 1
## 57 0 1
## 58 0 1
## 59 0 1
## 60 0 1
## 61 0 1
## 62 0 1
## 63 0 1
## 64 0 1
## 65 0 0
## 66 0 1
## 67 0 1
## 68 0 1
## 69 0 1
## 70 0 1
## 71 0 1
## 72 0 1
## 73 0 1
## 74 0 1
## 75 0 1
## 76 0 1
## 77 0 1
## 78 0 1
## 79 0 1
## 80 0 1sum_d <- sum_d |> filter(participant!=25) |> filter(participant!=65)
sum_d$group = factor(sum_d$group, levels=c(unique(sum_d$group)[2], unique(sum_d$group)[1]))
sum_d$subscale = factor(sum_d$subscale, levels=c(unique(sum_d$subscale)[2], unique(sum_d$subscale)[1]))
summary_d = sum_d |>
group_by(group, subscale) |>
summarize(MeanRating = mean(Rating),
sem = sd(Rating)/sqrt(length(Rating)),
upper = MeanRating + sem,
lower = MeanRating - sem,
.groups = 'drop')Descriptive statistics
Try to recreate Figure 2 (fig2.png, also included in the same folder as this Rmd file):
ggplot(summary_d, aes(x=group, y=MeanRating, fill=subscale)) +
geom_bar(position=position_dodge(width = 0.5), width = 0.5, stat="identity") +
geom_errorbar(aes(ymin=upper, ymax=lower), stat="identity", position=position_dodge(width = 0.5), width=0.1) +
scale_fill_manual(values=c('grey', 'black')) +
scale_y_continuous(limits=c(2,4),oob = rescale_none)
Inferential statistics
According to a mixed analysis of variance (ANOVA) with group (Spanish vs. Moroccan) as a between-subjects factor and temporal focus (past vs. future) as a within-subjects factor, temporal focus differed significantly between Spaniards and Moroccans, as indicated by a significant interaction of temporal focus and group, F(1, 78) = 19.12, p = .001, ηp2 = .20 (Fig. 2).
# reproduce the above results here
MANOVA(sum_d, subID = 'ID', dv = 'Rating', within = 'subscale', between = 'group', aov.include = TRUE) |>
EMMEANS('group', by = 'subscale')##
## ====== ANOVA (Mixed Design) ======
##
## Descriptives:
## ───────────────────────────────────────────
## "group" "subscale" Mean S.D. n
## ───────────────────────────────────────────
## young Spaniard PAST 2.691 (0.633) 39
## young Spaniard FUTURE 3.494 (0.408) 39
## Moroccan PAST 3.281 (0.715) 39
## Moroccan FUTURE 3.116 (0.563) 39
## ───────────────────────────────────────────
## Total sample size: N = 78
##
## ANOVA Table:
## Dependent variable(s): Rating
## Between-subjects factor(s): group
## Within-subjects factor(s): subscale
## Covariate(s): –
## ───────────────────────────────────────────────────────────────────────────────
## MS MSE df1 df2 F p η²p [90% CI of η²p] η²G
## ───────────────────────────────────────────────────────────────────────────────
## group 0.440 0.201 1 76 2.192 .143 .028 [.000, .114] .008
## subscale 3.966 0.497 1 76 7.979 .006 ** .095 [.016, .211] .070
## group * subscale 9.119 0.497 1 76 18.346 <.001 *** .194 [.078, .322] .147
## ───────────────────────────────────────────────────────────────────────────────
## MSE = mean square error (the residual variance of the linear model)
## η²p = partial eta-squared = SS / (SS + SSE) = F * df1 / (F * df1 + df2)
## ω²p = partial omega-squared = (F - 1) * df1 / (F * df1 + df2 + 1)
## η²G = generalized eta-squared (see Olejnik & Algina, 2003)
## Cohen’s f² = η²p / (1 - η²p)
##
## Levene’s Test for Homogeneity of Variance:
## ────────────────────────────────────────
## Levene’s F df1 df2 p
## ────────────────────────────────────────
## DV: FUTURE 1.831 1 76 .180
## DV: PAST 0.102 1 76 .751
## ────────────────────────────────────────
##
## Mauchly’s Test of Sphericity:
## The repeated measures have only two levels. The assumption of sphericity is always met.
##
## ------ EMMEANS (effect = "group") ------
##
## Joint Tests of "group":
## ───────────────────────────────────────────────────────────────
## Effect "subscale" df1 df2 F p η²p [90% CI of η²p]
## ───────────────────────────────────────────────────────────────
## group PAST 1 76 14.871 <.001 *** .164 [.056, .290]
## group FUTURE 1 76 11.491 .001 ** .131 [.036, .254]
## ───────────────────────────────────────────────────────────────
## Note. Simple effects of repeated measures with 3 or more levels
## are different from the results obtained with SPSS MANOVA syntax.
##
## Multivariate Tests of "group":
## ─────────────────────────────────────────────────────────────────────
## Pillai’s trace Hypoth. df Error df Exact F p
## ─────────────────────────────────────────────────────────────────────
## FUTURE: "group" 0.131 1.000 76.000 11.491 .001 **
## PAST: "group" 0.164 1.000 76.000 14.871 <.001 ***
## ─────────────────────────────────────────────────────────────────────
## Note. Identical to the results obtained with SPSS GLM EMMEANS syntax.
##
## Estimated Marginal Means of "group":
## ────────────────────────────────────────────────────────
## "group" "subscale" Mean [95% CI of Mean] S.E.
## ────────────────────────────────────────────────────────
## young Spaniard PAST 2.691 [2.476, 2.907] (0.108)
## Moroccan PAST 3.281 [3.066, 3.496] (0.108)
## young Spaniard FUTURE 3.494 [3.337, 3.650] (0.079)
## Moroccan FUTURE 3.116 [2.959, 3.273] (0.079)
## ────────────────────────────────────────────────────────
##
## Pairwise Comparisons of "group":
## ──────────────────────────────────────────────────────────────────────────────────────────────────
## Contrast "subscale" Estimate S.E. df t p Cohen’s d [95% CI of d]
## ──────────────────────────────────────────────────────────────────────────────────────────────────
## Moroccan - young Spaniard PAST 0.590 (0.153) 76 3.856 <.001 *** 0.595 [ 0.288, 0.903]
## Moroccan - young Spaniard FUTURE -0.377 (0.111) 76 -3.390 .001 ** -0.381 [-0.605, -0.157]
## ──────────────────────────────────────────────────────────────────────────────────────────────────
## Pooled SD for computing Cohen’s d: 0.991
## No need to adjust p values.
##
## Disclaimer:
## By default, pooled SD is Root Mean Square Error (RMSE).
## There is much disagreement on how to compute Cohen’s d.
## You are completely responsible for setting `sd.pooled`.
## You might also use `effectsize::t_to_d()` to compute d.Moroccans showed greater agreement with past-focused statements than Spaniards did, t(78) = 4.04, p = .001,
# reproduce the above results here
# see the table above
# where t(76) = 3.856, p < .001and Spaniards showed greater agreement with future-focused statements than Moroccans did, t(78) = −3.32, p = .001.(de la Fuente et al., 2014, p. 1685)
# reproduce the above results here
# see the table above
# t(76) = -3.390, p = .001Step 5: Reflection
Were you able to reproduce the results you attempted to reproduce? If not, what part(s) were you unable to reproduce?
NO. I excluded two participants with missing values on many items so the results are a little bit different. Also, I don’t know why the participant variable has the same value for two groups since the group is the between-participant variable.
How difficult was it to reproduce your results?
ANSWER HERE
What aspects made it difficult? What aspects made it easy?
Plot.