Introduction

[No abstract is needed.] Each replication project will have a straightforward, no frills report of the study and results. These reports will be publicly available as supplementary material for the aggregate report(s) of the project as a whole. Also, to maximize project integrity, the intro and methods will be written and critiqued in advance of data collection. Introductions can be just 1-2 paragraphs clarifying the main idea of the original study, the target finding for replication, and any other essential information. It will NOT have a literature review – that is in the original publication. You can write both the introduction and the methods in past tense.

In Study 2 of the report “Differentiate to Regulate: Low Negative Emotion Differentiation Is Associated With Ineffective Use but Not Selection of Emotion-Regulation Strategies”, Kalokerinos and colleagues, (2019) examined how the ability to make fine-grained distinctions between emotional states (i.e. emotion differentiation) was related to the selection and efficacy of emotion regulation (ER) strategies amidst a high-intensity emotional event.

The authors initiated students on a nine day experience sampling method (ESM) regimen two days prior to the students receiving a highly-consequential test result. The student participants completed ten surveys per day for the nine days delivered via an application on their smart phones or via a special mobile phone provided by the experimenters. Since the present study concerns regulation after and emotional event, only data collected after the results were released are analyzed, resulting in 7 days of ESM data.

The data collected in each survey were:

  • ratings of six discrete negative emotions (sad, angry, disappointed, ashamed, anxious, stressed) on a scale of 1-100 in response to thinking about the participant’s grade on the exam

  • ratings of the use of six ER strategies (rumination, distraction, reappraisal, expressive suppression, social sharing, acceptance) on a 7-point scale

  • score on the students’ test, (dichotimized in the analysis into passing and failing groups).

I chose to reproduce this result for two reasons. The first and primary reason is that I am collecting EMA data where I plan to do a similar type of lagged panel model. Reproducing this analysis will provide helpful practice in this type of modeling. The second reason is that I am interested in relationships between emotion differentiation abilities and emotion regulation processes. Thus, the results of this paper are of personal importance to me.

The Github repository containing all the materials for this reproduction can be found here. The original research report PDF can be found here.

Methods

Power Analysis

Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.

Sample

Planned sample size and/or termination rule, sampling frame, known demographics if any, preselection rules if any.

The sample for this dataset consisted of first-year Belgian undergraduate students. The planned N was 100 participants which would have 80% power to detect a medium effect (r = .30, α = .05). The final N for the study was 101 participants (14 males; age: M = 18.64, SD = 1.45). Participans were recruited through a university research participation program and through social media. The authors had a predetermined rule for omission of participants with less than 50% completion, but no participants met this criterion so all were included in analysis. Participants were awarded compensation on a basis commensurate with their study completion percentage.

Materials

All materials - can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.

The materials for the study are taken directly from the original paper and copied below:

Negative emotion.

Six emotions (sad, angry, disappointed, ashamed, anxious, stressed) were assessed on a 100-point scale (1 = not at all, 100 = very much). The item stem was “When you think about your grades right now, how [emotion] are you feeling?” (RKF = .99, RC = .74). In this study, we updated this measure to include emotions relevant to the context of receiving learning outcomes (Pekrun, 2006). We kept “sad,” “angry,” “anxious,” and “stressed” from Study 1, as the former three are also learning-related emotions (Pekrun, 2006), and continuity across studies allowed for comparison. However, differentiation should replicate across the inclusion of different emotions if each of the emotions provides new information. We added “disappointed” and “ashamed” because of their centrality in retrospectively evaluating learning outcomes (Pekrun, 2006).

Negative-emotion differentiation.

As in Study 1, we took the ICC between negative emotions within-person across measurement occasions, applied a Fisher’s z transformation, and then reverse scored it so higher numbers equaled higher differentiation. There were no negative ICCs.

Emotion-regulation strategies.

We assessed six strategies on a 7-point scale (0 = not at all, 6 = very much). The item stem was “Since the last beep, have you . . .” Five strategies from Study 1 were reworded to assess grade-relevant regulation: rumination (“ruminated about your grades?”), distraction (“distracted yourself from your grades and the associated emotions?”), reappraisal (“looked at your grades or the emotions that go with them from another perspective?”), expressive suppression (“suppressed the outward expression of your emotions about your grades?”), and social sharing (“talked to others about your grades and the associated emotions?”). We also included acceptance (“accepted your emotions about your grades the way they are?”).

Percentage passed.

For each subject, participants reported scores out of 20, with 10 and above being a passing grade and below 10 a failing grade. Failing requires retaking the exam later in the year or, in the case of too many failures, termination of enrollment. Given the clear emotional line at passing, we dichotomized scores on each subject as fail (1–9) or pass (10–20) and calculated the percentage of subjects passed across all subjects taken. This percentage variable was highly correlated with mean score out of 20 across exams (r = .90), and we found no differences in reported results when using mean score instead of percentage passed. In the baseline survey, we assessed participants’ expectations about their upcoming exam grades using the same measure. We used this to compute an expected-percentage-passed variable. Including both expected and actual passing percentage, or the difference between actual and expected passing percentage, did not substantively change our results. Thus, we focus on actual passing percentage.

Procedure

Can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.

The procedure for data collection is copied from the original article below:

Three days before receiving results, participants came to a lab session where they were trained on the ESM protocol. Participants were told that the study was about emotions and exams but were not given details about specific hypotheses. They then completed the ESM phase. On results-release day, within a 2-hr period, students were notified by e-mail that results were available in an online portal and asked to check them immediately. On this day, participants were sent a link to an online survey asking them to report their grade for each subject. For the ESM protocol, participants with a compatible personal Android phone installed mobileQ (N = 28). Other participants were given a research-only smartphone (N = 73). Participants completed 9 consecutive days of experience sampling: 2 days before the results release and 7 days after. We used a stratified random-interval scheme that sent a random signal within 10 equal intervals between 10:00 a.m. and 10:00 p.m. There was some variability in when results were released: Participants received their results between surveys 21 and 28 of 90. We were interested in regulation in response to results, and thus we included only post-results surveys, meaning that participants received between 63 and 70 surveys (M = 68.69). Participants received a signal on average every 71.9 min (SD = 29.8) and completed an average of 90.5% of signals (SD = 7.8%).

Analysis Plan

Can also quote directly, though it is less often spelled out effectively for an analysis strategy section. The key is to report an analysis strategy that is as close to the original - data cleaning rules, data exclusion rules, covariates, etc. - as possible.

Clarify key analysis of interest here You can also pre-specify additional analyses you plan to do.

The data analysis conducted in the original study consisted of two model structures built separately with each of the six emotion regulation strategies, resulting in twelve total models. The description of the data analytic strategy from the original article is copied below:

Data-analytic strategy

As in Study 1, we used lme4 (Bates et al., 2015) to fit mixed-effects models and standardized variables for analyses. We ran two-level models, with measurement occasions (N = 6,282) nested within persons (N = 101). In these models, we included percentage pass as a proxy for the emotional intensity of the stimulus. However, because we did not have the necessary statistical power, we did not estimate a three-way interaction with this variable. Strategies and negative emotion were measured at the occasion level, and differentiation and percentage passed at the person level. We found no substantive differences in either model when person-level negative emotion was included, but we included this variable in Model 1 to replicate Study 1.

Model 1: emotion differentiation as a predictor of emotion-regulation strategies.

In Model 1, we used differentiation, percentage passed, and negative emotion, which were grand-mean centered, to predict each strategy separately (six models). We included random intercepts per participant.

Model 2: Emotion Differentiation × Emotion Regulation Strategies predicting negative emotion.

In Model 2, we used differentiation, regulation, their cross-level interaction, and percentage passed to predict negative emotion (separately for each strategy; six models). We included lagged negative emotion (at the previous time point) to model emotional change, again excluding overnight lags. We person-mean-centered regulation and lagged emotion and grand-mean-centered differentiation and percentage passed. We included random intercepts per participant. For each participant, we included random slopes for regulation and lagged emotion, and we allowed these slopes to covary. There was one exception to this strategy: The acceptance model would not converge until we removed the random slope for acceptance, so we report this model with this random slope omitted.

Differences from Original Study

Explicitly describe known differences in sample, setting, procedure, and analysis plan from original study. The goal, of course, is to minimize those differences, but differences will inevitably occur. Also, note whether such differences are anticipated to make a difference based on claims in the original article or subsequent published research on the conditions for obtaining the effect.

Since this is an analysis reproduction, the data will be exactly the same. I plan to run this analysis using lme4 just as the authors did. I will also conduct the same analysis using the brms Bayesian Regression package in R so I can compare the credible intervals and expected values of the posterior distributions with the confidence intervals and point estimates given by the lme4 linear mixed model framework.

Methods Addendum (Post Data Collection)

You can comment this section out prior to final report with data collection.

Actual Sample

Sample size, demographics, data exclusions based on rules spelled out in analysis plan

Differences from pre-data collection methods plan

Any differences from what was described as the original plan, or “none”.

Results

Data preparation

Data preparation following the analysis plan.

Load Relevant Libraries and Functions

rm(list = ls())
library(tidyverse)
## ── Attaching packages ────────────────────────────── tidyverse 1.2.1 ──
## ✔ ggplot2 3.2.1     ✔ purrr   0.3.3
## ✔ tibble  2.1.3     ✔ dplyr   0.8.3
## ✔ tidyr   1.0.0     ✔ stringr 1.4.0
## ✔ readr   1.3.1     ✔ forcats 0.4.0
## ── Conflicts ───────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
library(haven)
library(psych)
## 
## Attaching package: 'psych'
## The following objects are masked from 'package:ggplot2':
## 
##     %+%, alpha
library(here)
## here() starts at /Users/ashish/files/phd_coursework/fall_2019/psy_251_methods/replication/kalokerinos2019
library(broom.mixed)
library(glue)
## 
## Attaching package: 'glue'
## The following object is masked from 'package:dplyr':
## 
##     collapse
library(lmerTest)
## Loading required package: lme4
## Loading required package: Matrix
## 
## Attaching package: 'Matrix'
## The following objects are masked from 'package:tidyr':
## 
##     expand, pack, unpack
## 
## Attaching package: 'lmerTest'
## The following object is masked from 'package:lme4':
## 
##     lmer
## The following object is masked from 'package:stats':
## 
##     step
theme_set(theme_minimal())

Import data

df_raw <- read_sav(here("data/Study 2_exam.sav"))

Data exclusion / filtering

Prepare data for analysis - create columns etc.

Compute ICC of negative emotions as a measure of emotion differentiation for each participant

Column names of particular emotion valence

neg_emo_cols <- c("emotion_sad", "emotion_angry", "emotion_disapp",
                  "emotion_ashamed", "emotion_anxious", "emotion_stressed")
pos_emo_cols <- c("emotion_proud", "emotion_happy", "emotion_content", "emotion_relief")

Create negative and positive composite scores Create lagged emotion intensity variables excluding overnight lags

df_raw_post_exam <- df_raw %>%
  # ---- Keep only reports after exam results were received ---- #
  filter(exam_beepnum >= 0)
df <- df_raw_post_exam %>% 
  # # ---- Grand mean center emotion ---- #
  # mutate(
  #   cntr_emotion_sad = scale(emotion_sad, T, F),
  #   cntr_emotion_angry = scale(emotion_angry, T, F),
  #   cntr_emotion_disapp = scale(emotion_disapp, T, F),
  #   cntr_emotion_ashamed = scale(emotion_ashamed, T, F),
  #   cntr_emotion_anxious = scale(emotion_anxious, T, F),
  #   cntr_emotion_stressed = scale(emotion_stressed, T, F),
  #   cntr_emotion_proud = scale(emotion_proud, T, F),
  #   cntr_emotion_happy = scale(emotion_happy, T, F),
  #   cntr_emotion_content = scale(emotion_content, T, F),
  #   cntr_emotion_relief = scale(emotion_relief, T, F),
  # ) %>% 
  # ---- Create lagged emotion vars ---- #
  mutate(
    lag_emotion_sad = lag(emotion_sad),
    lag_emotion_angry = lag(emotion_angry),
    lag_emotion_disapp = lag(emotion_disapp),
    lag_emotion_ashamed = lag(emotion_ashamed),
    lag_emotion_anxious = lag(emotion_anxious),
    lag_emotion_stressed = lag(emotion_stressed),
    lag_emotion_proud = lag(emotion_proud),
    lag_emotion_happy = lag(emotion_happy),
    lag_emotion_content = lag(emotion_content),
    lag_emotion_relief = lag(emotion_relief)
  ) %>% 
  # ---- Create lagged strategy vars ---- #
  mutate(
    lag_ER_rumination = lag(ER_rumination),
    lag_ER_distraction = lag(ER_distraction),
    lag_ER_acceptance = lag(ER_acceptance),
    lag_ER_reapp = lag(ER_reapp),
    lag_ER_supp = lag(ER_supp),
    lag_ER_soc_sharing = lag(ER_soc_sharing)
  ) %>% 
  # ---- Create variable for the day of month to use for removing overnight lags ---- #
  mutate(
    date = strftime(ScheduledTime, "%d")
  ) %>% 
  # ---- Create emotion intensity composite scores ---- #
  mutate(
    neg_emo_mean = rowMeans(df_raw_post_exam[,neg_emo_cols], na.rm = T),
    pos_emo_mean = rowMeans(df_raw_post_exam[,pos_emo_cols], na.rm = T)
  ) %>% 
  group_by(Participant) %>% 
  mutate(c_neg_emo_mean = scale(neg_emo_mean, T, F),
         c_pos_emo_mean = scale(pos_emo_mean, T, F)) %>% 
  ungroup %>% 
  # ---- Create lagged emotion intensity composite scores ---- #
  # mutate(
  #   lag_neg_emo_mean = rowMeans(df[,paste0("lag_", neg_emo_cols)]),
  #   lag_pos_emo_mean = rowMeans(df[,paste0("lag_", pos_emo_cols)])
  # ) 
  mutate(
    lag_neg_emo_mean = rowMeans(select(.,paste0("lag_", neg_emo_cols))),
    lag_pos_emo_mean = rowMeans(select(.,paste0("lag_", pos_emo_cols)))
  ) %>% 
  # ---- Create person level mean negative emotion score ---- #
  group_by(Participant) %>% 
  mutate(person_neg_emo_mean = mean(neg_emo_mean, na.rm = T)) %>% 
  
  
  # ---- Person scale lagged regulation strategies ---- #
  mutate_at(vars(matches("^lag_ER.*")), scale) %>% 
  # ---- Person scale regulation strategies ---- #
  mutate_at(vars(matches("^ER_.*")), scale) %>%
  # ---- Person scale lagged emotion ---- #
  mutate_at(vars(matches("^lag_.*_emo.*")), scale) %>% 
  ungroup %>% 
  # ---- Remove overnight lags ---- #
  arrange(Participant, beepnum) %>% 
  group_by(Participant, date) %>% 
  mutate_at(vars(matches("^.*lag.*$")),
            function(x){ifelse(row_number() == 1, NA, x)}) %>% 
  ungroup

Function to calculate ICC stats in row form

compute_icc <- function(dat) {
  dat %>%
    select(neg_emo_cols) %>% 
    irr::icc(model="twoway", unit = "average") %>%       
    unlist() %>%        
    t() %>%              
    as_tibble() %>%      
    select(              
      stimuli = subjects,
      raters, 
      icc = value,       
      lbound, 
      ubound
    ) %>%
    mutate_at(vars(stimuli, raters), as.integer) %>% 
    mutate_at(vars(icc:ubound), as.numeric)
}

Create dataframe of ICCs for each participant

icc_df <- df %>%
  group_by(Participant) %>%              
  nest() %>%                             
  mutate(icc = map(data, compute_icc)) %>%    
  unnest(icc) %>%                        
  select(-data) %>% 
  ungroup %>% 
  mutate(
    ed_score = (fisherz(icc)*-1)
    )
  
# icc_df %>% View

Merge ICC dataframe to full data

df <- icc_df %>% 
  select(Participant, icc, ed_score) %>% 
  right_join(df, by="Participant")

Initial data viz

Visualizing participant compliance

df %>% 
  group_by(Participant) %>% 
  summarize(proportion_complete = sum(!is.na(ExecutionTime))/90) %>% 
  ungroup %>% 
  ggplot(aes(x = proportion_complete)) + 
  geom_histogram(binwidth = .02) +
  # geom_density() + 
  labs(x = "Proportion complete", x = "Number of participants")

Visualizing emotion characteristics

To what degree did people experience different emotions?

df %>% 
  group_by(Participant) %>% 
  summarize_at(vars(starts_with("emotion_")), mean, na.rm=T) %>% 
  pivot_longer(cols = -Participant, names_to = "emotion_type", values_to = "emotion_rating") %>% 
  mutate(emotion_type = str_replace_all(emotion_type, "emotion_", ""),
         emotion_type = reorder(emotion_type, emotion_rating, mean)) %>% 
  
  ggplot(aes(x = emotion_type, y = emotion_rating)) +
  geom_bar(stat="summary") +
  theme(axis.text.x = element_text(angle=90)) +
  labs(x = "Emotion type", y = "Mean emotion intensity")
## No summary function supplied, defaulting to `mean_se()

How were different emotions related to the percentage of exams passed?

df_emotion_long <- df %>% 
  group_by(Participant, perc_pass) %>% 
  summarize_at(vars(starts_with("emotion_")), mean, na.rm=T) %>% 
  pivot_longer(cols = c(-Participant, -perc_pass), 
               names_to = "emotion_type", values_to = "emotion_rating") %>% 
  mutate(emotion_type = str_replace_all(emotion_type, "emotion_", ""),
         emotion_type = reorder(emotion_type, emotion_rating, mean))

df_emotion_long  %>% 
  ggplot(aes(x = perc_pass, y = emotion_rating, color = emotion_type)) +
  geom_jitter(alpha = .2) +
  geom_smooth(se=F) +
  theme(axis.text.x = element_text(angle=90)) +
  labs(x = "Proportion of exams passed", y = "Emotion intensity", color = "")
## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

df_emotion_long  %>% 
  ggplot(aes(x = perc_pass, y = emotion_rating, color = emotion_type)) +
  facet_grid(emotion_type ~.) +
  geom_jitter(alpha = .2) +
  geom_smooth() +
  theme(axis.text.x = element_text(angle=90)) +
  labs(y = "Emotion intensity", x = "Percantage of exams passed")
## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

How did different emotions unfold over time?

df %>% 
  filter(exam_beepnum >= 0) %>% 
  group_by(Participant) %>% 
  ggplot(aes(x = exam_beepnum, y = neg_emo_mean)) + 
  geom_jitter(alpha = .2, size = .4) +
  geom_smooth() + 
  geom_smooth(method="lm", linetype="dotted") +
  labs(x = "EMA beep number", y = "Negative emotion mean")
## `geom_smooth()` using method = 'gam' and formula 'y ~ s(x, bs = "cs")'
## Warning: Removed 656 rows containing non-finite values (stat_smooth).

## Warning: Removed 656 rows containing non-finite values (stat_smooth).
## Warning: Removed 656 rows containing missing values (geom_point).

perc_pass_summary <- df %>% 
  group_by(Participant) %>% 
  summarize(perc_pass = mean(perc_pass, na.rm = T)) %>% 
  summarize(mean = mean(perc_pass, na.rm = T),
            sd = sd(perc_pass, na.rm = T))

df %>% 
  mutate(
    perc_pass_group = case_when(
      perc_pass > (perc_pass_summary$mean + perc_pass_summary$sd) ~ "high",
      perc_pass < (perc_pass_summary$mean - perc_pass_summary$sd) ~ "low",
      perc_pass < (perc_pass_summary$mean + perc_pass_summary$sd) &
        perc_pass > (perc_pass_summary$mean - perc_pass_summary$sd) ~"mid"
    )
  ) %>% 
  filter(perc_pass_group == "mid") %>% 
  group_by(Participant, exam_beepnum) %>% 
  summarize_at(vars(starts_with("emotion_")), mean, na.rm=T) %>% 
  pivot_longer(cols = c(-Participant, -exam_beepnum), names_to = "emotion_type", values_to = "emotion_rating") %>% 
  mutate(emotion_type = str_replace_all(emotion_type, "emotion_", ""),
         emotion_type = reorder(emotion_type, emotion_rating, mean))  %>% 
  
  ggplot(aes(x = exam_beepnum, y = emotion_rating, color = emotion_type)) +
  geom_jitter(alpha = .2, size = .1) +
  geom_smooth(se=F) +
  theme(axis.text.x = element_text(angle=90)) +
  labs(x = "Ping number relative to exam result", y = "Emotion intensity", color = "")
## `geom_smooth()` using method = 'gam' and formula 'y ~ s(x, bs = "cs")'
## Warning: Removed 3720 rows containing non-finite values (stat_smooth).
## Warning: Removed 3720 rows containing missing values (geom_point).

Emotion over time broken down by emotion type

df %>% 
  group_by(Participant, exam_beepnum) %>% 
  summarize_at(vars(starts_with("emotion_")), mean, na.rm=T) %>% 
  pivot_longer(cols = c(-Participant, -exam_beepnum), names_to = "emotion_type", values_to = "emotion_rating") %>% 
  mutate(emotion_type = str_replace_all(emotion_type, "emotion_", ""),
         emotion_type = reorder(emotion_type, emotion_rating, mean))  %>% 
  
  ggplot(aes(x = exam_beepnum, y = emotion_rating, color = emotion_type)) +
  facet_grid(emotion_type ~.) +
  geom_jitter(alpha = .1, size = .2) +
  geom_smooth() +
  theme(axis.text.x = element_text(angle=90)) +
  labs(x = "Ping number relative to exam result", y = "Emotion intensity")
## `geom_smooth()` using method = 'gam' and formula 'y ~ s(x, bs = "cs")'
## Warning: Removed 6560 rows containing non-finite values (stat_smooth).
## Warning: Removed 6560 rows containing missing values (geom_point).

What is the relationship between percentage of exams passed and negative emotion?

df %>% 
  group_by(Participant, perc_pass) %>% 
  summarize_at(vars(starts_with("emotion_")), mean, na.rm=T) %>% 
  pivot_longer(cols = c(-Participant, -perc_pass), 
               names_to = "emotion_type", values_to = "emotion_rating") %>% 
  filter(emotion_type %in% neg_emo_cols) %>% 
  mutate(emotion_type = str_replace_all(emotion_type, "emotion_", ""),
         emotion_type = reorder(emotion_type, emotion_rating, mean))  %>% 
  summarize(mean_neg_emo = mean(emotion_rating),
            perc_pass = perc_pass[1]) %>% 
  
  ggplot(aes(x = perc_pass, y = mean_neg_emo)) +
  geom_jitter(alpha = .2) +
  geom_smooth() +
  labs(x = "Percentage of exams passed", y = "Mean negative emotion")
## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

How is negative emotion differentiation related to the amount of negative emotion experienced?

Interestingly, it appears that average emotion intensity goes up and emotion differentiation skills go up. One might expect that we would see lower emotional intensity for better differentiators since emotion differentiation is considered an adaptive skill.

df %>% 
  # ---- get a mean emotion score for each emotion for each participant ---- #
  group_by(Participant, ed_score) %>% 
  summarize_at(vars(starts_with("emotion_")), mean, na.rm=T) %>% 
  select(-pos_emo_cols) %>% 
  # ---- elongate ---- #
  pivot_longer(cols = c(-Participant, -ed_score), names_to = "emotion_type", values_to = "emotion_rating") %>% 
  mutate(emotion_type = str_replace_all(emotion_type, "emotion_", ""),
         emotion_type = reorder(emotion_type, emotion_rating, mean)) %>% 
  
  ggplot(aes(x = ed_score, y = emotion_rating, color = emotion_type)) +
  geom_point(alpha = .2) +
  geom_smooth(method = "loess", se=F) +
  geom_smooth(aes(group = 1), linetype = "dotted") +
  labs(y = "Emotion intensity", x = "Emotion differentiation score", color = "Negative emotion")
## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

Visualizing regulation characteristics

To what degree did people use different strategies?

df %>% 
  group_by(Participant) %>% 
  summarize_at(vars(starts_with("ER_")), mean, na.rm=T) %>% 
  pivot_longer(cols = -Participant, names_to = "strategy_type", values_to = "strategy_rating") %>% 
  mutate(strategy_type = str_replace_all(strategy_type, "ER_", ""),
         strategy_type = reorder(strategy_type, strategy_rating, mean)) %>% 
  
  ggplot(aes(x = strategy_type, y = strategy_rating)) +
  geom_bar(stat="summary") +
  theme(axis.text.x = element_text(angle=90)) +
  scale_y_continuous(limits = c(0,6)) +
  labs(x = "Regulation strategy", y = "Mean strategy use")
## Warning: Removed 304 rows containing non-finite values (stat_summary).
## No summary function supplied, defaulting to `mean_se()

How were different strategies related to the percentage of exams passed?

df %>% 
  group_by(Participant, perc_pass) %>% 
  summarize_at(vars(starts_with("ER_")), mean, na.rm=T) %>% 
  pivot_longer(cols = c(-Participant, -perc_pass), names_to = "strategy_type", values_to = "strategy_rating") %>% 
  mutate(strategy_type = str_replace_all(strategy_type, "ER_", ""),
         strategy_type = reorder(strategy_type, strategy_rating, mean))  %>% 
  
  ggplot(aes(x = perc_pass, y = strategy_rating, color = strategy_type)) +
  geom_jitter(alpha = .2) +
  geom_smooth(method = "lm", se=F) +
  theme(axis.text.x = element_text(angle=90)) +
  labs(x = "Percentage of exams passed", y = "emotion intensity")
## Warning: Removed 41 rows containing non-finite values (stat_smooth).
## Warning: Removed 41 rows containing missing values (geom_point).

How does emotion intensity at time t-1 relate to ER strategy at time t

df %>% 
  pivot_longer(cols = starts_with("ER_"), names_to = "strategy_type", values_to = "strategy_rating") %>% 
  
  ggplot(aes(x = lag_neg_emo_mean, y = strategy_rating)) +
  facet_grid(strategy_type~.) +
  geom_jitter(alpha = .05, size = .5) +
  geom_smooth(method = "lm") +
  labs(x = "Emotion intensity at t-1", y = "Use of ER strategy")
## Warning: Removed 12372 rows containing non-finite values (stat_smooth).
## Warning: Removed 12372 rows containing missing values (geom_point).

Does ED score predict use of strategies?

df %>% 
  pivot_longer(cols = starts_with("ER_"), names_to = "strategy_type", values_to = "strategy_rating") %>% 
  
  ggplot(aes(x = ed_score, y = strategy_rating)) +
  facet_grid(strategy_type~.) +
  geom_jitter(alpha = .05, size = .5) +
  geom_smooth(method = "lm") +
  labs(x = "Emotion differentiation score", y = "Use of ER strategy")
## Warning: Removed 6427 rows containing non-finite values (stat_smooth).
## Warning: Removed 6427 rows containing missing values (geom_point).

### Confirmatory analysis

The analyses as specified in the analysis plan.

Side-by-side graph with original graph is ideal here

Starting the modeling …

build_model1 <- function(x){
  lmer(
    as.formula(glue::glue(
    "scale({x}) ~ scale(ed_score) + 
         scale(perc_pass) + 
         scale(person_neg_emo_mean) +
         (1 | Participant)")), df) %>%
    broom.mixed::tidy() %>% 
    mutate(strategy = x) %>% 
    select(strategy, everything())
}
map(names(df)[str_detect(names(df), "^ER_")], build_model1) %>%
  bind_rows 
## boundary (singular) fit: see ?isSingular
## boundary (singular) fit: see ?isSingular
## boundary (singular) fit: see ?isSingular
## boundary (singular) fit: see ?isSingular
## boundary (singular) fit: see ?isSingular
## boundary (singular) fit: see ?isSingular
## # A tibble: 36 x 9
##    strategy  effect group term   estimate std.error statistic    df p.value
##    <chr>     <chr>  <chr> <chr>     <dbl>     <dbl>     <dbl> <dbl>   <dbl>
##  1 ER_rumin… fixed  <NA>  (Int… -5.44e-18    0.0128 -4.26e-16 6147.   1.000
##  2 ER_rumin… fixed  <NA>  scal… -2.50e-17    0.0128 -1.95e-15 6147.   1.000
##  3 ER_rumin… fixed  <NA>  scal…  3.38e-17    0.0162  2.08e-15 6147.   1.000
##  4 ER_rumin… fixed  <NA>  scal…  1.05e-17    0.0161  6.50e-16 6147.   1.000
##  5 ER_rumin… ran_p… Part… sd__…  6.20e-17   NA      NA          NA   NA    
##  6 ER_rumin… ran_p… Resi… sd__…  1.00e+ 0   NA      NA          NA   NA    
##  7 ER_distr… fixed  <NA>  (Int… -7.00e-18    0.0135 -5.19e-16 5521.   1.000
##  8 ER_distr… fixed  <NA>  scal… -1.36e-17    0.0133 -1.02e-15 5521.   1.000
##  9 ER_distr… fixed  <NA>  scal…  4.06e-17    0.0170  2.38e-15 5521.   1.000
## 10 ER_distr… fixed  <NA>  scal…  5.39e-17    0.0166  3.24e-15 5521.   1.000
## # … with 26 more rows
build_model2 <- function(x){
  # person mean center regulation & lagged emotion
  df %>% 
  lmer(
    as.formula(glue::glue(
    "scale(c_neg_emo_mean) ~ scale(ed_score) +
    scale(ed_score)*{x} +
    scale(perc_pass) + 
    lag_neg_emo_mean +
    (lag_neg_emo_mean + {x} | Participant)")), .) %>%
    broom.mixed::tidy() %>% 
    mutate(strategy = x) %>% 
    select(strategy, everything())
}
map(names(df)[str_detect(names(df), "^ER_")], build_model2) %>%
  bind_rows %>% 
  mutate(estimate = round(estimate, 2))
## boundary (singular) fit: see ?isSingular
## Warning: Model failed to converge with 2 negative eigenvalues: -1.8e-02
## -3.8e+01
## boundary (singular) fit: see ?isSingular
## boundary (singular) fit: see ?isSingular
## Warning: Model failed to converge with 1 negative eigenvalue: -6.1e-02
## boundary (singular) fit: see ?isSingular
## Warning: Model failed to converge with 1 negative eigenvalue: -4.0e+01
## boundary (singular) fit: see ?isSingular
## boundary (singular) fit: see ?isSingular
## Warning: Model failed to converge with 2 negative eigenvalues: -6.7e-02
## -1.4e+01
## # A tibble: 78 x 9
##    strategy effect group term  estimate std.error statistic     df
##    <chr>    <chr>  <chr> <chr>    <dbl>     <dbl>     <dbl>  <dbl>
##  1 ER_rumi… fixed  <NA>  (Int…     0.01    0.0114     0.758 5046. 
##  2 ER_rumi… fixed  <NA>  scal…     0       0.0115    -0.268 5025. 
##  3 ER_rumi… fixed  <NA>  ER_r…     0.2     0.0296     6.77    92.8
##  4 ER_rumi… fixed  <NA>  scal…     0.01    0.0116     0.995 4855. 
##  5 ER_rumi… fixed  <NA>  lag_…     0.35    0.0344    10.2    101. 
##  6 ER_rumi… fixed  <NA>  scal…    -0.08    0.0316    -2.48   102. 
##  7 ER_rumi… ran_p… Part… sd__…     0      NA         NA       NA  
##  8 ER_rumi… ran_p… Part… sd__…     0.31   NA         NA       NA  
##  9 ER_rumi… ran_p… Part… sd__…     0.24   NA         NA       NA  
## 10 ER_rumi… ran_p… Part… cor_…   NaN      NA         NA       NA  
## # … with 68 more rows, and 1 more variable: p.value <dbl>

Exploratory analyses

Any follow-up analyses desired (not required).

# df_scaled <- df %>% 
#   mutate_at(vars(matches(".*ER.*|.*emotion.*")), scale)
# 
# model <- '
#     level: 1
#         neg_emo =~ lag_emotion_angry + lag_emotion_disapp + lag_emotion_ashamed + lag_emotion_sad + lag_emotion_anxious
#         neg_emo ~ ER_distraction
#     level: 2
#         neg_emo =~ lag_emotion_angry + lag_emotion_disapp + lag_emotion_ashamed + lag_emotion_sad + lag_emotion_anxious
# '
# fit <- lavaan::sem(model, data = df_scaled, cluster = "Participant", fixed.x = FALSE)
# summary(fit)

Discussion

Summary of Replication Attempt

Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.

Commentary

Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.

Analysis requirements

sessionInfo()
## R version 3.5.3 (2019-03-11)
## Platform: x86_64-apple-darwin15.6.0 (64-bit)
## Running under: macOS Mojave 10.14.6
## 
## Matrix products: default
## BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] lmerTest_3.1-0    lme4_1.1-21       Matrix_1.2-15    
##  [4] glue_1.3.1        broom.mixed_0.2.4 here_0.1         
##  [7] psych_1.8.12      haven_2.1.1       forcats_0.4.0    
## [10] stringr_1.4.0     dplyr_0.8.3       purrr_0.3.3      
## [13] readr_1.3.1       tidyr_1.0.0       tibble_2.1.3     
## [16] ggplot2_3.2.1     tidyverse_1.2.1  
## 
## loaded via a namespace (and not attached):
##  [1] Rcpp_1.0.3        lubridate_1.7.4   lattice_0.20-38  
##  [4] utf8_1.1.4        assertthat_0.2.1  zeallot_0.1.0    
##  [7] rprojroot_1.3-2   digest_0.6.22     R6_2.4.0         
## [10] cellranger_1.1.0  plyr_1.8.4        backports_1.1.5  
## [13] evaluate_0.14     coda_0.19-2       httr_1.4.1       
## [16] pillar_1.4.2      rlang_0.4.1       lazyeval_0.2.2   
## [19] readxl_1.3.1      rstudioapi_0.10   minqa_1.2.4      
## [22] nloptr_1.2.1      rmarkdown_1.16    labeling_0.3     
## [25] splines_3.5.3     foreign_0.8-71    TMB_1.7.15       
## [28] munsell_0.5.0     broom_0.5.2       numDeriv_2016.8-1
## [31] compiler_3.5.3    modelr_0.1.5      xfun_0.10        
## [34] pkgconfig_2.0.3   mnormt_1.5-5      mgcv_1.8-27      
## [37] htmltools_0.4.0   tidyselect_0.2.5  lpSolve_5.6.13.3 
## [40] fansi_0.4.0       crayon_1.3.4      withr_2.1.2      
## [43] MASS_7.3-51.1     grid_3.5.3        nlme_3.1-137     
## [46] jsonlite_1.6      gtable_0.3.0      lifecycle_0.1.0  
## [49] magrittr_1.5      scales_1.0.0      cli_1.1.0        
## [52] stringi_1.4.3     reshape2_1.4.3    xml2_1.2.2       
## [55] generics_0.0.2    vctrs_0.2.0       boot_1.3-20      
## [58] irr_0.84.1        tools_3.5.3       hms_0.5.2        
## [61] parallel_3.5.3    yaml_2.2.0        colorspace_1.4-1 
## [64] rvest_0.3.4       knitr_1.25