[No abstract is needed.] Each replication project will have a straightforward, no frills report of the study and results. These reports will be publicly available as supplementary material for the aggregate report(s) of the project as a whole. Also, to maximize project integrity, the intro and methods will be written and critiqued in advance of data collection. Introductions can be just 1-2 paragraphs clarifying the main idea of the original study, the target finding for replication, and any other essential information. It will NOT have a literature review – that is in the original publication. You can write both the introduction and the methods in past tense.
In the study “Ecological Momentary Assessment of Stressful Events and Negative Affect in Bulimia Nervosa”, (Goldschmidt et al., 2014) sought to examine the time course of events and affect preceding binging and purging in bulimia nervosa (BN) patients. The authors distributed surveys querying stressful events, negative affect, and bulimic events throughout the day in BN patients’ daily lives. The authors found that an increase in negative affect (rather than levels of state affect) following stressful events mediates the relationship between stress and bulimic events. This was tested through a multilevel structural equation mediation model where negative affect at time 2 mediated the relationship between stressful events at time 1 and bulimic events at time 2 while controlling for levels of negative affect at time 1. The present reproduction effort aims to reproduce this finding in the lavaan package in R.
I chose to reproduce this result for two reasons. Firstly, this project will aid me in my research by providing a practice case for a useful longitudinal, within-subject analysis technique. I am currently collecting ecological momentary assessment (EMA) data examining within-subject variation in cognitive appraisals of emotion-triggering events and the impact of various emotion regulation strategies. Since there are multiple dependent variables, the SEM framework is well-suited to modeling this dataset. Secondly, this project will allow me to validate the lavaan R package as a suitable alternative to MPlus. The authors of the original paper built their model in MPlus. However, every aspect of the model should be reproducible in lavaan. By reproducing this finding, I can compare the results of MPlus’ model coefficients to the coefficients provided by lavaan.
The Github repository containing all the materials for this reproduction can be found here. The original research report PDF can be found here.
Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.
Planned sample size and/or termination rule, sampling frame, known demographics if any, preselection rules if any.
All materials - can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.
Can quote directly from original article - just put the text in quotations and note that this was followed precisely. Or, quote directly and just point out exceptions to what was described in the original article.
Participants completed a phone-administered structured interview to determine preliminary BN status. Eligible participants came to an in-person informational session where they completed a baseline assessment of BN symptoms using the SCID-I/P (First, Spitzer, Gibbon, & Williams, 1997). For two weeks, participants filled out six semi-randomly delivered EMA surveys each day. In addition, the participants filled out daily diary surveys prior to sleep and event-contingent surveys each time they engaged in an episode of binging or purging. These surveys assessed stressful events that occurred throughout the day, levels of negative affect, and bulimic events (i.e. binging, purging by vomiting or use of laxatives).
Stressful events were meaured using the Daily Stress Inventory (DSI; Brantley, Waggoner, Jones, and Rappaport, 1987). This instrument asks participants to select stressful events that occurred from a list of 60 events and to record when and how stressful these event were. Negative affects was measured with an abridged version of the Positive and Negative Affect Schedule (PANAS; Watson, Clark, & Tellegan, 1988).
Can also quote directly, though it is less often spelled out effectively for an analysis strategy section. The key is to report an analysis strategy that is as close to the original - data cleaning rules, data exclusion rules, covariates, etc. - as possible.
Clarify key analysis of interest here You can also pre-specify additional analyses you plan to do.
Explicitly describe known differences in sample, setting, procedure, and analysis plan from original study. The goal, of course, is to minimize those differences, but differences will inevitably occur. Also, note whether such differences are anticipated to make a difference based on claims in the original article or subsequent published research on the conditions for obtaining the effect.
You can comment this section out prior to final report with data collection.
Sample size, demographics, data exclusions based on rules spelled out in analysis plan
Any differences from what was described as the original plan, or “none”.
Data preparation following the analysis plan.
The analyses as specified in the analysis plan.
Side-by-side graph with original graph is ideal here
Any follow-up analyses desired (not required).
Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.
Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.