Reproducibility Report for Uncovering the structure of self-regulation through data-driven ontology discover by Eisenberg et al. (2019, Nature Communication)
Author
Mia Jimenez-Fuentes (miafuen@stanford.edu)
Published
December 14, 2025
Introduction
In this report I attempt to reproduce 2 key findings presented in Uncovering the structure of self-regulation through data-driven ontology discover, where Eisenberg et al. (2019) explore the idea of defining and measuring the concept of self-regulation. The overarching motivation for this paper was to explore the inconsistency in the way that concepts like self-regulation are defined in the literature. Given the various ways that researchers define and measure self-regulation has led to theoretical fragmentation in the field. The key analyses of interest here are factor anlalyses which identify latent dimensions underlying nearly 200 task and survey variables, and the clustering analyses, which map how those measures group together.
Justification for choice of study
In Uncovering the structure of self-regulation through data-driven ontology discover, Eisenberg et al. (2019) provides a compelling example of how large-scale, complicated, data-driven methods can be used to clarify the meaning and measurement of psychological constructs that are otherwise incredibly difficult to pin down. In this paper, the authors explore the idea of self-regulation. This concept has been tied to various outcomes ranging from health to education, but the field has consistently debated how to measure it and, more broadly, if it is a coherent construct at all. This idea comes up frequently in discussions around constructs like these, especially so in conversations around skills of executive functions. This paper tackles this issue by integrating a broad battery of tasks, surveys, and real-world outcomes, offering a unique opportunity to examine how different measures related to one another.
Reproducing the results of this study will be valuable for several reasons. To begin, the authors’ methods are broad and complex - involving large-scale data collection, exploratory factor analysis, clustering, and predictive modeling. Reproducing their approach offers an opportunity to practice key skills in dimensional reduction, psychometrics, and prediction. Furthermore, this paper raises important conceptual issues about the reliability and validity of common measures in psychology - issues central to my own developing research interests. Overall, this work is methodologically complex and incredibly theoretically relevant. It will allow me to deepen my understanding of advanced data techniques while also informing my ongoing exploration about the measurement and meaning of related constructs in research.
Anticipated challenges
One challenge I anticipate surrounds the complexity of the analytic pipeline - the authors rely on several advanced methods and small deviations in parameter choices or implementation could lead to different results. Secondly, the dataset includes nearly 200 independent variables derived from the survey and behavioral tasks and reproducing the results will require careful preprocessing to match the authors’ original approach.
Links
The following links are for the project repository on Github, the original paper (which lives in the Github folder), as well as the link to the OSF preregistration of the project. * Project repository: https://github.com/psych251/eisenberg2019 * Original paper: https://github.com/psych251/eisenberg2019/blob/main/original_paper/eisenberg2019pdf.pdf * Preregistration link: https://osf.io/ecbvf/overview
Methods
Description of the steps required to reproduce the results
In order to reproduce the findings from Eisenberg 2019, there are a few key steps. First, I will need to organize all data acess and setup steps. This means, identifying the data used in the original study. The processed data file is called meaningful_varialbes_imputed.csv where the authors have done some preprocessing already to the data. From here, I will confirm descriptives about the data - that there are the right number of variables, participants, and that the data matches the file used in the original paper. Furthermore, in order to have a good foundation for the methods used and decisions made, I will identify and explore the original scripts- those within the dimensional_structure and graph_analysis folders will be most relevenat. Second, I will reproriduce Figure 2 which is their dimensional structure plot. To do this, I will compute an inter-variable correlation matrix and run exploratory factor analysis using extraction and rotation methods. Using parallel analysis, I’ll then determine the dimensional structure and plot. Finally, I will reproduce figures 3 and 4, the ontological graphs. For this, I will construct network representaiotn using the correlation matrix, apply graph-based cluserting and hierarchical grouping, and visualize the structures.
Differences from original study
The main differences from the original study surround the computing environment. The original authors use a mix of Python and R, whereas here I will stick to R for my computing. Furthermore, original imputation was not reproduced as part of this report. Instead, I am starting from the authors’ imputed dataset.
Project Progress Check 1
Measure of success
The primary success criterion is whether I can produce the two figures – specifically the sparse partial-correlation network with a clearly separated task vs survey DV structure and the subsequent factor/cluster based ontology which involves extracting latent dimensions of hierarchical communities from the space constructed in figure 2. Both outcomes will be evaluated by structure/ visual alignment with the original paper and quantitatively (by checking counts of things like dependent variables (DVs) and subjects).
Pipeline progress
All findings have been roughly reproduced! In terms of data access and setup, I have identified and loaded the author’s finalized imputed data, confirmed participant count and dependent variable structure are consistent with the original report. In terms of figure 2, I have successfully computed the partial correlation network, confirmed dependent variable count and split, and have plotted. Originally, the dependent variable split was incorrect and I found that the authors used grepl (the method I use below) to identify the surveys, but then in a later, more processed file of the data have mislabled 5 surveys as tasks. I have gone ahead and followed their labeling. Finally, I have also roughly reproduced figures 3 and 4.
original_fig2 <-here("results", "figure2", "figure2original.png")reproduced_fig2 <-here("results", "figure2", "figure2unlabeled.png")original_fig2 <-image_read(original_fig2)reproduced_fig2 <-image_read(reproduced_fig2)# viewing side by side image_append(c(original_fig2, reproduced_fig2))
original_fig3 <-here("results", "figure3", "fig3_original.png")reproduced_fig3 <-here("results", "figure3", "figure3reproduction.png")original_fig3 <-image_read(original_fig3)reproduced_fig3 <-image_read(reproduced_fig3)# viewing side by side image_append(c(original_fig3, reproduced_fig3))
Figure 3
Figure 4 setup
Figure 4
Rough reproduction of Fig 4!
df_task <- df_task_paper_match # 5 factor EFAefa_task <-fa( df_task,nfactors =5,fm ="minres",rotate ="oblimin")L_raw <-as.matrix(efa_task$loadings)dv_names <-rownames(L_raw)# cluster dvvs in factor-loading space using dynamictreecut like authors did dist_task <-dist(L_raw)hc_task <-hclust(dist_task, method ="ward.D2")clusters <-cutreeDynamic(dendro = hc_task,distM =as.matrix(dist_task),deepSplit =2,pamRespectsDendro =FALSE,minClusterSize =3)
..cutHeight not given, setting it to 3.25 ===> 99% of the (truncated) height range in dendro.
..done.
original_fig4 <-here("results", "figure4", "fig4_original.png")reproduced_fig4 <-here("results", "figure4", "figure4reproduction.png")original_fig4 <-image_read(original_fig4)reproduced_fig4 <-image_read(reproduced_fig4)# viewing side by side image_append(c(original_fig4, reproduced_fig4))
Eisenberg et al. (2019) demonstrated that self-regulation is not a unitary, unified construct but instead is comprised of a multi-dimensional ontology that can be recovered in a data-driven manner. Using exploratory factory analysis across a large battery of both survey and behavioral data, the authors identified district latent factors and organized the task and survey level measures into interpretable clusters. The goal of this reproduction exercise was to evlaute if the survey and task ontologies that were built by Eisenberg et al. (2019) could be recovered using their general analytic framework - exploratory factor analysis and hierarchical clustering. using the matched survey and task datasets provided by the authors in their repo, we successfully reproduced the core qualitative structure of both ontologies. For survey data, a 12-factor solution was supported by parallel analysis and BIC and hierarchical clustering yielded interpretable clusters that aligned closely with the themes reported in the original paper. For task data, a 5-factor solution similarly produced coherent clusters like those reported in the original piece.
While there are some visual and aesthetic features of the figures that were not reproduced here - ordering of the clusters and the apparent staircase nature of the factor loading, the overall concepts were consistent. Furthermore, the task analysis yielded one additional cluster prior to post-hoc inspection, but even then the ontology structure (defined like the original authors by factor projections, cluster coherence, and conceptual interpretation), was consistent with the original report. Thus, the primary results of the Eisenberg et al. (2019) report were success reproduced with minor deviations.
Commentary
Several insights emerged from follow-up exploration of the reproduced ontologies. First, both survey and task related analyses revealed closely related constructs that can split into multiple empirically distinct clusters. In the task ontology, two inhibition-related clusters emerged that differed in whether they were dominated by threshold parameters versus perceptual or response related drift components. This highlights that self-regulation as a concept may occupy graded space rather than discrete categories. Importantly, there were minor differences in this reproduction that do not change the final findings or interpretations (that self regulation is best characterized as a multi-dimensional ontology spanning separable. ut related processes). One broader challenge that emerged for me from this reproduction was that presentation-level decisions (ie cluster ordering, label consolidation) play a significant role in how ontologies are interpreted, and are often under-specified or deeply burried in methodological desciprtions.
