1. Background 1.1. Introduction to the ABCD dataset The Adolescent Brain Cognitive Development (ABCD) study represents a monumental, longitudinal research initiative aimed at elucidating the intricate interplay of biological, environmental, and social factors that shape brain development and cognitive functioning in children and adolescents (Volkow et al., 2018). The ABCD study spans various research domains, exploring the intricate interplay of gene-environment interactions, substance use repercussions, and identifying neurodevelopmental markers linked to mental health outcomes (Casey et al., 2018). With data from over 10,000 participants aged 9-10 at their baseline assessment, the ABCD dataset offers an unparalleled, multidimensional resource consisting of neuroimaging, cognitive, behavioral, and genetic information and comprehensive data on participants’ family, social, and educational backgrounds. This wealth of data enables researchers to examine the myriad factors that contribute to both typical and atypical neurodevelopment in an unprecedented manner. The study of brain development in children and adolescents is of utmost importance. This critical period lays the foundation for establishing cognitive, emotional, and social competencies that shape individuals’ trajectories (Giedd & Rapoport, 2010; Perlman & Pelphrey, 2010). Moreover, by uncovering the complex mechanisms that drive typical and atypical neurodevelopment, the ABCD study offers invaluable insights for understanding, preventing, and treating cognitive and behavioral dysfunctions (Leong et al., 2021). Consequently, the ABCD dataset serves as a cornerstone for advancing our knowledge of the intricate processes underlying human brain development and cognitive maturation, ultimately informing public policy, education, and healthcare interventions to promote the well-being of children and adolescents (Goyal et al., 2022).
2. Overview The substantial complexity and heterogeneity of the developing brain (T. Insel et al., 2010; Kapur et al., 2012) pose considerable challenges when studying the relationships between child and adolescent brain development, behavior, and cognitive functioning, especially in large-scale datasets like the ABCD Study. To tackle these challenges, researchers are increasingly employing machine learning-based subtyping methods to explore child and adolescent brain development and clarify the relationships between subtypes, behavior, and cognitive functioning (Nikolaidis et al., 2021). In addition, subtyping approaches allow for a more accurate and reliable representation of underlying population heterogeneity, leading to improved prediction of outcomes (Nikolaidis et al., 2022). Stratifying the children in the ABCD dataset into subgroups based on their similarities across multiple neuroimaging modalities, like resting-state functional connectivity (RSFC), structural magnetic resonance imaging (sMRI), and diffusion tensor imaging (DTI), becomes possible through the application of advanced machine learning algorithms. These algorithms can identify patterns in the high-dimensional data and partition individuals into subtypes, representing distinct groups with shared neuroimaging characteristics (Wen et al., 2022). Theoretically, these subtypes may reflect unique neurodevelopmental profiles associated with specific cognitive and psychopathology outcomes. Hence, subtyping can provide a more nuanced and precise understanding of the associations between cognitive functioning and psychopathology outcomes by revealing the distinct neural underpinnings for each subgroup (Lichenstein et al., 2022). This research aims to investigate the potential utility of subtyping approaches in addressing the question, “Does subtyping provide any meaningful information we could not have obtained before?”. Harnessing the power of machine learning-based approaches to explore functional connectivity subtypes in child and adolescent brain development within the ABCD dataset, this research aims to uncover novel insights into the intricate relationships between brain development, behavior, and cognitive functioning, ultimately contributing to a deeper understanding of typical and atypical neurodevelopmental trajectories. By examining subtypes derived from RSFC, sMRI, and DTI, this research aims to elucidate the unique and shared contributions of functional and structural brain networks to cognitive functioning and psychopathology. This multimodal approach overcomes the limitations of studying individual imaging measures in isolation, paving the way for a more holistic understanding of the brain’s role in development. 2.1. The Rationale for Utilizing Resting-State Functional Connectivity, Structural Magnetic Resonance Imaging, and Diffusion Tensor Imaging in Deriving Neuroimaging Subtypes 2.1.1. Resting-State Functional Connectivity RSFC has emerged as a robust and reliable tool for investigating the brain’s functional organization. Numerous studies have demonstrated its stability over time, making it an ideal candidate for longitudinal research designs (Reineberg et al., 2018; Shehzad et al., 2009; Van Dijk et al., 2010). Furthermore, RSFC is not affected by task performance or state changes, unlike task-based fMRI, which can be influenced by factors such as fatigue, motivation, and attention (Biswal et al., 1995). The relatively long acquisition time of resting-state data allows for a higher signal-to-noise ratio, leading to increased statistical power in detecting functional networks (Fox & Raichle, 2007). These factors contribute to the growing consensus among neuroimagers that RSFC is the most stable and reliable form of functional neuroimaging data available (Damoiseaux et al., 2006). To fully leverage RSFC’s power, it is critical to use robust and reliable methods for data analysis and interpretation, such as subtyping approaches. These approaches can help address the underlying heterogeneity of the population and achieve more accurate and reliable predictions. 2.1.2. Structural Magnetic Resonance Imaging sMRI provides detailed information about brain anatomy, including gray and white matter volumes, cortical thickness, and surface area. These measures are sensitive to individual differences in cognitive functioning and psychopathology (Bernanke et al., 2022; Kanai & Rees, 2011). Theoretically, structural imaging can capture developmental changes in brain structure that are not directly observable through functional connectivity data, offering complementary information on neurodevelopment. Subtyping approaches applied to sMRI data can reveal unique neuroanatomical profiles underlying different subtypes, further informing the understanding of the biological basis of cognitive and psychiatric disorders. Incorporating structural imaging data enhances our ability to address the heterogeneity within populations and improve the accuracy of predictions related to typical and atypical development, offering a more comprehensive understanding than functional connectivity data alone can provide. 2.1.3. Diffusion Tensor Imaging DTI measures the diffusion of water molecules along axonal fibers, enabling the visualization and quantification of white matter tracts in the brain. As white matter development is a critical aspect of brain maturation, DTI offers a relatively pure measure of brain development concerning white matter (Bihan, 2003; Vaher et al., 2022). Unlike functional connectivity data, DTI provides valuable information about the brain’s structural connectivity, crucial for efficiently transferring information between functionally specialized brain regions. The reliability and stability of DTI measures make them suitable for investigations of brain connectivity (Yuan et al., 2019). Subtyping approaches applied to DTI data can capture the heterogeneity in white matter connectivity patterns, which may underlie differences in cognitive functioning and contribute to our understanding of typical and atypical development. These insights into structural connectivity’s role in shaping developmental trajectories complement the information obtained from functional connectivity data and help elucidate the complex interplay between brain structure and cognitive outcomes. 2.2. Improving the Quality of Neuroimaging Subtyping through the IVEPR Framework: A Comprehensive Approach Data-driven clustering has led to significant advancements in identifying patterns within complex datasets (Lee et al., 2022). However, the absence of a standardized approach results in inconsistencies regarding the derived clusters’ reliability, stability, and reproducibility (Yu et al., 2019). This proposal presents a comprehensive framework for data-driven clustering, encompassing Identification, Validation, Evaluation, Prediction, and Replication (IVEPR) to ensure that subtyping provides meaningful information beyond traditional analyses. This framework aims to improve the quality of clustering research, guide future studies, and foster meaningful advancements in the field.
3. Identification: The first step involves identifying meaningful clusters within the dataset using appropriate clustering algorithms, feature extraction techniques, and parameter settings. Establishing this standardized procedure ensures the derived clusters are biologically or theoretically relevant and comparable.
4. Validation: Using a demographically matched split sample approach allows for the consistency of findings across samples to be appropriately evaluated. This direct comparison supports the initial validation of the identified clusters, ensuring they are reproducible and not merely artifacts of the data or methodological choices. Furthermore, replication across both samples demonstrates that the clusters are consistent across different samples, proving their robustness and reliability.
5. Evaluation: Using statistical techniques such as ANOVA, the framework emphasizes objective evaluation metrics to assess if meaningful differences exist across the clusters on measures of interest.
6. Prediction: Assessing the predictive utility of clusters involves comparing predictions made with and without subtypes using conditional random forests (CRFs), a powerful ensemble learning method (Breiman, 2001). CRFs are particularly suitable for this task due to their ability to model complex and non-linear relationships without distribution assumptions, reduced risk of overfitting, and the capability to account for interactions between features. Furthermore, by comparing prediction errors between CRF models with and without the subtypes, the analysis can determine if the subtypes provide meaningful additional information for better predictions, demonstrating the practical utility and generalizability of the clustering solutions.
7. Replication: Reproducibility is vital in scientific research. The IVEPR framework underscores the importance of replication through bootstrap resampling and down-sampling processes to confirm the robustness and generalizability of the identified clusters. These standardized procedures minimize the likelihood of false discoveries and enhance the credibility of the findings.
8. Research Questions 3.1. Can subtypes based separately on resting state functional connectivity, structural imaging, and diffusion tensor imaging measures improve our understanding of the underlying neural mechanisms contributing to typical and atypical cognitive functioning and psychopathology across development? Answering this question is of paramount importance for several reasons. First, subtyping allows researchers to account for the vast heterogeneity in brain structure and function, often observed in typical and atypical development (T. R. Insel & Cuthbert, 2015). By identifying distinct subgroups of individuals with similar neurobiological profiles, this question aims to better understand the complex interplay between neural mechanisms and the profiles of children that emerge from them. Finally, this approach can help refine our models of brain development to enable a more precise understanding of the environmental, genetic, and social factors that influence the emergence of these neurodevelopmental subtypes. 3.1.1. Data-driven Subtyping via Bootstrap Enhanced Leiden Community Detection The first step in the analysis involves data-driven clustering analysis that uses a reproducibility-motivated method called bootstrap-enhanced Leiden Community Detection (LCD) (DeRosa et al., 2023; Traag et al., 2019). This approach will identify subgroups of individuals with similar functional connectivity or structural characteristics based on baseline measures of RSFC networks (Gordon et al., 2016; Marek et al., 2019), diffusion tensor imaging, and sMRI. Bootstrap-enhanced LCD offers several advantages over traditional clustering methods, such as hierarchical and k-means clustering. First, it accounts for the uncertainty in the estimated connectivity matrices by employing a bootstrapping procedure, which enhances the stability and reproducibility of the identified subtypes (Nikolaidis et al., 2022, 2021). Second, it is a data-driven approach, meaning it does not rely on a priori assumptions about the number or structure of subtypes, making it less prone to biases and more suitable for exploratory analyses. 3.1.2. Subtype Classification and Feature Importance using Gradient Boosted Decision Trees and SHAPLEY Additive Explanations Once the subtypes have been identified, the next step is to evaluate their stratification using gradient-boosted decision trees (Chen & Guestrin, 2016), a robust machine learning algorithm used for multiclass classification, which has been shown to outperform other classification methods in terms of accuracy and computational efficiency (Fernández-Delgado et al., 2014). Gradient-boosted decision trees are well-suited for this analysis for several reasons. First, it can handle large, high-dimensional data sets, which is common in neuroimaging research (Woo et al., 2017). Second, it can automatically handle multicollinearity, a common challenge in neuroimaging studies (Hastie et al., 2009). Finally, to further understand the importance of each feature in the classification process, the Shapley additive explanations (SHAP) method is used (Lundberg & Lee, 2017). SHAP values provide a suitable measure of feature importance, allowing researchers to interpret the impact of each feature on the classification outcome in a consistent and interpretable manner. 3.1.3. Assessing Subtype Overlap using Chi-square Tests and Odds Ratios After classifying the individuals into different subtypes based on their functional, structural, and DTI profiles, assessing the degree of overlap between them is crucial. This overlap will be assessed using chi-square tests and odds ratios, statistical methods for determining the association between categorical variables. Using chi-square tests and odds ratios will provide an intuitive way to quantify the degree of association between different subtypes, allowing us to determine whether the identified subtypes are distinct or overlap significantly.
   3.1.4. Split-Sample Subtype Validation Validation is crucial in this analysis process to assess the identified subtypes’ reproducibility, reliability, and robustness. By validating these subtypes, we can increase our confidence that the identified subgroups are distinct and not merely the result of random fluctuations in the data. In addition, this procedure ensures that the findings are more likely to generalize to other samples and populations, thus increasing the utility of the subtypes for understanding the neural mechanisms underlying cognitive functioning and psychopathology. Validation in this context refers to using a split sample approach, which divides the sample into two parts. Then, the subtypes and their profiles obtained in Sample 1 are compared to those obtained in Sample 2 to assess the consistency of the results. This direct comparison supports the initial validation of the identified subtypes, ensuring that the subtypes are reproducible and not merely artifacts of the data or methodological choices. Furthermore, replication across both samples would demonstrate that the identified subtypes are data-driven and consistent across different samples, thus proving evidence for their robustness and reliability. 3.2. How can subtyping based on neuroimaging measures help us better understand the heterogeneity of cognitive and behavioral functioning? Several analytical tools are employed sequentially to address this question. First, subtype confirmatory factor analysis (CFA) invariance testing will evaluate whether the same cognitive and psychopathology constructs are being measured across the subtypes. This analysis ensures that any observed differences across subtypes are not due to measurement artifacts but rather reflect underlying differences in the constructs of interest (Millsap, 2011). Establishing measurement invariance is crucial before comparing the subtypes on various outcomes, as it provides a solid foundation for the subsequent analyses. If measurement invariance has been established, analysis of covariance (ANCOVA) will test for differences across the subtypes of cognitive functioning and psychopathology measures of interest. 3.3. How can subtyping based on neuroimaging measures improve our ability to predict cognitive and behavioral outcomes in children and adolescents, and do these subtypes provide meaningful information for better prediction? Conditional random forests (CRFs) are a powerful tool for addressing this question. CRFs are an ensemble learning method that combines multiple decision trees to generate more accurate and stable predictions. There are several reasons why CRFs are particularly suited to assess feature importance with factors and investigate the predictive utility of neuroimaging-based subtypes. First, unlike parametric models that make assumptions about the underlying data distribution, CRFs can model complex and non-linear relationships between predictors and outcomes without such assumptions (Cutler et al., 2007). This flexibility makes them well-suited for analyzing the heterogeneous and high-dimensional neuroimaging data used to derive subtypes. In contrast, parametric models may fail to capture the actual relationships in the data, leading to biased and less accurate predictions. Second, CRFs are less prone to overfitting than single decision trees, as they rely on aggregating multiple trees, each constructed using a random subset of the data (Breiman, 2001). This “bagging” approach reduces the overall model variance and increases stability, ensuring that the models generalize well to new, unseen data. Overfitting is a significant concern when evaluating the predictive utility of subtypes. Models that perform well on the training data but poorly on new data are of limited practical use. Third, CRFs provide an intuitive and interpretable measure of feature importance by quantifying the decrease in prediction accuracy when a specific feature is permuted or randomly shuffled (Strobl et al., 2008). This “permutation importance” allows us to assess the relative importance of subtypes in predicting cognitive and behavioral outcomes compared to other factors, such as demographic and SES variables. Understanding the relative importance of these factors can help inform targeted interventions and policy decisions. Finally, CRFs can naturally account for interactions between features by allowing decision trees to split on multiple features at different levels of the tree hierarchy (Friedman & Popescu, 2008). This capability makes them an ideal tool for investigating the complex relationships between neuroimaging-based subtypes, demographic factors, and cognitive and behavioral outcomes, providing a more comprehensive understanding of the factors contributing to these outcomes. The prediction errors (i.e., the differences between actual and predicted scores) will be compared between CRF models with and without the subtypes to evaluate whether subtypes enhance our predictive ability. A significant reduction in prediction error when subtypes are included in the model would indicate that they provide meaningful information for better prediction. Compared to gradient-boosted decision trees (GBDTs), which were used to classify subtypes and derive the most important features for distinguishing them, CRFs are a more appropriate model for assessing feature importance in this context. GBDTs are designed to boost weak learners to improve overall model performance, but they can be prone to overfitting when dealing with complex interactions and high-dimensional data (Friedman, 2001). In contrast, CRFs are more robust to overfitting and can better handle the complexity of the relationships between categorical factors. 3.4. How reproducible are these subtypes, their differences, and their predictive ability? The pervasive heterogeneity of cognitive and behavioral difficulties in children and adolescents necessitates the development of reliable and robust methods to uncover meaningful subtypes. Therefore, it is essential to demonstrate that the subtypes identified are reproducible, reliable, and robust across different samples and analyses. Furthermore, these rigorous methods ensure that this data-driven clustering approach enhances the findings’ reproducibility, reliability, and robustness. To address the overarching question of how reproducible these subtypes are, their differences, and their predictive ability, two key strategies are employed: 3.4.1. Bootstrapping Bootstrapping, a widely used statistical resampling method will be employed to evaluate the reliability, reproducibility, and stability of statistical analyses in the ABCD dataset. Each original sample will first be partitioned to have an equal number of subtypes to protect against increasing the imbalance in the number of subtypes pulled during bootstrap resampling. Next, each partitioned sample will be resampled with replacement (bootstrapping), with 66% of the individuals from the original sample going into each bootstrapped iteration. For each of the 1000 bootstrapped samples, an ANOVA will be performed on the cognitive functioning measures. Furthermore, the partitioned samples from each iteration will be combined to create a “full sample,” which will then be resampled with replacement and subjected to ANOVA analyses. This approach will allow for the estimation of the variability in the sample and the quantification of the uncertainty and robustness of the results, as well as the calculation of confidence intervals and hypothesis tests without relying on assumptions about the underlying distribution. 3.4.2. Subsampling Subsampling is a technique for evaluating the reliability, reproducibility, and stability of statistical analyses by randomly selecting different subsets of the data, ranging from 10% to 90% from both split samples (Madigan et al., 2014). This allows for a systematic assessment of the robustness of the subtype results obtained from the original split sample datasets. In addition, by examining how sensitive the results are to changes in the size and composition of the sample, we can determine the degree of confidence we have in our findings. This is especially important when working with large datasets, where small changes can significantly impact the results.
9. Intellectual Merit Using ABCD’s large-scale dataset, this research employs the Identification, Validation, Evaluation, Prediction, and Replication (IVEPR) framework to explore the heterogeneity of brain structure and function in children and adolescents. The IVEPR framework, with its advanced, data-driven techniques, works to identify meaningful patterns, make robust predictions, and enhance interpretability within complex, high-dimensional datasets. By applying this framework, this research proposal aims to deepen our understanding of brain structure and function heterogeneity during development. In addition, it establishes a versatile methodological foundation for data-driven neuroimaging subtyping adaptable to other datasets. This comprehensive approach fosters advancements in the broader field of neuroimaging research, promoting scientific rigor across studies. By elucidating meaningful and reproducible subtypes associated with cognitive and behavioral outcomes, our research sheds light on the neural mechanisms underlying typical and atypical neurodevelopment, offering new insights for targeted interventions and treatment approaches. Identifying potential neural markers related to typical and atypical cognitive functioning and psychopathology contributes to understanding developmental trajectories, paving the way for early detection, and creating tailored interventions and support strategies.
10. Broader Impacts The proposed research offers the possibility of substantial contributions to our understanding of typical and atypical neurodevelopment. By standardizing data-driven clustering procedures, this work will enhance the reproducibility and reliability of research in the field, promoting more accurate and consistent insights into the complex heterogeneity of brain structure and function across development. Furthermore, identifying neuroimaging-based subtypes has the potential to elucidate the neural markers for cognitive functioning and psychopathology, paving the way for targeted interventions and personalized treatment approaches. Notably, the longitudinal exploration of these subtypes offers an invaluable opportunity to investigate developmental trajectories and the dynamic changes in brain structure and function over time. Such longitudinal analyses can reveal critical windows of vulnerability and resilience in typical and atypical development, deepening our understanding of the factors contributing to cognitive and psychiatric disorders and ultimately informing preventative strategies and early interventions to promote healthy neurodevelopment.