Co-occurring internalizing and externalizing psychopathology in childhood and adolescence: a network approach

Data were from the Avon Longitudinal Study of Parents and Children (ALSPAC). The ALSPAC is a prospective cohort study of children born in the English county of Avon between April 1st 1991 and December 31st 1992 (N = 14,062). The sample is broadly representative of the overall population of children in the UK [27, 28]. Data were collected using self-report postal questionnaires (completed by the study mothers and mother’s partners) and via yearly clinics for the study children from the age of 7.5 years [27, 28]. The study website contains details of all the data that are available through a fully searchable data dictionary (http://​www.​bris.​ac.​uk/​alspac/​researchers/​data-access/​data-dictionary/​). Ethical approval for the study was obtained from the ALSPAC Ethics and Law Committee and the Local Research Ethics Committees. Further detailed descriptions of the ALSPAC can be found elsewhere [27].

Disorders were assessed using maternal report versions of the Development and Wellbeing Assessment (DAWBA) [29]. This structured clinical interview is used to assign psychiatric diagnoses to 5–16 year olds. It is used to assess fourteen distinct symptom profiles corresponding to ICD-10 and DSM-IV diagnostic criteria. The symptom profiles used in the present analysis were those assessed consistently across the three waves; specific phobia (SPP), social phobia (SOP), posttraumatic stress (PTSD), generalized anxiety (GAD), depression (DEP), attention deficit/hyperactivity disorder (ADHD), oppositional/defiant disorder (ODD) and conduct problems (CD). Official diagnoses were only available at the 7.5 year assessment. To make the use of data from subsequent time points, a comprehensive recoding strategy was employed. Mirroring the structure of the DSM, the DAWBA employs skip patterns; mothers are first asked whether children display core symptoms, followed by questions related to distress and burden associated with symptoms. To create quasi-diagnostic variables that closely mirrored DSM-IV diagnoses, disorders were deemed present if study mothers reported the requisite symptom profiles (including core symptoms) and significant burden or distress associated with these symptom profiles. In the case of ODD, teacher complaint was used in place of distress. For CD, a binary variable reflecting ‘any frequent/definite troublesome behaviour’ was computed, as per ALSPAC codebook guidelines. This recoding process resulted in eight binary quasi-diagnostic variables at each of the three time points. More detailed descriptions of this recoding process are available in the online supplementary materials.

There is a lack of consensus as to how missing data should best be handled in network analysis [30]. The present study used the most common current practice, listwise deletion [14, 30, 31]. Complete data were available for 4405 maternal reports (DAWBA) at ages 7.5, 10.5 and 14, and this sub-sample was used for analysis.

Networks were constructed using the R package ‘Isingfit’ [32] which was developed to construct weighted undirected networks using binary data. This package uses the elasso method; based on the Ising [33] model, each variable is regressed on all other variables iteratively with a lasso penalty (Ɩ1) imposed on the regression coefficients that helps identify the simplest network by balancing sparsity and goodness of fit [18]. The Ɩ1 process identifies the best fitting network structure by specifying competing models with different levels of sparsity, and comparing the models using the extended Bayesian information criteria (EBIC) [34]. The edges in these networks are the mean values of the two logistic regression coefficients (i.e. node A predicting node B, and node B predicting node A), which can be interpreted similar to partial correlations. ‘Isingfit’ was used to construct networks using the 8 binary psychological disorder variables at ages 7.5, 10.5 and 14 years. The resultant networks were then graphically illustrated using the ‘qgraph’ package [35]. This package uses the Fruchterman–Reingold algorithm to place nodes with stronger and/or more connections closer together [36].

The relative importance of each node to the overall network structure was quantified using three common measures of node centrality. Strength is calculated for each node by summing its weighted connections with other nodes [37]. A node that is high in strength strongly and directly transmits its effects throughout the network [37]. Closeness reflects the average distance from a node of interest to all other nodes in a given network [37]. High closeness means a node is strongly influenced by changes in other nodes in the network [37]. Betweenness is calculated by counting the number of times a node of interest lies on the shortest path between two other nodes [37]. Nodes that are high in betweenness are important for transmitting effects between other nodes in the network. For all measures of centrality, higher values (presented as z-scores) are indicative of greater importance to the network as a whole [38].

Recently, the accuracy and stability of networks have received attention in the literature [39]. Accuracy and stability refer to the degree of certainty with which we can interpret the rank ordering the various edge weights and centrality indices. Network accuracy and stability were assessed using the guidelines of Epskamp, Borsboom, and Fried [39]. First, bootstrapped 95% confidence intervals (CIs) were used to examine the accuracy of network edges. Second, the stability of the order of the centrality indices (strongest to weakest) was examined using a subsetting bootstrap method, i.e. by re-estimating the network based on increasingly smaller subsets of the original sample. The underlying logic of this method is that if the order of centrality estimates from a network based on a small subset is highly correlated to the order of the centrality from the original network, the centrality estimates can be considered stable [39]. Stability analyses were conducted using the R package ‘bootnet’ [39].

To test for changes in the relationships between internalizing and externalizing disorders over time, structural invariance was examined using the ‘Network Comparison Test’ (NCT) package in R [40]. NCT tests the null hypothesis A₁ = A₂, where A₁ and A₂ are matrices containing the strengths of connections in two separate networks [40]. The NCT procedure involves non-parametric permutation testing and is conducted in three phases [40]. First the two networks in question are estimated and the maximum difference in edge strength between two given networks (M) is calculated and serves as the test statistic [40]. For the second step, cases are repeatedly randomly swapped between groups and the networks and test statistics re-estimated. Third, a reference distribution is created from these test statistics and statistical significance is determined, with the p value equal to the proportion of test statistics that have an equal or higher value than the observed test statistic [40].

Table 1 shows the frequencies and relative percentages of the disorders at the different time points. Across time, GAD was the most common disorder, followed by SPP. PTSD was the least endorsed disorder. Bivariate correlations between disorders are presented in the online supplementary materials (Table S1).

The association networks, constructed separately at each time point, are presented in Fig. 1. Similar patterns of association were observed; two distinct clusters of nodes emerged, reflecting internalizing and externalizing disorders. Within the internalizing cluster, GAD was placed centrally and demonstrated moderate to strong associations with all other disorders. Within the externalizing cluster, thick edges indicated strong associations between ODD and ADHD at all three time points. Relationships between ADHD and CD, however, were comparatively weak. Indeed, ODD appeared to bridge the associations between CD and ADHD at the different time points.

GAD appeared to lie at the heart of the network as a whole, due to its central placing at all three time points. An inspection of the centrality indices (Fig. 2) corroborated this observation; across time GAD consistently scored highest on the measures of strength, closeness and betweenness. Within the externalizing cluster, ODD was the most central node at ages 10.5 and 14 years, whereas ADHD demonstrated greater betweenness and closeness at age 7.5 years. Cross-cluster associations were common, although generally of smaller magnitude compared to the within-cluster associations. The most consistent bridging edge was DEP-ODD, which consistently appeared within the 10 strongest edges (Fig S1). The edge GAD-ADHD was also significant at each time point. Other edges were less consistent. For example, the edge SOP-ADHD was moderately strong at ages 10.5 and 14 years, but non-significant at age 7.5.

The bootstrapped 95% confidence intervals for the edges are presented in the online supplementary materials (Fig S1). Although there was considerable overlap among the CIs, the strongest edges demonstrated little overlap, suggesting statistically significant differences in the strengths of these associations. As such, the networks were moderately accurately estimated, and the order of the edge weights in each of the three networks can be interpreted with some degree of confidence [48]. The results from the subsetting bootstrap method are presented in the online supplementary materials (Fig S2). Overall, the network centrality indices appeared highly stable; the correlations between the measures of centrality using the full sample and a subset of 30% ranged from approximately 0.5–0.85. This indicates that, even when 70% of the sample were randomly removed, the order of the centrality indices remained stable, suggesting a robust estimation of centrality. Overall, strength was the most stable of the three indices, and betweenness the weakest.

With regard to structural invariance, after applying a Bonferroni adjustment to account for multiple testing, the non-parametric permutation tests found no significant difference in overall network structure (Fig S3). As such, the network structure was considered broadly stable over time.

Borsboom and colleagues [41] suggested that, if modelled using network techniques, traditional hierarchical measurement models of psychopathology would be reflected in clusters of highly associated nodes, analogous to the higher order dimensions of internalizing and externalizing. The findings of the present study support this claim. While this finding may not be surprising (both network and latent variable models are derived from covariance), it offers a different interpretation of psychiatric comorbidity. This perspective suggests that, rather than a single causal factor (or amalgam of causal factors) driving the association between symptoms/disorders, causal factors spread their effects throughout psychopathological networks via local interactions and reinforcement. In other words, higher order dimensions (e.g. p) may be capturing a plethora of local-level interactions. Under this interpretation, the dimensions of internalizing and externalizing are correlated due to certain disorders (which themselves are comprised of networks of associated symptoms) acting as ‘bridges’ between these two broad spectra.

The nature of these interactions (i.e. edges), however, is far from clear and serves to further emphasise the complexity of psychiatric comorbidity. Given that the analysed data were cross-sectional (within-time points), a significant edge could represent a multitude of possible relationships. First, it is possible that disorders directly influence each other. For example, there is a long history of research looking at the comorbid anxiety and depression, with evidence suggesting that anxiety tends to precede and lead to subsequent depression [42]. One proposed mechanism for such a relationship is that cognitive/neurophysiological processes (e.g. sustained heightened physiological arousal) may lead to an exhaustion of the body which manifests as depression [43]. Other edges may reflect more indirect associations, e.g. the edges ODD-DEP and ADHD-GAD in the current networks. Developmental cascade models have long suggested that externalizing behaviour may indirectly lead to internalizing problems through mediating variables [44‐46]. To illustrate, frequent disruptive behaviour in childhood/adolescence may lead to negative reactions from parents, teachers and/or peers, e.g. shouting, punishment, ostracisation from peer group, and/or academic failure. Such negative outcomes may in turn foster feelings of irritability, distress and worthlessness within the child, and if left unchecked, these experiences may eventually progress to levels of clinical significance [45, 46]. Edges in any given network may also represent spurious associations due to an unmeasured common cause. For example, shared biological (e.g. genes) or environmental (e.g. trauma) risk factors may influence the development of multiple symptom domains simultaneously [42]. If these risk factors are not represented in a given network, this may give rise to non-causal associations between disorders. Edges are further complicated by the possibility of bidirectional feedback loops, equifinality (multiple risk factors, pathways and processes leading to similar outcomes), and multifinality (specific risk factors leading to multiple outcomes) [47].

Given that the networks in the present study were undirected and cross-sectional, any causal interpretations, such as those described above, are purely speculative. The aim of the present study, however, was not to infer strict causal relationships but to demonstrate how network analysis can be used to quantify the importance of disorders and identify key associations between disorder pairs and as such generate hypotheses regarding the complex mechanisms that drive psychiatric comorbidity. For example, in the present study, GAD and ODD were identified as the two most influential nodes within wider comorbidity networks, suggesting that symptoms within these disorders may be influential in the initiation and/or maintenance of comorbid psychopathology. Furthermore, a number of significant edges bridged the spectra of internalizing and externalizing, the strongest of which was the edge ODD-DEP. This suggests that local-level interactions between these two disorders may go some way to explaining the correlations between internalizing and externalizing when modelled as continuous dimensions [24].

The main strength of the present study was the large sample size relative to the number of parameters estimated [49]. With regard to limitations, diagnostic data were not available for the participants at all three of the time points; therefore, a comprehensive recoding strategy was adopted. It must be noted, however, that the diagnostic algorithms used to create the quasi-diagnostic variables were based on skip patterns (i.e. those who did not endorse ‘core’ symptoms were scored as having no disorder, and subsequent distress and burden were not assessed). This likely introduces an element of bias to the data in favour of DSM scoring conventions. Similar bias likely affects all analyses that employ DSM skip structures.

Skip patterns are particularly problematic when using symptom-level data, as they introduce deterministic dependencies (i.e. secondary symptoms can only be endorsed if primary symptoms are first endorsed, thus artificially inflating correlations between these symptom pairs). However, this was not the case in the present study, as a diagnosis of one disorder was not dependent on the diagnosis of another disorder. Finally, any study that employs network analysis comes with the caveat that this approach is highly divisive. Indeed, this has been evidenced by a series of back-and-forth papers focussed on the replicability of networks [50‐52]. A methodological critique of network analysis and/or traditional latent variable models is beyond the scope of this article (for recent reviews, see [13, 48, 50‐52]); however, our findings do appear to support the replicability of psychopathological networks within a cohort of children and adolescents assessed repeatedly over a period of significant development.