Reducing youth suicide: systems modelling and simulation to guide targeted investments across the determinants

The North Coast PHN supports a population of 502,524 (as at 2016) [43], distributed over a geographic area of approximately 35,570 km² and taking in both coastal and inland rural communities. Aboriginal and/or Torres Strait Islander people represent approximately 5.0% of the North Coast population, which is higher than the proportion for both New South Wales (2.9%) and Australia (2.6%) [44]. The region has a high proportion of the population aged over 65 years (20.4%), and is more socio-economically disadvantaged than the national average [44]. Unemployment is high across the North Coast region, with some local government areas (LGAs) reporting rates as high as 9.7% compared to the national average of 5.9% in 2018 [44]. Domestic and family violence rates are higher than the NSW average, with some LGAs reporting incidence of domestic assaults as high as 757.1 per 100,000 population [45], as are rates of homelessness, with some LGAs reporting rates as high as 57.5 per 10,000 population [46]. Rates of suicide are also well above the NSW average, with incidence rising from 7.9 per 100,000 in 2006 to 16.6 per 100,000 population in 2017 [47]. As of mid-2016, young people aged 15–24 years made up 10.4% of the north Coast PHN population, which was lower than the proportions of young people in the NSW population (13.9%) and the Australian population (13.2%). In 2016–2017, the rate of children and young people being in out of home care was higher in the North Coast PHN region (16.3 per 1000 children aged 0–17 years) compared to 11.4 per 1000 children in NSW [44]. The rate of children and young people reported as being at risk of significant harm was also higher in the region (79.7 per 1000 children aged 0–17 years) compared to 52.3 per 1000 children in NSW [44].

A system dynamics model was developed using a participatory modelling approach that involved approximately 50 local stakeholders, including representatives from health and social policy agencies, non-government organisations, primary care providers, emergency services, research institutions, community groups, and, importantly, people with lived experience of suicide. The process employed a broad systems perspective drawing on the deep tacit knowledge and diverse perspectives of these system actors. Input from stakeholders was provided through a series of workshops, meetings, priority setting surveys and system mapping activities conducted in 2019 (see video: http://​nccforbetterlive​s.​com.​au/​systems-modelling). The participatory modelling process undertaken for the current study followed the approach detailed elsewhere [48‐50]. In summary, workshop 1 (full day) included discussion and prioritisation of the key outcomes of interest, the mapping of pathways and drivers of those outcomes, and the prioritisation of interventions to be included in the model. Following the workshop, the map was converted to a conceptual model which was synthesised with best available research evidence and data to inform the development of the initial system dynamics model. In addition, a draft structure representing the service pathways of the local mental health system developed from workshop 1 was disseminated to broader community stakeholders for verification and input. Stakeholders returned their modifications to the draft structure, and feedback from this process was synthesised with missing pathways subsequently integrated into the system dynamics model. Several months after workshop 1, the 50 stakeholders reunited for workshop 2 (full day) where the draft structure, logic, and key assumptions of the model were presented for verification, discussion, feedback, and consensus. In addition, participants were supported to map the mechanisms of effect of prioritised interventions (priorities determined via survey) onto the structure of the system dynamics model. Reviews of the literature following workshop 2 further informed intervention mechanisms and parameters, and an interactive interface was developed. A final (half-day) workshop 3 was conducted several months after workshop 2 where the penultimate version of the model was presented to the stakeholders for verification, discussion, feedback, and consensus. Stakeholders were provided with the opportunity to interact directly with the model interface to run scenarios, test alternative assumptions, discuss and question results, and provide feedback on interface design and functionality. The key insights derived from the model and their implications for service planning, commissioning of programs and services, and evaluation were presented and discussed to ensure face validity.

Model development does not follow a linear process; it is an iterative knowledge feedback process that entails continuous hypothesis development, testing, and refinement—a process shared with the multidisciplinary group both through the workshops and additional out-of-session meetings as required with stakeholder sub-groups. Model structure, parameter estimates, and other numerical inputs were informed (where possible) by published research or available regional, state, and national data or were estimated via constrained optimisation (see Additional file 1). Model construction and analysis were performed using Stella Architect ver. 1.9.4 (www.​iseesystems.​com). The model was validated by (i) testing whether the model could replicate historic data across a range of key indicators (namely; time series of psychological distress, psychiatric hospitalisations, emergency department (ED) presentations, youth and total population self-harm hospitalisations, and suicide deaths—see Additional file 1: Fig. S5, Fig. S7, Fig. S17, and Fig. S22) and (ii) ensuring face validity of the model structure and performance among stakeholders working in or interacting with different parts of the system.

The core model structure included (1) a population component, capturing changes over time in the size and composition of the population resulting from births, migration, ageing, and mortality; (2) a psychological distress component that models flows of people to and from states of low psychological distress (Kessler 10 [K10] score 10–15), moderate psychological distress (K10 score 16–21), and high to very high psychological distress (K10 score 22–50); (3) a series of components capturing the interdependent dynamics of key social determinants, namely, early life exposures, substance abuse, domestic violence, homelessness, and unemployment, and their influence on levels of psychological distress; (4) a mental health services component that models the movement of psychologically distressed people through one of several possible service pathways involving (potentially) general practitioners, psychiatrists and allied mental health professionals (including psychologists and mental health nurses), emergency department and psychiatric inpatient care, community- and hospital-based outpatient care, and online services; and (5) a suicidal behaviour component that captures self-harm hospitalisations (used as a proxy for suicide attempts—see the “Limitations” section) and suicide deaths.

Figure 1 provides a high-level overview of the causal structure and pathways of the model, with arrows denoting unidirectional or bidirectional relationships between each component. The structure and assumptions relating to each component and their interactions are detailed in Additional file 1. The model captures changes over time (dynamics) within each component and between the components of the model. For example, within the health system component, the proportion of the population waiting for services, receiving services, or disengaging from services changes over time based on service system capacity and the rates of flow into, within, and out of the service system. Dynamics also occur between the model components, for example, as unemployment rises, not only does it directly act to increase the incidence of high to very high psychological distress in the modelled population (which has flow-on effects on rates of substance misuse, and adverse early life exposures), but it also increases rates of domestic violence and homelessness, both of which further increase the rate psychological distress. Such dynamics makes it difficult to anticipate what impacts might occur across the system and on key outcomes if we intervene on one or multiple components of the model.

Primary model outputs included total (cumulative) numbers of self-harm hospitalisations and suicide deaths for the youth population (15–24 years) and total population. The model also provided estimates of the prevalence of low, moderate, and high to very high psychological distress by age categories (15–24 years, 25–64 years, 65 years, and above) and a range of measures of mental health service usage (e.g. mental health-related general practice consultations, psychiatric and allied mental health service consultations, services waiting times, numbers of psychologically distressed consumers that have disengaged from services, Emergency Department (ED) presentations, and psychiatric hospitalisations). All outputs were calculated every 0.4375 days (i.e. one sixteenth of a week) over a period of 30 years, starting from 1 January 2011, permitting comparisons of model outputs with historic data from 2011 to 2017 (see Additional file 1) and forecasts of the impacts of intervention scenarios described below simulated from the time of implementation (2021) to the start of 2041. This longer-term forecast horizon was deemed necessary to allow impacts of variations in the social determinants of mental health to be seen, as well as to encourage transition to a long-term strategic outlook in assessing the value of investment decisions rather than the current short-term perspective that induces more reactive decision making.

Parameter values that could not be derived directly from available data or published research were estimated via constrained optimisation, using historical time series data on a wide range of mental health and social outcomes, including psychological distress prevalence, self-harm hospitalisation and suicide rates, rates of mental health services usage (general practice consultations, specialised psychiatric services, ED and hospital inpatient care, community-based mental health services), substance abuse disorder prevalence, unemployment and labour force participation rates, domestic violence incidence, and the prevalence of homelessness. Powell’s method [51] was employed to obtain the set of (optimal) parameter values minimising the sum of the mean absolute percent error calculated for each time series separately (i.e. the mean of the absolute differences between the observed time series values and the corresponding model outputs, where each difference is expressed as a percentage of the observed value).

We modelled the potential impacts on suicidal behaviour of a set of interventions identified by the PHN and stakeholder group as most relevant to the North Coast NSW context. Interventions were identified based on alignment with current investment priorities and feasibility of implementation and included (1) six mental health and suicide prevention and (2) five social determinants modifiers. Details of each intervention (and assumptions) are provided in Additional file 1: Table S1. Twenty-one alternative scenarios (Table 1) were compared with a baseline (business as usual), in which existing policies and programs remain in place and current per capita growth in mental health services capacity is maintained until the end of the simulation.

Sensitivity analyses were performed to assess the impact of uncertainty in estimates of the direct effects of each intervention and forecasted growth in services capacity (i.e. GP mental health services, psychiatrists and allied services, community mental health services, psychiatric hospital care, and alcohol and drug services) on the simulation results. We used Latin hypercube sampling to draw 100 sets of values for the selected model parameters from a uniform joint distribution spanning ± 20% of the default values (see Additional file 1: Table S2). Differences in projected (cumulative) numbers of self-harm hospitalisations and suicide deaths between the baseline and intervention scenarios were calculated for each set of parameter values and summarised using simple descriptive statistics. This research was approved by the University of Sydney Human Research Ethics Committee (Project number: 2018/833).

Programs effective in increasing social connectedness had the single greatest impact on suicidal behaviour in the youth population and total population, reducing the numbers of self-harm hospitalisations by 11.4% and 13.5%, respectively, and suicide deaths by 12.4% and 14.5% respectively. Relatively large reductions were also forecasted with the implementation of technology-enabled coordinated care (reducing self-harm hospitalisations and suicide deaths by 5.6% each), and post-attempt assertive aftercare (reducing self-harm hospitalisations and suicide deaths by 5.8% each) in the youth population, with slightly higher reductions across the total population. The combined impact of these three interventions were forecast to reduce self-harm hospitalisation by 21.3% and suicide deaths by 22.2% in the youth population with relatively rapid reductions over the first decade that plateau over the second decade (Fig. 4). Implementing all the specific mental health and suicide prevention interventions failed to achieve substantial additional reductions in suicidal behaviour across the total or youth populations than achieved through the ‘best combination.’

The model projected a substantial reduction in self-harm hospitalisations (28.5%) and suicide deaths (29.3%) in a youth population by combining social connectedness programs, technology-enabled coordinated care, and post-attempt assertive aftercare with a reduction in childhood adversity by 50% and an increase in youth employment initiation by 50%. Similar impacts were achieved across the total population with this combination.

Of the scenarios targeted at reducing the social determinants of mental health, reducing childhood adversity by 50% had the single greatest impact on suicidal behaviour in the youth population, reducing the numbers of self-harm hospitalisations and suicide deaths by 6.7% each. Combining reductions in childhood adversity with a 50% increase in youth employment initiation is forecast to deliver reductions in self-harm hospitalisations and suicide deaths of 9.0% each and an effect that is amplified over time (Fig. 4). The impact of this combination on the total population is far less reducing self-harm hospitalisations and suicide deaths by 2.9% each. Implementing all scenarios across the social determinants of mental health did not produce substantial additional benefits in reducing suicidal behaviour across the total or youth populations than achieved through the ‘best combination.’

There are a number of limitations that require consideration when interpreting the findings of this study. There is potential measurement bias in the range of secondary data used to parameterise the model including the population health surveys, Medicare claims data, and PHN and Local Health District (LHD) datasets. The model acknowledges these potential sources of measurement bias and a number of commonly used strategies were employed to address them, including the triangulation of multiple data sources, parameter estimation via constrained optimisation, and local verification to identify plausible estimates.

In addition, there is potentially an under-enumeration of suicide cases used to calibrate the model, due to the misclassification of suicides to ICD codes relating to unintentional injury and events of ‘undetermined intent’ [64]. Suicide attempts identified from hospital admissions data likely only capture those cases serious enough to warrant medical intervention, and instances of self-harm where the intent was not clear may be not coded as suicide attempts. However, this under-enumeration is consistent across simulations of the baseline case and intervention scenarios and as such are unlikely to affect the forecast estimates of impact (i.e. the % reduction in suicidal behaviour) of intervention strategies or the strategic insights derived from the model. Ongoing systematic monitoring and evaluation can determine the extent to which the model forecasts are corresponding with real-world outcomes over time, allowing refinement of model parameters to improve forecasting capabilities. Finally, as the impacts of simulated scenarios are subject to the population, demographic, behavioural, and service dynamics of the modelled region, they are not necessarily generalisable to other regions; however, depending on contextual similarity to the modelled region, the qualitative insights are likely to relevant.