Using health administrative data to model associations and predict hospital admissions and length of stay for people with eating disorders

Mode of separation is the way in which a person leaves the hospital or service setting, and this is utilised as the main outcome measure for a patient admitted or not admitted from the EDD. For the APD, the outcome measure is defined as an episode of care with length of stay to be the total length of stay from when a person is admitted to when they were discharged. It is determined that this would be the most appropriate and clinically relevant definition for this study to explore factors that impact on the total duration of a patient’s whole episode of care.

All variables available from routinely collected health data were considered for inclusion in the analysis. Initial data collection from the EDD and APD were reduced to clinically significant variables based on consideration of descriptive statistics and clinical relevance/importance. The APD included diagnostic variables so initially, broad diagnostic groupings or index scores such as the Charlson Comorbidities Index and Elixahauser Index were considered, however from careful exploration of the weighting and the evidence [22, 23] in the clinical context of ED, it was decided that broad weighted groupings and indexing would not be clinically appropriate or meaningful. Therefore, diagnoses were grouped into diagnostic categories. Each diagnostic category was revised multiple times taking into account descriptive statistics, ICD-10-AM categorisations and clinical context/meaning. The final diagnostic groupings consisted of 10 mental health comorbidities and 9 physical health comorbidities. (Supplementary 1)

Data was analysed using STATA Version 15 (StataCorp, College Station, TX, USA). Alpha of 0.05 was considered statistically significant. A full model for exploratory purposes as well as a main effects model for prediction were developed for both the EDD with the binary outcome of admission or discharge and for the APD with length of stay as outcome. A multivariable logistic regression model was chosen for modelling the odds of an admission from emergency department from the EDD and a multivariable linear regression model was chosen for modelling length of stay from the APD. Due to skewness of the APD data, further sensitivity analyses were conducted using a multivariable negative binomial regression model.

For both multivariable initial exploratory models, variable selection was based on clinical context, appropriateness and importance of categorisation and transformation of covariates in the context of statistical analyses. For both final prediction models, the model building strategy followed the “Purposeful Selection of Covariates” as outlined by Hosmer et al. [24] The general principle of the model building strategy involved careful exploration and selection of each of the covariates based on clinical relevance and importance followed by statistical analyses in a cyclical iterative process. All potential interactions were considered clinically and then statistical analyses of potential interactions were explored. Investigators representing different disciplines with expertise in their area reviewed each step of this process, until a consensus was reached.

Validation of the final main effects models for prediction included tests on residuals, heteroscedasticity, goodness of fit tests, variation inflation factors testing for multicollinearity and exploration of outliers. A regular bootstrap procedure was followed for internal validation of the predictions in both prediction models. A regular bootstrap including 100% of the sample, with 1000 repetitions was used for internal validation [25, 26].

There were 228 presentations from the EDD identified as diagnosed with ED between 2011 and 2020. The mean age is 30.0 years (SD = 14.7, range 16 to 87). Referrals were from self, family or friends (70.2%) or ‘other’ health, aged care and community services (29.8%). The most presentations to emergency department occurred in 2019, with almost half of presentations identified with a diagnosis of Anorexia Nervosa. Most patients were female (92.1%) and presented to RPA (84.6%) compared to other hospitals in SLHD. Patient characteristics are available in Table 1.

The multivariable model (Table 2) shows weak evidence of a difference in odds of admission across year (p = 0.089). Odds of admission in 2020 ([OR] 0.18, 95% Confidence Interval [95%CI] 0.05 to 0.64; p = 0.008) and 2012 ([OR] 0.26, 95% Confidence Interval [95% CI] 0.08 to 0.85; p = 0.026) are lower compared to 2018. This year was chosen as the reference year as data collection and policy changes were implemented in 2016, improving consistency in the years 2017 to 2019, so the median from these years was chosen. This is consistent with descriptive statistics (Table 1) and univariate analyses (Supplementary 2). There is also weak evidence of a difference in odds of admission for varying diagnosis, and the odds of admission for patients diagnosed with Bulimia Nervosa is lower ([OR] 0.31, 95% Confidence Interval [95% CI] 0.10 to 0.95; p = 0.040) compared to Anorexia Nervosa. For every unit increase in age there is increase in odds of admission ([OR] 1.04, 95% Confidence Interval [95% CI] 1.01 to 1.08; p = 0.015). Across triage categories (increasing urgency) there is decrease in odds of admission ([OR] 0.37, 95% Confidence Interval [95% CI] 0.21 to 0.66; p = 0.001) for each level of category declining in urgency. There is also difference in odds of an admission dependent on source of referral (self/family or other) and if arriving by ambulance as opposed to self-presenting to Emergency.

For testing goodness of fit, the Hosmer Lemeshow Test had a p-value of 0.784, indicating the model fits well. When compared to the minimal model there is a significant difference to the multivariable model with p < 0.001, which demonstrates a better fit. These tests are also supported by the area under the receiver operating characteristic curve (AUC) of 0.788 which indicates acceptable discrimination. The pseudo R² is 0.201 which suggests approximately 20.1% proportional improvement compared to the minimal model.

A final prediction model for predicting ED admissions in emergency department was developed with results of the model in Table 3. This has a pseudo R² of 0.135 which suggests approximately 13.5% proportional improvement compared to the minimal model. For goodness of fit, the Hosmer Lemeshow Test has a p-value of 0.249, which provides evidence the model fits well. Furthermore, this final prediction model is internally validated by bootstrapping with 1000 repetitions. (Supplementary 3) Also, compared to the minimal model there is a significant difference to the multivariable model with p < 0.001. These tests are also supported by the AUC of 0.724 which indicates acceptable discrimination. Although not statistically significant at the 5% level of significance as a predictor (p = 0.09), ED diagnoses are retained as a variable as it is considered a clinically important variable.

There were 613 patients from the APD identified as diagnosed with ED between 2014 and 2020. The mean age is 30.7 years (SD = 13.4, range 16 to 86). Peak number of admissions were in 2019 with over two thirds of presentations identified with a diagnosis of AN. Most patients were recorded as female (93.3%) and presented to RPA (81.2%). A number of diagnostic categories were identified from grouping ICD-10-AM codes and whilst some had low prevalence, all clinically important and relevant diagnostic categories were retained for analysis. The complete characteristics are in Table 4.

There is strong evidence of a difference in average length of stay across admission years in the initial exploratory model (p = 0.003). Compared to 2018, there is evidence of an average increase of 14.29 days length of stay in 2014 (95% Confidence Interval [95% CI] 2.78 to 25.80; p = 0.015). Also, the average length of stay increased by 0.96 (95% Confidence Interval [95% CI] 0.37 to 1.55; p = 0.001) for every unit increase in diagnosis count. Increased length of stay is also associated with admissions that included a mental health ward or Specialist ED Ward. There is also evidence of a difference in length of stay dependent on the ED diagnoses (p = 0.023) and mode of separation (p < 0.001).

There is an increase in average length of stay if a patient has delirium/dementias 21.83 days (95% Confidence Interval [95% CI] 5.81 to 37.86; p = 0.008), effects of chronic malnutrition 13.77 days (95% Confidence Interval [95% CI] 8.03 to 19.50; p < 0.001), refeeding syndrome markers (hypophosphatemia/oedema) 10.93 days (95% Confidence Interval [95% CI] 3.90 to 17.96; p = 0.002), hypoglycemia 6.77 days (95% Confidence Interval [95% CI] 1.00 to 12.53; p = 0.021), and anxiety disorders 5.68 days (95% Confidence Interval [95% CI] 0.10 to 11.26; p = 0.046). In contrast there is decreased average length of stay if a patient has adjustment disorders − 11.6 days (95% Confidence Interval [95% CI] -20.06 to -3.13; p = 0.007), substance disorders − 6.66 days (95% Confidence Interval [95% CI] -11.45 to -1.88; p = 0.006), and personality disorders − 6.02 (95% Confidence Interval [95% CI] -11.45 to -0.58; p = 0.030).

The R² for this initial exploratory model (Table 5) is 0.587, which indicates that 58.7% of variability in length of stay is explained by the variables in this model. It is observed from the validation tests that there is heteroscedasticity and some minor departure from normality of errors. However, with a large sample (n > 500), it has been shown that even the most skewed data can be analysed using least squares regression [27]. Sensitivity analyses were also performed comparing this model to negative binomial models. (Supplementary 5) Despite some differences in output, the majority of the output were similar, further supporting the linear model.

A final prediction model for length of stay for patients admitted with ED was developed with results of the model in Table 6. The R² for this final prediction model is 0.562, which indicates that 56.2% of variability in length of stay is explained by inclusion of all these variables in this final prediction model. This final prediction model is internally validated by bootstrapping. (Supplementary 6) Furthermore, the same validation process and sensitivity analyses were conducted as the initial exploratory model with similar results providing support for a linear model. (Supplementary 7)

In this study modelling emergency department admissions utilising EDD, the initial exploratory model consists of 10 variables and after purposeful selection of covariates, the final prediction model consists of 4 variables. In other studies on prediction models on emergency department admissions, variables included in these models varied significantly such as diagnoses, functional status and quality of life [28, 30]. The source of data utilised also varied, including administrative, clinical and survey data [31‐33]. It has been found that administrative or clinical data have greater predictive ability compared to survey data [28].

In the initial exploratory model (Table 2), ED diagnoses, age, triage category, source of referral and mode of arrival are observed to be statistically significant factors impacting on admission. For example, with every year increase in age there is 4% greater odds of admission (95% Confidence Interval [95% CI] 1.01 to 1.08; p = 0.015). Whilst there are limitations with the validation by Hosmer Lemeshow test, results from the model are clinically plausible and valid both in magnitude and direction. Investigators believe this contributes to the validity of the model and provides evidence administrative data can be utilised to understand factors that impact on ED patient admissions in emergency.

However, the same cannot be inferred from the final prediction model (Table 3) as it consists of only 4 variables, ED diagnoses, age, triage category and mode of arrival. Whilst each variable is clinically valid, they would not be considered adequate to predict admissions, supported by the pseudo R² of 0.135, which is relatively low. Several other studies have shown multiple comorbidities and polypharmacy have a significant impact of health care utilisation [34, 35], which were not available in the EDD and may have improved the utility/validity of the final prediction model.

The initial exploratory model for length of stay has clinically plausible and valid factors that impact on the outcome of length of stay. This initial exploratory model includes 34 variables in total in which 19 are diagnostic categorisations based on ICD-10-AM. It is clear that the APD contains much further information than EDD, including the important variable of diagnoses, which is not available in the EDD.

Diagnoses are important variables in health care outcomes and utilisation. A large number of studies on length of stay have utilised Charlson Comorbidity Index and other methods of indexing diagnoses with mixed results [31]. There are numerous adaptations of Charlson Comorbidity Index [36‐38] however none that exist for ED and given clinical heterogeneity in this population, utilising such an index would require adaptation and validation. Therefore, diagnoses were included in a number of ways including diagnosis counts, eating disorder diagnoses and clinically appropriate and meaningful groupings and categorisations of comorbid diagnoses. Furthermore, from the initial exploratory model in the results Table 5, there is further evidence of clinical utility as a number of clinically plausible variables are associated with length of stay in addition to diagnoses related variables including admission year, type of wards admitted and mode of separation.

In the final prediction model in Table 6, diagnoses of substance disorder, personality disorder, anxiety disorder, adjustment disorder, refeeding syndrome markers (hypophosphatemia/oedema) and effects of chronic malnutrition provide predictive value. As well as the diagnoses, predictors for length of stay include diagnosis counts, ED diagnoses, type of wards admitted, marital status and socioeconomic disadvantage. The variables in the final prediction model for length of stay are all considered to be clinically plausible and valid and this is again supported statistically by the model with R² of 0.562 as well as internal validation via bootstrapping (Supplementary 6). Based on the R², this model performed relatively well compared to numerous other studies on prediction models for length of stay, which had R² between 0.3 and 0.6 [29]. Many studies are now utilising machine learning models used for predicting length of stay, however the model in this study performed better than most of the machine learning models thus far [39].

The purpose of this study is to assess the clinical utility of health administrative data but there are several limitations in using these data. These data are not purposefully collected for this study or for clinical purposes. It is collected for resource utilisation and allocations with data entry initially, by administrative staff and clinicians with limited training on data entry. There is a lack of standardisation in data collection in EDD in contrast to APD, where data collection is conducted by highly trained coders reviewing all the documentation. This, in part, explains the improved validity and utility of modelling length of stay compared to admissions in emergency.

For modelling admissions in emergency, the sample size was much smaller than anticipated as the search criteria was based on SNOMED diagnosis codes. It is known that patients with ED present to emergency department for numerous physical or mental health issues such as electrolyte disturbance or self-harm, which would not have been identified and this is a known limitation of SNOMED codes [40]. Therefore, the population was defined as patients with confirmed ED diagnoses on presentation to emergency for this study to better understand limitations of EDD data with the view to inform follow up studies, further analyses and models.

From modelling length of stay, the diagnostic groupings were based on ICD-10-AM coding, which is not consistent with diagnostic criteria utilised in clinical practice and did not include severity. This could potentially impact predictive value for length of stay. However, some limitations were mitigated by meticulously reviewing each of the ICD-10-AM codes with these diagnostic groupings both statistically and clinically, with multidisciplinary clinical input in a cyclical iterative process.

Health administrative data can be utilised to describe factors that impact on health outcome measures as well as predict health outcome measures in adults with ED. Both the initial exploratory models provide useful and clinically valid information. The final prediction model for admissions in emergency is not statistically or clinically valid for prediction, but there is potential with inclusion of further variables such as diagnoses. This is demonstrated by the final prediction model for length of stay, which performed well in comparison to other studies on prediction models for length of stay. Further exploration and evaluation of this type of data with inclusion of more information from data linkage and clinical information evaluating various statistical methods are the next steps currently in progress following this study. Furthermore, future studies will also include external validation, translation and implementation with prospective studies to further validate and optimise the final prediction models.

It is also evident from this study that there would be numerous benefits to inclusion of further variables and standardising data collection. Utilising health administrative data in this way is an efficient process for assessing impacts of multiple factors on patient care and predicting their health care outcomes, which has significant potential in not only guiding effective resource allocation and utilisation but also improving care provided to people with a lived experience of ED, their families and carers. It is even more important now to use all available data to optimise outcomes and provide efficacious and cost-effective services to meet the growing demand for treatment and care in the community, especially given the ongoing increasing prevalence and impacts of COVID-19 in mental health and eating disorders.