Introduction
Since the implementation of Millennium Development Goal 4 to reduce childhood mortality, the number of deaths for children under five years of age has dropped from 11.9 million in 1990 to 6.3 million in 2013 [
1,
2]. Despite considerable success in reducing childhood mortality globally, it remains concentrated in the world’s poorest regions, with nearly half of under-five deaths in 2012 occurring in sub-Saharan Africa. Infectious diseases are important causes of death in children with pneumonia, malaria, measles, meningitis and HIV/AIDS accounting for over one-quarter of all under-five deaths in 2012 [
3].
In-hospital deaths often occur within the first 48 hours of admission [
4]. Implementation of simple and effective clinical tools to rapidly identify and treat the sickest children is urgently needed to reduce morbidity and mortality. Clinical scores can be used to assess disease severity in patients admitted to hospital, compare mortality rates between different institutions and regions, and evaluate the efficacy of different interventions to improve patient outcomes. A number of clinical severity scoring systems have been developed in pediatric populations, including PRISM (Pediatric Risk of Mortality Score) [
5], PIM (Pediatric Index of Mortality) [
6], sMODS (simplified Multi-Organ Dysfunction Score) [
7], PELOD (paediatric logistic organ dysfunction) [
8], PEWS (Pediatric Early Warning System Score) [
9], bedside PEWS [
10], and PAWS (Pediatric Advanced Warning System Score) [
11]. However, many of these scores rely on laboratory data that are not available in many resource-constrained settings and are not practical for routine assessment of disease severity.
Recently, prognostic scoring systems have been specifically developed for pediatric populations in low-resource settings. These scores can be easily computed following patient assessment by front-line health care workers, without the need for specialized equipment, laboratory testing or onerous paperwork. Signs of Inflammation in Children that Kill (SICK) was developed in India as a practical triage tool based on data from 1,099 children (44 deaths) [
12,
13]. The Lambaréné Organ Dysfunction Score (LODS) was developed as a simple clinical prediction tool to identify African children with malaria in need of referral or close monitoring using data from 23,809 children (1,004 deaths) with severe
Plasmodium falciparum malaria [
14]. Finally, a score was developed for early death prediction in Kenyan children admitted to hospital (presented here as the Pediatric Early Death Index for Africa (PEDIA)) [
15]. PEDIA was developed in a cohort of 8,091 children (436 deaths) admitted to hospital in Kenya and validated in a cohort of 4,802 children [
15].
In this study we prospectively evaluated the ability of admission LODS, SICK and PEDIA to predict outcome in an observational study of febrile children admitted to hospital in Uganda. We assessed the discrimination and calibration of the scores and performed sub-group analysis to compare score performance in malaria versus non-malaria febrile illness (NMFI), and in early (≤48 hours) versus late (>48 hours) deaths.
Discussion
Clinical scoring systems are used in intensive care to compare performance between units and assess mortality in different patient groups. A similar system would be useful in resource-constrained settings where there is considerable heterogeneity in the quality of medical care. Clinical scores could be used to prioritize resources (at an individual patient level (that is, triage) and nationally/regionally), track changes in hospital performance over time, and monitor changes in disease patterns, in order to facilitate early identification of outbreaks. In this study, we validated SICK, LODS and PEDIA as prognostic scores for in-hospital mortality in a regional referral hospital in eastern Uganda. While all three scores were able to discriminate between survivors and non-survivors, LODS and PEDIA showed better discrimination and calibration in both malaria and NMFI. Of the three scores, LODS is the easiest to compute with only three variables compared to seven variables in SICK and eight variables in PEDIA, and had less missing data. As LODS had good discrimination and calibration, does not require equipment or specialized knowledge to generate, and is the most parsimonious, we believe it holds the most promise as a practical, ‘real world’ prognostic scoring system.
LODS was developed using the Severe Malaria in African Children (SMAC) network that collected data from six research sites (five countries) across Africa, including only children with laboratory-confirmed severe malaria [
19]. Although the study validated the prognostic ability of LODS between disparate patient and parasite populations, and within different health care systems, it was not evaluated in NMFI. In this study, we present the first evaluation of LODS in the context of NMFI and show that each LODS sign (coma, prostration, deep breathing) was associated with a fatal outcome, irrespective of the etiology of fever. The performance of LODS declined significantly when trying to predict later deaths, which is consistent with recent SMAC reports showing considerable variability in model performance in predicting later deaths [
20]. Although the predictors of early, intermediate and late death varied in subsequent analyses from the SMAC network, deep breathing, prostration and coma were significantly associated with intermediate and late deaths when data from all sites were analyzed together [
20]. Differences in mortality kinetics and predictors of late mortality between sites may reflect regional differences in disease and quality of care. In our population, LODS had better sensitivity than the original cohort (94% versus 85%, LODS >0) and comparable specificity (98% versus 98%, LODS <3). Overall, LODS had good discrimination and calibration in malaria and NMFI, suggesting it may have widespread utility.
SICK was developed as a childhood triage score at a single tertiary care hospital in New Delhi [
12,
13]. The variables included in SICK were defined
a priori based on physical manifestations of the systemic inflammatory response syndrome with weightings determined using multiple logistic regression analyses. In our study, not all variables included in SICK were associated with fatal outcome, suggesting a more parsimonious model could be developed. These findings are consistent with the original model where two variables (heart rate and respiratory rate) did not differ between surviving and non-surviving children, but were still included in the final model. The optimal cut-off in our study was >2.4 and was fairly consistent in sub-analyses, and similar to the cut-off derived from the development cohort (>2.5). Age was not associated with increased odds of death in our study; however, we only included children over 2 months of age. The neonatal period (first four weeks of life) carries one of the highest risks of death and accounts for over 40% of under-five deaths [
21]. Had we included children of all ages in our cohort, age would likely have been an important predictor of death, as it was in the SICK development cohort. According to the developers of SICK, missing data should be treated as normal; however, in the context of our cohort we found better model discrimination if we considered missing data as abnormal since a number of variables were not missing at random. Despite these limitations, the discrimination of the modified SICK score was still less than LODS and PEDIA, suggesting that additional modifications to the SICK would be required before SICK should be considered as a practical scoring system in resource-constrained settings.
PEDIA is the only scoring system that was developed in Africa in children with both malaria and non-malaria illness [
15]. In the original cohort used to develop the prognostic score, 56% of children admitted to hospital were positive for malaria. Thus, the score was developed in a large cohort of children with fever of mixed etiology. The original publication focused on developing prognostic models for immediate, early and late deaths. In this study, we elected to evaluate the early death score rather than the immediate score because it does not require any laboratory testing, whereas the immediate death score required assessment of anemia. PEDIA was comparable to LODS in predicting mortality among all children, children with malaria, NMFI, and those who died early (<48 hours) versus late. Generally, model calibration was also good. However, PEDIA is more complex than LODS without offering additional predictive/prognostic value, and does not provide additional clinical information that could be used to direct interventions (for example, fluids or oxygen).
To avoid subjective bias and explore consistency in score cut-offs under different conditions, we used a statistical method to define the optimal score cut-off for each analysis. We elected to use the Youden index, which gives equal weight to sensitivity and specificity. However, alternate methods could be used to establish cut-offs that would favor sensitivity or specificity. In the case of patient triage, increased sensitivity would be desirable. While the Youden index indicated a LOD score >1 had the best overall score performance in our cohort, selection of ≥1 as a cut-off would increase the sensitivity of the score from 81.8% to 93.9%. Likewise, the thresholds for SICK and PEDIA could be changed to reflect the desired sensitivity or specificity of the score. This will be an important consideration for future studies when assessing the score performance in different populations and disease conditions.
A limitation of this study was the inability to determine outcomes for all children, as approximately 15% of patients were lost to follow up. The reasons for the high rate of abscondment in the unit are unknown, but consistent with ongoing surveillance studies in the hospital. Clinically, the children who absconded were more likely to receive a blood transfusion (42.3%) than children who were discharged (33.7%) but less likely to receive anti-malarial treatment (quinine or artemisinin-based therapy). There were no differences in clinical diagnoses (malaria, pneumonia, sepsis and meningitis), pre-treatments or treatments (IV fluids, glucose, antibiotics) in hospital between children who were discharged and absconded. The absconders had higher LOD scores than survivors but lower LOD scores than non-survivors. In order to reliably assess patient outcomes in resource-constrained settings, it will be important to understand why patients abscond (for example, quality of care, direct/indirect costs, social reasons). Rates of missing data increased with increasing score complexity reaching 10.3% in the SICK score. While all variables in LODS and PEDIA were missing at random, several of the variables in SICK were more likely to be missing in children who died. When we evaluated which variables were more likely to be missing (Additional file
1: Table S1), it appears that variables requiring equipment (blood pressure, oxygen saturation, temperature) or direct assessment (heart rate, respiratory rate) had higher rates of missingness. These findings may reflect the limited resources of the site (in terms of both equipment and personnel), as well as the severity of illness, where documentation of vital signs was not prioritized in situations of critical illness.
Overall, 4.7% of the children in our study died, which is consistent with the mortality rates observed in the cohorts used to develop the LOD score (n = 23,890 children, 4.2% mortality) and PEDIA score (n = 8,091, 5.1% mortality). The sites used to develop these scores represent heterogeneous sites across Africa where malaria transmission intensity and the etiology of disease vary.
Acknowledgments
We thank all the patients and their families, the medical officers, nurses and research assistants that cared for the patients and collected study data, and the medical superintendent of the Jinja Regional Referral Hospital. This work was supported by a kind donation from Kim Kertland, the Tesari Foundation, the Canadian Institutes of Health Research (CIHR) MOP-115160, −13721 and −136813 (KCK), a Canada Research Chair in Molecular Parasitology (KCK), Canada Research Chair in Infectious Diseases and Inflammation (WCL), CIHR Clinician-Scientist Training Award (MH), CIHR Post-Doctoral Research Award (ALC), and the Sandra Rotman Centre for Global Health. The funders had no role in study design, data collection, data analysis, data interpretation, writing of the report, or decision to submit the article for publication.
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
The study was conceptualized and designed by KCK with input from WCL. Patient recruitment and data collection were obtained by SN, ROO, MH and ALC. Statistical analysis was performed by ALC, with input from KH, MH, CCJ, WCL and KCK. ALC and KCK wrote the manuscript with input from all authors. All authors read and approved the final version.