Developing practical clinical tools for predicting neonatal mortality at a neonatal intensive care unit in Tanzania

The world has made substantial progress in tackling infant mortality over the past 50 years [1], and neonatal (up to 28 days of age) mortality rates (NMR) are steadily falling in high-income countries (HICs). Of 2.4 million neonatal deaths in 2019 worldwide, the World Health Organization (WHO) [2] estimated that most occurred in low- and middle-income countries (LMICs). In Tanzania, NMR is falling at a much lower rate than mortality among children under 5 years [3] and mortality is especially high in preterm neonates [4]. Tanzania has made substantial efforts in improving neonatal care and reduce mortality by implementation of various guidelines and policies recommended by WHO. This includes essential new-born care, kangaroo mother care, integrated management of childhood illness (IMCI), growth monitoring, and care for childhood development. The national coverage of these interventions remains low (40%) compared to that recommenced by WHO (80%). To address the gap the government has consolidated national guidelines, which are available in all levels of health facilities. While improved thermal regulation of neonates using incubators, better formulas for nutrition, surfactants to treat respiratory distress syndrome, and antimicrobials to fight infections have significantly reduced new-born deaths in HICs, such as the UK [1], these are often unavailable or unaffordable to families in low-resource settings. Limited human resource and health systems capacity are factors that prevent the scale up of health interventions in LMICs [5]. Indeed, the main causes of neonatal deaths include prematurity, intrapartum-related complications, such as birth asphyxia, and preventable and treatable infections, such as early- and late-onset neonatal sepsis [2, 6]. Reducing the mortality of neonates remains an urgent challenge [7].

The United Nations Children’s Fund (UNICEF, [8]) estimated 19 deaths per 1000 live births in Tanzania in 2015, primarily caused by birth asphyxia (29.3%), prematurity (24.7%) and sepsis (19.7%). Two large studies in hospitalised neonates have also found the major causes of neonatal mortality to be birth asphyxia, preterm deliveries, infections (sepsis), respiratory distress syndrome and congenital malformations [9, 10]. However, no study has explored the relationship between mortality, causes of mortality and changes in vital signs, such as temperature or heart rate. These data are crucial when drafting policy recommendations regarding management of neonatal conditions, such as neonatal sepsis, and reducing mortality.

Early warning systems that indicate patients at high risk or with deteriorating conditions are increasingly recognised as valuable tools for clinicians [11]. Although such systems are used in medicine for adults and children, they are not well-developed for neonatal care, even in high-resource settings, with existing tools often complex to use, demanding in terms of data needed or specific to certain infant groups [12, 13]. A review of early warning- and track and trigger systems used for infants by Mortensen et al. [13] found that no such scoring systems existed for neonates in NICU or those born prematurely. Mitchell et al. [12] explored the feasibility of using an early warning score for neonates in Kenya and concluded that such scores could be useful to identify at-risk neonates. However, they also highlighted the issues with limited data recording in low-resource settings, which the score relies on.

Currently, the guidelines typically used in neonatal medicine are based on cut-offs for individual vital signs [14, 15]. For example, low temperature (< 36.5 °C) is often considered an indicator for clinical deterioration, but it is not normally used in combination with other measurements, such as oxygen saturation or heart rate, to predict neonatal outcomes. The Apgar score is a simple evaluation system to assess neonatal health including five easily evaluated vital signs: appearance, heart rate, reflexes, muscle tone, and respiration [16]. A tool that combines vital signs (throughout hospitalisation) and other risk factors associated with mortality could aid effective clinical decision making.

There is a demand not only for early warning systems for neonatal care, but also for tools that are easily usable in low-resource settings [12]. Computer-based clinical decision support tools are increasingly used in healthcare and may involve statistical and artificial intelligence or machine learning approaches [17]. Machine learning can identify patterns in complex datasets, predicting clinical outcomes, and has recently been explored in other areas of medicine, including diagnostics and prescribing practices [18‐21]. However, the most of these machine learning tools are a black box approaches that generate algorithms not readily interpretable by a clinician or usable without access to computational resource, limiting their usefulness in a low-resource setting. For this reason, clinician-friendly tools, such as generalised linear models or decision trees whose results can be shown graphically, should be considered. We hypothesise that tools that use a combination of models, rather than individual measurements, give a more accurate prediction of neonatal mortality. These tools would need an appropriate balance between accuracy and ease of use in a low-resource setting.

Here, we aimed to i) identify risk factors associated with neonatal mortality in the NICU in Bugando Medical Centre, Mwanza, Tanzania, ii) identify the most accurate GLM or decision tree for predicting mortality, and iii) provide a tool that provides clinically relevant cut-offs for predicting mortality that is easily used by clinicians in a low-resource setting.

Early warning systems in medicine are being increasingly adopted but remain insufficiently well-developed in neonatal medicine [13]. Moreover, there is a demand for early warning systems for neonatal mortality that are readily usable in low-resource settings [12]. Our research addresses the gap in availability of early warning systems for neonatal mortality that combine underlying conditions and vital signs and that are readily usable by clinicians in low-resource settings. Here, we used data from a cohort of neonates at Bugando Medical Centre, in Mwanza, Tanzania to first identify risk factors, including vital signs, for mortality and second, to use these findings to develop a potential clinical tool.

Using a standard statistical approach (GLMs), we identified the risk factors associated with neonatal mortality in neonates admitted to the NICU at BMC, including changes in vital signs. Low birthweight strongly associated with mortality – a finding consistent with previous studies [23]. Babies born preterm with low birth weight may die because of acute complications like hypothermia and hypoglycaemia [28‐30], due to a lack of feasible, cost-effective care, such as breastfeeding, warmth support and basic care for breathing difficulties and infection. Low body temperature (defined as a core temperature below 36.5 °C) was also a risk factor. It is often associated with low birthweight and is known to increase mortality and morbidity [31]. Oxygen saturation and heart rate are vital signs that can change rapidly and are often monitored continuously. However, our analyses show that the first measurements (taken upon admission) were nevertheless significantly associated with mortality. Perinatal asphyxia, a condition known to account for much of neonatal mortality [31, 32], was also a significant risk factor.

Using this information, we considered the development of a tool for clinicians, to help identify neonates at risk of mortality. Due to their relative ease of interpretation, we compared GLMs and decision trees as potential tools that can be presented visually. Decision trees are flowcharts that have been used to aid classifications in medical problems [33‐35] and could be used to formulate an early warning system, such as the one used in the UK to monitor deterioration of paediatric patients [11]. We then found that our final decision tree contained only birthweight, with a cut-off of 1.325 kg and an accuracy of 0.69. When comparing the accuracy of predictions across models, we found that the decision tree approach was inferior to a GLM, even when the GLM only included one variable (birthweight). More sophisticated machine learning tools such as random forests are likely to be more accurate but cannot be presented visually and therefore did not offer a viable option for our research.

Currently used guidelines in neonatal care typically focus on individual cut-offs but the inclusion of multiple variables improves accuracy. We used ROC curves and AUC to compare the performance of our binary classifier and obtain the cut-off corresponding to the sensitivity and specificity that optimises accuracy of classification. The best single risk factor in our GLM was birthweight, and using a ROC curve, we identified the cut-off to be at 1.750 kg with a corresponding accuracy of 0.69. Existing guidelines with a 2.5 kg cut-off correspond to a lower accuracy of 0.66. For temperature, the cut-off was 36.5 °C with an accuracy of 0.67, matching the WHO’s guidelines that define normal temperature range to be between 36.5 °C and 37.5 °C [15]. Accuracy increased when multiple variables were included in the model. For the best two-variable model (birthweight and temperature), accuracy was 0.70. Accuracy increased to 0.72 for the 3, 4 and 5 variable models. We selected the three-variable GLM (birthweight, temperature, and heart rate) as providing the best balance between accuracy and ease of use and showed an effective way of displaying the tool by plotting heart rate against temperature, with several cut-offs for different birthweights.

This tool could be effective in different settings but has been fine-tuned to fit our dataset obtained from a NICU with 48.5% mortality rate. It may require modification in other settings and should be supplemented with local data where possible. In addition, the trade-off between sensitivity and specificity must be kept in mind. A high sensitivity value, or the proportion of true positives, ensures that most neonates at risk of dying are detected. Given the seriousness of the outcome, it is important that at-risk neonates are not missed. However, a high number of false positives may put extra burden on healthcare staff and facilities in low-resource settings, and so desired specificity (which determines the number of false positives) should be considered.

Here, we presented a proof of concept for a simple tool that could be fine-tuned using additional data and ultimately lead to an evidence base that could be used to support policy recommendations around neonatal care. A larger sample size would improve accuracy of prediction and could also provide a fuller picture of risk at extremes of the measured variables where we had few datapoints available. Only the first set of recorded vital signs were used in this analysis, but incorporation of multiple measurements if available could also improve accuracy of predictions. Complete data on Apgar scores may also help to improve this tool.

The incompleteness of data underpinning our medical charts was a limitation of our study. The Apgar score is an important indicator of neonatal health, but it was not consistently recorded and therefore we excluded it from analysis. Our sample size was small, and we may have encountered a selection bias as enrolment relied on maternal consent.

Currently, there is a lack of early-warning systems for infant mortality in LMIC settings. A recent study in Kenya [12] demonstrated enthusiasm by stakeholders for an early warning scoring system for low birth weight or pre-term infants, with the ease of use of chart-based approach to capture multiple measurements being met with enthusiasm. We anticipate therefore that a decision tool based on our approach could be a useful addition to daily practice. Infants can be rapidly assessed against the chart whenever vital signs are measured to help identify those infants at risk, or deteriorating, so that additional care can be provided. However, we would recommend that more extensive data be collected to refine the tool before deployment in the clinical environment.

Vital signs are often excluded from analyses investigating risk factors associated with neonatal mortality. This study shows the importance of including these, along with any underlying conditions, as well as adding them to local guidelines concerning neonatal care. Low birthweight and temperature are an especially strongly associated with adverse outcomes, suggesting that greater use of incubators or kangaroo mother care could be useful [36]. We report that GLMs are superior to decision trees when assessing risk factors associated with mortality, and that ROC curves are useful tools for identifying cut-offs for clinical guidelines. This study shows how to combine multiple risk factors, including vital signs, from a GLM into a graphical tool that could be used in various low-resource settings to provide early warning of risk of mortality for infants in the first 28 days of life.