Prediction modelling in the early detection of neonatal sepsis

Articles were searched from distinct databases such as PubMed, Scopus, and CINAHL using the following keywords “Prediction model” “neonatal sepsis” “neonatal sepses”. Earliest ten-year papers were searched till September 2020 for the study purpose in all the databases.

Papers were comprehended if they fit the following inclusion criteria: (1) Operational definition of neonatal sepsis: Neonates having positive blood culture before 72 h of life were considered as early neonatal sepsis, whereas neonates having positive blood culture report after 72 h of life were termed as late neonatal sepsis. Meanwhile, neonates having positive blood culture report before 30 days of life were considered as neonatal sepsis patient [17]; (2) Case–control, cohort—prospective/retrospective studies predicting the prognosis of having neonatal sepsis (early/late onset) which is to be culture positive; (3) Prediction model developed through various statistical procedure like machine learning/logistic regression/artificial intelligence for early detection of neonatal sepsis; (4) The prediction developmental model should be either internally or externally validated, and (5) Articles published in English peer reviewed journal.

However, randomised clinical trial, review/systematic review/meta-analysis related to early/late neonatal sepsis were excluded from the study. Implementation of the prediction models pertaining to neonatal sepsis or prognosis of the disease was also ruled out from the study.

The study screening was conducted in accordance with the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines [18]. The first reviewer [P.S] evaluated all the search results. Further title, abstract and  subsequently full text screening were cross checked by the rest two reviewers (EAR, VK).  Title, abstract and full text screening was consummated on the ground of study design/disease/outcome/intervention/irrelevant. Any difference in the opinion regarding final included articles was resolved through discussions.

Data extraction was carried out by all the reviewers based on Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist [19, 20]. Extricate data consisted of objective, study design, patient characteristics, type of statistical model, predictors in the model, outcome measure, sample size and location. Significant predictors were identified by relative risk or odds ratio. Outcome measures were assessed by area under the curve, sensitivity, specificity, positive predictive and negative predictive value. Prediction model risk of bias assessment tool (PROBAST) was applied to gauge the risk of bias of the included articles and were classified into high, low, and unclear risk of bias. PROBAST risk of bias consists of questions from four domains which include participants, predictors, outcome and analysis. A subtotal of 20 questions were present for the judgement of the paper. These questions were answered as yes/no/probably yes/probably no/ unclear, where “yes” implies the absence of bias, while “no” signifies the presence of bias. An article is denoted as “low risk of bias”, if all the desirable answers to the 20 questions were yes or else it is designated as high risk. An unclear answer to any of the 20 questions represents the article to be of unclear risk of bias [21].

Systematic analysis of all the included studies was conducted to generate all the desired data. Statistical measure employed for building the model was determined to figure out the type of it. Predictors were viewed to identify its potential in the model. Performance of the prediction model were determined by noting the major indices like Area Under the curve (AUC), Specificity, Sensitivity, Negative Predictive Value (NPV) and Positive Predictive Value (PPV). External and internal validation of the models was also scrutinised to determine the efficiency of the model.

Majority (five studies) of the included articles were performed in United State of America [24, 26, 29, 30], while the remaining each were from Israel, China, Mexico, United Kingdom and Canada [22, 23, 27, 28, 31]. The highest sample size of all the included studies was 4794, while the minimal is 36, which are illustrated in Fig. 2. Most of the studies were single centre (six studies), while 30% (three studies) were of multicentre origin. Greater number of studies (seven studies) had applied logistic regression to build a prediction model, while two of the studies had applied artificial intelligence. Autoregressive hidden Markov model (AR-HMM) was applied by a single study for prediction model building [28]. Partly, the studies (five studies) had applied split validation (70:30 ratio) concepts for the derivation and validation of the model, while one of the studies had applied external validation [21]; while the other four studies had utilised bootstrap or concordance statistics methodology for internal validation.

Neonatal sepsis detection was the primary outcome of considerable number of studies (five studies) [27‐31]. Early-onset and late-onset sepsis detection were encountered as the main outcome in three and two different studies, respectively [22‐26]. Sheer single study had stratification of the early-onset neonatal sepsis patients based on the degree of illness as the defined outcome [25]. The length of the age of all the participants during outcome assessment varied from 0 to 30 days of life. The major characteristics of the included studies are summarised in Tables 1, 2 and 3. Predictors adopted in the comprehended studies for constructing the model were classified into four domains such as neonatal factors, maternal factors, clinical, laboratory parameters and miscellaneous. Table 4 represents the predictors notified in each study for the final model prediction.

Prediction model risk of bias assessment tool (PROBAST) was applied to appraise the risk of bias of the included articles. Six articles were of low risk of bias while two were of high due to their minimal sample size. Remaining two articles were of unclear risk of bias as “analysis domain” of the PROBAST table was unable to give a clear conclusion. Since, one of the domains was unclear, we labelled the paper as the same as per the PROBAST guidelines [21]. The detailed assessment of risk of bias for the included articles using PROBAST tool, based on individual domain is provided in the supplementary table 1.

Predictors’ assessment of the included articles was evaluated on the basis of neonatal factors, maternal factors, clinical/laboratory parameters with sign and symptoms and miscellaneous factor for quick detection of early, late or neonatal sepsis as a whole.

Maternal factors like maternal age, cervicovaginitis were the potential predictors in detection of sepsis as whole [27]; whereas, group B streptococcus status, duration of premature rupture of membrane aids in evaluating early-onset sepsis [24].

Sepsis detection can be eased out by neonatal predictors like fever, birth weight, foetal morbidity and gender [27]. Birth weight, endotracheal intubation, thyroid hypofunction, umbilical venous catheter (UVC) was the promising neonatal factors for predicting late-onset sepsis [23]; while gestational age, intrapartum temperature and antibiotics treatment were utilised as budding prognosticator for early-onset sepsis detection [24].

Clinical and laboratory parameters suchlike sick appearance, neutrophil–lymphocyte ratio > 1.5, and C-Reactive Protein > 0.75 were able to detect late-onset sepsis at the earliest [22]. Signs and symptoms such as increase in the number of bradycardia and mini-bradycardia events was a reflection on fast onset of neonatal sepsis [28]. Moreover, platelet count, bradypnea, band cells heart rate (maximum, minimum), SpO₂ minimum, blood pressure systolic (minimum, mean), thyroid, acyl carnitine, amino acids and adrenal functions were few of the indicators which also showed significant results in the early detection of neonatal sepsis as whole [27‐30].

Miscellaneous factors like oximeter error did not show any significant result [28]; while, catheter presence showed positive sign in fast detection of neonatal sepsis [25].

The AUC (derivation model AUC = 0.92, validation model AUC = 0.90) noted for the model developed through clinical and laboratory parameters for the prognosis of having late-onset neonatal sepsis was quite satisfactory [22]. Formation and testing of various nomogram from the derivation and validation set (derivation AUC = 0.855, validation AUC = 0.834) for prediction of late-onset neonatal sepsis was also successful in doing so [23].

Intrapartum maternal risk factors were utilised to form a prediction model for early-onset sepsis detection with AUC of 0.807 and 0.794 from derivation and validation set, respectively [24]. Early onset sepsis identification by neural network model had AUC 92.5, accuracy of 86.74, sensitivity 80.32, and specificity of 90.4 [26]. Quantitative stratification algorithm for early-onset sepsis detection had classified neonates with sepsis risk at birth into three categories, i.e. clinical illness, equivocal and well appearing [25].

Autoregressive hidden Markov model with AUC of 0.74 clearly exhibited that physiological change in events such as bradycardia or mini-bradycardia can detect neonatal sepsis prior to culture report [26]. Maternal and neonatal factor model had predicted the neonatal sepsis disease with an AUC of 97.4, specificity 80%, sensitivity 93.33%, PPV 82.35 and NPV of 92.3 which had surpassed the performance of physician also [27]. Non-invasive model developed for early detection of neonatal sepsis through logistic regression showed better results when compared to that of the invasive model (AUC of invasive model 0.777, AUC of non-invasive model 0.824) [30]. Neonatal sepsis detection by another team of authors also displayed similar results (AUC 0.879) [29]. Newborn screening results like thyroid and adrenal function, acyl carnitine and amino acids also improved the model fit for detection of neonatal sepsis [31].

Half of the included studies had utilized 2:1 ratio, i.e. split method of derivation and validation set for evaluating the accuracy, precision and validity of model [22, 24, 25, 29, 30]; meanwhile, mere one study had used the external validation procedure of conducting the same study in different set of samples at different period of time [23]. 70:15:15 (70 derivation set, 15 validations set, 15 testing set) ratio was applied for validation of model developed by maternal and neonatal risk factor for identification of neonates with sepsis. Back propagation and neural network methods were applied to do so. The performance of the model was calculated on three metrics, i.e. root mean square error (RMSE), regression coefficient and statistical slope and intercept [27]. Similar back propagation method was applied by another study for validation of model for early-onset sepsis detection [26]. Average precision and F score was generated by precision recurve call for validation purpose in model built by autoregressive hidden Markov model [28]. Bootstrap method of internal validation (200 samples) was applied to validate model which predicted neonatal sepsis through the analysts of newborn screening [31].

This systematic review illustrated that few articles were available on prediction modelling for fast detection of early/late/neonatal sepsis as whole. Out of those, most articles included had displayed satisfactory results in terms of prediction of the disease. Maternal, neonatal, clinical and laboratory predictors identified through these models will assist health-care professionals in the rational management of the disease [22‐24, 27, 29, 30]. Model performance for quick detection of sepsis was almost equivalent to that of physician performance [27]. Stratification of early-onset sepsis into clinically ill, equivocal and well appearing using prediction modelling had helped in modifying the treatment approach for individual categories [25]. Physiological events like increased number of bradycardia or mini-bradycardia were identified as good predictors to detect the disease prior to the culture report [28]. Thus, it can be concluded that prediction modelling measures can be used as an additional approach to clinician decision, if not used alone.

Included articles in our study had either utilised clinical/laboratory parameters, neonatal/maternal risk factors, or physiological events of the neonates for building the prediction model [22‐31]. Birth weight characteristics, gestational age features, role of setting, sampling strategy and concomitant interventions were some of the primary predictors assessed for late-onset neonatal sepsis by routine neonatal screening [34]. Analogous results were found in one of the included articles where birth weight, endotracheal intubation, thyroid hypofunction and umbilical venous catheter (UVC) were the promising neonatal factors for predicting late-onset sepsis [23]. Study conducted by Seidel et al. for predicting late-onset sepsis by body surface screening had pooled sensitivity of 41% (95% CI 17–70%) and specificity of 56% (95% CI 34–76%), whereas another included study in our review for predicting late-onset sepsis by clinical and laboratory evaluation had shown sick appearance (OR 5.7, 95% CI 1.1–29.1), CRP > 0.75 (OR 5.4, 95% CI 1.1–26.3)), neutrophil to lymphocyte ratio > 1.5 (OR 6.7, 95% CI 1.2–38.5) as the chief predictors with model AUC of 0.92 [22]^.

Another systematic review was designed to evaluate the outcome of early neonatal sepsis by sepsis calculator, and had shown reduced antibiotic usage and fewer surges in mortality and readmission to hospital [32, 33]; whereas, one of the included articles in the review had different objective and had shown gestational age, intrapartum temperature and antibiotics treatment as budding prognosticator for early-onset sepsis detection [24]. Yet another included article had classified early-onset sepsis patients into various categories based on the degree of illness. Stratification of those patients had enabled the health-care professionals to render distinct treatment to various categories, thus ensuring proper management [25].

Study conducted by Liang et al. illustrated that prematurity, low birth weight and young age at presentation were some of the major factors which were associated with mortality in neonates with sepsis [35]. Artificial neural network approach was employed by another included article for analysing various maternal and neonatal predictors for early detection of neonatal sepsis and had suggested that maternal age, neonatal fever, apnoea, platelet count was the most valuable predictor of sepsis followed by cervicovaginitis, gender, bradypnea, band cells, catheter presence, birth weight, neutrophil counts and foetal morbidity.

Apart from the previously mentioned predictors, novel diagnostic marker such as neutrophil CD64, platelet to lymphocyte and neutrophil to lymphocyte can also be added to the prediction model for increasing the accuracy of diagnosis of neonatal sepsis as neutrophil CD64 when combined with standard biomarkers like CRP and WBC increases the sensitivity and accuracy of diagnosis [10, 37]. Likewise, inclusion of platelet to lymphocyte and neutrophil to lymphocyte ratio as predictors in the model will increment the diagnosing capacity of early-onset neonatal sepsis in particular [37]. Thus, addition of novel biomarker along with the standard biomarkers in the model will potentiate the predicting ability of the model.

Among the two of the included articles used for the quick detection of the late-onset neonatal sepsis, Huang et al. had the most appropriate methodology which can be used in other clinical settings for building prediction model [22, 23]. Logistic regression adopted for building the model had several advantages in comparison to the other approaches as it provide probability prediction of each predictor unlike only classifying it into different levels [38]. Models generated by this approach are less likely to be over fit even with the smaller datasets. Unlike other approaches like support vector machines and decision tree, it can easily accommodate larger data set and can refurbish easily to reflect new data set [39]. External validation approach was employed by Huang et al. to measure the accuracy of the model. It ensures reproducibility and transportability of the model in other clinical settings [40, 41]. Being a multicenter origin study, it provides the build model with satisfactory power to detect late-onset sepsis in heterogeneous subjects [42]. Validation cohort AUC of 0.963 of the study also highlighted similar results [23].

Out of all the included articles for fast detection of early-onset neonatal sepsis, Escobar et al.’s study can be considered to have remarkably relevant methodology [24‐26]. Similar kind of methodology was adopted by Puopolo et al. and Martinez et al. [22, 23]; however, Escobar et al.’s study can be primarily considered as it had stratified early-onset sepsis neonates into clinical illness, equivocal and well appearing. Stratification method adopted in the study will assist the clinician in providing appropriate treatment modalities and strategy for improved patient care in various subgroups [43]. Logistic regression, multicenter origin, and internal validation employed by this study had made it a desirable prediction model; however, external validation was necessary for replicating the methodology in other clinical settings.

Among the five selected articles for early detection of neonatal sepsis, three had used the same statistical measure, i.e. logistic regression [29‐31], while the remaining one had used autoregressive hidden Markov model approach [28]. Autoregressive Markov model is better than simple Markov model as it identifies a greater number of sequences. It also allows various sequences to be modelled rather than only inserting or deleting it. Unlike simple Markov models, these are quite expensive and require larger seed sequences to be trained thus, is difficult to replicate [44]. Study conducted by Thakur et al. 2019, 2018 was quite similar based on sample size, validation measure and outcome viewed. However, replicating it in other settings is still questionable as external validation was not performed [29, 30]. Bootstrapping method applied by Fell et al. is one of the efficient ways of internal validation when compared to that of cross or split validation as it utilises the entire data set for validating which produces results of low variance [45] still, cannot surpass the advantages of external validation. Thus, external validation being one of the inevitable parts of prediction modelling should be taken into consideration in the future research for the applicability quotient.

Articles containing model validation part (internal/external) were particularly included in the review, as models without validation were of low robustness and accuracy and difficult to replicate in other settings [46‐48].

Studies conducted in English language were only considered for the review due to which a number of articles had to be excluded and thus, can be considered as one of the shortcomings. Meanwhile, scarcity of data also affected the outcome of the review. Further research (both interventional and observational studies) is required in this field to generate more evidence.

Thus, the chief findings of this review are that prediction modelling can be considered as a novel way of approach towards the disease, which not only will help the clinician in making rationale and critical decision but will help the young researchers in considering prediction modelling as an important tool for investigation.