Prognostic accuracy of age-adapted SOFA, SIRS, PELOD-2, and qSOFA for in-hospital mortality among children with suspected infection admitted to the intensive care unit

While the prevalence and mortality of paediatric sepsis has become comparable to figures reported in adult ICUs in high-income countries [1‐3], defining sepsis in the absence of a gold standard remains a challenge [4]. Following the 2001 consensus statement of the Society of Critical Care Medicine, paediatric sepsis was defined as infection in the presence of at least two out of four criteria of the systemic inflammatory response syndrome (SIRS) [5, 6]. The 2005 Consensus definition for paediatric sepsis maintained the requirement for SIRS, providing further specification on organ failure definitions [6]. The validity of SIRS criteria to identify and risk-stratify patients with sepsis has been challenged in adults, where insufficient sensitivity and specificity were demonstrated [7, 8]. While tachycardia and tachypnoea represent adaptive mechanisms commonly seen in febrile childhood infections, including diseases with near-zero mortality (e.g. bronchiolitis [9]), the face, and construct validity and sensitivity of SIRS criteria have not been studied in large cohorts of critically ill children.

The recent Sepsis-3 consensus definition emphasized that sepsis is differentiated from uncomplicated infection by the presence of life-threatening organ dysfunction as a result of a dysregulated host response to infection [10]. The Delphi process, systematic reviews, and development and validation cohorts leading to Sepsis-3 were based on adult populations and the task force recognized “the need to develop similar updated definitions for pediatric populations” [11]. However, current paediatric sepsis definitions remain essentially based on Sepsis-2, representing a major obstacle towards research, benchmarking, coding, and quality monitoring [11, 12]. The operationalization of clinical criteria to identify individuals meeting outcomes consistent with sepsis in Sepsis-3 is based on the Sequential Organ Failure Assessment (SOFA) score, however neither SOFA nor quick SOFA (qSOFA) were developed for children.

We hypothesized that in children admitted to ICU with infection, the presence of organ dysfunction would better identify patients at substantially higher mortality in comparison to the presence of ≥ 2 SIRS criteria. We compared the performance of SIRS criteria with measures of organ failure to characterize outcomes of children with sepsis.

In this multicentre cohort of 2594 children aged < 18 years admitted to ICU with infection, we externally validated and assessed the discriminatory capacities of SIRS, severe sepsis, SOFA, PELOD-2, and qSOFA. We observed superior prognostic accuracy of SOFA and PELOD-2, both for in-hospital mortality and for the composite outcome of in-hospital mortality or ICU length of stay of ≥ 3 days, in comparison with SIRS, severe sepsis, or qSOFA. SIRS lacked specificity to identify children with infection at substantially higher mortality risk.

Key features underlying the Sepsis-3 consensus definition relate to the differentiation of sepsis from non-life-threatening infection, operationalization of the definition, and establishment through a data-driven process using large cohorts [18]. In contrast, paediatric Sepsis-2 definitions focus on systemic inflammation, applying non-validated criteria commonly seen outside sepsis, and have specific requirements for individual organ dysfunctions, attributing more weight to cardiovascular or respiratory organ dysfunction [19, 20]. The paradigm of SIRS as a feature of paediatric sepsis has been maintained for two decades, but neither SIRS nor the particular organ dysfunction criteria—which overlap with multiorgan dysfunction—to define severe sepsis in children have been externally validated in large ICU cohorts. Previous studies reported that SIRS criteria are met in > 90% of febrile children presenting to ED, of which < 5% require ICU admission [21]. We demonstrate that ≥ 2 SIRS criteria are present in 81.8% of paediatric patients admitted to ICU with infection, resulting in poor specificity and poor positive predictive value to capture patients at risk for adverse outcomes. While each additional SIRS criterion was associated with an increase in the relative odds of the primary, and secondary outcomes, significance was only reached when 3 or 4 SIRS criteria were present. Furthermore, the sensitivity of ≥ 2 SIRS criteria to discriminate in-hospital mortality was inferior to an incremental increase by ≥ 2 points in SOFA or PELOD-2, challenging the common notion that SIRS has excellent sensitivity for sepsis. Our findings are supported by a study in critically ill adults, demonstrating shortcomings in sensitivity, specificity, and validity of ≥ 2 SIRS criteria to define sepsis [7]. The limited utility if SIRS is further illustrated by substantial differences when applying Sepsis-3 versus Sepsis-2 criteria to infected children [11]. In a recent study on children with bloodstream infection, 30-day mortality was 1% in the presence of bacteraemia and SIRS without organ dysfunction, but increased to 17% when organ dysfunction was present [22]. Hence, our findings support abandoning SIRS since operationalising inflammation performs inferior to operationalising organ dysfunction when predicting death or prolonged ICU stay as outcomes.

Both age-adapted SOFA and PELOD-2 were superior to SIRS in identifying patients with infection at greater risk of mortality. Given the limited evidence on optimal blood pressure thresholds [23, 24], and to avoid overfitting of models, we applied the validated PELOD-2 age-specific cut-offs for cardiovascular and renal dysfunction, which may partially account for the similar performance observed between SOFA and PELOD-2 [25]. When simplifying the discrete scores (SIRS, SOFA, PELOD-2, qSOFA) to binary categorizations, PELOD ≥ 8 performed best. Notably, Sepsis-3 defined an increase in SOFA by ≥ 2 points based on the a priori requirement to identify presence of ≥ 1 (new) organ dysfunction to characterize sepsis, and not by post hoc derivation of optimal cut-offs. While a PELOD-2 ≥ 2 will capture patients with ≥ 1 organ dysfunction, PELOD ≥ 8 performed best in our study but such scores will predominantly reflect multiorgan dysfunction. Our findings support the operationalization of clinical criteria to paediatric patients with sepsis [10] and are highly comparable to a large external validation cohort in critically ill adults captured by the same database [8]. Leclerc and colleagues assessed PELOD-2 in 862 children with infection recruited in the original PELOD cohort and reported a high in-sample performance [23]. Our findings are further supported by a recent single centre PICU study including patients up to 21 years which tested a paediatric SOFA adaptation [26] based on the same PELOD-2 cut-offs for arterial hypotension and renal dysfunction but applying a more granular score increase, resulting in excellent performance. In contrast to the paper by Matics et al. [26] which did not report on SIRS or severe sepsis, we analyzed multicentre data of patients < 18 years, using mortality as the primary and mortality and/or PICU length of stay ≥ 3 days as the composite secondary outcome, and applied crude and adjusted analyses using similar methodology to adult Sepsis-3 validation cohorts [8].

qSOFA has been proposed as a screening tool in adults with infection, prompting assessment for evidence of organ dysfunction on hospital floors or in the ED [10]. In our study, the performance of our adapted qSOFA score to identify children who subsequently died or had prolonged length of stay was only moderate, which potentially could reflect the use of ventilation, sedation, and inotropes in ICU, altering respiratory rate, blood pressure, and GCS. In comparison, other PICU sepsis scores and paediatric Early Warning Tools have reported AUROCs of > 0.80 [12, 27]. Specific features of paediatric sepsis and septic shock [28], such as late development of arterial hypotension and a higher proportion of fulminant presentations [12, 22, 29, 30], warrant improved rapid identification of infected paediatric patients with organ dysfunction [30] across emergency department, hospital ward, and ICU settings. Given the high proportion of patients with organ dysfunction (74% with SOFA ≥ 2) in our cohort, future studies are needed to test the discriminatory performance in cohorts of lower average acuity.

The key strengths of this study relate to the application of stringent data-driven validation procedures to allow comparison of the prognostic accuracy of SIRS, severe sepsis, SOFA, PELOD-2 and qSOFA. Moreover, given the limitations of using mortality as an outcome, analyses included the composite outcome of mortality or ICU length of stay of 3 days or longer, aligned with adult Sepsis-3 studies [8, 13]. We deliberately did not restrict the study to admissions coded as sepsis but instead included all children admitted to ICUs with suspected and confirmed infection. Diagnoses were based on assessment by trained ICU specialists and not administrative coding.

Our study carries several limitations. First, it is based on a binational prospective dataset of adult and mixed adult-paediatric ICUs, whereas the main PICUs contribute to a different registry, which may have led to selection bias. In contrast to the paediatric ANZPIC registry, the ANZICS APD has been prospectively collecting data on SIRS, APACHE and SOFA score in patients admitted to the contributing units which ensured consistent practices of organ dysfunction assessment and data monitoring. Due to the high centralization of PICU services in Australia and New Zealand, it is common for critically ill children outside large metropolitan areas to be admitted to mixed ICUs. In the present study, 149 (5.8%) children required secondary interhospital transfer to a PICU, all of which were tracked to capture the primary and secondary outcome. Second, capturing the worst parameters within the first 24 h of ICU admission may not capture peak disease severity, resulting in lower performance of the scores [26]. However, prediction requires early assessment by definition and the passage of time is associated with competing risk bias due to death or discharge from ICU. In a recent study on a different cohort of septic children admitted to specialized PICUs we have demonstrated that a small set of clinical variables available within the first hour of PICU admission allows to establish robust severity stratification for paediatric sepsis mortality [12]. Third, although data collection for this binational ICU registry had been monitored using regular quality controls and mandatory audits, data had not been primarily captured for sepsis studies. Fourth, the SOFA score was modified as detailed information on vasopressor type and dose was not consistently available. Finally, two items of the PELOD score, including pupil size and serum lactate levels, were not available in the database, which may have reduced the performance of PELOD. Several studies have identified lactate as one of the best predictors of paediatric sepsis severity [12, 31, 32].

In conclusion, the two SIRS variables based sepsis criteria had poor specificity and diagnostic performance to discriminate children with infection at substantially higher mortality risk. In contrast, SOFA and PELOD-2 had significantly greater prognostic accuracy for in-hospital mortality. Our findings indicate that age-specific translation of Sepsis-3 definitions to critically ill children using validated measures of organ dysfunction should be considered in the next revision of paediatric sepsis definitions. In contrast, the performance of qSOFA to identify patients with organ dysfunction at risk for worse outcomes was poor, and may not be of sufficient clinical value to be recommended as a screening tool for paediatric age groups within the ICU.