Derivation and validation of a prognostic model for postoperative risk stratification of critically ill patients with faecal peritonitis

The Genetics of Sepsis and Septic Shock in Europe (GenOSept) and Genomic Advances in Sepsis (GAinS) are prospectively gathered cohorts of critically ill septic patients with FP recruited from multiple centres in Europe. They include data from patients with various degrees of illness severity, including potential risk modifiers and confounding factors (such as comorbidities, indices of acute physiological derangement, organ support, radiological and laboratory findings, origin of FP) [12, 13]. These diagnostically homogeneous cohorts of FP patients, gathered primarily for the purposes of studying genetic epidemiology in sepsis, also provide high-quality data well suited to the development and testing of a prognostic model specific to this postoperative patient population.

The primary aim of this study was to develop and validate a prognostic modelling tool able to stratify postsurgical critically ill patients with FP, by quantifying their mortality risk in the short- (28 day) and long-term (6 month), independently from intra-operative surgical findings, using prospectively collected data from the GenOSept and GAinS cohorts.

The same inclusion and exclusion criteria were used for both cohorts. Inclusion criteria: adult patients (>18 years) admitted to a High Dependency Unit (HDU) or Intensive Care Unit (ICU) with FP, defined as visible inflammation of the serosal membrane that lines the abdominal cavity, secondary to contamination by faeces, as diagnosed by the operating surgeon at laparotomy. All critically ill patients in this cohort, therefore, were recruited after the diagnosis was established during surgical source control. Exclusion criteria: peritonitis due to gastric or upper GI-tract perforation (e.g. gastric or duodenal ulcer perforation, small bowel perforation), patient or legal representative unwilling or unable to give consent; patient pregnant; advanced directive to withhold or withdraw life-sustaining treatment or admitted for palliative care only; patient already enrolled in an interventional research study of a novel/unlicensed therapy (patients enrolled in interventional studies examining the clinical application or therapeutic effects of widely accepted, “standard” treatments, were not excluded); patient immunocompromised (known regular systemic corticosteroid therapy, exceeding 7 mg/kg/day of hydrocortisone or equivalent, within 3 months of admission and prior to acute episode; known regular therapy with other immunosuppressive agents, e.g. azathioprine; known to be HIV positive or have acquired immunodeficiency syndrome as defined by the Centre for Disease Control; neutrophil count less than 1000 mm⁻³ due to any cause, including metastatic disease and haematological malignancies or chemotherapy, but excluding severe sepsis; organ or bone marrow transplant receiving immunosuppressive therapy).

The definition of sepsis was based on the International Consensus Criteria: “the clinical syndrome defined by the presence of both infection and a systemic inflammatory response” [14]. Patients were followed for up to 6 months from enrolment or until death.

The case report form (CRF) was developed and tested by CH, CG, AG, JDC and Dr J. Millo, together with other members of the GenOSept Consortium. Variables recorded included demographic, clinical and outcome data. A specific electronic case report form (eCRF) was developed by Lincoln, Paris, France, using software developed in collaboration with JDC. The database was password-protected, allowing investigators to enter data into the eCRF online, and included audit trail capability for data entry and subsequent modifications. To minimise errors, logical range checks were in place so that the investigators would be alerted if an attempt was made to enter data values outside the expected ranges.

Quality assurance (QA) was performed by P.H., C.G., A.W., A.G. and C.H, who systematically reviewed all data. Data queries (DQs) were generated within the eCRF for missing or erroneous data and sent electronically to the relevant investigators for action, where necessary. Up to the end of January 2011, an estimated 3986 valid DQs had been generated, with a response rate by the investigators of approximately 92%. Common reasons for DQs were missing information, particularly the Charlson Index, antimicrobial use, estimated day of onset of FP before ICU admission, information about circumstances of GCS assessment and outcome data.

All patients’ eCRFs were reviewed by experienced critical care physicians. Where the patient’s eligibility for inclusion in the relevant cohort was unclear, clarification was sought from the investigators. Regular QA reports were provided to the relevant Management Committee for review; the National Investigators were contacted regarding quality issues if necessary.

The derivation cohort included 462 patients with FP recruited in the UK. Their median (inter-quartile range, IQR) age was 69.4 (58.6–77.2) years. The geographic validation (non-UK) cohort included 515 FP patients recruited to the GenOSept study from the other European countries. Their median (IQR) age was 69.1 (58–77) years. The temporal validation cohort included 323 FP patients recruited in the UK between January 2011 and March 2015. Their median (IQR) age was 68.3 (57.6–77.2) years. For details of the recruiting centres, please see Additional file 1.

The baseline characteristics and the outcomes of the three cohorts are presented in Tables 2 and 3, respectively.

The age distribution was not significantly different across the cohorts, although the derivation cohort had a higher proportion of patients aged over 75. Males predominated in all cohorts. The racial distribution was more heterogeneous in the geographic validation cohort, while the derivation and the temporal validation cohorts were almost entirely Caucasian. Among the comorbidities diabetes, previous serious infections and other illnesses were more prevalent in the geographic validation cohort, compared to the other cohorts. The underlying causes for FP varied across cohorts, with anastomotic breakdown being particularly common in the geographic validation cohort. Baseline Sequential Organ Failure Assessment (SOFA) and Acute Physiology and Chronic Health Evaluation II (APACHE II) scores and prevalence of mechanical ventilation on day one were comparable across the cohorts. The occurrence of acute renal failure on day one was more frequent in the geographic validation cohort, with differences with the other cohorts (32.7, 42.8 and 23.3% for the derivation, geographic and temporal validation cohorts, respectively), accompanied by a difference in the utilisation of renal replacement therapy (21, 21.3 and 7.5% for the derivation, geographic and temporal validation cohorts, respectively) on day one. The geographic validation cohort was characterised by higher mortality rates (at all time points) and longer ICU stay, compared to the other two cohorts; this latter feature was also reflected, although to a lesser extent, in the length of hospital stay.

When evaluated using a receiver operating characteristics (ROC) curve, the discriminatory performance of the 6-month prognostic model in the UK derivation sub-cohort yielded an AuROC of 0.81 (95% CI 0.76–0.86) as indicated in Fig. 1a. At geographic validation in the non-UK sub-cohort, the 6-month prognostic model produced an AuROC of 0.73 (95% CI 0.69–0.78; Fig. 1b). At temporal validation, the 6-month model yielded an AuROC of 0.76 (95% CI 0.69–0.83; Fig. 1c).

The 28-day prognostic tool also performed similarly, yielding an AuROC 0.82 (95% CI 0.77–0.88; Fig. 2a) for the derivation UK sub-cohort. At geographic validation in the non-UK sub-cohort, the 28-day prognostic model produced an AuROC of 0.75 (95% CI 0.69–0.80; Fig. 2b). In the temporal validation cohort, the 28-day model yielded an AuROC of 0.79 (95% CI 0.71–0.87; Fig. 2c).

The 6-month FP prognostic score produced numerical values which can be stratified within 5 intervals (0–2; above 2–4; above 4–6; above 6–12; above 12) corresponding to five levels of 6-month mortality risk. The 28-day mortality FP score produces values classified within 5 intervals, corresponding to different risk categories for the outcome (0–2; above 2–4; above 4–8; above 8–16; above 16). The observed mortality rates corresponding to each class of risk for the two scoring systems are presented in Table 4 for all three cohorts (Additional file 1: Figs. S1 and S2 display the corresponding histograms of mortality). A 6-month FP score above 12 is consistently associated with a greater than 50% mortality risk at 6 months across all cohorts. A 28-day FP score above 16 is associated with a greater than 40% mortality risk for the 28-day outcome for the derivation and geographic validation cohorts, but not for the temporal validation cohort, in which the highest observed mortality risk was around 22%.

To assess how the FP models compare, as prognostic tools, to the routinely used SOFA and APACHE II scores, we calculated AuROCs for these scoring systems, to predict 6-month and 28-day mortality, in order to compare each tool across all cohorts and for both outcomes. For 6-month mortality, the SOFA score produced AuROCs of 0.73 (95% CI 0.68–0.78), 0.68 (95% CI 0.63–0.72) and 0.62 (95% CI 0.54–0.7) in the derivation, geographic and temporal external validation cohorts, respectively, while the APACHE II score yielded AuROCs of 0.74 (95% CI 0.7–0.79), 0.71 (95% CI 0.66–0.75) and 0.69 (95% CI 0.62–0.77) for those cohorts, respectively. For the 28-day mortality outcome, the SOFA score produced AuROCs of 0.76 (95% CI 0.7–0.82), 0.66 (95% CI 0.6–0.73) and 0.67 (95% CI 0.58–0.77) in the derivation, geographic and temporal external validation cohorts, respectively, while the same AuROCs for the APACHE II score were 0.71 (95% CI 0.64–0.77), 0.69 (95% CI 0.63–0.75) and 0.75 (95% CI 0.67–0.83), respectively.

The AuROCs obtained using the FP scores were consistently superior to those obtained with the SOFA score, with statistical significance across all cohorts (derivation, geographic and temporal external validation) and for both 6-month and 28-day mortality outcomes (Additional file 1: Figs. S3 and S4, respectively).

The AuROCs obtained using the FP scores were also superior to those derived using the APACHE II score for both outcomes, although statistical significance was not consistently achieved across all cohorts (Additional file 1: Figs. S5 and S6, for 6-month and 28-day mortality, respectively).

The SOFA score was developed in a mixed (medical and surgical) ICU population [28] and has been subsequently externally validated in various populations [1], such as cardiac surgical patients [29] and critically ill burn patients [30].

While the SOFA score was originally developed for the purpose of describing the evolution of organ dysfunction, rather than for prognostic purposes, we previously found that both admission SOFA and trends in the global SOFA scores were closely associated with mortality [25]. Many studies have reported the use of the SOFA score both in isolation [31‐35] and in combination with other variables [36, 37], for the purpose of outcome prediction. In our study, neither the SOFA nor the APACHE II scores, when used in isolation, performed as well as the tools developed here. Furthermore, day one SOFA performed particularly poorly in the temporal validation group, while the APACHE II risk model (which was developed for the purpose of outcome prediction) performed more consistently across the three cohorts, both for the 6-month and the 28-day outcome. This finding suggests that the value of SOFA lies primarily in describing temporal changes in organ function. Nevertheless, a single SOFA score can be successfully integrated with other parameters, to provide a prognostic tool with improved accuracy [36, 37], as we have done for day one SOFA in these analyses. While the confidence intervals for the AuROCs were relatively wide, when the FP models were compared to SOFA, statistically significant differences were found across all cohorts. This was not always the case for comparisons with APACHE II, further highlighting the superior prognostic accuracy of this severity score compared to an isolated, day one SOFA score.

The adverse effect of hypothermia on the outcome of critically ill patients has been described by other authors, although data on the relevance of hypothermia to outcomes remain conflicting [38, 39]. Laupland and co-authors studied 10,962 medical, non-scheduled and scheduled surgical patients admitted to critical care with varying degrees of hypothermia and fever. Hypothermia was, after controlling for confounding factors, significantly and independently associated with mortality in medical patients [38]. Tiruvoipati et al. reported data from 175 elderly ICU patients, identifying lower temperatures and the Simplified Acute Physiology Score II (SAPS II) during the first day of ICU admission as being independently associated with higher hospital mortality [39, 40]. An association between severe hypothermia and the risk of ICU acquired infections has also been reported among medical patients [41].

An increased heart rate is a physiological response to infection and sepsis, and part of the systemic inflammatory response syndrome (SIRS). Sprung and colleagues found that the presence of SIRS predicts infection, severity of illness, organ failure and outcome, with the two most common SIRS criteria met during ICU stay being respiratory rate (82%) and heart rate (80%) [42]. Morelli and co-workers randomised a total of 154 septic shock patients to receive a continuous infusion of esmolol (targeting a heart rate of 80–94 bpm) or standard treatment in an open label trial. The patients in the esmolol arm achieved lower heart rates, without an increase of adverse events. Interestingly, an improvement in survival and other secondary outcomes was also reported [43]. Others have found that a high daily mean heart rate was a significant predictor of ICU mortality [44].

Anaemia in surgical patients undergoing both cardiac and non-cardiac procedures has previously been reported to be associated with worse outcomes [45‐49]. Beattie and co-workers performed a retrospective observational study of 7759 non-cardiac surgical patients to establish the relationship between preoperative anaemia and postoperative mortality and found that preoperative anaemia was common and strongly linked with postoperative mortality, even after adjustment for major confounders [49].

All of the patients with FP included in the analyses reported here underwent laparotomy (the diagnosis of FP was based on the intra-operative finding of faecal soiling of the peritoneal cavity). In addition, a significant proportion of patients (40%) were documented to have cardiovascular co-morbidity, a group in which anaemia has been shown to be associated with worse survival and major adverse cardiovascular events. Although anaemia may be associated with a poor outcome, data on the effects of blood transfusion are conflicting, with most reports not demonstrating benefit from transfusion aimed at achieving a higher haemoglobin threshold [50, 51].

One limitation of the current study is that we were unable to test the performance of other scoring systems such as the colorectal POSSUM, the MPI, PIA II or the PSS in our dataset, as these systems all require some intra-operative or preoperative findings, which were not available to us. On the other hand, the fact that our scores do not require any intra-operative findings could be viewed as an advantage.

A further limitation is the lack of comparison with alternative and more recent versions of severity scores, such as the Simplified Acute Physiology Score (SAPS) 3, the APACHE III or IV or the Mortality Prediction Model (MPM) III. We consider this unlikely to have a significant impact on the validity of our results, as multiple studies have shown that the performance of such tools, even in their more recent versions, is not significantly improved [52]. A pragmatic decision was made to rely on the APACHE II (rather than more recent versions of APACHE) in view of its practicality, the fact that it is the only available non-proprietary version in widespread clinical use [1, 2, 4] and the comparator of choice in multiple other recently published studies [53, 54].

The SOFA score may be a less than ideal comparator, as the SOFA was not originally developed for prognostication. Multiple previous studies have, however, reported using the SOFA score, both in isolation [31‐35] and in combination with other parameters [36, 37], for outcome prediction.

Another limitation is that our study was not designed to evaluate the influence on outcome of the timing and adequacy of source control or antibiotic treatment. All patients included in the study reported here received source control via surgical laparotomy prior to recruitment and the overwhelming majority of the patients (91.8%) received antimicrobial therapy deemed to be adequate [24].

Mortality differed markedly between the cohorts, even though they were recruited using the same inclusion and exclusion criteria. Whilst it is impossible to identify with certainty which factors explain these differences, multiple potential reasons can be postulated. Firstly, the variation in mortality rates strongly correlates with the occurrence of acute renal failure on day one. Acute renal dysfunction and deteriorating renal function have both been consistently associated with poor outcome in this specific subset of patients [24, 25]. The effects of random variability and the fact that in the UK the centres recruiting to GenOSept and those recruiting to GAinS were not always the same may have also contributed. Finally, improvements in the management of sepsis over the years may have influenced the incidence of renal failure and outcomes.