Evaluation of the INCREMENT-CPE, Pitt Bacteremia and qPitt Scores in Patients with Carbapenem-Resistant Enterobacteriaceae Infections Treated with Ceftazidime–Avibactam

Enterobacteriaceae are among the most frequent cause of bacterial infections in patients of all ages in both community and inpatient settings [1]. The remarkable ability of these organisms to acquire a growing array of mechanisms to evade the activity of broad-spectrum antibiotics therefore presents a growing challenge. In particular, the emergence and spread of carbapenem-hydrolyzing beta-lactamases (carbapenemases) limits our ability to treat many life-threatening infections [2]. Due to the well-rehearsed challenges of conducting randomized controlled trials (RCTs) in patients with carbapenem-resistant Enterobacteriaceae (CRE) infections, current management strategies are largely informed by observational data and expert opinion [3]. Observational studies can provide valuable insight into the real-world effectiveness of treatment alternatives in circumstances where execution of RCTs is not feasible. However, because treatment is not randomly assigned, confounding by indication may lead to a biased estimate of the treatment effect. Covariate adjustment is especially important in CRE studies due to the wide spectrum of disease severity and underlying health status of the patients who acquire these infections, which clearly influences both the management approach and outcomes. Risk scores are useful for covariate adjustment in these circumstances, and they can also facilitate comparisons of populations across different studies. Furthermore, some risk scores have been used clinically to define high-risk subgroups for whom more intensive management strategies may be targeted [4, 5]. A number of risk scores have recently been developed and/or validated specifically in patients with CRE infections [4, 6, 7]. The INCREMENT-CPE score (ICS) was developed to predict 14-day mortality in patients with carbapenemase-producing Enterobacteriaceae (CPE) bacteremia [4]. It was subsequently modified and validated for 30-day mortality in patients with bacteremic and non-bacteremic CPE infections [6]. The Pitt bacteremia score (PBS) and an abbreviated version of the PBS, the qPitt, were also recently validated in a large cohort of patients with bacteremic and non-bacteremic CRE infections [7]. Notably, these analyses included no [4, 6] or few [7] patients treated with newer antibiotics with activity against CRE. Their performance in the era of more effective and safer antibiotics [8, 9] is unclear. Therefore, the objective of this study was to evaluate the predictive performance of the ICS, PBS, and qPitt for mortality among patients with CRE infections treated with ceftazidime–avibactam.

A total of 109 patients were included. The median age was 63 (53–74) years and 34.9% were admitted from a skilled nursing facility or long-term acute care hospital (Table 1). Over half (54.1%) of patients were in the intensive care unit (ICU) at infection onset and 16.5% were admitted during the remainder of their admission. The most common infection sources were respiratory (34.9%), intra-abdominal (21.1%) and urinary (20.2%). Nine (8.3%) patients had a positive CRE blood culture. A total of 113 CRE strains were isolated. Klebsiella pneumoniae was the most commonly identified pathogen, isolated in 71 (65.1%) patients followed by Escherichia coli in 16 (14.7%) and Enterobacter spp. in 12 (11.0%). Regarding antimicrobial susceptibility, among K. pneumoniae isolates tested for ceftazidime-avibactam susceptibility (n = 48), two were resistant, including one that carried New Delhi metallo-beta-lactamase and OXA enzymes.

Overall, 30-day mortality was 16.5%, and was highest in patients with primary bacteremia (42.9%) or pneumonia (23.7%) and lowest in patients with urinary tract infection (4.5%). Chronic pulmonary disease was significantly more prevalent among patients who died compared to those who survived (p = 0.004). In addition, patients who died were significantly more likely to reside in the ICU, require mechanical ventilation, or have severe sepsis/septic shock at infection onset (Table 1). There were no associations between 30-day mortality and early active antibiotic therapy, early ceftazidime–avibactam, nor the use of combination therapy in univariate analyses. Patients who died were significantly more likely to have their ceftazidime–avibactam dose adjusted for decreased renal function (72.2% vs. 42.9% in non-survivors vs. survivors, respectively). The median ICS, PBS, and qPitt were significantly higher in patients who died compared to those who survived (Table 1).

Discrimination for 30-day mortality for the ICS, PBS, and qPitt was 0.704, 0.689, and 0.685, respectively (Fig. 1; Table 2). No significant differences in discrimination were found for all pairwise comparisons (Supplementary Appendix 2). All models showed adequate calibration for 30-day mortality according to the Hosmer–Lemeshow goodness-of-fit test (Table 2). Precision, as measured by the Brier score, ranged from 0.128 for the ICS score to 0.132 for the qPitt score for 30-day mortality.

The performance characteristics of the ICS, PBS, and qPitt as binary classification tools for 30-day mortality at selected cut-points are summarized in Table 3. Using an ICS cutoff ≥ 11 to indicate high mortality risk provided the best performance with a sensitivity, specificity, PPV, NPV, PLR, and NLR of 72.2%, 60.4%, 26.5%, 91.7%, 1.82, and 0.46, respectively. ICS ≥ 11 was significantly associated with 30-day mortality (RR 3.184, 95% CI 1.35–8.930). The post-test probability of mortality increases from 16.5% to 26.5% and decreases from 16.5% to 8.2% among patients with ICS ≥ 11 and < 11, respectively. The optimal cutoff scores to indicate high 30-day mortality risk for the PBS and qPitt were ≥ 3 and ≥ 2, respectively. The RR of mortality was 3.068 (95% CI 1.094–8.606) and 2.850 (95% CI 1.016–7.994) for patients with PBS ≥ 3 and qPitt ≥ 2, respectively. RRs and 95% CIs for the individual components of each score are shown in Supplementary Appendix 3. Although the confidence intervals were wide, all components of the ICS had RRs > 2. With regard to the components of the PBS and qPitt, altered mental status and respiratory failure/mechanical ventilation consistently had RRs > 2, while the association between temperature and mortality was variable. Survival curves for the dichotomized ICS, PBS, and qPitt are shown in Fig. 2a–c. Early and sustained separation between curves was seen for all scores, and the log rank tests were significant for ICS ≥ 11 and PBS ≥ 3 (p = 0.0315 and p = 0.0242, respectively) but not qPitt ≥ 2 (p = 0.0521).

No significant associations between 30-day mortality and early active, early ceftazidime–avibactam, combination ceftazidime–avibactam therapy nor ceftazidime–avibactam renal dose adjustment were found after controlling for each risk score (data not shown).

We conducted a retrospective, multicenter study evaluating the predictive performance of the ICS, PBS, and qPitt for mortality in patients with CRE infections treated with ceftazidime–avibactam. Overall, the scoring systems demonstrated adequate calibration and precision in our cohort. According to conventional thresholds to categorize score discrimination [16, 17], the ICS demonstrated satisfactory discrimination while the PBS and qPitt had poor discrimination. However, differences between scores were very small and not statistically significant.

RCTs to help guide treatment selection for patients with CRE infections are scarce. Most available clinical studies are observational and retrospective with important limitations, including selection bias and inadequate control for confounding. Risk scores are useful for confounding adjustment when analyzing observational data. However, they may perform less well in external cohorts, and, although a model may be successful at one point in time, antimicrobial resistance patterns, pathogen virulence, and standards of care change with time and therefore models must be updated. Our findings do not fully align with those of Henderson et al., who evaluated the performance of the PBS and qPitt for 14-day mortality in 475 patients with non-bacteremic and bacteremic CRE infections from the CRACKLE-1 database [7]. In their evaluation, the discriminatory ability of the PBS and qPitt was considerably higher than in our cohort (0.853 and 0.851, respectively). With regard to the ICS, which was recently validated in 42 patients with non-bacteremic and bacteremic K. pneumoniae carbapenemase-producing K. pneumoniae infections, 30-day mortality was substantially higher than in our cohort (57.1% vs. 16.5%), but discrimination was similar (AUC 0.78 vs. 0.70). The optimal cut-point to identify patients at high risk for 30-day mortality was 11 in our cohort versus 8 in the validation study by Cano et al. [6].

Several factors may account for these discrepancies. First, there are important differences between the cohorts with regard to infection source (less bacteremia and more respiratory in our cohort) and pathogen species (more diverse in this study vs. primarily K. pneumoniae in the other studies). It is notable, however, that Henderson et al. found the performance of the PBS and qPitt to be similar when analyses were restricted to the subgroup of patients with non-bacteremic CRE infections [7]. Furthermore, an important property of the most useful scoring systems is that they perform similarly across different target populations. Second, patients who died within 72 h of infection onset were excluded from our study (patients had to receive ≥ 72 h of ceftazidime–avibactam), and variables such as severe sepsis/septic shock may best predict very early versus later deaths. However, observational studies evaluating antibiotic alternatives typically have inclusion criteria based on receipt of ≥ 48–72 h of the antibiotics of interest [9, 22, 23]. Prediction scores that discriminate for later deaths may be more suited for adjustment in these studies. The ICS was developed and validated specifically in patients with CPE infections. We did not confirm carbapenemase production, and a proportion of patients were likely infected with non-carbapenemase-producing CRE which have been shown to confer a lower risk of poor outcomes compared to CPE [24]. However, as was the case in our study, these data are not always available for observational analyses.

As noted previously, the validation studies for the ICS, PBS, and qPitt were conducted largely in the era before the introduction of newer antibiotics with activity against CRE [6, 7]. Two recent observational studies found improved survival in patients with CRE infection treated with ceftazidime–avibactam compared to historical controls treated with colistin-, aminoglycoside-, and carbapenem-based regimens [8, 9]. It is plausible that the use of ceftazidime–avibactam in all patients in our cohort may have partly contributed to the observed differences in score performance. However, it is important to remember that other changes have occurred in recent years that may have influenced the relationship between baseline variables and outcomes. Rapid genomic and phenotypic methods are now available to accelerate the identification of CRE [25]. A great deal of progress has also been made with regard to our understanding of key aspects of the complex pharmacokinetics of polymyxins enabling the design of dosing regimens more likely to achieve therapeutic concentrations [26]. Ten percent of patients in this study received a polymyxin with ceftazidime–avibactam. Furthermore, at most institutions, the use of new antibiotics must be approved by the antimicrobial stewardship team or infectious diseases consult service. All patients in our study were managed by the infectious diseases consult service. The value of specialist involvement in the management of multidrug-resistant infections is well established [27‐31].

The PBS and qPitt scores are based on the important severity of illness-related variables, while the ICS also adds comorbidities and site of infection. None of the scores incorporate key treatment-related factors that could influence outcomes. We observed that mortality was significantly higher in patients who had their ceftazidime–avibactam dose renally adjusted. The association was no longer significant after adjustment based on the scores, raising the possibility that need for dose adjustment may be a surrogate for other prognostic variables. Renal impairment and the need for renal replacement therapy have been identified by other investigators as risk factors for poor outcomes in patients treated with ceftazidime–avibactam [32‐34] Antibiotic blood and tissue concentrations as well as minimum inhibitory concentrations may impact patient response, and inclusion in prognostic modeling could improve predictive performance at the expense of model simplicity. Conversely, it may be worthwhile to revisit and revise protocols pertaining to ceftazidime–avibactam renal dose adjustment criteria [35].

Our study has several important limitations. First, the study is subject to inherent bias with its retrospective, observational design. However, in-depth EMR reviews allowed us to obtain detailed patient level data that may not be available in validation studies that obtain data from large healthcare administrative databases. Second, although our data spanned 4 years and we enrolled patients from six medical centers, the number of patients included was relatively small owing to the infrequency of CRE infections and slow incorporation of new antibiotics into clinical practice. Our event rate was also low, which may partly explain the variable performance of the models. We calculated the scores using the worst physiological measurements within 24 h of culture collection as a proxy for infection onset. However, for patients with community-onset infections as well as those with chronic relapsing infections, the time-frame used may not have reflected the true infection onset. We included only patients treated with ceftazidime–avibactam. Evaluations of scoring system performance in patients treated with other novel agents would be valuable.

There is an ongoing trend in infectious diseases toward algorithmic approaches for risk stratification and treatment. With respect to clinical use, the ICS, PBS, and qPitt all have the advantages of being based on readily available variables and being simple to calculate compared to the more time-intensive APACHE II or SOFA scores. However, none performed well enough in our cohort to be used for clinical decision-making in individual patients. Updating the scores to more accurately predict outcomes in the era of novel antibiotics, rapid diagnostics, and infectious diseases specialist involvement would be worthwhile, not only to improve their utility as tools in observational research but also so that clinicians can use them as supplementary information when making appropriate decisions about the management of patients with CRE infections.