Application and comparison of scoring indices to predict outcomes in patients with healthcare-associated pneumonia

Healthcare-associated pneumonia (HCAP), a relatively new category of pneumonia, refers to infections that occur prior to hospital admission in patients with contact or exposure to a healthcare environment [1]. Compared to community-acquired pneumonia (CAP), HCAP is a distinct type of pneumonia with unique microbiological and epidemiological characteristics and outcomes [2‐6].

In the current era of rising healthcare costs, the decision to hospitalize adults with CAP has received considerable attention and many pneumonia severity prediction rules have been designed to stratify patients with CAP into risk groups [7, 8]. Severity assessment is not only the key to deciding the site of care but also in guiding both general management and antibiotic treatment. Of the prominent tools for this purpose are the Pneumonia Severity Index (PSI) developed by Fine and colleagues [9] and the CURB (confusion, urea, respiratory rate, blood pressure) score proposed by the British Thoracic Society, and Infectious Diseases Society of America/American Thoracic Society Consensus Guidelines on the Management of Community-Acquired Pneumonia in Adults [10]. Other clinical prediction rules for severe community-acquired pneumonia, like the severe community acquired pneumonia (SCAP) score were also developed, and were seemingly better at identifying severe CAP. The SCAP is validated to predict 30-day mortality among two cohorts of consecutive adult patients with CAP and identifies more patients as low risk for potential outpatient care [11]. The need for ICU care was better identified with the SOAR (systolic blood pressure, oxygenation, age, respiratory rate) model compared to the other scoring rules (CURB (confusion, urea, respiratory rate, blood pressure), CURB-65 (confusion, urea, respiratory rate, blood pressure, age 65), CRB-65 (confusion, respiratory rate, blood pressure, age 65)) in patients with nursing home acquired pneumonia [12], a subgroup of HCAP.

Each scoring system has its strengths and weaknesses. As demonstrated by the studies on heterogeneous populations, validation studies of algorithms for HCAP therapy will be difficult [13]. It would be very helpful if we can apply the existing scoring systems to HCAP. However, to the best of our knowledge, none of these prediction rules has been validated in patients hospitalized with HCAP. Therefore, we sought to compare the performance of the current scoring indices to predict mortality and ICU admission in patients with HCAP.

HCAP is a heterogeneous disease that includes patient populations with varying severities of illness [22]. The mortality associated with HCAP was similar to that of nosocomial pneumonia, higher than that of CAP, and lower than ventilator-associated pneumonia [13]. As shown in Table 1, each subgroup contributes to different parts of overall HCAP mortality. There is increased mortality of groups II (34.4%) and III (47.3%) of patients with HCAP, indicating that HCAP is a heterogeneous disease. As has already been reported by Brito and Niederman, all patients with HCAP should be identified and then divided on the basis of severity of illness to guide initial therapy [13]. Severe pneumonia has been defined by the requirement for admission to an ICU [16]. The decision to admit a patient with HCAP to an ICU depends on subjective clinical views and the peculiarities of the local healthcare setting. The availability of valid criteria for defining severe pneumonia would provide a more reliable basis for improving patient risk assessments. The severity on admission can affect hospital mortality, the need for ICU admission, and even 90-day mortality after hospital discharge [23]. A number of prognostic scoring tools have been developed to predict mortality and the need for ICU care for patients with CAP; the two tools that have been studied the most are the PSI and CURB-65. However, they are not ideal for assessing the need for ICU care, and other scoring systems, such as those developed by the IDSA/ATS guideline group, and the SMART-COP tool, are available for this purpose [24]. So far, and to the best of our knowledge, no severity index has been developed and validated for patients with HCAP.

The AUC is a measure of the accuracy of a test to correctly classify patients with and without a particular outcome and is used frequently in studies of severity assessment in CAP. The AUC describes the relationships between sensitivity and specificity, a higher AUC implies a less steep trade-off between sensitivity and specificity. An AUC is considered to have moderate discriminating power from a value of 0.70 on up. We conducted this retrospective chart review of 444 records and assessed the validity of PSI, CURB-65, SCAP, and others and constructed an ROC.

The PSI scoring system has been shown to be a powerful tool for assigning the risk of death from CAP in different populations [17]. This scoring system was primarily designed to identify patients with a low mortality risk who could safely be treated as outpatients. However, it is complicated to use, requiring computation of a score based on 20 variables. To ensure that the final prediction rule remained simple to use and practical, prognostic features not usually available at the time of initial assessment post hospital admission were excluded from the CURB-65 model [21]. The CURB-65 model does not consider decompensated co-morbidity due to CAP and results in limited application in the elderly [24]. Since the majority of patients were elderly, the data are not much different from what is published in the literature regarding CAP; that is, CURB-65 may not be a good index for predicting mortality in this population.

The modified ATS rule provides simple clinical criteria for those patients who require ICU admission [16]. According to the authors' description, the modified ATS rule can serve as a useful counterpart to the prediction by Fine et al. The modified ATS rule was good in terms of sensitivity (89.9%) and the area under the receiver operator curve graph (0.823) for predicting 14-day ICU admission in HCAP patients. The modified ATS severe CAP definition published in 2001 was superseded by the 2007 IDSA-ATS severe CAP definition (IDSA/ATS). The newer definition was based on a series of papers and on re-evaluation by the guideline committee of data published since the 2001 definition was made. Therefore, we also tested the two indices and found that modified ATS as well as IDSA/ATS can be applied for defining severe HCAP.

The strongest clinical predictors of SCAP were pH <7.30 and systolic pressure <90 mmHg [19]. A depressed pH, which is likely a side effect of metabolic acidosis derived from sepsis, is not included in other prediction rules, such as CURB-65 or modified ATS. In our series, a low pH was associated with poor outcomes in patients with HCAP. The SCAP score is as accurate as, or better than, other current scoring systems (for example, CURB-65 and PSI) in predicting adverse outcomes in patients hospitalized with CAP [25]. We found that SCAP also works well with HCAP. The discriminatory power of SCAP, as measured by AUC, was 0.81 for ICU admission in our HCAP patients, compared with the 0.75 in CAP patients from another study [25].

The PSI and CURB-65 have been used to guide the need for ICU care, but they are not ideal for this purpose [24]. Some of these indices were originally designed to assess ICU admission rather than mortality. Therefore, a poor performance could be found if applied in predicting mortality. Compared to PSI, modified ATS, IDSA/ATS, SCAP, and SMART-COP were easy to calculate. For predicting ICU admission (Day 3 and Day 14), modified ATS (AUC: 0.84, 0.82), SMART-COP (AUC: 0.84, 0.82), SCAP (AUC: 0.82, 0.80) and IDSA/ATS (AUC: 0.80, 0.79) performed better (showing a statistically significant difference) than PSI, CURB-65, SOAR and SMRT-CO.

The main strength of the study is the relatively large sample size. The limitations of the study include possible selection bias as all patients who were included in our analysis consist of a heterogenic variety of sources. There may be different patient characteristics in each study site. On the other hand, it can reflect the reality of HCAP coming from heterogeneous populations. In addition, there are a huge number of patients that received microbiologically adequate therapy (sensitive to the antibiotic administered) and their clinical conditions do not improve because of other possible factors (for example, incorrect dosing, interval of administration, pharmacokinetic/pharmacodynamic features, hypoalbuminemia in critically ill patients) which were not investigated in this study. However, those were beyond the scope of the study.

The utility of the scoring indices for risk assessment in patients with healthcare-associated pneumonia shows that the scoring indices originally designed for CAP can be applied to HCAP. The promising results offer the clinician an adjunctive tool when making site-of-treatment decisions for patients and when stratifying patients with HCAP into risk groups.