Introduction
The prevalence of nosocomial invasive candidiasis has increased dramatically since the 1990s, with
Candida spp. being the fourth most common cause of bloodstream infections in the United States, accounting for 9% of such infections, over half of which occur among ICU patients [
1,
2]. Some risk factors commonly associated with invasive candidiasis include prior surgery, acute renal failure, receipt of total parenteral nutrition (TPN) and presence of a central venous catheter (CVC) [
3]. The clinical and economic burden of invasive candidiasis is significant, as these infections are associated with increased mortality, longer hospital stay and higher cost [
4]. Crude mortality in patients with candidemia is approximately 40%, with the highest mortality among those infected with
C. krusei (approaching 60%) and the lowest among those infected with
C. parapsilosis (approaching 30%) [
2]. Delayed initiation of appropriate antifungal therapy for candidemia has been associated with increased mortality [
5‐
7].
In an effort to minimize the detrimental effects of nosocomial candidiasis and delayed appropriate therapy, prophylactic antifungal therapy has been utilized with varied results [
8]. Some studies of ICU patients have shown a beneficial effect of antifungal prophylaxis [
9‐
11], but others have not [
12]. Two meta-analyses have shown conflicting results, one showing decreased mortality with antifungal prophylaxis [
13] and the other showing no survival advantage [
14]. Nevertheless, universal antifungal prophylaxis is not clinically or fiscally responsible, as many patients are at low risk for developing infection and inappropriate use of antifungals can promote resistance and increase costs. Therefore, a prediction rule for identifying patients most likely to benefit from prophylaxis would be advantageous.
In 2005, Paphitou and colleagues [
15] published a prediction rule for invasive candidiasis in ICU patients that incorporated risk factors of hemodialysis, TPN, diabetes mellitus and broad-spectrum antibiotics. In 2007, Ostrosky-Zeichner and colleagues [
16] sought to improve upon the previously published rule. They found that a CVC or broad-spectrum antibiotics in combination with at least two minor risk factors (pancreatitis, major surgery, TPN, immunosuppressants, corticosteroids or dialysis) was predictive of invasive candidiasis.
Limited data regarding external validation of the Ostrosky-Zeichner rule exist [
17,
18], and no published evidence indicates external validation of the Paphitou rule. The aim of our study was to evaluate the Paphitou [
15] and Ostrosky-Zeichner [
16] clinical prediction rules and retrospectively validate them in ICU patients at our institution.
Discussion
As shown in our study, invasive candidiasis is significantly associated with increased mortality, longer overall hospital stay and higher costs. Due to the appreciable morbidity, mortality and costs associated with invasive candidiasis and the implications of delayed appropriate antifungal therapy, the use of clinical prediction rules to help determine in which high-risk patients prophylactic antifungal therapy may be beneficial is of interest [
2,
4‐
7]. This study served as an external validation and comparison of two previously published clinical prediction rules for invasive candidiasis [
15,
16]. The variables utilized in this study were applied exactly as published in the original rules. In our patient population, several risk factors used in the Paphitou and Ostrosky-Zeichner rules, including surgery, pancreatitis, hemodialysis and diabetes, were not shown to have statistically significant correlations to invasive candidiasis. Conversely, abdominal surgery and pre-ICU length of stay, which are risk factors that were not used in the Paphitou and Ostrosky-Zeichner rules, were found to be significantly associated with invasive candidiasis. These factors were used in combination with other significant risk factors, such as the use of broad-spectrum antibiotics, presence of a CVC and use of TPN and systemic corticosteroids to develop a clinical prediction rule for our patient population.
The Paphitou and Ostrosky-Zeichner prediction rules performed similarly. Paphitou rule 1, involving the fewest risk factors, demonstrated the lowest sensitivity but the highest specificity. As the rules utilized more risk factors, the sensitivity increased with a corresponding drop in specificity. The NMC rule demonstrated a higher sensitivity than and similar specificity to the Ostrosky-Zeichner rule with the same number of risk factors used in the rule (Tables
3 and
4). Depending on the rule, only 4.1% to 5.4% of patients meeting rule criteria developed invasive candidiasis. This low PPV suggests a limited role for the prediction rules in identifying patients most likely to benefit from antifungal prophylaxis in the ICU. Because of a relatively low overall incidence of invasive candidiasis, all of the prediction rules had a low PPV, which means that if the NMC rule had been applied to our study population, the percentage of ICU patients who would have received antifungal prophylaxis would have been 4.7%. In contrast, the rules demonstrated a high NPV. The NPVs indicate that fewer than 2% (fewer than 1% under the NMC rule) of patients not meeting rule criteria developed invasive candidiasis. This suggests that when the rule equations are applied to a patient, if the resultant value is below the rule breakpoint (Table
4), the patient will not likely develop invasive candidiasis (for example, a probability of 99% using the NMC rule) and therefore will not benefit from antifungal prophylaxis.
Paphitou's rule was developed in 2005 on the basis of a study of 327 patients in a surgical ICU [
15]. The overall incidence of invasive candidiasis in the Paphitou study was 7.1%, which is higher than the observed overall incidence of 2.3% in our study. However, our study was not conducted exclusively in a surgical ICU population. Direct comparison of the sensitivity, specificity, PPV and NPV is not possible, because these values were not reported in the Paphitou study.
In 2007, Ostrosky-Zeichner and colleagues [
16] expanded on the results of the Paphitou study to include a broader range of risk factors in both medical and surgical ICU patients in a multicenter study of nearly 3,000 patients. In contrast to the Paphitou study, the overall incidence of invasive candidiasis among the Ostrosky-Zeichner population, at 3%, was more similar to that of our patient population. Likewise, the Ostrosky-Zeichner patient population was more comparable to ours in that medical and surgical ICU patients were included. Sensitivity and specificity were 34% and 90%, respectively, and the PPV and NPV were 1% and 97%, respectively. In comparison, our study demonstrated higher sensitivity, PPV and NPV, but lower specificity.
More recently, Ostrosky-Zeichner and colleagues [
17] further refined their rule in a retrospective study of 597 ICU patients by including the combination of mechanical ventilation for at least 48 hours and the presence of a CVC and broad-spectrum antibiotic use on D1 to D3 plus an additional minor risk factor and found that this changed the sensitivity, specificity, PPV and NPV to 50%, 83%, 10% and 97%, respectively. The investigators also applied their original rule to this patient population and found a sensitivity and specificity of 27% and 93%, respectively, and a PPV and NPV of 13% and 97%, respectively. In comparison, our study demonstrated higher sensitivity and NPV, but lower specificity and PPV. The overall incidence of invasive candidiasis in the Ostrosky-Zeichner population was 3.7%. Notably, 80% of patients in the Ostrosky-Zeichner study were surgical ICU patients, and 78% of them were mechanically ventilated in comparison to approximately 60% who were mechanically ventilated in our study. Moreover, mechanical ventilation was not found to be significantly different among cases and controls in our study population (Table
2). Because our study was completed before the new rule was published, comparison of the sensitivity, specificity, PPV and NPV of this rule in our patient population was not possible.
Notably, a large number of solid organ and hematopoietic stem cell transplants are performed at our institution. Because the specific equations used in the Paphitou and Ostrosky-Zeichner studies were not reported, we had to derive our own equations for their rules. Our patient population's characteristics may have caused various factors, such as use of immunosuppressants, systemic corticosteroids and broad-spectrum antibiotics, to be weighted differently in the equations for the rules. Consequently, these factors might have changed the performance of the rules.
The AUC ROC curve is a measure of the discriminative power of the rule. A value of 1 would be equivalent to perfect discriminative power; that is, the higher and closer a value is to unity, the better the rule discriminates patients with invasive candidiasis from patients without invasive candidiasis. Paphitou rule 1, incorporating the fewest risk factors, achieved the lowest AUC ROC curve and therefore the lowest discriminative power. Our institution-specific rule had the highest AUC ROC curve at 0.770, indicating the best discriminative power among the various rules.
Playford
et al.[
18] recently evaluated three clinical prediction rules in a prospective cohort of 615 Australian ICU patients. The rules evaluated included the 2007 and 2011 Ostrosky-Zeichner rules plus an additional rule that included
Candida colonization [
16,
17,
21]. Fungal surveillance cultures of the throat, rectum and/or perineum and urine were performed 72 hours after ICU admission and twice weekly thereafter, enabling the investigators to add
Candida colonization as a factor to the two Ostrosky-Zeichner rules. The overall incidence of invasive candidiasis in their study was 2.4%, which is similar to the incidence in our study. However, mechanical ventilation, CVC use and surgical procedures were more common and systemic corticosteroid and immunosuppressant use were less common in their study than in our patient population. Their study demonstrated that the specificities and PPVs of the two Ostrosky-Zeichner rules were improved by the addition of colonization to the rule, although this resulted in lower sensitivity.
Our study has several limitations. First, the study was a retrospective, single-center study. Additionally, our institution has a large population of solid organ transplant and hematopoietic stem cell transplant patients. Thus, the results of our study may not be generalizable to those conducted at other institutions. Data from our study as well as that of Playford
et al.[
18] suggest that clinical prediction rules can be applied only to patient populations other than the derivation population if the potential for case-mix variability is taken into consideration. The application of the rules was limited to ICU patients, as this was the original population for which the rules were developed. The development and application of a prediction rule for use in the non-ICU patient population remains an unaddressed issue. Also, previous studies supporting the effectiveness of antifungal prophylaxis were conducted in populations that had an incidence of invasive candidiasis over 10% among the control populations [
9‐
11]. In contrast, our study, along with those in which the Ostrosky-Zeichner and Paphitou rules were developed [
15‐
17], occurred in populations with lower infection rates. In this setting, it may be more appropriate to focus attention on early empiric antifungal therapy, although early empiric therapy with fluconazole was not shown to be beneficial in an ICU population with an incidence of invasive candidiasis of 9% in the control population [
22]. Last, clinical prediction rules utilizing colonization with
Candida spp. as a risk factor [
21,
23] could not be evaluated in our study, because routine surveillance cultures to detect such colonization are not performed at our institution. While colonization has been shown to be a risk factor for invasive candidiasis [
24], the predictive value of colonization for invasive candidiasis is variable [
21,
23,
25].
Competing interests
EDH is currently an employee of Cubist Pharmaceuticals; is a shareholder in Cubist Pharmaceuticals; has been a consultant for Ortho-McNeil Pharmaceuticals, Cubist Pharmaceuticals and Forest Laboratories; and has received research funding from Pfizer and TheraDoc. MER has received research funding from Cubist Pharmaceuticals, Sanofi-Pasteur, Cardinal Health Foundation, 3 M and Molynlcke. AGF has received research funding from Merck and Pfizer. MKZ, MM and ACK have no conflicts of interest to declare.
Authors' contributions
EDH contributed to the study concept and design, the execution of the study and manuscript preparation. MKZ and MM contributed to data acquisition and manuscript preparation. MER and AGF contributed to the study design and manuscript preparation. ACK contributed to the study design, statistical analysis and manuscript preparation. All authors read and approved the final manuscript.