Introduction
The American Thoracic Society and Infectious Diseases Society of America (ATS/IDSA) published guidelines for the management of hospital-acquired pneumonia (HAP), ventilator-associated pneumonia (VAP), and healthcare-associated pneumonia (HCAP) in 2005 [
1]. The guidelines emphasize several major principles. First, HAP, VAP, and HCAP should be treated promptly and adequately because patients who experience delays in appropriate therapy have increased mortality. Second, local microbiologic data should be used to customize management guidelines within centers. Third, excessive antibiotic use should be avoided by accurately diagnosing the infection, using culture results to de-escalate initial therapy, and minimizing the duration of therapy. Fourth, prevention strategies based on modifiable risk factors for HAP should be implemented.
The goal of the ATS/IDSA guidelines is to provide an organizational framework for initial evaluation and management of adults with bacterial HAP, VAP, or HCAP; however, the authors acknowledged that the guidelines had several limitations [
1]. For example, individual recommendations were based on the best available evidence, but the impact of the collective guidelines on clinical outcome had not been validated. Similarly, the guidelines provide algorithms and management strategies, but they do not provide specific tools for implementation at the local level.
Inspired by the ATS/IDSA guidelines [
1] and the desire to assess and improve outcomes in adults with HAP in the intensive care unit (ICU), we initiated a performance improvement project designated Improving Medicine through Pathway Assessment of Critical Therapy in Hospital-Acquired Pneumonia (IMPACT-HAP). Our objectives were to develop, implement, and assess this performance improvement project. To assess compliance with management guidelines, we developed a series of quality indicators and herein report our findings before and after implementation of this performance improvement project. Preliminary findings have been previously reported [
2‐
4].
Materials and methods
Participants
We conducted a multicenter performance improvement project at four academic tertiary care centers in the United States. Participating centers were The Ohio State University Medical Center, Columbus, Ohio; Henry Ford Health System, Detroit, Michigan; University of Miami Miller School of Medicine/Jackson Memorial Hospital, Miami, Florida; and University of Louisville, Louisville, Kentucky. The project was conducted in selected adult medical ICUs at two centers (Columbus, 57 participating ICU beds and Miami, 18 beds) on the basis of local staffing to conduct the project; and in all adult ICUs at two centers (Louisville, 61 beds and Detroit, 144 beds). The project was approved by the institutional review board at each participating center; each waived the need for informed consent.
We conducted a prospective performance improvement project in three phases over 18 consecutive months from 1 February 2006 through 31 July 2007. The first phase was a three-month pre-implementation period. Representatives from each center planned the project. The IMPACT-HAP investigators evaluated the ATS/IDSA guidelines [
1] and reviewed the clinical literature to develop a consensus pathway for the management of HAP, VAP, and HCAP for ICU patients. We then defined a series of quality performance indicators to assess compliance with management guidelines. We created a form to collect patient-level data (demographics, laboratory, treatment, outcomes) and an Internet-based repository to transfer the data to the IMPACT-HAP study center at the University of Louisville.
The second phase was a six-month implementation period. Principal investigators at each center formed multidisciplinary teams to customize ATS/IDSA guidelines and create a local consensus pathway based on their respective order sets, hospital formulary, and local epidemiology (unit-specific antibiograms), including consideration of center-specific resistance patterns. Educational efforts were initiated during implementation. The third phase was a nine-month post-implementation period with continued education and data collection.
Patient inclusion criteria and assessment
Adults in participating ICUs were eligible for inclusion in the database if they met ATS/IDSA definitions for HAP, VAP, or HCAP [
1], including clinical suspicion of evolving pneumonia while in the ICU, with new or progressive infiltrates on chest radiograph and at least two of the following: new or increased cough, sputum production, tracheal secretions, or shortness of breath; fever or hypothermia; leukocytosis, left shift, or leukopenia; or deterioration of pulmonary function [
5]. Patients were followed until hospital discharge, death, or Day 28, whichever occurred first.
Comorbid conditions were prospectively defined and extracted from patient records. Respiratory disease was defined as a history of chronic obstructive pulmonary disease. Renal disease was defined as a history of chronic renal disease, or abnormal blood urea nitrogen and creatinine. Cardiac disease was defined as systolic or diastolic ventricular dysfunction by history, physical examination, chest radiogram, or echocardiogram. Immunosuppression was defined as active malignancy; AIDS; end-stage renal, liver, or lung disease; steroids (prednisone ≥10 mg for >7 days); or active chemotherapy or radiotherapy within 30 days. Severe sepsis was determined by calculation of the sepsis criteria and organ dysfunction criteria [
6], which were abstracted from the chart at the time of enrollment or on Day 0.
Investigators completed a data collection form for each patient. Principal investigators at each site reviewed each form, added any missing data, and internally validated the information before transferring it via the Internet to the repository. Validation of data quality was also performed at the IMPACT-HAP study center. Initial empiric therapy was classified as appropriate if the isolated pathogen was susceptible to at least one prescribed antibiotic. For the purpose of this analysis, data from the first two phases were combined into one nine-month period (that is, pre-implementation and implementation (hereafter, pre-implementation period)) and compared with data from the third nine-month phase (that is, post-implementation period).
Selection of empiric antibiotics was based on ATS/IDSA guidelines [
1] and the presence of risk factors for multidrug-resistant organisms (MDROs). For example, therapy was considered guideline compliant if, within one day of pneumonia onset, patients with risk factors for MDROs received dual gram-negative therapy plus either linezolid or vancomycin for methicillin-resistant
Staphylococcus aureus (MRSA). Reasons for noncompliance were recorded. Clinical outcome was categorized on Day 14 as cure (complete resolution of signs and symptoms of pneumonia), improvement (partial resolution), or failure (deterioration of signs and symptoms of pneumonia). Clinical success was defined as cure or improvement.
Statistics
Descriptive statistics were calculated for baseline demographics and severity-of-illness scores, compliance with guidelines, and reasons for noncompliance in patients enrolled during the pre-implementation and post-implementation periods. Compliance with each quality performance indicator was calculated as the percentage of patients who met the criteria for each indicator based on the total number evaluable for each quality indicator. Between-implementation differences were compared using the Chi-square test or, if applicable, Fisher's exact test for categorical variables and, for continuous variables, the Student's t-test for normally distributed variables and the Wilcoxon rank sum test for non-normally distributed variables. P- values of ≤0.05 were considered to be statistically significant. All data analyses were performed using SAS software, version 9.2 (SAS Institute Inc., 100 SAS Campus Drive, Cary, NC 27513-2414, USA).
Discussion
The IMPACT-HAP project demonstrated the feasibility of developing a performance improvement project for HAP, VAP, and HCAP based on the ATS/IDSA guidelines [
1], while operationalizing and executing it at the local level in multiple centers. The project included features of other successful critical pathways [
8‐
10], such as appointing leaders to champion the project, actively involving relevant stakeholders, and benchmarking. Initial planning by representatives from each center required three months to evaluate the clinical literature, prepare a consensus pathway based on the ATS/IDSA guidelines [
1], and develop a data collection form and Internet-based data repository. The pathway was intentionally flexible to accommodate inter- and intra-center differences. Multidisciplinary teams from each center adapted the empiric therapy recommendations based on their local epidemiology and hospital formulary, another feature of other successful projects [
11‐
14]. Implementation at the local level required approximately six months. Different educational programs were offered to pulmonary/critical care and infectious diseases attending physicians and fellows, house staff, pharmacists, nursing and respiratory therapy staff, including slide presentations for formal grand rounds and lectures -- all of which were supplemented with printed materials to lead clinicians through the process for evaluating suspected nosocomial pneumonia, assessing signs and symptoms, prescribing empiric antibiotics, de-escalating antibiotics, and stopping therapy.
We had initially planned to complete our educational series at the end of the implementation period in order to assess the impact of our initiative. Upon implementation, however, we recognized the need for continual education to accommodate monthly rotations by house staff, some of whom had not attended the formal grand rounds or lectures. This is not unexpected, as recurrent education is yet another feature of other successful interventions [
8,
9,
15,
16]. This approach, however, has implications for assessing compliance with quality performance indicators and may have confounded our ability to detect differences between pre- and post-implementation periods.
Compliance with the quality performance indicators related to making the clinical and microbiologic diagnosis of HAP, VAP, and HCAP (QI-1 and QI-2) exceeded 80%. Compliance was high before implementation of our project and remained high during post-implementation. These findings are consistent with the aggressive management of critically ill patients, the high quality of care generally practiced in ICUs, and the knowledge base at these four academic centers.
We found a statistically significant improvement in administration of initial empiric therapy compliant with ATS/IDSA guidelines (QI-3: pre- vs post-implementation, 30.7% vs 43.7%;
P = 0.01). Notably, improvement became evident during implementation and, importantly, was sustained throughout post-implementation. The most common reason for noncompliance was failure to administer a secondary gram-negative agent, which is consistent with the selective implementation of the consensus pathway at participating centers in the IMPACT-HAP study. Specifically, two centers did not use the secondary gram-negative agent because their local antibiograms indicated adequate coverage with the primary gram-negative agent or reserved the secondary gram-negative agent for patients with VAP. Therefore, initial empiric therapy could be compliant with center-specific guidelines but not with ATS/IDSA guidelines. Others have also questioned the need for dual gram-negative coverage in patients with risk factors for MDROs because of concerns regarding potential renal toxicity associated with aminoglycoside use and lack of an additional benefit based on the local microbiology [
17]. A recent survey of more than 800 physicians provides further insight [
18]. Although 71% were aware of published guidelines, participants chose guideline-concordant antibiotic regimens for HCAP only 9% of the time. Consistent with our experience, participants failed to choose a secondary antipseudomonal agent for HCAP up to 36% of the time [
18].
The next two quality performance indicators addressed measures to avoid excessive antibiotic use. Compliance with short-course therapy doubled during post-implementation (QI-4: 25.0% vs 52.9%); however, this difference was not statistically significant and less than 10% of our patients met the criteria for short-course therapy. Antibiotics were de-escalated on Day 3 based on microbiologic findings only in about one-third of candidates for de-escalation during both periods (QI-5b: 32.4% vs 37.5%). We did not capture reasons for failure to de-escalate, which merits further evaluation as this is an additional area for potential improvement. The last quality performance indicator, clinical success at Day 14, remained unchanged (QI-6, 68.0% vs 66.4%). Potential reasons for the inability to show changes in clinical outcomes despite improvement in compliance with empiric therapy include early adoption of the protocol during the implementation phase and the decision to continue educational efforts throughout the study despite our initial plans to complete our educational series at the end of the implementation phase, as previously discussed. Another consideration is the high clinical success rate observed during the pre-implementation period (reasonable for critically ill patients in the ICU with HAP, VAP, and HCAP), making it difficult to demonstrate further improvement.
While the compliance rates for certain quality performance indicators including compliance with empiric antibiotics appear low, our collective compliance rates were generally consistent with those in studies comparing outcomes before and after implementation of guidelines for severe HAP and VAP [
12,
13,
19] or studies validating guidelines for HAP in the ICU [
20‐
22]. Previous studies had several limitations, such as being conducted at single centers and usually having limited numbers of patients, although Dellit
et al. evaluated 819 patients [
19]. The three validation studies [
20‐
22] did not include pre-implementation data. Most studies evaluated the older 1996 ATS guidelines [
23] or center-specific guidelines [
13,
19,
22]; only Ferrer
et al.
[
20] evaluated the 2005 ATS/IDSA guidelines [
1]. Compliance rates were variable partly because of differences in study methods. In validation studies, compliance rates for empiric therapy were 49% [
21] and 58% [
20]. In the third validation study [
22], overall compliance with standard operating procedures (SOPs) was reported as either higher or lower than 70%. Only 34% of patients were in the high compliance group [
22]. Compliance with empiric antibiotics was not reported in the other studies [
12,
13,
19].
Further studies are needed to validate the ATS/IDSA guidelines for patients with HAP, HCAP, and VAP. Some studies have shown that compliance with guidelines shortens the duration of mechanical ventilation and ICU stay [
22] or is associated with a higher percentage of adequate antibiotics, which in turn leads to reduced mortality [
12]. In another study of HCAP, receipt of empiric therapy not recommended by guidelines was independently associated with mortality after adjusting for other variables [
24]. Most studies, however, fail to demonstrate significant correlations between compliance and outcomes [
13,
19‐
21]. Our findings on the relationships between compliance and outcomes are reported separately [
25]. Additional clinical findings from IMPACT-HAP, such as a new score to predict mortality [
26] and the relationship of vancomycin minimal inhibitory concentration (MIC) to mortality [
27], are also reported separately.
Our project had several limitations. Most importantly, the participants were adults in multiple ICUs, an inherently complex group with many comorbidities. This was a non-randomized, observational study with the natural limitations of real-world academic medical center practices. However, data from IMPACT-HAP were for the most part prospectively collected and validated before entry into the data repository. Another limitation was the inability to capture all patients admitted to these ICUs during the study period. Specifically, we enrolled as many patients as possible but missed some, especially those admitted on weekends or discharged from the ICU before we could learn of them. Furthermore, patient enrollment was not evenly distributed among centers. For these reasons, our findings may not be generalizable to other ICU populations or patients treated outside the ICU setting. Finally, feedback was not provided to investigators until after the post-implementation period was completed. Real-time feedback might have improved compliance with ATS/IDSA guidelines and other quality indicators.
Acknowledgements
We thank the following investigators and collaborators for their contributions to IMPACT-HAP: Carol Myers, RN, CIC, David Taylor, PhD, Lindsay Pell, PharmD, and Kari Mount, PharmD (The Ohio State University, Columbus OH); Ennie Cano, PharmD, Cynthia M. Cely, MD, Andrea S. Castelblanco, MD, G. Fernando Cubillos, MD, Andrew A. Quartin, MD, MPH (University of Miami/Jackson Memorial Hospital/VAMC, Miami FL); and Nadia Z. Haque, PharmD (Henry Ford Health System/Wayne State University School of Medicine, Detroit, MI). We also thank Cindy W. Hamilton, PharmD, ELS (Hamilton House, Virginia Beach, VA) for assisting with manuscript preparation. Hamilton House received compensation from Pfizer Inc. for its contributions.
Competing interests
Funding for this study was provided by Pfizer Inc., US Medical. The University of Louisville Foundation was responsible for project oversight and distribution of funds to participating institutions. JEM has served on advisory boards for Madcat Healthcare, Pfizer, Astellas, and Merck; and received educational grants from Fallon Medica. DHK has received research support from Pfizer, and has served as a consultant to and is on the speakers' bureaus of Astellas, Cubist, Glaxo Smith Kline, and Pfizer. MJZ has received honoraria for lectures from Pfizer, Cubist, and Astellas, as well as grant support from Henry Ford Hospital, Pfizer, Johnson and Johnson, and Cubist. JAR has received research support from Pfizer, is a consultant for Pfizer, and has received honoraria from Pfizer, Merck, and Wyeth for lectures. PP has no conflicts of interest to disclose. VLW, KDF and EGS are employees of Pfizer, Inc.
Authors' contributions
JEM, DHK, MJZ, JAR, PP, EGS and KDF contributed to project development and implementation. JAR, PP and VLW had full access to all data and take responsibility for the integrity of data and accuracy of data analysis. All authors contributed to analysis and interpretation of data, and to drafting of the manuscript and critical revision for important intellectual content. All authors read and approved the final manuscript.