Background
Antibiotic overuse promotes the emergence of resistant microorganisms [
1] and contributes to preventable morbidity and costs [
2]. In outpatient settings, most antibiotic prescriptions are directed at acute respiratory infections (ARI) [
3]. Misutilization of antibiotics for uncomplicated ARI remains common, years after publication of widely endorsed prescribing guidelines [
4]. The rationale for the persistence of the problem is not fully understood, and may include lack of knowledge of the treatment guidelines, [
5] lack of appreciation for the development of resistance, [
6] and pressure to expediently satisfy perceived patient expectations [
6]. Provider education can decrease needless antibiotic prescriptions for ARI, but no information delivery method has proven consistently effective, scalable and sustainable [
7,
8].
The broad deployment of electronic medical records (EMR) is opening new educational avenues to medical providers. EMR-based clinical decision support systems (CDSS) can be integrated into the workflow and change clinical practice by providing timely, guideline-based, patient-specific recommendations [
9]. A CDSS with a light footprint could conceivably remain in place indefinitely [
10]. However, our knowledge on how to assure that such a tool remains useful over the long-term is still maturing.
In an effort to improve outpatient antibiotic stewardship at a large, statewide Veterans Affairs health system, a CDSS was interposed at the time of electronic order entry for two antibiotics commonly used for ARI: gatifloxacin, the mainstream formulary respiratory fluoroquinolone when it was available, and azithromycin. This intervention nearly eliminated unwarranted ARI prescriptions for the two agents for a period of at least 3.5 years [
11]. When gatifloxacin was withdrawn from the market in May 2006 due to reports of dysglycemias [
12], we considered the possibility that, after 3.5 years, the guideline information delivered by the CDSS with each attempt to prescribe an antibiotic had become assimilated into our standard of practice, thereby rendering the CDSS unnecessary. To gain insight into this possibility, we did not reapply the ARI CDSS to moxifloxacin, the formulary replacement for gatifloxacin. Meanwhile, the CDSS for azithromycin was left in place for a period of more than 4.5 years. In the current work, we take advantage of these conditions to simultaneously ask if the CDSS retained its effectiveness over the long-term and if its effect on decreasing antibiotic overuse for ARI could persist despite withdrawing the intervention.
Methods
This is a retrospective, observational study designed to assess the long-term effects of CDSS use and the consequence of CDSS withdrawal on concordance of antibiotic usage with widely-endorsed ARI treatment guidelines. The study was performed using medical record data from the Veterans Affairs Maryland Health Care System (VAMHCS) during a period extending from January 1, 2004, to December 31, 2011. The ARI CDSS was prompted each time an attempt was made to electronically prescribe azithromycin for the entire study period, or gatifloxacin from the beginning of the study period until May 2006 (pre-withdrawal period), when gatifloxacin was removed from the market. The CDSS did not target other antibiotics, including moxifloxacin which replaced gatifloxacin on the formulary after May 2006 (post-withdrawal period). Only one oral formulation of a respiratory fluoroquinolone was listed on the national VA formulary at any given time, with minimal overlap of listing during transitions. No non-formulary respiratory fluoroquinolones, such as levofloxacin, were prescribed for any of the ARI visits included in our study sample.
Description of the intervention
The ARI CDSS has been previously described [
11]. For targeted antibiotics, the CDSS was deployed when electronic (e-) prescription was attempted. For azithromycin and gatifloxacin, the CDSS included treatment paths for the following diseases: community-acquired pneumonia, acute bronchitis, acute sinusitis, non-specific acute respiratory infection (ARI), and exacerbations of chronic obstructive pulmonary disease (COPD). An “Other” option was included for provider-supplied rationales. The CDSS used data mined from the EMR and information entered by the provider to assess whether an antibiotic prescription would be consistent with published ARI guidelines. Diagnostic criteria and antibiotic prescribing criteria have been previously defined [
11]. For appropriate rationales, the CDSS led to order entry for the antibiotic and generated documentation describing why the drug was being used. If antibiotics could be safely withheld, the CDSS did not lead to a prescription. Instead, it provided guidance on how to maintain patient satisfaction when withholding antibiotics. For both safety and tolerability, providers could override the system by providing their own rationale in the “Other” pathway or by prescribing antibiotics not subject to the CDSS.
Study participants
An automated case-detection algorithm (CDA) previously found to identify 73% of patients with ARI was applied to EMR-derived relational databases [
13]. The algorithm flagged outpatient visits if providers assigned an ARI-related diagnostic code OR prescribed a cough suppressant AND if the clinical notes documented at least two ARI symptoms, as assessed by automated text analysis [
13].
Record review
A record review was completed for 1) all flagged visits associated with azithromycin, gatifloxacin, or moxifloxacin prescription, 2) a random sample of all flagged visits associated with “All Other Antibiotics”, limited to 50% of flagged visits to retain feasibility of the manual record review, and 3) a random sample of 200 flagged visits where no antibiotics were given, as such visits had rarely been found to be discordant in the past [
11].
Sampled flagged visits were manually abstracted for data elements needed to assign diagnosis and treatment for pneumonia, acute bronchitis, or sinusitis [
11]. Three independent reviewers, including a pulmonary medicine specialist, examined all free-text EMR entries on the day of index for the flagged visits, with the exception of notes issued by the CDSS itself. Presence or absence of individual ARI symptoms and related time courses were abstracted. A stated diagnosis of pneumonia or inclusion of pneumonia in the list of differential diagnoses was coded as pneumonia. Chest imaging was not reviewed. Reviewers cross-validated 10% of each other’s work. Inter-rater reliability was determined for: a) the application of inclusion/exclusion criteria (kappa = 81.0%); and b) for the final assessment of concordance of antibiotic prescribing with guidelines (kappa = 85.5%).
Exclusion, diagnostic and concordance criteria
After the automated CDA identified possible ARI visits, pre-determined exclusion criteria were applied using manually abstracted information. These criteria included: 1) Stated diagnosis of COPD; 2) Did not meet case definition for outpatient sinusitis, acute bronchitis, or pneumonia; 3) Not an initial visit for a given ARI episode; 4) No provider documentation; 5) Antibiotic unequivocally prescribed for a non-ARI diagnosis; 6) Records could not be accessed for review. Cases of pharyngitis were excluded from our study according to criteria 2 because it was not an available pathway for the CDSS and because the CDSS-targeted antibiotics were found not to be misused in patients whose only ARI diagnosis was pharyngitis [
11].
Abstracted information for each visit was reassembled into ARI diagnoses and visit concordance was determined according to the previously described case definitions and treatment guidelines [
11]. Except for pneumonia, ARI diagnoses were not mutually exclusive. A visit was considered “concordant” if antibiotic use followed guidelines for at least one ARI diagnosis.
Antibiotic usage
The Veterans Affairs Pharmacy Benefits Management Services database was queried to compare national to VAMHCS usage of azithromycin and fluoroquinolones (ProClarity software, Microsoft Corp, Redmond CA).
Statistical analysis
Based on the accuracy of the screening case-detection algorithm [
13] and allowing for attrition due to misclassification, we calculated that abstracting 1800 flagged outpatient encounters would result in at least an 80% power to detect a 10% change in concordance after CDSS withdrawal.
The potential confounders for the relationship between study time periods and guideline concordance were identified using T-tests for continuous variables and Chi-square tests for categorical variables. Patient-specific covariates were sex, self-reported race, age, and documented ARI symptoms. Covariates were included in the multivariable logistic regression if they were found to have an interaction, according to standard practice. Multivariable logistic regression and odds ratios were calculated to determine the association between CDSS withdrawal and guideline concordant antibiotic prescribing (SAS software v. 9.2, SAS Institute Inc., Cary, NC). A chi-square test was used to compare guideline concordance between pre-withdrawal and post-withdrawal periods. A p-value of .05 or less was considered statistically significant.
Discussion and conclusions
We studied the long-term effect of a CDSS interposed at the time of e-prescription on concordance with ARI treatment guidelines. The sustained effectiveness of the CDSS was demonstrated in two ways. First, over an implementation period of 3.5 years (for gatifloxacin) to 8 years (for azithromycin), the proportion of concordant visits was higher with the CDSS than without. Second, concordance decreased when the CDSS was withdrawn for the fluoroquinolone, without a corresponding change for antibiotics either continuously (azithromycin) or never subjected to the intervention. Together, these data indicate that a CDSS at the time of prescription can be an effective, long-term strategy for curtailing misuse of the drugs that it targets. This effect may not persist after the CDSS is removed, but CDSS can be designed to be a durable part of provider workflow.
We took several steps to strengthen this study’s quasi-experimental design: a) to minimize sampling bias, we obtained a population-based sample of ARI visits through a previously validated ARI CDA; b) we used a manual reference review and explicit criteria to identify ARI diagnoses and concordance with treatment guidelines; c) we compared the change in guideline concordance when the CDSS was withdrawn against two control groups, one continuously exposed to the CDSS during the whole study period (azithromycin), and one never exposed to the intervention (“All Other Antibiotics”); d) we did not implement complementary educational interventions, thereby increasing the likelihood that the observed effects on prescribing practices could be attributed to the CDSS itself; and e) we compared VAMHCS to national drug utilization to show that the local increase in fluoroquinolone use following CDSS withdrawal ran contrary to national trends, and thus was unlikely to simply be the result of perceived advantages of moxifloxacin over gatifloxacin. Our findings of stable azithromycin use and increased fluoroquinolone use during the post-withdrawal period also ran contrary to those observed in a national study of one million VA ARI visits, where azithromycin usage increased and fluoroquinolone usage decreased in 2012 compared to 2005 [
14].
The sustainability of the CDSS intervention distinguishes it from other attempts to address antibiotic stewardship for ARI. Financial incentives, clinician workshops, or audits with providers can all decrease unnecessary antibiotics for ARI [
15‐
19]. Because these interventions are resource intensive, they usually cannot be continued indefinitely. The durability of their effect therefore depends upon changing the antibiotic prescribing culture. Of the three published examples of outpatient interventions whose effect was maintained for multiple years, two featured voluntary participation and may thus have selected for particularly motivated providers [
20,
21]. Another was conducted within small, stable practices, i.e. in environments expected to be amenable to cultural changes [
22]. Our results suggest that it is possible to foster the judicious use of antibiotics in health care settings that experience a continuous influx of new trainees and part-time staff, i.e. where it may be difficult to provide timely information about prescribing guidelines through conventional means [
16]. Our results suggests that the effectiveness of the CDSS required its continued implementation. This is nevertheless possible because as the CDSS becomes an integral part of the workflow and providers become familiar with its informational content, it ultimately consumes minimal resources,
We do not know what propelled the rate of discordant antibiotic prescriptions once the CDSS was discontinued. Some providers may not have assimilated the ARI treatment guidelines, either because of insufficient CDSS exposures, or because they limited their interaction with the CDSS to what was strictly needed to generate a prescription thus missing the scientific arguments that would lead to a permanent change in behavior. Other providers may have internalized the content of the CDSS, but this knowledge may not have been sufficient to maintain concordant practices in the presence of external pressures such as time expediency, patient or subspecialist demands, or ongoing performance measures of patient satisfaction [
23]. Given these realities, the process-of-care into which the CDSS operated included non-informational elements that may indeed prove crucial to antibiotic management, such as providing implicit institutional support for the decision to withhold antibiotics, or requiring providers to commit a written justification for the use of antibiotics to the EMR, where it is potentially vulnerable to subsequent peer review.
Our study has some limitations that warrant consideration. The biases associated with a pre−/post- intervention study design have been described [
24]. The intervention was deployed in a veteran population that is mostly male, is older, and has more co-morbidities than the general population [
25]. Providers within the VA system have long used the EMR for virtually all aspects of patient care, and this familiarity may have an impact on the acceptance and effectiveness of the CDSS. We did not formally reassess the performance characteristics of the ARI CDA. However, the CDA yielded a validated ARI case density commensurate with past performance measurements. Moreover, the structured components of the algorithm (cough suppressant prescriptions, ARI diagnostic codes) are not expected to be susceptible to changes in language patterns. In the unlikely possibility that the free-text describing common respiratory symptoms changed over the years, there is no reason to believe that these changes would systematically create a bias for or against concordance for any specific antibiotic group. The retrospective data collection meant that symptoms and signs were assumed to be absent if they were not documented in the medical record. ARI episodes could therefore have been missed or mislabeled, along with the rationale justifying the use of antibiotics. Our study focused mostly on ARI visits for which antibiotics were prescribed, and so could not assess possible shunting of utilization between different antibiotics. However, a prior assessment that included all ARI visits indicated that the CDSS intervention did not transfer inappropriate use from the CDSS-targeted agents to alternative antibiotics, nor did it promote the diagnosis of pneumonia to justify prescribing antibiotics [
11]. If we excluded visits where the antibiotic was clearly prescribed for a non-ARI diagnosis, we did not evaluate how frequently providers supplied rationales other than those provided by ARI guidelines to prescribe CDSS-targeted agents. We limited CDSS targeting to azithromycin and fluoroquinolones, because they were the two most commonly prescribed antibiotics for ARI and were not commonly used for diseases other than ARI [
11]. Different effect sizes may therefore have been observed if different or additional antibiotics had been targeted, or if the CDSS was implemented in health systems with different prescribing patterns [
14,
16]. Our study did not collect information about emergence of resistant organisms, clinical efficacy or patient safety. However, no patient with provider-assigned diagnosis of pneumonia failed to receive antibiotics, a finding consistent with our past work with 2125 patients [
11]. Other studies have demonstrated no increases in patient follow-up visits, need for antibiotic prescription after the index visit, or mortality in cohorts where antibiotic stewardship programs successfully decreased over-prescribing for ARI [
26‐
29]. Taken together, these limitations represent design opportunities for future studies aimed at extending e-prescription-based approaches to other antibiotics, drugs, diseases or settings.
There is a continued need for better methods to promote responsible antibiotic use, and any intervention must match the scale and historical resilience of the problem. A CDSS systematically interposed at the time of e-prescription provided significant and durable improvements in the use of antibiotics for outpatients with ARI. The approach could be adapted to optimize the use of any medication and take advantage of the increasing integration of the EMR in medical practices.
Acknowledgements
The authors declare financial support from the Department of Veterans Affairs (TQ,CF, and SD) and the Centers for Disease Control and Prevention (EV,KR,LW, and SD). The funding bodies had no role in the collection, analysis, and interpretation of data; in the writing of the manuscript; and in the decision to submit the manuscript for publication.