A multifaceted strategy for implementation of the Ottawa ankle rules in two emergency departments
BMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b3056 (Published 12 August 2009) Cite this as: BMJ 2009;339:b3056
- Taryn Bessen, NHMRC NICS-RANZCR fellow 1,
- Robyn Clark, NHMRC fellow (former heart Foundation-NHMRC NICS scholar) 2,
- Sepehr Shakib, NHMRC NICS-SA DoH fellow3,
- Geoffrey Hughes, director of critical care services4
- 1Department of Medical Imaging, Royal Adelaide Hospital, Adelaide, SA 5000, Australia
- 2Sansom Institute, Faculty of Health Sciences, University of South Australia, Adelaide, SA 5000, Australia
- 3Department of Clinical Pharmacology, Royal Adelaide Hospital
- 4Emergency Department, Royal Adelaide Hospital
- Correspondence to: T Bessen Taryn.Bessen{at}health.sa.gov.au
- Accepted 27 May 2009
Abstract
Problem Despite widespread acceptance of the Ottawa ankle rules for assessment of acute ankle injuries, their application varies considerably.
Design Before and after study.
Background and setting Emergency departments of a tertiary teaching hospital and a community hospital in Australia.
Key measures for improvement Documentation of the Ottawa ankle rules, proportion of patients referred for radiography, proportion of radiographs showing a fracture.
Strategies for change Education, a problem specific radiography request form, reminders, audit and feedback, and using radiographers as “gatekeepers.”
Effects of change Documentation of the Ottawa ankle rules improved from 57.5% to 94.7% at the tertiary hospital, and 51.6% to 80.8% at the community hospital (P<0.001 for both). The proportion of patients undergoing radiography fell from 95.8% to 87.2% at the tertiary hospital, and from 91.4% to 78.9% at the community hospital (P<0.001 for both). The proportion of radiographs showing a fracture increased from 20.4% to 27.1% at the tertiary hospital (P=0.069), and 15.2% to 27.2% (P=0.002) at the community hospital. The missed fracture rate increased from 0% to 2.9% at the tertiary hospital and from 0% to 1.6% at the community hospital compared with baseline (P=0.783 and P=0.747).
Lessons learnt Assessment of case note documentation has limitations. Clinician groups seem to differ in their capacity and willingness to change their practice. A multifaceted change strategy including a problem specific radiography request form can improve the selection of patients for radiography.
Problem
A gap remains between what we know and what we practise.1 2 3 Blunt ankle trauma is a common presentation to an emergency department; traditionally, radiographs are ordered for virtually all such patients, and typically 85% of these examinations do not find a fracture.4 According to clinical research findings, radiography is not always needed to exclude an ankle or foot fracture.4 5 The Ottawa ankle rules (box 1) are a clinical decision tool that aids the efficient use of radiography in acute ankle injuries.5 A systematic review published in 2003 confirmed that the Ottawa ankle rules accurately exclude ankle and mid-foot fractures in patients with ankle injuries and can reduce the number of unnecessary radiographs by 30-40%.6
The Ottawa ankle rules
An ankle radiographic series is required only if there is pain in the malleolar zone and any of these findings:
Bone tenderness at the posterior edge or tip of the lateral malleolus (distal 6 cm), or
Bone tenderness at the posterior edge or tip of the medial malleolus (distal 6 cm), or
Inability to bear weight both immediately and in the emergency department
A foot radiographic series is required only if there is pain in the mid-foot zone and any of these findings:
Bone tenderness at the base of the fifth metatarsal, or
Bone tenderness over the navicular bone, or
Inability to bear weight both immediately and in the emergency department
The Ottawa ankle rules are well known and widely used internationally.7 In the emergency departments in this study, they were either not being used or their application varied considerably. We aimed to identify strategies to improve the uptake of the Ottawa ankle rules in the emergency departments, and to determine if these strategies were effective in improving selection of patients with ankle injuries for radiography.
Background and setting
This study began at a major tertiary referral hospital serviced by a public radiology provider. After 12 months, the implementation strategies developed at the tertiary hospital were transferred to a community hospital serviced by a private radiology contractor, to assess if they would work in another setting. The study continued for a total of 20 months at the tertiary hospital and six months at the community hospital. These timeframes were determined by our funding. The two sites were chosen because clinical champions facilitated access to their radiology and emergency departments for the project, and because they had similar throughput of ankle injuries.
The case mix of the emergency departments differed substantially; the adult tertiary hospital saw proportionally greater volumes of higher acuity patients than did the community hospital, which saw both adults and children with predominantly lower acuity presentations.
Only radiographs ordered from the emergency departments were included in this study. At both hospitals, radiography requests were made on paper forms.
The target group consisted of all clinicians working in the emergency departments who were able to order radiographs—that is, triage nurses who had completed accreditation to order radiographs, nurse practitioners, and all medical staff. Over the study period, the target group comprised 315 clinicians at the tertiary hospital and 62 at the community hospital.
Appropriate ethical approval was obtained from the Human Research Ethics Committees of each hospital involved in the study.
Strategy for change
The study was underpinned by concepts for changing clinical practice and the implementation of evidence based guidelines.8 9 10 11 12 In particular, we adopted key strategies outlined by Grol, which included a barriers analysis, engagement of the target group in design and implementation of a plan which linked the change strategies to barriers, and finally evaluation of the outcomes.10 13
Baseline audit
A baseline audit established the gap between evidence and practice and provided a starting point from which to measure the extent of change in practice. The audit included all consecutive eligible patients over a retrospective six month period who presented to the emergency departments with an ankle or mid-foot injury.
Barriers analysis
An initial analysis identified potential barriers to evidence being used in routine clinical practice. We completed process mapping of several ankle patients from triage to discharge or admission, along with key informant14 interviews of emergency department and radiology staff.
Barriers fell into three groups:
Individual clinician—Knowledge of the Ottawa ankle rules, concern about missing a fracture, lack of confidence in clinical ability to exclude a fracture without a radiograph, and lack of knowledge about which radiograph to order were key factors. Perceptions that “ankles are not important,” “the patient expects a radiograph,” and “it’s better to carry out radiography, otherwise the patient will re-present” were common.
Social context—Many staff felt obliged to order a radiograph if patients were unhappy with lengthy waits to be seen.
Organisational context—High staff turnover due to rotating staff, staff on shift work, no negative consequences of ordering unnecessary radiographs, and, at the tertiary hospital, triage nurses ordering radiograph without clinical examination at busy times to improve flow of patients within the emergency department were all important barriers.
Engaging the target group
At the tertiary hospital, the results of the baseline audit and barrier analysis were presented to an emergency department consultants’ meeting to gain support in principle. We identified change champions and clinical opinion leaders in both the emergency department and radiology, and convened a multidisciplinary steering group, which helped design the change strategies and drive the changes.
At the community hospital, we presented the results of the baseline audit and barrier analysis to a general staff meeting with the director and acting manager of the emergency department present, to provide support and endorsement.
Linking change strategies to barriers
The change plan was a multifaceted change strategy designed to deal with the identified barriers and included:
Education
A new problem specific radiography request form (fig 1)⇓
Reminders
Audit and feedback
Using radiographers as “gatekeepers” (tertiary hospital only).
Barriers at the individual clinician level and in the social context were met by education during in-service tutorials. Barriers within the organisational context were met by introducing a new request form incorporating the Ottawa ankle rules (fig 1⇑), reminders via posters and lanyard cards, audit and feedback, and empowering the radiographers to reject the old request forms or any incomplete new request forms (“gatekeeping”). Dealing with the lack of negative consequences for ordering unnecessary radiographs was outside the scope of this study. At the tertiary hospital, it had been common practice for triage nurses to order ankle radiographs without clinical examination during busy times to streamline the flow of patients through the emergency department. During the study period, a process of fast tracking all low acuity patients was independently implemented to reduce the time to definitive assessment (fast tracking was already in place at the community hospital). This meant there was less need for triage nurses to order ankle radiographs without clinical examination and this barrier no longer needed to be tackled. Strategies were tailored to each hospital, and apart from audit and feedback, were all introduced simultaneously (table 1⇓).
Multifaceted implementation
Evaluating outcomes
Assessment of documentation of the Ottawa ankle rules on the request form and in the case notes was used as a marker of the use of the rules in clinical practice. The hospital’s radiology imaging system was searched to determine if imaging had occurred for each ankle presentation. If there was no record, case notes were reviewed to confirm that no imaging had been performed. Request forms were reviewed for all presentations throughout the study, and case notes were reviewed for all patients in the baseline audit. Because of frequent delays in availability, case notes were reviewed only in the implementation period for patients who had no imaging requested, or those without documentation of the Ottawa ankle rules criteria on the request form. Definition of acceptable documentation of Ottawa ankle rules criteria was defined a priori by consensus of a group of senior emergency department consultants at the tertiary hospital and was applied consistently during the study period at both sites.
We assessed the proportion of patients sent for radiography in each emergency department, and we calculated fracture rates as the proportion of radiographs showing a definite fracture. The missed fracture rate was calculated as the proportion of patients who had no radiograph initially but subsequently presented to a public hospital in the metropolitan area within seven days and had a fracture demonstrated on radiography. These presentations were captured through the metropolitan-wide public hospital electronic clinical information system.
For all measures, we compared the results from the baseline and implementation periods by using Fisher’s exact test. The P value for statistical significance was set at 5%.
Effects of change
During the study, 1561 patients with ankle injuries presented to the two hospitals: 215 during the baseline period and 813 in the 20 month implementation period at the tertiary centre, and 244 during the baseline period and 289 in the six month implementation period at the community hospital. Age and sex of the patients at each hospital was similar, with most injuries occurring in the 20-29 year old age group in both sexes.
Documentation of the Ottawa ankle rules
Table 2⇓ shows that documentation of the Ottawa ankle rules increased significantly on both the request forms (tertiary hospital 42.2%, community hospital 31.2%; P<0.001 for both) and in the case notes (tertiary hospital 37.2%, community hospital 29.2%; P<0.001 for both).
Documentation of Ottawa ankle rules, fracture rate, and use of new request form. Values are numbers (percentages) unless otherwise indicated; varying denominators indicate availability of request forms and case notes
Proportion of patients referred for radiography
The proportion of patients referred for radiography fell by 8.6% at the tertiary teaching hospital (95.8% v 87.2%; P<0.001) and by 12.5% at the community hospital (91.4% v 78.9%; P<0.001) compared with the baseline audit (table 2).
Proportion of radiographs showing a fracture
The increase in the proportion of radiographs that showed a fracture was not significant at the tertiary hospital (increased by 6.7% from 20.4% to 27.1%) but was statistically significant at the community hospital (increased by 12.0% from 15.2% to 27.2%; P<0.01; table 2).
In the baseline period no patients re-presented with a missed fracture. During the implementation period, three of the 104 patients (2.9%) at the tertiary hospital and one of the 61 patients at the community hospital (1.6%) did not undergo radiography on initial presentation and subsequently re-presented with a missed fracture. This compared with a missed fracture rate of zero in the baseline group (table 2).
Compared with baseline, a greater proportion of patients with ankle injuries were seen during the implementation period by nurse practitioners at the tertiary hospital (30/215 v 252/813) and by resident medical officers at the community hospital (6/244 v 70/289), and a smaller proportion of patients were seen by triage nurses in both emergency departments (56/215 v 126/813 at the tertiary hospital, 48/244 v 8/289 at the community hospital).
The effect of the change strategies on radiography referrals also varied by clinician group at both sites. The greatest reduction in proportion of patients referred for radiography was in the nurse practitioner groups (30/30 v 208/252 (decreased by 17.5%) at the tertiary hospital; 11/11 v 19/28 (decreased by 32%) at the community hospital) and resident medical officers at the community hospital (6/6 v 55/70; decreased by 21%), but these reductions were not statistically significant.
The proportion of referrals made on the new request form was 88% at the tertiary hospital and 41% at the community hospital. The proportion of new request forms completed correctly was 85.7% and 87.1% at the tertiary and community hospital, respectively (table 2).
Discussion
The Ottawa ankle rules were well accepted, but it was difficult to stimulate enthusiasm about the need for change in practice because minor ankle injuries were not perceived as important or life threatening, and because ordering unnecessary radiographs had no negative consequences. Guidelines that recommend the elimination of an established behaviour (such as ordering radiographs) are more difficult to implement than guidelines that recommend adding a new behaviour.15
Despite this, the change strategies implemented in this study achieved a significant increase in the use of the Ottawa ankle rules at both the tertiary hospital (37.2%) and the community hospital (29.2%). This change was three to four times higher than in a systematic review of 235 guideline dissemination and implementation strategies, which showed a median 10% improvement.9
However, the fall in the number of unnecessary radiographs at the tertiary hospital was considerably less than the 30-40% predicted by Bachman et al.6 This may reflect the hospital’s higher fracture rate at baseline (20% v Bachman’s 15%). The reason for the greater effect of the change strategy on documentation than on reduction in radiography referrals is not entirely clear. It is noteworthy that the reduction in unnecessary ankle radiographs could reduce costs in a public hospital setting, while it could potentially reduce revenue for a private radiology provider.
The number of missed fractures at both sites was small and not statistically significant. A more robust measurement of missed fracture rates could be achieved with longer periods of data collection.
Change in practice
Clinicians
The greatest change in practice was seen with the nurse practitioners at both sites and the resident medical officers at the community hospital. Nurse practitioners became champions of the implementation of the Ottawa ankle rules, and among the junior medical staff uptake may have increased because the new request form gave a clear, evidence based framework for clinical assessment. The reason for lack of change in radiography referral patterns by triage nurses (non-nurse practitioners) is not entirely clear. The shift in radiography referrals away from the triage nurses could account for some of the outcomes, but not for change in practice in other clinician groups.
Clinicians initially expressed concern that patients expected radiography to be performed. During the implementation period, staff reported increased confidence in explaining to patients why radiography was not needed. Only nine of the 165 patients who did not undergo radiography re-presented to the same or different hospital for a second assessment.
Workplace culture can be an important facilitator or barrier for change. Cultural aspects related to leadership, teamwork, and readiness to change were not evaluated in this study. The early adopters and change champions were crucial to our success, and their support allowed smooth passage of new practices through hospital committees and line management: hospital executives, directors of emergency and radiology departments, and all tiers of medical, nursing, and radiology staff.
New radiography request form
The new request form was well adopted at the tertiary hospital (88% of referrals were made on the new form). The staff acknowledged that it served not only as a memory aid but also described the appropriate population in which the Ottawa ankle rules should be used. Given the large volume of staff and the small proportion of permanent staff in the tertiary hospital’s emergency department, the new request form worked well as a change strategy and was used successfully by both rotating and shift working staff. At both sites, the new form was used correctly in over 85% of radiography referrals.
The reduction in unnecessary ankle radiographs was greater at the community hospital, even though the problem specific radiography form was used in less than half of presentations (41%). This may be due to the smaller size and stability of the community emergency department workforce.
A key strategy for implementing the new request form at the tertiary hospital was assigning a “gatekeeper” role to the radiographers. The radiographers accepted and performed this role, but for some it was confrontational and challenging.
Limitations
The observational nature of this study limits interpretation of the data. In addition, the study period was confined to the duration of the lead author’s fellowship.
Assessing change in practice by assessing documentation of the Ottawa ankle rules in the case notes was hampered by inconsistencies in the quality of documentation. Although most clinicians will carefully document the results of an electrocardiogram, they seem to be less motivated to document examination findings for an injury that is not life threatening.
The rate of missed fractures may have been higher than reported, as patients who subsequently presented to their general practitioner may have been referred to community based private radiology contractors; this would not be captured by the hospitals’ electronic clinical information system. The differing lengths of implementation periods and external factors such as seasonality may also have influenced these findings.
It is impossible to disentangle the outcome effects of the separate components of the change strategy when all, apart from audit and feedback, were started concurrently. A cost benefit analysis would further evaluate this multifaceted change strategy. The effect of this change strategy on length of stay in the emergency department, change in practice in requesting of other radiographs, and patient satisfaction could also be considered.
Lessons learnt
Several key learning experiences arose from this study.
Using documentation as a surrogate measure for clinical practice has limitations and is both time consuming and prone to error, but it is often the only way of evaluating clinical practice.
Different groups of clinicians seem to differ in their capacity or willingness to change their practice. In this study, nurse practitioners in both emergency departments and resident medical officers at the community hospital showed the greatest uptake of the Ottawa ankle rules.
A multifaceted change strategy including education; a new, problem specific, radiography request form; reminders; and audit and feedback, and using radiographers as “gatekeepers,” can result in improved selection of patients for radiography.
Notes
Cite this as: BMJ 2009;339:b3056
Footnotes
TB is a NHMRC National Institute of Clinical Studies-Royal Australian and New Zealand College of Radiologists (NHMRC NICS-RANZCR) fellow 2006-2008. She acknowledges the important contribution of her mentors, Virginia Deegan and Judith Dwyer. We also acknowledge our “change champions,” without whose support this study would not have been possible. These include (but are not limited to) Dianne King, director, emergency department, Flinders Medical Centre; Sue Farr, acting manager emergency department, Noarlunga Health Services; and staff in both emergency departments, who we hope are now our new change champions. We also acknowledge the department of medical imaging at the Royal Adelaide Hospital, and Dr Jones and Partners Medical Imaging, Noarlunga Health Services. Special thanks to Rebecca McMahon for research assistance.
Funding: This work was funded by NHMRC NICS and RANZCR.
Contributors: TB conceived and designed the study and won the funding for the study; TB, RC, and SS analysed and interpreted data. All authors drafted and revised the article and approved the final version. GH is guarantor.
Competing interests: None declared.
Ethical approval: Ethics approval was granted from all participating centres.
Provenance and peer review: Not commissioned; externally peer reviewed.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.