Background
Electronic audit and feedback
Use of theory
Aim and objectives
Methods
Criteria for considering studies for this review
Types of studies
Types of participants
Types of intervention
Types of outcome measures
Exclusion criteria
Data collection and analysis
Data sources and search strategy
-
MEDLINE and Ovid (1946 to August week 3 2016)—searched 12 August 2016
-
EMBASE and Ovid (1974 to 2016 week 35)—searched 12 August 2016
-
Cochrane Central Register of Controlled Trials (CENTRAL) 2016, Issue 8, part of The Cochrane Library. http://www.thecochranelibrary.com/, including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register—searched 12 August 2016
-
CINAHL and EBSCOhost (1981 to present) searched 12 August 2016
-
Science Citation Index and Social Sciences Citation Index, ISI Web of Science (1975 to present)—searched 12 August 2016
Selection of studies
Data extraction and management
Identifying TDF domains
Data synthesis
Results
Description of studies and e-A&F interventions
Study ID | Linder et al. 2010 [38] | Thomas et al. 2007 [39] | Carlhed et al. 2006 [34] | Guldberg et al. 2011 [36] | Gude et al. 2016 [35] | ||
---|---|---|---|---|---|---|---|
Study design | CRCT | CRCT | CRCT | RCT | RCT | CRCT | CRCT |
A&F domain | Prescribing and management of drugs, acute care management | Prescribing and management of drugs, guide primary prevention and screening | Chronic disease management | Guide primary prevention and screening | Acute care management | Chronic disease management | Chronic care management |
Feedback recipients | Physicians | Physicians | Physicians | Physicians | Interdisciplinary clinical teams | Interdisciplinary clinical teams | Interdisciplinary clinical teams |
Direction of change required | Reduce current behaviour | Increase current behaviour | Unclear | Reduce current behaviour | Increase current behaviour | Increase current behaviour | Increase current behaviour |
A&F implementation | EHR-integrated tools | Web panel screens | Software panel screens | Web panel screens | Web panel screens | Distributed as software update | Web panel screens |
Interactive A&F component | Drill down to patient level; select indicators for feedback | Drill down to patient level; select indicators for feedback | Choose presentation mode | Select indicators for feedback | Select indicators for feedback | Drill down to patient level; select indicators for feedback | Select indicators for action plan |
Feedback presentation mode | Graphical summaries | Graphical summaries | Text summaries | Unclear | Graphical summaries | Tabular data, graphical summaries | List summaries, with ‘traffic light’ icons |
Frequency of feedback updates | Monthly (automatically) | Bi-monthly (automatically) | Bi-monthly (automatically) | Every login | Real-time | 3 times (month 1, month 3, month 12) during the trial by software update | Quarterly |
Target goals basis | National CDC recommendations | Australian medical guideline recommendations | National evidence-based guideline recommendations | Radiologist defined their own goals | National evidence-based guideline recommendations | National evidence-based guideline recommendations | National evidence-based guideline recommendations |
Action planning used | No | No | No | Yes | Yes | No | Yes |
Benchmarks and comparators | Clinician vs peer average performance vs national benchmark (identified through expert analysis) | Peer-ranked performance data benchmarked against participating trial sites | Clinician vs aggregate average resident performance (based on Institute for clinical systems improvement targets) | Radiologist performance vs trial peers’ performance vs achievable national benchmarks | Local team vs average trial peers’ performance; local team vs national average performance | Local practice performance vs average performance of trial peers | Local team-based performance vs. peer performance (calculated as achievable benchmarks) |
Intervention duration (months) | 9 | 12 | 12 | 21 | 18 | 14 | 19.8 and 22.5a
|
Participants in study (n) | Primary care practices (27) | Primary healthcare centres (60) | Clinical groups (residents) within hospital | Clinical groups (radiologists) between regional hospitals | Multi-disciplinary teams between hospitals (38) | General practices (86) | Cardiac rehabilitation Centres (18) |
Outcome type | Dichotomous process measures | Dichotomous process measures | Dichotomous process measures | Dichotomous process measures | Dichotomous process measures | Dichotomous process measures | Dichotomous process measures and patient outcomes |
Comparator arm | A&F vs current practice | A&F vs current practice | A&F vs current practice | A&F vs current practice | Head-to-head comparison of A&F | A&F vs current practice | Head-to-head comparison of A&F |
Role of e-A&F in overall QI strategyb
| Core | Moderate | Minimal | Core | Moderate | Core | Core |
Baseline performance known | Unclear | Yes | Yes | Yes | Yes | Yes | Yes |
Risk of bias | Medium | Low | Medium | Unclear | High | Medium | Low |
Effect of e-A&F interventions on quality of care
Study ID | Intervention sample size; control sample size | Outcome of interest | Odds ratio (95% CI) |
---|---|---|---|
Linder et al. 2010 [38] | I: 258 clinicians; C: 315 clinicians | (1) Antibiotic prescribing rate for acute respiratory infection | 0.97 (0.92–1.03) |
Peiris et al. 2015 [41] | I: 19385 patients; C: 19340 patients | (1) Patients who received appropriate screening of cardiovascular risk factors by the end of study | 1.47 (1.41–1.53) |
I: 5335 patients; C: 4846 patients | (2) Prescription rate of recommended medications for high-risk cohort | 1.25 (1.16–1.35) | |
Thomas et al. 2007 [39] | I: 252 patients; C: 231 patients | (1) Diabetes care metrics for all participating residents’ patients at study inception and completion including haemoglobin monitoring in the prior 6 months | 1.72 (1.20–2.47) |
(2) Diabetes care metrics for all participating residents’ patients at study inception and completion lipid monitoring in the prior 12 months | 1.75(1.18–2.59) | ||
Carlhed et al. 2006 [34] | I: 3786 patients; C: 2940 patients | (1) Lipid-lowering therapy at discharge | 3.26 (2.49–4.27) |
(2) Angiotensin-converting enzyme (ACE) inhibitors at discharge | 10.08 (7.31–13.90) | ||
(3) Clopidogrel at discharge | 1.96 (1.77–2.18) | ||
(4) Heparin or low-molecular weight heparin (LMWH) during hospitalisation | 3.47 (2.89–4.16) | ||
(5) Performed coronary angiography (or, for hospitals lacking in-house coronary angiography, referral to another hospital) | 3.05 (2.57–3.63) | ||
Guldberg et al. 2011 [36] | I: 1196 patients; C: 1050 patients | (1) Haemoglobin measurement sustained | 0.86 (0.59–1.25) |
I: 121 patients; C: 91 patients | (2) Haemoglobin measurement initiated if no measurement at baseline | 0.77 (0.45–1.33) | |
I: 1109 patients; C: 887 patients | (3) Cholesterol measurement sustained | 1.74 (1.35–2.24) | |
I: 208 patients; C: 258 patients | (4) Cholesterol measurement initiated if no measurement at baseline | 2.07 (1.38–3.12) | |
Carney et al. 2011 [33] | I: 23 radiologists: C: 9 radiologists | (1) Mean recall rates at time T1(0–9 months) | 1.12 (1.00–1.27) |
(2) Mean recall rates at time T2(9–18 months) | 1.10 (0.96–1.25) | ||
Gude et al. 2016 [35] | I: 7341 patients; C: 4591 patients | (1) Complete data on psychological functioning | 1.07 (0.46–2.5) |
I: 7341 patients; C: 4591 patients | (2) Complete data on social functioning | 7.95 (0.54–116.3) | |
I: 7341 patients; C: 4591 patients | (3) Complete data on lifestyle factors | 1.11 (0.45–2.75) | |
I: 4934 patients; C: 4071 patients | (4) Disease specific education completeda
| 0.57 (0.31–1.06) | |
I: 5580 patients; C: 4591 patients | (5) Lifestyle modification programme completeda
| 1 (0.48–2.04) | |
I: 4591 patients; C: 7341 patients | (6) Improved quality of live after CR | 0.99 (0.84–1.19) | |
I: 7341 patients; C: 4591 patients | (7) Successful smoking cessation | 1.02 (0.86–1.2) | |
I: 7341 patients; C: 4591 patients | (8) Patients receive a discharge letter with remaining lifestyle goals | 0.87 (0.27–2.81) | |
I: 4591 patients; C: 7341 patients | (9) Complete data on physical functioning | 1.32 (0.45–3.84) | |
I: 4591 patients; C: 7341 patients | (10) Complete data concerning cardiovascular risk factors | 1.2 (0.65–2.23) | |
I: 2922 patients; C: 7341 patients | (11) Exercise training completeda
| 1.64 (0.57–4.71) | |
I: 4071 patients; C: 4898 patients | (12) Relaxation and stress management training completeda
| 0.44 (0.14–1.41) | |
I: 4591 patients; C: 7341 patients | (13) Cardiovascular risk factors evaluated at discharge | 1.22 (0.4–3.76) | |
I: 4591 patients; C: 7341 patients | (14) Improvement in exercise capacity | 0.86 (0.69–1.07) | |
I: 4591 patients; C: 7341 patients | (15) Successful work resumption | 1.04 (0.86–1.24) | |
I: 4591 patients; C: 7341 patients | (16) Moderately active lifestyle norm met at discharge | 1.03 (0.82–1.29) | |
I: 4591 patients; C: 7341 patients | (17) Vigorously active lifestyle norm met at discharge | 0.88 (0.74–1.04) |
Covariate |
I
2 residual heterogeneity/unaccounted variability |
R
2 amount of heterogeneity accounted for |
P value (moderator) |
---|---|---|---|
Null model | 99.12% | – | – |
Size of clinical teams | 98.97% |
0.00%
| 0.884 |
Intervention duration | 98.03% | 51.70% | 0.001** |
Interdisciplinary teams | 98.96% |
10.40%
| 0.142 |
Real-time feedbacka
| 97.81% | 58.04% | 0.001** |
Graphical feedback | 99.24% |
0.00%
| 0.825 |
TDF | |||
No. of intervention domains coded | 98.79% |
0.00%
| 0.723 |
No. of control domains coded | 97.65% | 60.81% | 0.001** |
Common aspects of TDF employed as change behaviour mechanisms in e-A&F
Study | Intervention arm | Control arm | ||
---|---|---|---|---|
Domain | Support statement/action | Domain | Support statement/action | |
Linder et al. 2010 [38] | (1) Knowledge | (1) CDCa recommendations (statements having factual or procedural knowledge) | Usual care | |
(4) Nature of the behaviours | (4) Targeted behaviour was reduction of inappropriate antibiotic prescribing; | |||
(5) Beliefs about consequences; | (5) Included billing data to provide a sense of a financial incentive to clinicians; incorrect beliefs that antibiotics are necessary to treat acute respiratory infections | |||
(6) Motivation and goals; (9) Social influences; | (6, 9) View displayed a clinician’s performance against his or her clinic peers and against national benchmarks; | |||
Peiris et al. 2015 [41] | (1) Knowledge | (1) Synthesis of recommendations from several screening and management guidelines for cardiovascular diseases, kidney disease and diabetes mellitus | Usual care | |
(2) Skills (8) Environmental context and resources; | (2, 8) Practices received an average of 48-min support per month comprising on-site training, remote clinical webinars and helpdesk access; | |||
(6) Motivation and goals (9) Social influences | (6, 9) Health services could view peer-ranked performance data benchmarked against other participating trial sites; | |||
(7) Memory, attention and decision process; (11) Behaviour regulation | (7, 11) Tool to allow health services to audit health records, identify performance gaps rapidly and establish recall/reminder prompts rapidly. Provided point of care recommendations based on cardiovascular diseases risk (11) Identification of screening and management gaps for the whole patient population built into a commonly used audit tool | |||
(12) Nature of behaviours | (12) Shifting prescribing behaviours | |||
Thomas et al. 2007 [39] | (1) Knowledge (2) Skills | (1, 2) Dashboard information was organised by evidence-based guidelines, highlighting relevant data; received usual clinic education consisting of faculty review of diabetes care among patients supervised with the resident. (1, 2) Two 1-h sessions introducing registries, describing their value in practice improvement, and providing instruction on registry use | (1) Knowledge; (2) Skills (8) Environmental context and resources | (1, 2, 8) Received usual clinic education consisting of faculty review of diabetes care among patients supervised with the resident and linked to access to the ‘electronic curriculum for diabetes care’; |
(6) Motivation and goals; (9) Social influence; | (6, 9) Feedback comparing their diabetes performance metrics to aggregate resident performance; | |||
(8) Environmental context and resources; | (8) Access to the ‘electronic curriculum for diabetes care’ linked to electronic registry feedback; | |||
(11) Behaviour regulation | (11) Registry-generated lists identifying patients not in compliance with guideline recommendations | |||
Carney et al. 2011 [33] | (1) Knowledge; (2) Skills; (8) Environmental context and resources; | (1,2,8) Continuous medical education modules | Usual care | |
(5) Beliefs about consequences | (5) Profiled breast cancer risk in each radiologist’s respective patient population; Information on the possible impact of medical malpractice concerns on recall rates | |||
(6) Motivation and goals | (6) Awarded 2 h of category I continuous medical education credit; | |||
(7) Memory, attention and decision process | (6, 7) Radiologists were able to insert their goals for changes; | |||
(9) Social influences | (6, 9) Audit data individualised for each participating radiologist with comparisons to both national benchmarks and to peers for the same measures during the same time period. | |||
(11) Behaviour regulation | (11) Illustrating the metrics in clinical performance that could be improved; reinforce change by assisting radiologists to develop goals that would improve their performance; | |||
(12) Nature of behaviours | (12) Reduce unnecessary recall from memory practice | |||
Carlhed et al. 2006 [34] | (1) Knowledge | (1) Educational on the content of National Acute Myocardial Infarction guidelines | (1) Knowledge | (1) Educational on the content of National Acute Myocardial Infarction guidelines |
(4) Belief about capabilities (6) Motivation and goals | (4, 6, 9) During and between learning sessions, the teams were requested to come up with action plans for appropriate local changes; (6, 9) Local performance feedback with comparisons to other centres and national average | (4) Belief about capabilities (6) Motivation and goals | (4, 6, 9) During and between learning sessions, the teams were requested to come up with action plans for appropriate local changes; | |
(8) Environmental context and resources | (8) Education training partly managed at a web-based portal and linked with the registry web tool. | (9) Social influences (11) Behaviour regulation | (9, 11) Frequent collaborative approach meetings to solve common problems between teams and results and lessons learnt were shared with other team members; | |
(9) Social influences (11) Behaviour regulation | (9, 11) Frequent collaborative approach meetings to solve common problems between teams and results and lessons learnt were shared with other team members; | |||
Guldberg et al. 2011 [36] | (1) Knowledge | (1) Guidelines concerning treatment and control of type 2 diabetes in general practice | ||
(5) Beliefs about consequences; | (5) Graphic treatment history of the individual patients by each variable | |||
(6) Motivation and goals; | (6) To provide an overview of the patient population as a basis for planning interventions if needed | |||
(7) Memory, attention and decision process | (7) Option to use data in patient consultations | |||
(9) Social influences; | (6, 9) Graphic presentations comparing each clinic with the other participating clinics by each variable | |||
Gude et al. 2016 [35] | (1) Knowledge | (1) Ineffectiveness partly explained by the fact that professionals were not able to translate their intentions into completed actions, i.e. the second step of the mechanism, before the study end. |
a
Exact replica of the intervention arm
| |
(6) Motivation and goals (intention) (11) Behaviour Regulation | (6) The intervention successfully encouraged teams to define local performance improvement goals (6, 11) Educational outreach visits were held with the local multidisciplinary team to set goals and plan actions and update existing action plans | |||
(7) Memory attention and decision processes | (7) Educational outreach visits were held with the local multidisciplinary team to reflect on the feedback. The team discussed and reflected upon their most recent feedback report and created or updated their QI plan. | |||
(8) Environmental context and resources | (8) Some of the persisting organisational barriers were related to lack of resources (e.g. budget ceilings imposed by insurers), competing interests between managers from different clinical disciplines, and poor attendance of clinical leadership (cardiologists and managers) at outreach visits. (8) Implementation of web tool to be used to develop a quality improvement plan by selecting indicator areas for improvement (related to quality indicators in the feedback report) | |||
(9) Social influences | (9, 6) Performance scores based on the centre’s performance score relative to peer performance was implemented using the concept of achievable benchmarks. | |||
(12) Nature of behaviours | (12) Update existing action plans following a continuous audit and feedback improvement cycle |
Study ID | Linder et al. 2010 [38] | Thomas et al. 2007 [39] | Carlhed et al. 2006 [34] | Guldberg et al. 2011 [36] | Gude et al. 2016 [35] | Total | |||||
---|---|---|---|---|---|---|---|---|---|---|---|
Domain | Intervention | Intervention | Intervention | Control | Intervention | Intervention | Control | Intervention | Intervention | Control | |
(1) Knowledge | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 10 |
(2) Skills | 1 | 1 | 1 | 3 | |||||||
(3) Social/professional role and identity | 0 | ||||||||||
(4) Beliefs about capabilities | 1 | 1 | 2 | ||||||||
(5) Beliefs about consequences | 1 | 1 | 1 | 3 | |||||||
(6) Motivation and goals | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 9 | |
(7) Memory, attention and decision process | 1 | 1 | 1 | 1 | 1 | 5 | |||||
(8) Environmental context and resources | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | |||
(9) Social influences | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 9 | |
(10) Emotion | 0 | ||||||||||
(11) Behaviour regulation | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | |||
(12) Nature of the behaviours | 1 | 1 | 1 | 3 | |||||||
Total | 5 | 8 | 6 | 2 | 9 | 6 | 5 | 5 | 6 | 6 |