Background
Methods
Development of an implementability framework
Approach
Data collection
Data analysis
Application of the implementability framework
Approach
Sampling
Data collection
Data analysis and interpretation
Results
Study | Design | Guideline features encouraging use |
---|---|---|
Brouwers 2009 Canada [48] | Survey of 756 physicians of various specialties between 1999 and 2005 on intended use of 84 cancer guidelines yielding 4,091 surveys | Strong supporting evidence, flexibility of recommendations to local context |
Wakkee 2008 Netherlands [49] | Questionnaire of 261 dermatologists on characteristics of specific guideline | Concise recommendations |
Nuckols 2008 United States [39] | Modified Delphi panel of 11 physicians of various specialties | Strong supporting evidence, flexibility of recommendations to patient needs and preferences |
Francke 2008 Netherlands [7] | Meta-review of 12 systematic reviews on guideline implementation: | |
1. 41 cross sectional pre-/post-test studies or controlled trials | Easily accessible, strong supporting evidence, explicit resource implications, flexibility of recommendations to local | |
2. 76 survey and qualitative studies | ||
3. 91 randomized, cross-over, balanced incomplete block design, controlled before/after, interrupted times series studies | context, concise recommendations | |
4. 61 mixed methods studies with focus on randomized or controlled trials | ||
5. 23 studies of various quantitative designs | ||
6. 235 randomized or controlled trials, controlled before/after or interrupted time series designs | ||
7. 40 randomized or controlled trials or before/after studies | ||
8. 15 randomized or controlled trials, pre-/post-test studies and one systematic review | ||
9. 59 studies of various quantitative or qualitative or mixed design | ||
10. 6 randomized controlled trials, time series or before/after studies and 8 studies of mixed design | ||
11. 18 ranodmized or controlled trials, before/after or interrupted time series studies | ||
12. 20 randomized or controlled trials, case series or case reports | ||
Cochrane 2007 United States [30] | Systematic review of 256 studies of guideline implementation (178 surveys, 16 focus group studies, 18 interview studies, 44 mixed methods studies) | Easily accessible, strong supporting evidence, flexibility of recommendations to local context, concise recommendations |
Carlsen 2007 Norway [50] | Qualitative analysis of six focus groups involving 27 general practitioners | Trustworthy, suit patients, recommended action is feasible |
Carlsen 2007 Norway [42] | Systematic review of 12 qualitative studies (7 focus group studies, 5 interview studies) evaluating general practitioner attitudes about guidelines | Authorship familiarity, flexibility of recommendations to patient needs and preferences, short and concise, include patient leaflets |
Jones 2007 Canada [51] | Qualitative analysis 28 interviews with physicians and nurses in four intensive care units | Easily accessible, accompanying tools such as checklists, strong supporting evidence, concise recommendations |
Thomason 2007 United States [52] | Survey and focus groups with 60 physicians and nurses who attended a national conferences | Strong supporting evidence, concise recommendations |
Sinuff 2007 Canada [53] | Qualitative analysis of interviews with 30 physicians and nurses at one hospital | Easily accessible, accompanying tools such as algorithms or pocket cards, concise recommendations |
McKinlay 2004 New Zealand [54] | Qualitative analysis of interviews with 13 general practitioners from five sites | Authorship familiarity, variety of print and electronic formats |
Shiffman 2003 United States [55] | Modified Delphi process involving representatives from 22 organizations active in guideline development | Explicit resource implications, suggestions for auxiliary documents for providers or patients, evaluative data collection tools |
Price 2001 United States [56] | Discourse analysis of laboratory study using clinical scenarios and guidelines of different formats involving three general practitioners and three endocrinologists | Algorithmic guidelines were useful for clinical problem solving, textual guidelines were useful for learning |
Vinker 2000 Israel [57] | Questionnaire of 293 general practitioners and family physicians participating in educational programs over two months | Strong supporting evidence, flexibility of recommendations to patient needs and preferences, concise recommendations |
Harris 2000 United States [42] | Questionnaire and focus groups with an undisclosed sample drawn from 304 general practitioners based at 16 sites | Accompanying tools such as checklists and standard orders, summaries such as algorithms or diagrams, navigational support such as color-coded tabs, evaluative data collection tools, accessible by computer, information guides for patients |
Shekelle 2000 United States [31] | Randomized controlled trial of questionnaire on intent to use guidelines among 545 general internists, neurologists and physical medicine specialists who received usual guideline or guideline modified with clinical vignettes | Clinical vignettes describing application of guidelines according to patient needs and preferences |
Cabana 1999 United States [58] | Systematic review of 76 journal articles on barriers to guideline adherence among physicians | Strong supporting evidence, authorship familiarity, easily accessible, concise recommendations, flexibility of recommendations to patient needs and preferences |
Grol 1998 Netherlands [37] | Observational study involving 12,880 decisions made by 61 general practitioners based on 12 guidelines with various attributes rated by participants | Strong supporting evidence, concise recommendations, explicit resource implications |
Domain | Definition |
---|---|
Adaptability | The guideline is available in a variety of versions for different users or purposes. |
Usability | Content is presented, organized, or formatted to enhance the ease with which the guideline can be employed. |
Validity | Evidence is summarized and presented such that its quantity and quality are apparent, and it can be easily reviewed, understood, and interpreted. |
Applicability | Contextual or supplementary clinical information is provided by which to interpret and apply the recommendations for individual patients. |
Communicability | Information is included to support discussions with patients, or patient involvement in decision making. |
Accommodation | Costs, resources, competencies and training, technical specifications, and anticipated impact required to accommodate use are identified. |
Implementation | Strategies for identifying barriers of use, and selecting, planning, and applying promotional strategies are described. |
Evaluation | Performance measures for audit or monitoring are included. |
Type of organization | Overall | Diabetes | Hypertension | Leg Ulcer | Heart Failure |
---|---|---|---|---|---|
Academic | 1 | 1 | --- | --- | --- |
Government | 6 | 2 | 2 | 1 | 1 |
Expert panel or consortium | 2 | 1 | --- | 1 | --- |
Professional association | 9 | 3 | 2 | 1 | 3 |
Private, nonprofit | 2 | 1 | --- | --- | 1 |
Total | 20 | 8 | 4 | 3 | 5 |
Domain | Element | Examples |
---|---|---|
Usability | Navigation | Table of contents |
Evidence format | Narrative, tabulated or both | |
Recommendation format | Narrative, graphic (algorithms) or both; Recommendation summary (single list in full or summary version) | |
Adaptability | Alternate versions | Summary (print, electronic for PDA); Patient (tailored for patients/caregivers); Published (journal) |
Validity | Number of references | Total number of distinct references to evidence upon which recommendations are based |
Evidence graded | A system is used to categorize quality of evidence supporting each recommendation | |
Number of recommendations | Total number of distinct recommendations (sub-recommendations considered same) | |
Applicability | Individualization | Clinical information (indications, criteria, risk factors, drug dosing) that facilitates application of the recommendations explicitly highlighted as tips or practical issues using sub-titles or text boxes, or summarized in tables and referred to in recommendations or narrative contextualizing recommendations |
Communicability | Patient education or involvement | Informational or educational resources for patients/caregivers, questions for clinicians to facilitate discussion, or contact information (phone, fax, email or URL) to acquire informational or educational resources |
Accommodation | Objective | Explicitly stated purpose of guideline (clinical decision making, education, policy, quality improvement) |
Users | Who would deliver/enable delivery of recommendations (individuals, teams, departments, institutions, managers, policy makers, internal/external agents), who would receive the services (patients/caregivers) | |
User needs/values | Identification of stakeholder needs, perspectives, interests or values | |
Technical | Equipment or technology needed, or the way services should be organized to deliver recommendations | |
Regulatory | Industrial standards for equipment or technology, or policy regarding their use | |
Human resources | Type and number of health professionals needed to deliver recommended services | |
Professional | Education, training or competencies needed by clinicians/staff to deliver recommendations | |
Impact | Anticipated changes in workflow or processes during/after adoption of recommendations | |
Costs | Direct or productivity costs incurred as a result of acquiring resources or training needed to accommodate recommendations, or as a result of service reductions during transition from old to new processes | |
Implementation | Barriers/facilitators | Individual, organizational, or system barriers that are associated with adoption |
Tools | Instructions, tools or templates to tailor guideline/recommendations for local context; Point-of-care templates/forms (clinical assessment, standard orders) | |
Strategies | Possible mechanisms by which to implement guideline/recommendations | |
Evaluation | Monitoring | Suggestions for evaluating compliance with organization, delivery and outcomes of recommendations, including program evaluation, audit tools, and performance measures/quality indicators |
Domain/Element | Statistic | Overall (n = 20) | Diabetes (n = 8) | Hypertension (n = 4) | Leg Ulcer (n = 3) | Heart Failure (n = 5) |
---|---|---|---|---|---|---|
Adaptability | ||||||
Journal version | n (%) | 10 (50.0) | 4 (50.0) | 3 (75.0) | 0 (0.0) | 3 (60.0) |
PDA version | n (%) | 5 (25.0) | 3 (37.5) | 1 (25.0) | 0 (0.0) | 1 (20.0) |
Short version | n (%) | 9 (45.0) | 0 (0.0) | 3 (75.0) | 2 (66.7) | 4 (80.0) |
Patient version | n (%) | 4 (20.0) | 0 (0.0) | 2 (50.0) | 0 (0.0) | 2 (40.0) |
Usability | ||||||
Table of contents | n (%) | 15 (75.0) | 6 (75.0) | 2 (50.0) | 3 (100.0) | 4 (80.0) |
Number of pages | mean | 120.2 | 199.5 | 45.8 | 60.7 | 88.4 |
med | 72.5 | 95.5 | 46.0 | 46.0 | 80.0 | |
min | 21.0 | 21.0 | 39.0 | 21.0 | 25.0 | |
max | 878.0 | 878.0 | 52.0 | 115.0 | 163.0 | |
Number of recommendations | mean | 71.7 | 120.8 | 10.5 | 41.3 | 60.4 |
med | 41.5 | 126.5 | 9.5 | 39.0 | 43.0 | |
min | 8.0 | 24.0 | 8.0 | 20.0 | 9.0 | |
max | 214.0 | 214.0 | 15.0 | 65.0 | 118.0 | |
Recommendation summary | n (%) | 11 (55.0) | 3 (37.5) | 4 (100.0) | 2 (66.7) | 2 (40.0) |
Recommendation algorithm | n (%) | 13 (65.0) | 2 (25.0) | 4 (100.0) | 2 (66.7) | 5 (100.0) |
Validity | ||||||
Number of references | mean | 452.0 | 849.9 | 128.8 | 111.7 | 278.2 |
med | 230.5 | 247.0 | 80.0 | 83.0 | 252.0 | |
min | 15.0 | 15.0 | 24.0 | 72.0 | 218.0 | |
max | 3,487.0 | 3,487.0 | 331.0 | 180.0 | 347.0 | |
Evidence graded | n (%) | 19 (95.0) | 7 (85.5) | 4 (100.0) | 3 (100.0) | 5 (100.0) |
Evidence format | narrative | 15 (75.0) | 6 (75.0) | 4 (100.0) | 2 (66.7) | 3 (60.0) |
narrative + tabular | 5 (25.0) | 2 (25.0) | --- | 1 (33.3) | 2 (40.0) |
Domain/Element | Overall (n = 20) | Diabetes (n = 8) | Hypertension (n = 4) | Leg Ulcer (n = 3) | Heart Failure (n = 5) | |||||
---|---|---|---|---|---|---|---|---|---|---|
n | % | n | % | n | % | n | % | n | % | |
Applicability | ||||||||||
Individualization | 18 | 90.0 | 6 | 75.0 | 4 | 100.0 | 3 | 100.0 | 5 | 100.0 |
Communicability | ||||||||||
Patient informed care | 10 | 50.0 | 4 | 50.0 | 2 | 50.0 | 1 | 33.3 | 3 | 60.0 |
Accommodation | ||||||||||
Objectives: | ||||||||||
Clinical | 20 | 100. | 8 | 100. | 4 | 100.0 | 3 | 100.0 | 5 | 100.0 |
Education | 1 | 0 | 1 | 0 | --- | --- | --- | --- | --- | --- |
Policy | --- | 5.0 | --- | 12.5 | --- | --- | --- | --- | --- | --- |
Quality improvement | 2 | --- 10.0 | 1 | --- 12.5 | --- | --- | --- | --- | --- | --- |
Users | 12 | 60.0 | 5 | 62.5 | 1 | 25.0 | 2 | 66.7 | 4 | 80.0 |
User needs/values | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.00 | 0 | 0.0 |
Technical | 9 | 45.0 | 3 | 37.5 | 1 | 25.0 | 1 | 33.3 | 4 | 80.0 |
Regulatory | 3 | 15.0 | 0 | 0.0 | 3 | 75.0 | 0 | 0.0 | 0 | 0.0 |
Human resources | 1 | 5.0 | 1 | 12.5 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
Professional | 4 | 20.0 | 0 | 0.0 | 1 | 25.0 | 2 | 66.7 | 1 | 20.0 |
Impact | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
Costs | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
Implementation | ||||||||||
Barriers | 3 | 15.0 | 1 | 12.5 | 2 | 20.0 | 0 | 0.0 | 0 | 0.0 |
Tailoring instructions | 2 | 10.0 | 0 | 0.0 | 0 | 0.0 | 2 | 66.7 | 0 | 0.0 |
Point-of-care tools/forms | 6 | 30.0 | 3 | 37.5 | 0 | 0.0 | 2 | 66.7 | 1 | 20.0 |
Implementation strategies | 9 | 45.0 | 4 | 50.0 | 1 | 25.0 | 2 | 66.7 | 2 | 40.0 |
Evaluation | ||||||||||
Evaluation instructions | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
Performance measures | 10 | 50.0 | 4 | 50.0 | 2 | 50.0 | 2 | 66.7 | 2 | 40.0 |