Background
Objective
Methods
Search strategy
Study eligibility criteria
Inclusion criteria | Exclusion criteria |
---|---|
• Date of Publication (October 2000-December 2011) | • Date of Publication (Prior to October 2000) |
• English Language | • Non-English Language |
• Empirical Study | • Non-Empirical Study |
• Evidence Use in Program Management | • Evidence Use in Clinical Decision-Making |
• OECD Country | • Evidence Use in Health Policy Decision-Making |
• Not OECD Country | |
• Non-Medical Services |
Data collection and analysis
Results
Overall characteristics of included studies
Study design
Primary author | Publication year | Study methods | Study design | Country | Setting | Participants |
---|---|---|---|---|---|---|
1 Belkhodja | 2007 | Telephone Survey | Quantitative | Canada | Ministries, Health Authorities, Hospitals | 928 decision-makers (managers and professionals) |
2 Bowen | 2009 | Interviews and Focus Groups | Quantitative | Canada | Health Authorities | 205 decision-makers (senior managers, middle managers and board members) |
3 Dobbins | 2007(a) | Interviews | Quantitative | Canada | Public Health Units | 16 decision-makers (6 program mangers, 6 directors, 1 Medical Officer of Health) |
4 Dobbins | 2001 | Telephone Survey and Questionnaire | Quantitative | Canada | Public Health Units | 141 decision-makers (medical and associate medical officers of health, program directors, program managers) |
5 Dobbins | 2007(b) | Telephone Survey | Quantitative | Canada | Community-based health organizations | 92 decision-makers (from any level from CEO to front-line clinicians, senior planners) |
6 Farmer | 2001 | Interviews | Quantitative | Scotland | Health Authorities | 15 decision-makers (7 Directors’and 8 physician advisors) |
7 Ham | 2003 | Interviews, Questionnaires, Case Studies | Qualitative and Quantitative | United Kingdom | Health Authorities | 257 decision-makers (152 managers, 44 medical specialists, 21 nurses, 12 administrative and clerical staff, 12 GPs, 16 other) 4 case studies |
8 Higgins | 2011 | Interviews | Qualitative | Canada | Health Authorities | 21 decision-makers (16 front-line staff 5 managers) |
9 Jbilou | 2007 | Survey | Qualitative | Canada | Health Organizations (Hospitals, Health Authorities, Ministries, Agencies) | 942 decision-makers (managers, professionals, in ministries, hospitals, boards and councils) |
10 McDiarmid | 2007 | Telephone Interview | Qualitative and Quantitative | Canada | Hospitals | 21 decision-makers (16 front-line staff 5 managers) |
11 Mitton | 2004 | Interviews and Focus Groups | Qualitative | Canada | Health Authority | 25 decision-makers (senior managers, clinicians) |
12 Niedzwiedzka | 2003 | Survey, Interviews, Focus Groups, Document Review | Qualitative and Quantitative | Poland | Hospitals and Departments of Health | 815 decision-makers (hospital CEOs, medical directors, head nurses, directors) (#s for interviews and focus groups unknown) |
13 Weatherly | 2002 | Survey, Interviews, Document Review | Qualitative and Quantitative | United Kingdom | Health Authorities | 102 Health Authorities (78 decision makers N 68 coordinators, 10 leaders) |
14 Wilson | 2001 | Online Survey | Quantitative | Canada | Community-based health organizations | 25 decision-makers (Executive Directors) |
Location and decision-making setting
Type of decision-makers
Quality of included studies
Qualitative | Qualitative | Mixed method | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Type-of-study | Methodological-quality-criteria | Bowen [22] | Dobbins [15] | Farmer [16] | Higgins [17] | Mitton [18] | Belkhodja [27] | Dobbins [24] | Dobbins [26] | Jbilou [28] | Wilson [21] | Ham [25] | Niedzwiedzka [19] | Weatherly [20] | McDiarmid [23] |
Screening Questions | Are there clear qualitative and quantitative research questions (or objectives), or a clear mixed methods question (or objective)? | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Do the collected data allow the research question (objective) to be appropriately addressed? | Yes | Yes | Yes | Unclear | Unclear | Unclear | Unclear | Yes | Unclear | Unclear | Yes | Yes | Unclear | Unclear | |
1. Qualitative | 1.1. Are the sources of qualitative data (archives, documents, informants, observations) relevant to address the research question (objective)? | Yes | Yes | Yes | Unclear | Unclear |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Unclear | Yes | Unclear | Yes |
1.2. Is the process for analyzing qualitative data relevant to address the research question (objective)? | Yes | Yes | Yes | Unclear | Unclear |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Unclear | Unclear | Unclear | Unclear | |
1.3. Is appropriate consideration given to how findings relate to the context, e.g., the setting, in which the data were collected? | Unclear | Unclear | Unclear | Unclear | Unclear |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Unclear | Unclear | Unclear | Unclear | |
1.4. Is appropriate consideration given to how findings relate to researchers’ influence, e.g., through their interactions with participants? | No | No | No | No | No |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Unclear | Unclear | Unclear | Unclear | |
4. Quantitative Descriptive | 4.1. Is the sampling strategy relevant to address the quantitative research question (quantitative aspect of the mixed methods question)? |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Yes | Yes | Yes | Yes | Yes | Unclear | Yes | Yes | Yes |
4.2. Is the sample representative of the population understudy? |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Unclear | Unclear | Unclear | Unclear | Unclear | Unclear | Unclear | Yes | Unclear | |
4.3. Are measurements appropriate (clear origin, or validity known, or standard instrument)? |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Unclear | Yes | Yes | Unclear | Yes | Unclear | Yes | Unclear | Yes | |
4.4. Is there an acceptable response rate (60% or above)? |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Yes | Yes | Yes | Yes | No | Unclear | Unclear | Yes | No | |
5. Mixed Methods | 5.1. Is the mixed methods research design relevant to address the qualitative and quantitative research questions (or objectives), or the qualitative and quantitative aspects of the mixed methods question (or objective)? |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Yes | Unclear | Yes | Unclear |
5.2. Is the integration of qualitative and quantitative data (or results) relevant to address the research question (objective)? |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Yes | Unclear | Yes | Unclear | |
5.3. Is appropriate consideration given to the limitations associated with this integration, e.g., the divergence of qualitative and quantitative data (or results) in a triangulation design? |
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
|
Not Applicable
| Unclear | Unclear | Yes | Unclear |
Barriers to evidence use
Primary author | Publication year | Information | Organization (structure & process) | Organization (culture) | Individual | Interaction |
---|---|---|---|---|---|---|
1 Belkhodja | 2007 | X | ||||
2 Bowen | 2009 | X | X | X | X | |
3 Dobbins | 2007(a) | |||||
4 Dobbins | 2001 | X | ||||
5 Dobbins | 2007(b) | X | X | X | ||
6 Farmer | 2001 | X | X | X | X | |
7 Ham | 2003 | X | X | |||
8 Higgins | 2011 | X | X | X | ||
9 Jbilou | 2007 | |||||
10 McDiarmid | 2007 | X | X | |||
11 Mitton | 2004 | X | X | X | X | |
12 Niedzwiedzka | 2003 | X | X | X | ||
13 Weatherly | 2002 | X | X | X | X | |
14 Wilson | 2003 | X | X | X | X |
Barrier theme | Types of barrier |
---|---|
Information | • Irrelevance of research |
• Unclear definition of evidence | |
• Negative perceptions of research | |
• Limited access to information | |
• Mismatch of research to complex reality | |
• Time required to produce research | |
• Excess quantity of information | |
Organization (Structure and Process) | • Time limitations |
• Lack of internal research resources | |
• Human resource constraints | |
• Financial constraints | |
• Lack of data and systems | |
• Deficient planning processes | |
• Absence of processes | |
• Poor support from senior management | |
• Rigid program silos | |
• Competing priorities | |
• Poor communication | |
Organization (Culture) | • Decision-making |
• Crisis management | |
• Resistance to change | |
• Politically influenced decisions | |
• Challenging the promotion of evidence use | |
Individual Skills | • Research literacy |
• Research utilization | |
• Management | |
Interaction | • Decision-maker/researcher gap |
• Mutual mistrust |
Barriers: information
Barriers: organization – structure and process
“Lack of time and resources emerged as key barriers. Under-resourcing was described as resulting in poor decisions, …an inability to allocate resources to research or evidence-related positions and (perhaps most importantly) workload pressures that were described as actively working against the thoughtful reflection essential for [evidence-informed decision making]” [22], p. 93.
Barriers: organization – culture
Barriers: individual
“Capacity was lowest for the domains related to: acquiring research (subsection I); assessing the reliability, quality, relevance, and applicability of research evidence (subsections III and IV); and summarizing results in a user-friendly way”[21], p. 3.
Barriers: interaction
Facilitators of evidence use
“Public health decision-makers value the use of systematic reviews to facilitate the decision-making process. They indicated that systematic reviews were particularly useful because they integrate the results of many studies into one, which allows them to bypass the stage of looking at individual studies. This saves them time and gives them more confidence knowing their decisions are based on the culmination of many studies instead of just a few”[26], p. 159.
Primary author | Publication year | Information | Organization (structure & process) | Organization (culture) | Individual | Interaction |
---|---|---|---|---|---|---|
1 Belkhodja | 2007 | X | X | X | X | X |
2 Bowen | 2009 | |||||
3 Dobbins | 2007(a) | X | X | |||
4 Dobbins | 2001 | X | X | X | ||
5 Dobbins | 2007(b) | X | X | |||
6 Farmer | 2001 | X | X | X | ||
7 Ham | 2003 | X | X | X | ||
8 Higgins | 2011 | |||||
9 Jbilou | 2007 | X | X | X | X | |
10 McDiarmid | 2007 | |||||
11 Mitton | 2004 | X | X | |||
12 Niedzwiedzka | 2003 | X | X | |||
13 Weatherly | 2002 | X | X | X | ||
14 Wilson | 2011 | X | X | X | X |
Facilitator theme | Types of facilitator |
---|---|
Information | • Access to information |
• Complex intervention evaluation methods | |
• Targeted dissemination | |
Organization (Structure and Process) | • Intra-organizational linkages |
• Expertise in research utilization | |
• Processes for integration of evidence | |
• Administrative support | |
• Operational data availability | |
Organization (Culture) | • Supporting evidence use |
• Human resources training and rewards | |
• Inter-organizational collaboration | |
• Visible research utilization | |
Individual Skills | • Researcher and decision-maker focus on application |
Interaction | • Contact between researchers and decision-makers |
• Mutual respect |
Facilitators: information
“research-producing organizations knowing not only who their target audience(s) are and what their needs are concerning research evidence, but also what questions require answers, and what kind of answers are optimal for different types of decisions”[15] p. 9.