Background
Methods
Identification and selection of studies
Study selection
Type of studies
Population
Interventions
Type of outcomes
Screening
Quality assessment
Data extraction
Data analysis
Results
Search results
Characteristics of included studies
Author, year, country | Study design | Health topic | Health organisation setting | Decision-maker population | Control group | Research implementation group | Outcome measure | ||
---|---|---|---|---|---|---|---|---|---|
Beynon et al. 2012, multi-national [46] | Randomised controlled trial | Health in low- and middle-income countries | Public health | Professions from government and non-government organisations and academia (n = 807) | Existing Institute of Development Studies publication from the In Focus Policy Briefing series | Basic 3-page policy brief | Basic 3-page policy brief plus an expert opinion piece | Basic 3-page policy brief plus an unnamed research fellow opinion piece | Online questionnaires (immediately, 1 week and 3 months post) Semi-structured interviews (in-between 1 week and 3 months and after 3 month questionnaires) |
Brownson et al. 2007, USA [47] | Quasi-experimental | Guidelines for promoting physical activity | State and local health departments (n = 8) | Health department program managers, administrators, division, bureau, or agency heads, and ‘other’ positions e.g. program planner, nutritionist (State n = 58) (Local n = 55) (Other n = 80) | Remaining states and the Virgin Islands served as the comparison group) | Workshops, ongoing technical assistance and distribution of an instructional CD-ROM | 25-item questionnaire survey (2 years) | ||
Bullock et al. 2012, UK [48] | Programme evaluation case study | Non-specific | NHS health service delivery organisations (n = 10) | Management fellows (n = 11) Chief investigators (n = 10) Additional co-applicants from the research teams (n = 3) Workplace line-managers (n = 12) (Total n = 36) | None | UK Service Delivery and Organisation (SDO) Management Fellowship programme | Semi-structured face-to-face interviews | ||
Campbell et al. 2011, Australia [49] | Program evaluation | Range of topics related to population health, health services organisation and delivery, and cost effectiveness | State-level policy agencies, including both the New South Wales and Victorian Departments of Health (n = 5) | Policymakers (n = 8) | None | ‘Evidence check’ rapid policy relevant review and knowledge brokers | Structured interviews (2–3 years) | ||
Chambers et al. 2012, UK [58] | Case study | Adolescents with eating disorders | Primary care | Local NHS commissioners and clinicians (n = 15) | None | Contextualised evidence briefing based on systematic review | Short evaluation questionnaire | ||
Champagne et al. 2014, Canada [59] | Case studies | Non-specific | Academic health centres (n = 6) | Extra fellows, SEARCHers, Colleagues, Supervisors, Vice-presidents and CEOs (n = 84) | None | Executive Training for Research Application (EXTRA) program Swift, Efficient, Application of Research in Community Health (SEARCH) Classic program | Semi-structured interviews and data from available organisational documents | ||
Courtney et al. 2007, USA [60] | Cohort study | Substance abuse treatment programs | Community-based treatment units (n = 53 units from n = 24 multisite parent organisations) | Directors and clinical supervisors (n = 309) | None | 2-day workshop (entitled “TCU Model Training-making it real”) | Compliance with early steps of consulting and planning activities (1 month) Organisational Readiness for Change (ORC) assessment (1 month) | ||
Dagenais et al. 2015, Burkina Faso [52] | Implementation evaluation | Maternal health, malaria prevention, free healthcare, and family planning | Public health | Researchers; Knowledge brokers; health professionals; community-based organisations; and local, regional, and national policy-makers (n = 47) | None | Knowledge broker | Semi-structured individual interviews and participant training session questionnaires | ||
Dobbins et al. 2001, Canada [61]. | Cross-sectional follow-up survey | Home visiting as a public health intervention, community-based heart health promotion, adolescent suicide prevention, community development, and parent-child health | Public health units (n = 41) | Public health decision-makers (n = 147) | None | Systematic reviews | Cross-sectional follow-up telephone survey | ||
Dobbins et al. 2009, Canada [9] | Randomised controlled trial | Promotion of healthy bodyweight in children | Public health departments (n = 108) | Front-line staff 35% Managers 26% Directors 10% Coordinators 9% Other 20% (n = 108) | Access to an online registry of research evidence | Tailored, targeted messages Access to an online registry of research evidence | Knowledge broker Tailored, targeted messages Access to an online registry of research evidence | Telephone-administered survey (knowledge transfer and exchange data collection tool) 1–3 months post completion of intervention (intervention lasted 12 months) | |
Dopp et al. 2013, Netherlands [55] | Mixed methods process evaluation | Dementia | Home-based community health | Managers (n = 20) Physicians (n = 36) Occupational therapists (n = 36) | None | Multifaceted implementation strategy | Semi-structured telephone interviews with managers (3–5 months) Semi-structured focus groups with occupational therapists (2 months) | ||
Flanders et al. 2009, USA [53] | Implementation evaluation | Patient safety | Teaching and nonteaching, urban and rural, government and private, as well as academic and community settings (n = 9) | Hospitalists or quality improvement staff, representatives from each institutions department of quality or department of patient safety (n = 9) | None | The Hospitalists as Emerging Leaders in Patient Safety (HELPS) Consortium | Web-based survey (post meetings) | ||
Gagliardi et al. 2008, Canada [56] | Mixed methods exploratory | Colorectal cancer | Not specified | Researchers (n = 6) Clinicians (n = 13) Manager (n = 5) Policy-maker (n = 5) (Total n = 29) | Review of Canadian health services research in colorectal cancer based on published performance measures 1-day workshop to prioritise research gaps, define research questions and plan implementation of a research study. | Participant survey (prior to workshop) Observation of workshop participants (during workshop) Semi-structured interviews and observation of workshop participants (during workshop) | |||
Kitson et al. 2011, Australia [50] | Project evaluation | 7 clinical topic areas identified in The Older Person and Improving Care (TOPIC7) project | Large tertiary hospital (n = 1) | Clinical nursing leaders (n = 14) Team members (n = 28) Managers (n = 11) | None | Knowledge translation toolkit | Semi-structured interviews and questionnaires | ||
Moat et al. 2014, multi-national, [63] | Survey evaluation | Health in low- and middle-income countries | Public health | Policy-makers, stakeholders and researchers (n = 530) | None | Evidence briefs Deliberative dialogues | Questionnaire surveys | ||
Traynor et al. 2014, Canada [57] | Single mixed-methods study and a case study | Child obesity | Canadian public health departments (n = 30) (Case studies n = 3) | Health department staff (RCT n = 108) (Case A n = 258) (Case B n = 391) (Case C n = 155) | Access to an online registry of research evidence | Knowledge brokering | Knowledge broker journaling (baseline, interim, follow-up) Qualitative interviews n = 12 (1 year) Case study interviews n = 37 (baseline, interim and 22 month follow-up) | ||
Uneke et al. 2015, Nigeria [54] | Implementation evaluation | Low- and middle-income country health | Public health | Directors from Ministry of Health (n = 9) Senior researchers from the university (n = 5) NGO executive director (n = 1) Director of public health in the local government service commission (n = 1) Executive secretary of the AIDS control agency (n = 1) State focal person of Millennium Development Goals (n = 1) (Total n = 18) | None | Training workshop (HPAC) Certificate course (HPAC) Policy brief and hosting of a multi-stakeholder policy dialogue (HPAC) | Semi-structured interviews (end of each intervention) Group discussions | ||
Waqa et al. 2013, Fiji [51] | Process evaluation | Overweight and obesity | Public health government organisations (n = 6) NGOs (n = 2) | Senior managers (n = 20) Middle managers (n = 22) Junior managers (n = 7) (Total n = 49) | None | Policy brief and hosting of a multi-stakeholder policy dialogue (HPAC) | Semi-structured interviews Process diaries | ||
Wilson et al. 2015, Canada [64] | Process evaluation | Non specific | Policy analysts (n = 9) Health department units (n = 6) | Senior analysts (n = 8) Junior analysts (n = 1) | None | Access to an online registry of research evidence | Semi-structured telephone interviews |
Study design
Participants and settings
Research implementation strategies
Study (author, year) | Implementation strategy | Theoretical framework | Summary description |
---|---|---|---|
Access to online registry of research evidence | Dobbins framework | Reference offered a link to a short summary and full text of each review | |
Tailored, targeted messages and access to online registry of research evidence | Title of systematic review and link to full reference, including abstract sent via email Reference offered a link to a short summary and full text of each review | ||
Knowledge broker, tailored messages, and access to online registry of research evidence | Knowledge brokers ensured relevant evidence was transferred in useful ways to decision-makers to assist skills and capacity development for translating evidence into local healthcare delivery. Activities included regular electronic and telephone communication, one face-to-face site visit, and invitation to a workshop. Title of systematic review and link to full reference, including abstract sent via email Reference offered a link to a short summary and full text of each review | ||
Beynon 2012, [46] | Basic 3-page policy brief | A simple theory of change for a policy brief | Link to policy brief sent via email |
Basic 3-page policy brief plus an expert opinion piece | Same basic 3-page policy brief plus an expert opinion piece credited and written by a sector expert, Lawrence Haddad. Link to policy brief sent via email | ||
Basic 3-page policy brief plus an un-credited expert opinion piece | Same basic 3-page policy brief and expert opinion piece but credited to an unnamed research fellow. Link to policy brief sent via email | ||
Brownson 2007, [47] | Workshops, ongoing technical assistance, and distribution of an instructional digital materials | Framework for a systematic approach to promoting effective physical activity programs and policies | Workshops included: formal presentations, case study applications, and ‘real-world’ examples Ongoing technical assistance included: strategic planning, grant writing, tuition waivers, consultation for effective strategy planning, and dissemination guidance Digital materials included: additional information, prominent public health leader interviews, and resource tools |
Courtney 2007, [60] | Workshop | The change book | Pre-workshop completion of organisational readiness for change assessment. Workshop included: conceptual overview presentations, personalised feedback, comparison with other agencies, and group work |
Bullock 2012 [48] | Fellowship program | Programme evaluation framework (adapted from Kirkpatrick) | Practicing managers work within research teams for the duration of a funded project |
Campbell 2011, [49] | ‘Evidence check’ rapid policy relevant review and knowledge brokers | Van Kammen et al.’s approach to knowledge brokering | Pre-meeting commissioning tool completed prior to knowledge broker meetings, which clarified research question. Then a rapid review summary of evidence on policy area is performed |
Chambers 2012, [58] | Contextualised evidence briefing based on systematic review | Facilitators of the use of research evidence identified by a systematic review (Innvaer et al. [28]) | Researcher attended meeting to clarify research question and prepared a concise evidence briefing on policy area |
Champagne 2014, [59] | Executive Training for Research Application (EXTRA) program | Knowledge creation logic model | Program included: residency sessions, projects, educational activities, networking, and post-program activities |
Swift, Efficient, Application of Research in Community Health (SEARCH) Classic program | Program included: modules, inter-module work, and application of knowledge to practice-based projects | ||
Dagenais 2015, [52] | Knowledge broker | Theoretical models for understanding health behaviour | Knowledge broker tasks included: liaison, information management and support, partner meetings, developing documentary research strategies, database set-up for relevant information, drafting summary documents, workshops, and developing and monitoring actions plans |
Dobbins 2001, [61] | Systematic reviews | – | Systematic reviews of the effectiveness of public health interventions disseminated to public health decision-makers |
Dopp 2013, [55] | Multifaceted implementation strategy | The model of Grol and Wensing | Educational materials, educational meetings, outreach visits, newsletters, and reminders |
Flanders 2009, [53] | The Hospitalists as Emerging Leaders in Patient Safety (HELPS) Consortium | – | Meetings on quality improvement methodology and substantiative patient safety-related topics, and a final half-day session drawing out learning’s and next steps |
Gagliardi 2008, [56] | Comprehensive review and workshop | Author’s conceptual model of factors influencing effectiveness of knowledge exchange | Comprehensive review of Canadian health services research in colorectal cancer based on published performance measures and workshop to prioritise research gaps, define research questions, and plan implementation of a research study |
Kitson 2011, [50] | Knowledge translation toolkit | – | Team recruitment, clarification, stakeholder engagement, pre-strategy evaluation, training, support meetings, communication and feedback, process evaluation, dissemination (e.g. posters and presentations), future planning, and program evaluation |
Moat et al. 2014, multi-national, [50] | Evidence briefs | Theory of planned behaviour | Evidence briefs and deliberative dialogues across a range of issues and low- and middle-income countries |
Deliberative dialogues | |||
Uneke 2015, [54] | Training, workshop, certificate course, policy brief, and hosting of a multi-stakeholder policy dialogue | – | Workshop featuring training on the role of research evidence, preparation of policy briefs, how to organise and use policy dialogues, and how to set priorities. Certificate course aimed to foster research capacity, leadership, enhance capacity for evidence-informed decision-making, and health policy monitoring/evaluation. Policy briefs were produced, and the multi-stakeholder policy dialogue between key stakeholders was then held |
Knowledge broker capacity building | – | Knowledge coordinated organisation recruitment, mapping policy environment, analysed organisational capacity and support for evidence-informed policymaking, developed evidence-informed policymaking skills, and facilitated development of evidence-informed policy briefs | |
Wilson et al. 2015, Canada [64] | Access to online registry of research evidence | Framework for assessing country-level efforts to link research to action | The ‘self-serve’ evidence service consisted only of database access |
Access to online registry of research evidence, email alerts, and full-text availability | The ‘full-serve’ evidence service included (1) database access for research evidence addressing questions about governance, financial and delivery arrangements within which programs, services and drugs are provided and about implementation strategies; (2) monthly email alerts about new additions to the database; and (3) full-text article availability |
Quality/risk of bias
Experimental studies
Non-experimental studies
Narrative synthesis results: effectiveness of research implementation strategies for promoting evidence-informed policy and management decisions in healthcare
Study (author, year) | Implementation strategy | Level 1: change in reaction/attitudes/beliefs | Level 2: learning | Level 3: behaviour |
---|---|---|---|---|
Randomised controlled trial | ||||
Beynon 2012 [46] | Basic 3-page policy brief | High-quality ratings Opinion about evidence strength or intervention effectiveness varies by health topic | – | Less likely to source other information and research related to the topic than control |
Basic 3-page policy brief plus an expert opinion piece | High-quality rating Opinion about evidence strength or intervention effectiveness varies by health topic. Increased intention to send policy brief to someone else and tell someone about key messages | – | Less likely to source other information and research related to the topic than control. Trend towards intentions persisting to actions. More likely to send policy brief to someone else | |
Basic 3-page policy brief plus an un-credited expert opinion piece | High-quality rating Opinion about evidence strength or intervention effectiveness varies by health topic | – | Less likely to source other information and research related to the topic than control | |
Dobbins 2009 [9] | Tailored, targeted messages | – | – | Improved use of public health policies and programs compared to control |
Tailored, targeted messages plus a knowledge broker | – | – | Addition of knowledge broker potentially reduced use of public health policies and programs. However, improvements may have occurred in organisations with low research culture | |
Non-randomised controlled trial | ||||
Brownson 2007 [47] | Workshops, ongoing technical assistance, and digital resources | Change in whether heard of recommendations and attended training. Less likely to report state legislators were supportive of physical activity interventions. No change in other outcomes from baseline | All knowledge and skill measurements improved. Change larger for local than state health department decision-makers in every category except methods in understanding cost. The largest change related to attitudes | Improvement in self-reported individual adapted health behaviour change. No difference in other behaviour change outcomes |