Background
Methods
Preparing the issue brief
Convening the stakeholder dialogue
Results
Key findings from the issue brief
The problem
-
limited number of formalized supports in place to provide decision-makers with rapid syntheses of the best available research evidence about problems, options and/or implementation considerations related to health system challenges (i.e. providing the right product at the right time);
-
inconsistent interaction between researchers and decision-makers to ensure that the priorities of decision-makers are addressed (i.e. having the right people developing products on the right issues); and
-
uncertainty about what success looks like given the long chain of potential causal relationships between an intervention/program (e.g. a rapid-response program) and relevant outcomes (e.g. whether decision-makers’ needs are met and/or their use of research evidence).
Three broad features of a program to address the problem
Organizational feature | Possible approaches to operationalizing each feature | Criteria meta |
---|---|---|
Governance (structure, scope and rules) | ● Administer the rapid-response program through the McMaster Health Forum under its existing governance structure that prioritizes strong links with and involvement of policymakers and stakeholders in the programs it delivers | ● 1, 7 |
● Operationalize this approach to governance by convening a rapid-response program steering committee consisting of federal, provincial and territorial health system decision-makers and stakeholders who can provide strategic guidance about administering the program | ● 1, 9 | |
● Establish that the rapid-response program: | ● 2 | |
○ addresses topics requested by health system decision-makers (requests will be submitted to the Forum through email and the questions will be refined by the Forum in collaboration with the requestor where necessary); | ||
○ ensures that the findings of the syntheses are based on the available research evidence and not the personal views of those who requested or developed it; | ||
○ identifies whether any potential conflicts of interest exist in any product produced through the rapid-response program; and | ||
○ disseminates completed syntheses (e.g. through the existing Forum Update newsletter and/or through a dedicated email list to the program partners) and makes them available through a dedicated repository on the Forum’s website (but without the requestor’s jurisdiction attached to the synthesis to provide some level of anonymity) | ||
Management and staffing | ● Allocate authority to the organizational leadership of the Forum for ensuring the accountability of the program in relation to its mandate | ● 3 |
● Use effective project management processes to make the best use of available resources and to sequence and prioritize tasks in a way that allows for all requests to be completed within specified timelines | ● 6 | |
● Implement minimum training standards (e.g. completing an online training course about finding and using research evidence) and provide ongoing mentorship for staff contributing to the rapid-response program (this includes both those at the Forum and from partner networks or organizations) | ● 4 | |
Program resources | ● Seek external (but not user-pay) and long-term funding (e.g. from a Partnerships for Health System Improvement grant from the Canadian Institutes of Health Research) that will allow for both the delivery and ongoing evaluation of the program | ● 5 (if successful) |
● Prioritize some requests over others in times when demand exceeds available resources, which could be accomplished through one or more of the following approaches: | ● 6 | |
○ completing requests from those who have not recently accessed the program; | ||
○ requesting a resubmission at a later date for topics that are deemed less urgent (either by the requestors themselves, by the steering committee or both); and/or | ||
○ engaging the program steering committee to help decide which requests should be prioritized (e.g. through a voting or ranking process over email) | ||
Collaboration | ● Engage trusted national, provincial and territorial partner networks or organizations (where possible and necessary) to | ● 8 |
○ identify whether a synthesis has already been completed on the topic (e.g. by establishing a listserv that can be used to efficiently contact all partners when a request is received) and | ||
○ collaborate with the Forum to conduct syntheses (or build on existing products identified) to ensure relevance to particular provincial and territorial contexts |
Timeline | What can be done | What cannot be done |
---|---|---|
Three business days | ● Identify systematic reviews and economic evaluations relevant to health systems from key databases (e.g. Health Systems Evidence) | ● Identify primary research studies (e.g. published studies and unpublished reports) |
● Provide summary tables that outline | ● Conduct quality appraisals for reviews that are not available through Health Systems Evidence | |
○ key findings from relevant systematic reviews, | ||
○ quality appraisals of systematic reviews (for reviews that are available through Health Systems Evidence) and | ● Prepare a detailed summary of key findings | |
○ countries in which studies included in systematic reviews were conducted (for reviews that are available in Health Systems Evidence) | ● Engage experts to conduct a merit review of the findings to ensure scientific rigour and system relevance | |
● Conduct jurisdictional scans of what is being done nationally and internationally | ||
● Conduct a full systematic review | ||
Ten business days | ● Identify systematic reviews and economic evaluations relevant to health systems from key databases (e.g. Health Systems Evidence) | ● Identify grey literature (e.g. unpublished reports) that is not already contained in key databases (e.g. Health Systems Evidence) |
● Identify relevant primary research studies when limited evidence is available from systematic reviews | ● Prepare a detailed summary of key findings | |
● Provide summary tables that outline | ● Incorporate feedback from experts engaged in the merit-review process within the 10-day timeline (but a final summary that incorporates reviewers’ feedback will be sent within another five business days) | |
○ key findings from relevant systematic reviews, | ||
○ quality appraisals of systematic reviews (for reviews that are available through Health Systems Evidence) and | ||
○ countries in which studies included in systematic reviews were conducted (for reviews that are available in Health Systems Evidence) | ||
● Prepare a brief summary of the key findings from systematic reviews (and primary research studies where relevant) | ● Conduct jurisdictional scans of what is being done nationally and internationally | |
● Engage experts to conduct a merit review of the brief summary to ensure scientific rigour and system relevance (a draft summary will be submitted to the requester before merit reviewer feedback is received and then a final summary that incorporates reviewers’ feedback will be submitted within another five business days) | ● Conduct a full systematic review | |
30 business days | ● Identify systematic reviews and economic evaluations relevant to health systems from key databases (e.g. Health Systems Evidence) | ● Conduct a full systematic review |
● Identify relevant primary research studies when limited evidence is available from systematic reviews | ||
● Conduct jurisdictional scans of what is being done nationally and internationally through targeted searches of databases for published literature, and websites of relevant jurisdictions and stakeholders for grey literature that is not already contained in key databases (e.g. Health Systems Evidence) | ||
● Consult with experts with knowledge of the topic to identify additional relevant research evidence (contingent on locating relevant experts) | ||
● Provide summary tables that outline | ||
○ key findings from relevant systematic reviews | ||
○ quality appraisals of systematic reviews (for reviews that are available through Health Systems Evidence) and | ||
○ countries in which studies included in systematic reviews were conducted (for reviews that are available in Health Systems Evidence) | ||
● Prepare a detailed summary of the key findings from systematic reviews (and primary research studies where relevant) | ||
● Engage experts to conduct a merit review of the detailed summary to ensure scientific rigour and system relevance and incorporate reviewers’ feedback in the final report within the 30-business-day timeline |
Where to measure success | Possible approaches to measuring whether we have been successful |
---|---|
Program organization | ● Brief survey asking the requestor to evaluate key features of the rapid-response program (administered after receipt of rapid synthesis) |
● Short qualitative interviews with requestors (conducted approximately 6 months following receipt of rapid synthesis) | |
Final product (i.e. did the rapid synthesis meet the requestor’s needs?) | ● Brief survey asking the requestor to evaluate key features of the rapid synthesis |
● Short qualitative interviews with requestors asking questions about what was most and least helpful about the synthesis (6 months following receipt of rapid synthesis) | |
Influence on behavioural intention to find and use research evidence | ● Assessment of behavioural intention (and the attitudes, social norms and perceived behavioural control that influence whether such intention translates into action) after receiving the rapid synthesis and 6 months later (assessed in survey administered after receipt of rapid synthesis and again during the short qualitative interviews 6 months later) |
Whether and how the synthesis was used (i.e. did it support evidence-informed decision-making?) | ● Short qualitative interviews with requestors about how they used the rapid synthesis (conducted 6 months following receipt of rapid synthesis) |
Implementation considerations
Levels | Potential barriers |
---|---|
Individual | ● No barriers identified at the citizen or patient level for any of the program features |
Service provider |
Program feature 1 - organizing a rapid-response program
|
● Existing providers of rapid-response programs may overlap to some extent with the scope of a new program focused on producing rapid syntheses for health system decision-makers about problems, options and/or implementation considerations related to a specific health system challenge | |
Program feature 2 - deciding what can be done in what timelines
| |
● None identified | |
Program feature 3 - defining success and measuring it
| |
● None identified | |
Organization |
Program feature 1 - organizing a rapid-response program
|
● Organizations may still lack the skills, structures, processes and a culture to promote and use research findings in decision-making | |
Program feature 2 - deciding what can be done in what timelines
| |
● None identified | |
Program feature 3 - defining success and measuring it
| |
● None identified | |
System |
Program feature 1 - organizing a rapid-response program
|
● Decision-makers may be reluctant to rely on a rapid-response program established in another jurisdiction | |
● Decision-makers may be reluctant to make requests to an external rapid-response program for politically sensitive issues or to publicly disclose that they made a request | |
● Decision-makers may face difficulties in developing a shared vision for a rapid-response program given their constraints and competing priorities | |
Program feature 2 - deciding what can be done in what timelines
| |
● Decision-makers may not be inclined to make requests to an external rapid-response program for very short timeframes (e.g. 3 days) given that this may already be done internally on a routine basis | |
Program feature 3 - defining success and measuring it
| |
● Decision-makers may be reluctant to fully disclose the impact of the rapid-response program, especially on politically sensitive issues |