Background
Overview of the locally-driven collaborative projects (LDCP) priority setting process
Phase 1: Survey to prioritize subject areas | ||
Health units submitted a survey to the LDCP program identifying five subject areas of greatest interest. Interest was calculated by summing the total number of health units that selected each subject area. The top seven subject areas with greatest interest moved forward to phase 2. | ||
Phase 2: Workshop to prioritize 1 research question in each subject area | ||
Step 1 | Formulating potential research questions • Participants provided with an opportunity to formulate research questions of interest to their health unit within the prioritized subject area and topics • Participants shared questions generated by each group member | Number of research questions at the end of the step that move forward for further prioritization |
>20 | ||
Step 2 | Narrowing down potential research questions • Using a consensus-based decision-making process, research questions were refined and reduced • Six principles guided discussion and decision-making: duplications, already done, misalignment, out-of-scope, too big, too early • Approximately 12 research questions move forward to Step 2.3 – applying criteria of ‘interest’ and ‘impact’ | ~12 |
Step 3 | Applying criteria of ‘interest’ and ‘impact’ • Participants were asked to consider criteria of interest and vote for three research questions that best met this criteria • Top five questions with potential for greatest impact were ranked and moved forward to Step 2.4 – considering criteria of ‘balance’ | 5 |
Step 4 | Considering criteria of ‘balance’ • Participants engaged in a discussion about criteria of balance as it relates to the three remaining potential research questions • Comments captured on flip-chart paper • All five questions move forward to Step 2.5 – applying criteria of interest, impact, and balance to identify top research question | 5 |
Step 5 | Applying criteria of interest, impact and balance to identify top research question • Participants were asked to begin by considering criteria of interest and vote for one research question that their organization would be most interested in • Participants were next asked to consider criteria of impact and vote for one research question that relates to the most important public health issue • Finally, participants asked to consider the criteria of balance and vote for one research question that will have the most significant benefit on the public health system as a whole • The research question with the most votes moved forward to be collaboratively developed into an LDCP | 1 |
Phase 1 – Survey
Phase 2 – Workshop
Criteria | Definition |
---|---|
Interest | Alignment with the priorities and direction of health units and the public health system |
Impact | Ability to generate knowledge and evidence to support health units’ ability to meet the Ontario Public Health Standards and influence change in the public health system |
Balance | Address the priorities of health units from different regions and of various sizes with the goal of meeting the demands of the majority and the needs of the minority |
Methods
Theme | Description As outlined in the Checklist [11] | Evaluation questions Adapted from the Checklist [11] | Indicators Example of indicators used to inform analysis of LDCP research priority setting process | Data sources |
---|---|---|---|---|
Context | Articulating the contextual factors that underpin the process | 1. Will the established goals, underlying values and principles continue to be relevant the next time the program facilitates priority setting? 2. Are there changes to the number of resources available for the next priority setting cycle? | i) Extent to which findings from original stakeholder engagement processes remain relevant for next research priority setting process ii) Availability of ongoing financial and human resources | i) Consultation with senior leadership within PHO ii) Program documents |
Inclusiveness | Deciding who should be involved in setting research priorities | 3. Did appropriate stakeholders participate in the most recent priority setting cycle, and was there balanced representation? | i) Number and representativeness of health units who submit phase 1 survey ii) Percentage of workshop participants who are front-line staff, managers, and senior decision-makers iii) Percentage of workshop participants who agree or strongly agree that they had opportunity to express opinions and ideas | i) Health unit demographic characteristics obtained from Phase 1 survey ii) Workshop registration list iii) Workshop evaluation iv) Informal feedback from program participants |
Information gathering | Choosing what information should be gathered to inform the process | 4. Was the most recent priority setting exercise appropriately informed? 5. Did the provided information sources support decision making? | i) Types of technical information provided to workshop participants ii) Perceived usefulness of technical material provided to workshop participants to aid decision-making | i) Workshop facilitation materials ii) Informal feedback |
Planning for implementation | Establishing plans for translating research priorities into projects | 6. In previous cycles, were there challenges to translating the research priorities into research? | i) Challenges reported relative to implementing LDCP project proposals ii) Quality and amount of support available from LDCP program staff | i) Interim and final progress reports ii) Consultation with program staff |
Criteria | Selecting relevant criteria to focus discussion | 7. In the most recent priority setting cycle, were the criteria effective for decision making, and will the criteria continue to be relevant for the next cycle? | i) Percentage of workshop participants who agree or strongly agree that the process helped them successfully apply the criteria ii) Alignment of criteria with overarching goals of research priority setting process | i) Workshop evaluation ii) Consultation with program advisory committee |
Methods for deciding on priorities | Choosing a method for deciding on priorities | 8. In the most recent priority setting cycle, were the methods for deciding on priorities appropriate and effective for decision making? | i) Percentage of workshop participants who agree or strongly agree that the process used to select priorities helped to build consensus ii) Percentage of workshop participants who agree or strongly agree that the workshop was an effective way to help health units discuss and prioritize topics for research and evaluation projects | i) Workshop evaluation |
Use of a comprehensive approach | Assessing whether a comprehensive approach is necessary or if a tailored process and methods are required | 9. Are there elements in comprehensive approaches and priority setting methods? Specifically, the Listening Model, COHRED, Child Health and Nutrition Research Initiative, Essential National Health Research, Combined approach matrix and Delphi technique, which are transferable to the LDCP priority setting process | i) Extent to which elements of established comprehensive approaches and methods can be transferred to or would strengthen LDCP priority setting processes | i) Review of comprehensive approaches and methods described in the Checklist |
Transparency | Communicating the approach that was used to set priorities | 10. Did all stakeholders receive information about the process and outcomes of the most recent priority setting process? | i) Types of communication strategies used to share information about the LDCP priority setting process with stakeholders ii) Timeliness of communications about the priority setting process | i) Program document review |
Evaluation | Defining when and how evaluation of process and outcome will occur | 11. Are further evaluation activities required to assess the delivery and outcomes of the priority setting process? | i) Perceived usefulness of current evaluation activities for informing quality improvements to the LDCP priority setting process | i) Consultation with program advisory committee |