Background
Methods
Qualitative analysis of the results of the systematic review and interviews
Identification of guiding elements
GIN-McMaster GDC Topic | Elements identified from the systematic survey [7] | Elements identified from qualitative study interviews [8] | Additional elements identified during the validation process |
---|---|---|---|
1. Organisation, budget, planning and training | • Shortened timeframe for this work | • Define the amount of time allotted for the RG • Consider resources (time) needed for conducting the RG or standard systematic review • Consider resources (financial) needed for conducting the systematic review • Define the process for when resources (time and financial) are limited | • Have standard operating procedures specific for RGs in place and ready for use • Prepare training material appropriate for rapid training (e.g. online modules readily available for rapid viewing) • Have templates for RG ready for use • Identify peer reviewers early • Plan early for panel meeting • Is there a requirement or value to identify key data needs to support decision? The data needed to reach a decision may frame the project and question. For a chemical or other spill, the hazard or health effects are of great concern, but the first question may be to measure the extent of exposure (which will define some recommendations of the RG) |
2. Priority-setting | • No distinct elements identified for rapid guidelines | • Address whether or not there is a need for an interim/rapid guidance • Define the rationale motivating the rapid as opposed to the standard development (e.g. new evidence about efficacy/cost-effectiveness/safety, emergent/dangerous situations, etc.) | • Identify and assess published guidelines addressing the same topic (might help in prioritising issues not covered by those guidelines) |
3. Guideline group membership | • No distinct elements identified for rapid guidelines | • Consider involving a health economist • Involve representative from the clearance entity to expedite review and approval of final document | • Consider involving content experts with prior experience with guideline development methodology • Consider involving technical experts (systematic reviewers and methodologists) with prior experience with rapid reviews and RG methodologies; involve them early on • Organisation may develop a database of experts by area of expertise |
4. Establishing guideline group process | • No distinct elements identified for RGs | • Consider virtual meetings • Use a mix of face-to-face and virtual meetings | • None |
5. Identifying target audience and topic selection | • No distinct elements identified for RGs | • None | • Alert the target audience to the upcoming RG to increase engagement in development, review and uptake of the RG |
6. Consumer and stakeholder involvement | • No distinct elements identified for RGs | • No elements identified for RGs | • Chemical spill importance to stakeholders |
7. Conflict of interest considerations | • No distinct elements identified for RGs | • Exclusion from the guideline development group of participants reporting conflicts of interest | • Consider alleviating conflict of interest management-related restrictions when recruiting participants in a short timeframe is challenging |
8. Question generation | • Consider a limited scope | • Consider a narrow scope | • None |
9. Considering importance of outcomes and interventions, values, preferences and utilities | • Consider reducing the number of outcomes to a few critical ones | • Elicit values and preferences from qualitative literature | • Rely primarily on input of experts and stakeholders |
10. Deciding what evidence to include and searching for evidence | • No distinct elements identified for RGs | • Define a systematic review process for when evidence is limited • Address exclusion criteria (e.g. grey literature, non-English language, etc.) • Base RGs on evidence from systematic reviews | • Identify and assess published systematic reviews addressing the same topic (might help in prioritising reviews for questions not covered by those systematic reviews) • Consider conducting rapid scoping reviews in preparation for the rapid review • Consider conducting rapid reviews |
11. Summarising evidence and considering additional information | • No distinct elements identified for RGs | • None | • Rely on evidence solicited from experts to collect ‘additional information’ and identify relevant primary studies |
12. Judging quality, strength or certainty of a body of evidence | • No distinct elements identified for RGs | • None | • None |
13. Developing recommendations and determining their strength | • No distinct elements identified for RGs | • None | • Consider online meeting and pre-voting |
14. Wording of recommendations and of considerations about implementation, feasibility and equity | • No distinct elements identified for RGs | • Finalise wording of recommendations during the panel meeting(s) | • None |
15. Reporting and peer review | • Determine whether an expedited review process can be used for RGs | • Describe the review process, if it differs from PGs • Plan for a shorter review time | • Use pre-drafted templates for the final report, as well as automated reports produced by software like the GDT |
16. Dissemination and implementation | • No specific elements identified for RGs | • Address potential obstacles for implementation | • None |
17. Evaluation and use | • No specific elements identified for RGs | No specific elements identified for RGs | • None |
18. Updating | • Define a date for when the RG will be conducted as a standard PG | • If providing interim guidance, define when the RG or full PG will be finished or conducted | • Emergent or dangerous situations may have a ‘staged release’ of RGs in the following order: (1) first action to protect public health, respond to crisis, spill that is heavily weighted to protect against worst-case scenario; (2) the second action or recommendation based on planned update based on new or additional information may recommend a change in values leading to less conservative recommendations |
Validation of elements
Results
Guideline Development Checklist for standard guidelines | Process modification | |
---|---|---|
Topic | Description | Principles for RGs |
1. Organisation, Budget, Planning and Training | Organisation, budget, planning and training involves laying out a general but detailed plan describing what is feasible, how it will be achieved and what resources are required to produce and use the guideline. The plan should refer to a specific time period, and be expressed in formal, measurable terms. | 1. Define the amount of time available for development of the RG and the elements from the GDC that should be followed. 2. Develop RG-related standard operating procedures; develop templates for RGs; identify peer reviewers early on; and plan panel meetings as early as possible. |
2. Priority-setting | Priority-setting is the identification, balancing and ranking of priorities by stakeholders. It ensures that resources and attention are devoted to those general areas (e.g. chronic obstructive pulmonary disease, diabetes, cardiovascular disease, cancer, prevention) where healthcare recommendations will provide the greatest benefit to the population, a jurisdiction or a country. A priority-setting approach needs to contribute to future plans while responding to existing potentially difficult circumstances (citations provided in checklist extension). | 3. Define the rationale motivating the RG (e.g. new evidence about efficacy/cost-effectiveness/safety, emergent/dangerous situations, etc.). 4. Address whether there is a need for temporary and/or emergency guidance. |
3. Guideline Group Membership | Guideline group membership defines who is involved, in what capacity, and how the members are selected for the guideline development and at other steps of the guideline enterprise. | 5. Involve relevant individuals in the guideline oversight committee. 6. Develop a database of topic-specific experts by area of expertise to consult when establishing the guideline oversight committee. |
4. Establishing Guideline Group Processes | Establishing guideline group processes defines the steps to be followed, how those involved will interact, and how decisions will be made. | 7. When the timelines are short, greater emphasis should be placed on using virtual meetings (alone or along with face-to-face meetings). |
5. Identifying Target Audience and Topic Selection | Identifying target audience involves describing the potential users or consumers of the guideline. Topic selection defines the topics to be covered in the guideline (e.g. diagnosis of chronic obstructive pulmonary disease). | 8. Alert the target audience to the RG before release. |
6. Consumer and Stakeholder Involvement | Consumer and stakeholder involvement describes how relevant people or groups who are not necessarily members of the panel but affected by the guideline, e.g. as target audience or users, will be engaged. | None |
7. Conflict of Interest Considerations | Conflict of interest considerations focus on defining and managing potential divergence between an individual’s interests and their professional obligations that could lead to questioning of whether the actions or decisions are motivated by gain such as financial, academic advancement, clinical revenue streams or community standing. Financial or intellectual or other relationships that may impact an individual or organisation’s ability to approach a scientific question with an open mind are included. | 9. RG guideline development panels may need a rapid process for implementing conflict of interest policies. |
8. PICO Question Generation | PICO question generation focuses on defining key questions the recommendations should address, including the detailed population, intervention (including diagnostic tests and strategies) and outcomes that will be relevant for decision-making (e.g. should test A be used, or should treatments B, C, D or E be used in chronic obstructive pulmonary disease). | 10. RGs should address a limited number of questions. |
9. Considering Importance of Outcomes and Interventions, Values, Preferences and Utilities | Considering importance of outcomes and interventions, values, preferences and utilities includes integrating in the process of developing the guidelines, how those affected by its recommendations assess the possible consequences. These include patient and carer knowledge, attitudes, expectations, moral and ethical values, and beliefs; patient goals for life and health; prior experience with the intervention and the condition; symptom experience (for example, breathlessness, pain, dyspnoea, weight loss); preferences for and importance of desirable and undesirable outcomes; perceived impact of the condition or interventions on quality of life, well-being or satisfaction and interactions between the work of implementing the intervention, the intervention itself, and other contexts the patient may be experiencing; preferences for alternative courses of action; and preferences relating to communication content and styles, information and involvement in decision-making and care. This can be related to what in the economic literature is considered ‘utilities’. An intervention itself can be considered a consequence of a recommendation (e.g. the burden of taking a medication or undergoing surgery) and a level of importance or value is associated with that. | 11. Outcome prioritisation process for each PICO should be brief. 12. Information on patients’ values and preferences can be informed by multiple methods, such as qualitative literature or patient advocacy groups. |
10. Deciding what Evidence to Include and Searching for Evidence | Deciding what evidence to include and searching for evidence focuses on laying out inclusion and exclusion criteria based on types of evidence (e.g. rigorous research, informally collected), study designs, characteristics of the population, interventions and comparators, and deciding how the evidence will be identified and obtained. It also includes but is not limited to evidence about values and preferences, local data and resources. | 13. Consider the resources (both time and financial) needed and available for when defining the process for conducting the systematic review. Scoping or rapid reviews may inform eligibility criteria and prioritisation. |
11. Summarising Evidence and Considering Additional Information | Summarising evidence and considering additional information focuses on presenting evidence in a synthetic format (e.g. tables or brief narratives) to facilitate the development and understanding of recommendations. It also involves identifying and considering additional information relevant to the question under consideration. | 14. Relevant primary studies and evidence solicited from experts may be used to inform ‘additional information’ in the evidence to decision table. |
12. Judging Quality, Strength or Certainty of a Body of Evidence | Judging quality, strength or certainty of a body of evidence includes assessing the confidence one can place in the obtained evidence by transparently evaluating the obtained research (individual studies and across studies) and other evidence applying structured approaches. This may include, but is not limited to, evidence about baseline risk or burden of disease, the values and preferences, resource use (cost), estimates of effects and diagnostic test accuracy. | None |
13. Developing Recommendations and Determining their Strength | Developing recommendations focuses on integrating the factors that influence a recommendation using a structured analytic framework, and a transparent and systematic process. Determining the strength of the recommendations refers to judgments about how confident a guideline panel is that the implementation of a recommendation exerts more desirable than undesirable consequences. | 15. Use pre-meeting voting and virtual meetings to expedite the decision-making process. |
14. Wording of Recommendations and of Considerations of Implementation, Feasibility and Equity | Wording of recommendations refers to choosing syntax and formulations that facilitate understanding and implementation of the recommendations. Such wording is connected to considerations of implementation, feasibility and equity, which refer to the guideline panel’s considerations about how the recommendation will be used and what impact it may have on the factors described. | 16. Finalise the wording of the final recommendations during the panel meeting(s). |
15. Reporting and Peer Review | Reporting refers to how a guideline will be made public (e.g. print, online). Peer review refers to how the guidelines document will be reviewed and how it can be assessed (e.g. for errors), both internally and externally, prior to its publication by stakeholders who were not members of the guideline development group. | 17. Define and transparently record the process used when evidence is determined to be limited. 18. Expedited options for internal and external review of the RG should be explored, and if deemed possible, the process should be outlined in the RG. |
16. Dissemination and Implementation | Dissemination and implementation focuses on strategies to make relevant groups aware of the guidelines and to enhance their uptake (e.g. publications and tools such as mobile applications). | 19. RG implementation strategy should reflect the scope of the PICO. 20. RGs should outline and address any potential obstacles to implementation. |
17. Evaluation and Use | Evaluation and use refers to formal and informal strategies that allow judgments about evaluation of the guidelines as a process and product; evaluation of the use and/or uptake; and evaluation of impact and whether or not the guideline leads to improvement in patient or population health or other consequences. | None |
18. Updating | Updating refers to how and when a guideline requires revision because of changes in the evidence or other factors that influence recommendations. | 21. When developing an interim guideline, the date for when the RG or full practice guideline will be conducted should be defined. If developing an RG, the date for when the full practice guideline will be conducted should be defined. |