Background
Methods
Review question(s)
Literature search
Eligibility criteria
Quality appraisal
Data extraction and analysis
Results and discussion
Process principles, content, and context principles
Process principles
1. Prioritization | The guidance fits in properly and is consistent with current health system priority areas within all applicable system levels and sectors by targeting a priority topic/jurisdiction/population [4, 5, 53‐59]. The guidance addresses these specific local priority areas with a clearly documented/demonstrated need and also informs policy decisions on how to further prioritize across competing areas [4, 53, 54, 59, 60]. The origin of the mandate to develop the guidance [59] is also reported (for example, guidance that is mandated by a top official like the Minister of Health is considered to be of high priority). |
2. Relevance | The guidance recommendations should be relevant to, appropriate to, and valid for the health system issue being addressed and relevant to the target population [5, 55‐57, 60‐63]. The recommendations are relevant to the setting within which the guidance will operate, the institutional needs of that system/sub-system, as well as local, national, and potentially global needs [54, 64‐68]. |
3. Timeliness | The recommendations are available in a timely manner in relation to when the policy decisions are made or timely in relation to the health system issue being addressed [55, 56, 61, 69, 70]. The guidance is timely and usable by the broad range of health systems stakeholders [53, 68, 71] since some policy decisions are sometimes made within crucial corresponding time frames or as windows of opportunity open and close [4, 62]. |
4. Scope | The guidance is comprehensive and covers all relevant/appropriate (direct and indirect) health system levels, sub-systems, and sectors [5, 8, 56, 61, 63, 71, 72]. This also includes the various relevant sub-systems/components (hospitals, regional health authorities, and public health units etc.) within the health system. Identifying the scope is important because these various components are interlinked, interdependent, and interact at various interfaces for overall health system performance [53, 55, 58‐60, 71‐74]. |
5. Transparency | Systematic, replicable, and transparent processes are applied in developing and reporting the guidance [52, 61, 75, 76]. These processes are systematic and transparent enough for the methods of development/reporting of the guidance to be reproducible [59, 74]. In order to paint a clear picture to knowledge users and target populations, sufficient details on these processes are provided [5, 54, 63, 65, 67]. |
6. Evidence- based | The best available research evidence informs the recommendations [4, 5, 54, 55, 57, 58, 60, 61, 63, 69‐71, 73‐77]. The type(s) of evidence that was used to generate the guidance is/are stated, and this can range from well-established scientific methodologies or it can also be non-experimental (for example, colloquial evidence, anecdotal evidence, or preliminary models) [8, 62, 63, 70, 75, 78]. The evidence is context sensitive enough to resonate with local realities [4, 5, 54, 56, 57, 60, 61, 63, 69‐71, 74, 75]. |
7. Stakeholder involvement | Alternative views on the policy issue and the complementary expertise of a multidisciplinary group of relevant stakeholders are considered in the development of the guidance [4, 5, 8, 53‐55, 58‐64, 67‐69, 71, 75‐77, 79]. Guidance developers, those involved in the implementation and evaluation of the guidance, and those who will be affected by the guidance recommendations are involved in the development process [54, 56, 57, 66, 71, 74, 80]. |
8. Ethical | The recommendations reflect considerations of an ethical lens [4, 8, 54, 56, 76] and align with applicable ethical principles and values (for example, equity, equality, human rights, liberty, efficiency, autonomy, dignity, beneficence) [8, 55, 63, 65, 74, 80, 81]. The guidance adequately promotes fairness and equality in terms of age, ability, culture, gender, socioeconomic status, religion, occupation, language, ethnicity, race, or sexual orientation among the target population [60, 61, 66, 69, 79, 82]. |
9. Outcomes | The guidance describes all the anticipated effects/outcomes as well as the appropriate indicators that can be used to measure the effects/outcomes [8, 54, 57, 58, 66, 67, 74, 75]. Adequate rationale regarding the choice of the outcomes and the indicators selected is provided [8, 58]. Considering potential uncertainties that may result, alternative outcomes and outcome indicators are also identified. Performance thresholds, targets, and standards that are considered acceptable are also identified [8, 58, 65‐67, 76]. |
10. Competing interests | A declaration of competing interests (for example, financial, academic, professional) by the guidance developers, whether direct or indirect, is/are made in advance [57, 74, 80]. The author’s positions, roles, and affiliations are clearly stated [65]. Any reported or identified conflicts of interest are managed, with a description of the approaches used to curb any influence clearly documented [57, 80]. It is also clear that the views of any funding body involved have not influenced the development process of the guidance [65]. |
11. Presentation | The recommendations are clear, succinct, unambiguous, and presented in a readable and consistent format [57, 60, 76, 81], with key recommendations easily identifiable [53, 57, 58, 66]. The guidance is presented in a manner that is uniform, user-friendly, and easy to navigate [4, 53, 61, 64, 68, 80]. It contains an executive summary, full text, a complete list of relevant references, a glossary of terms, and full meaning of abbreviations and contact information of authors. Words or phrases denote an aspirational rather than a mandatory intent [80]. |
Content
12. Problem definition | The health systems challenge (for example, financial, governance, or delivery arrangements) and its causes are clearly articulated (including any links/integration with other policy problems on the government’s agenda) [5, 75, 83]. The nature, causes, magnitude, frequency, and intensity of the problem, the populations, and jurisdictions that are affected are clearly described [57, 60, 62, 64, 66, 71]. Appropriate rationale exists to justify that either new guidance is needed or existing guidance of acceptable quality can be adapted and used to address the problem [4, 56]. |
13. Operationalization | The recommended “solutions” are operationalized sufficiently with the conceptualization, operational guidance, and the mode of delivery of the options clearly stated [8, 54, 60]. For example, the guidance provides instructional support for their successful operation and staff training that corresponds with the guidance expectations. Training recommendations could be in the form of a course, a workshop, accompanying manuals, or consultancy services that staff can refer to during the implementation phase in order to standardize practice [8, 54, 58, 60, 61, 63, 73, 74]. If technical assistance (research institutes, consulting firms, NGOs) is required, this is identified and documented [8]. |
14. Costs | The guidance clearly documents a tentative budget required to implement the guidance recommendations [8, 64, 65, 74, 77, 84]. The potential financial costs (including downstream costs) of the operation are stated so that decision makers can assess the feasibility of the guidance implementation and evaluate whether the cost of implementing the guidance will be worth its potential impacts [5, 64, 73, 74]. |
15. Resources | The inputs and resources required to implement the recommendations are clearly defined and they have to be proportionate to the health system problem that is being addressed [55, 57, 61, 71, 73, 79]. Some of these resources could be time, infrastructure, administrative capacity, information, equipment, supplies, healthcare professionals, training etc. [55, 56, 58, 59, 61, 69, 76]. The guidance provides a description of the amount, frequency, and duration of the inputs and resources required [54, 57, 71]. |
16. Effectiveness | The guidance reports whether the anticipated goals and objectives have been achieved elsewhere or in a similar setting/condition, either through evidence from evaluation studies done at other sites (if available) or from expert opinion [54‐56, 69]. In describing this effectiveness, the guidance makes projections on how and why the objectives and goals will be achieved in the current setting [64, 69, 75]. |
17. Cost-effectiveness | The recommendations are attentive to value for money considerations [57, 63, 84]. Sound local or applicable evidence (wherever available) on the cost-effectiveness of the guidance recommendations are provided [54, 57, 60, 63, 79]. These traditionally report costs, direct and indirect program inputs/resources, and outcomes to guide health policy decisions and provide benchmark(s) or threshold(s) that the health system is willing to accept or support in relation to other competing health system priorities [54, 60, 73]. |
18. Benefits/harms | Description of the potential unintended consequences (positive and negative) of the guidance is provided or an assessment/judgment of the potential benefits/harms are made [60, 67, 75]. Descriptions of the populations or institutions that may experience significant impacts are identified [57, 60, 65]. |
19. Dissemination plan | Strategies for communicating the guidance are included with a clear dissemination framework, the mode of delivery, and the integrity of the avenue used for dissemination been properly reported [54, 65, 81]. The proposed strategies for disseminating the guidance are tailored to the relevant audiences (for example, a formal written report, user-friendly summary, oral presentation, poster, press release, booklet, workbook, films, pocket card) [57, 74]. |
20. Process evaluation | This involves recommendations for evaluating the structure and process of implementation as well as corresponding challenges [61, 68, 80]. This evaluation examines the extent to which the guidance recommendations were implemented as planned, and also provides a way to monitor the process and make adjustments and improvements to implementation strategies [74, 79]. It documents the inputs, services, and activities that were implemented, and can identify potential strengths, weaknesses, opportunities, and threats to the implementation process [62, 79]. |
21. Outcomes/impact evaluation | An assessment of the outcome/impact of the guidance is recommended to determine whether the course of action was a success or failure. There are recommendations on measuring the results, or outcomes of the guidance in a way that determines whether the changes observed in relation to the health system challenge being addressed can be attributed to the guidance [58, 61, 80, 82]. There are also recommendations for an impact evaluation to look at the short- and long-term deeper primary and secondary changes that resulted from the guidance [57, 69, 74, 77]. |
22. Updating | Recommendations for periodic updates are made and the procedure to update the guidance is provided with explicit timelines on anticipated review [57, 68], appropriate expiration date of the guidance, and an explanation of the rational for the proposed time frames [80]. Setting time frames for periodic updates ensures that guidance producers revisit the recommendations and respond accordingly to potential health system changes and emerging challenges. Also, the recommendations should be current, and the evidence (for example, systematic reviews) on which they are based is considered recent and up-to-date [76, 80]. |
Context principles
23. Feasibility | The guidance recommendations are realistic and the actions are pragmatic [4, 5, 53, 55, 57, 59, 61, 63, 68, 70, 81, 84]. The guidance describes facilitators and barriers for implementation [58, 69]. It is clearly demonstrated that the implementation of the guidance is feasible within the proposed practice environment, and the recommendations match local capacities and expectations [4, 8, 53, 61, 63, 69, 71, 80]. |
24. Affordability | The guidance recommendations are affordable within the financial structure and budgetary allocations of the health system [53, 54, 56, 69]. Potential sources of local government funding and donor organizations are identified. For policy issues in which there may be several sources of funding, the guidance also considers the level of coordination among the donors and between the donors and the local government [56, 58, 59, 61, 69]. |
25. Flexibility | The guidance is flexible and adaptable to the expertise of the user and the varying local conditions. It acknowledges the importance of professional judgment and discretion and provides recommendations that users can adapt in accordance with their own individual circumstances and needs [57, 71, 80]. The recommendations steer away from the adoption of rigid approaches so as not to inappropriately or unnecessarily limit those in charge of applying them [57, 71]. |
26. Socio- culturally acceptable | Considering the diversity of values in many regions, the recommendations are robust under societal and cultural scrutiny by adopting a socio-cultural perspective [54, 57, 61‐63, 69, 73]. It recognizes socio-cultural expectations and provides an understanding of the role that socio-cultural factors will play in the success of the guidance recommendations [54, 55, 69, 71, 73]. |
27. Politically sound | The political acceptability of the recommendations is considered in order to assess if they align with political interests/commitments [54, 57, 63, 64, 70, 75, 79, 81, 82]. Implementation of guidance can stir swings in the national mood, lead to changes in the balance of organized forces, such as interest groups, or influence outcome of events within the government, for instance an election [85]. Therefore, options proposed that are in sync with the political climate may garner adequate support from top policy/government officials [58, 73]. |
28. External factors | Determinants of health system performance that lie outside the formal architecture of the health system but will influence the performance of its functions are considered; for example, judicial system, social system, recession, corruption, state of the economy [48, 54, 73, 79]. These are non-health system factors originating from other local institutional organizations that impact on the usual operations of the health system [54, 59, 64, 80]. |
29. Generalizability | The recommendations are transferable to other settings with similar health system features; for example other countries or regions [59, 65, 66, 79]. Judgments are made about the applicability of the recommendations beyond its original context (setting or population) to ensure that contexts with similar institutional, socioeconomic, and political demographics facing an identical health system challenges can adapt and use the guidance [55, 60, 65, 74]. |
30. Sustainability | The guidance provides an indication of the sustainability of the effects of the recommendations to show that long-term outcomes can be continuously achieved and maintained at an acceptable level [56, 61, 64, 69]. Due to constantly evolving health system issues, looming budget cuts, fluctuating resources, rising costs of new technologies, an ageing population, shifting burdens of diseases etc., it is crucial to develop recommendations that will stand the test of time [56, 60, 69]. |
Synthesis across domains
-
➢ Uptake of guidance can be enhanced if it is addressing a priority area for which evidence-based reports of effectiveness and cost-effectiveness exist.
-
➢ Consultations with appropriate stakeholders are also crucial for feasibility of implementation and sustainability of the HSG because sometimes, the individuals tasked with implementing guidance recommendations may not be committed to them and this may influence adherence. Their input into the HSG process and support for the recommendations may alleviate this concern.
-
➢ HSG outcome(s) chosen will influence the way the problem is defined and provides information that will be useful for evaluating the HSG.
-
➢ Including an ethical viewpoint will also impact how the problem is defined and will influence cost-effectiveness thresholds.
-
➢ Systematic and transparent processes may provide an impetus for donor involvement (affordability).
-
➢ Information on affordability can determine whether the health system issues can be addressed in a timely manner and inform judgments on how to evaluate the process.
-
➢ The HSG should be flexible enough to accommodate constantly evolving evidence and changing health system priorities.
-
➢ The guidance should provide socio-culturally appropriate solutions that are relevant to the applicable levels/sectors of the health system.
-
➢ HSG that is in line with ethical principles, addresses priority issues, and is timely will be appealing to politicians.
-
➢ The external alignment of guidance may affect the effectiveness and impact the operational considerations.
-
➢ The outcomes selected, the system level/sectors involved, or the ethical values in place may expose the external factors that may be pertinent.
-
➢ Some determinants of generalizability of the guidance are the stakeholders involved, the evidence used, transparency of the process, ethical lens considered, and outcome/indicators selected