Assessing the health workforce implications of health policy and programming: how a review of grey literature informed the development of a new impact assessment tool

The attainment of national and global health goals can be achieved only with a health workforce that is adequate in number, distribution, skills, motivation and performance. This fact has been formally recognised by the inclusion of a ‘health worker density and distribution’ indicator within the monitoring framework for the sustainable development goals (SDGs) [1]. In 2016, the World Health Assembly (WHA) adopted Workforce 2030: the World Health Organization (WHO) Global Strategy on Human Resources for Health (HRH) [2], which sets out the policy agenda to ensure that the health workforce is adequate, well distributed and fit for purpose and thus allowing for the attainment of the SDGs, while also contributing to economic growth through the creation of qualified employment opportunities in the health sector, as underscored by the UN High Level Commission on Health Employment and Economic Growth [3]. The WHA resolution adopting the Global Strategy on HRH includes three key actions relating to assessing the health workforce implications of health policy and programming:

These provisions in the Global Strategy and the accompanying resolution aim to address challenges that have become apparent over the last decade. For instance, the impact of investments by global health initiatives on HRH is mixed, with results failing to achieve their full potential and unintended but sometimes detrimental effects caused by distortionary incentives and practices [5‐7].

Development initiatives may potentially lead to a number of impacts on the health workforce which occur at various levels of the health system and over different time periods (short, medium and long term), many of which can easily be overlooked. Examples include:

This approach is conceptually similar to the Health in All Policies (HiAP) approach, which expects all public policies to take into account the health implications of decisions, and avoid harmful health impacts. This approach recognises that health is one of a number of competing priorities, but maintains that health considerations must always be taken into account when policy decisions are made [14]. Similarly, the WHA resolution relating to the Global HRH Strategy recognises that HRH may not be a central element of a grant, investment or resolution of an international organisation, but the HRH implications should be assessed regardless. This will help to ensure that the implications (if there are any) are taken into account, even if they are unintended or indirect.

The HiAP approach involves the undertaking of systematic health impact assessments [14]. Impact assessment (IA) is a process that prepares evidence for decision-makers concerning the advantages and disadvantages of initiatives, based on an assessment of their potential outcomes and impacts [15, 16]. Ex ante IAs aim to identify, predict, evaluate and mitigate the consequences of development proposals prior to major decisions and commitments being made [17]. They represent an attempt to provide, in advance, a coherent and transparent analysis of the reasoning that lies behind, and the foreseeable effects of, any proposed policy or intervention, and thus have the potential to improve the quality and/or efficiency of these policies and interventions [18‐22].

There were calls for the mainstreaming of ex ante health workforce impact assessments (HWIAs) as long ago as 2002 [23‐25]. At that time, the specific benefits of such assessments were seen to be as follows: (i) they would draw the attention of decision-makers to the potential consequences of their decisions for HRH, (ii) they would help steer organisational and financing decisions towards minimising negative effects on the workforce and enhancing positive ones, and (iii) they would help to build up documentation on how HRH are affected by new policy initiatives [23].

Fifteen years later, these issues remain highly relevant to national and international development agendas, yet HWIAs are still not common practice and there are no established tools and techniques for them. If the actions contained within the 2016 WHA resolution are to be implemented systematically, there is a need to develop such tools and techniques. A well-designed and well-implemented HWIA is a positive opportunity to make better decisions [16, 26] as well as being a way to fulfil the requirements of the Global HRH Strategy.

This paper presents the findings of an exploratory analysis of IA tools and guidance developed by the health and other sectors. The aim of this analysis was to locate examples of guidance and tools for IA on a wide range of topics, and thus to inform the development of an HWIA tool, a draft of which is appended to this paper. Reflecting the three key actions highlighted in the WHA resolution, the tool’s audience is expected to be technical, managerial and political decision-makers representing development partners, global health initiatives and the WHO Secretariat.

Given the type of materials sought, the search was limited to grey literature. The following search terms were entered into the Google search engine by a researcher based in the UK:

The above search strategy yielded hundreds of thousands of results. For each search term, the first six to eight pages of results were reviewed (there were approximately 10 results per page), excluding any sponsored or promoted web pages, until the results became mostly duplicates or irrelevant. This stage of the review consisted of clicking on the URL and assessing (a) whether or not the link included an IA tool, report or guidance, and if so (b) whether or not these had been developed or used by or on behalf of national or local governments (or their agencies), United Nations (UN) agencies, the European Union (EU), the Organisation for Economic Cooperation and Development (OECD), a national health service, a funder of global health initiatives, or the International Association for Impact Assessment (IAIA). If yes, a note was made of the URL so that the documents could be read in more detail later. A snowballing technique was also used; if one of the located documents cited other relevant guidance, tools or reports, these other documents were also sought and added to the review if they met the above criteria. The search was conducted in English, between August and November 2016.

The initial search yielded 51 examples of guidance, tools and reports relating to IAs. The additional snowballing yielded a further 21 documents, bringing the total to 72. Information was located about a wide range of types of IA, as shown in Table 1. The earliest examples were from the late 1990s, and the latest from 2016.

Out of the 72 documents, 44 were tools or guidance/reports that included a tool. Table 2 shows that most of these originated from the European Union (EU), Organisation for Economic Cooperation and Development (OECD) or the United Kingdom (UK). This is probably partly a function of the search being conducted from the UK, and partly due to the fact that the EU and the OECD were early adopters of IA as part of their approach to developing new policies and programmes [18, 27].

Each of the 72 identified documents was read, and notes made on their contents when these were considered to be potentially relevant to the development of an HWIA tool. Based on these notes and printouts of the tools located during the above search, a draft HWIA tool was developed. The process of extracting information from the identified documents was an inductive one, with themes becoming apparent as the review progressed.

To ensure that the tool was comprehensive and structured, a number of potentially relevant conceptual frameworks was considered as a basis for the review, including a ‘right to health’ framework [20], a social determinants framework [28], and an equity framework [9]. Given the need to make the tool specific to HRH, the WHO health labour market framework (Fig. 1) was considered to be the most relevant, since it will help to ensure that all aspects of HRH and all relevant ‘policy levers’ are considered during an assessment.

The review indicated that there is no single model used widely for IAs [8, 29, 30]. A limited amount of literature explains how to carry out an ex ante IA, and there is a particular dearth in the field of international development and low- and middle-income country contexts [30]. Table 1 shows that literature and documentation about health workforce assessments is lacking: none of the documents located via the review related to a health workforce assessment. Nevertheless, the reviewed documents contain a number of common and relevant themes, which could form the basis of guidance for the design and execution of an ex ante HWIA, and these are described in this section.

The established conceptual framework on which the draft HWIA is based is the WHO health labour market framework (see Fig. 1). Three main objectives of an ex ante HWIA were identified, via adaptation of the objectives of IAs in other fields: (1) to ensure that HRH are explicitly addressed and incorporated into the decision-making process, and to anticipate the likely impacts on HRH, (2) to generate more positive outcomes, by enhancing the positive impacts on HRH, avoiding/minimising/offsetting negative impacts on HRH and/or improving efficiency, and (3) to ensure that the HRH implications are not overlooked, especially when they are not obvious.

A distinction is made in the HWIA objectives between HRH impacts and implications. Impacts can be defined as the effects on the health workforce of the initiative being assessed. Implications come into play when there is a degree of dependency on the health workforce for successful implementation of the initiative, so HRH requirements need to be considered as part of the planning and implementation process. Unlike some of the other topics for which IAs have been carried out, an initiative could have HRH impacts or implications (or both). This indicates that an HWIA tool must be able to capture both, so the draft tool addresses implications in Sections A4 and B1 and impacts in Sections A5 and B2. The concept of implications aligns with the concept of ‘risk’, which many commentators recommend including in an IA—if an initiative is in any way dependent on the health workforce, an IA must consider whether there is a risk that the workforce will be unable or unwilling to contribute to implementing it. In the case of HRH, the risks could include insufficient health workers in the right locations, insufficient skills or equipment to deliver the intervention, perverse or insufficient incentives, low health worker morale or lack of motivation [50].

Table 4 presents the different sections of the draft tool and explains how and why these sections follow the guidance obtained via the literature review. The draft tool itself can be found in Additional file 1.

The draft HWIA tool begins by establishing the objectives of the initiative being assessed, an explanation of what problem(s) or issue(s) it is designed to address, and the causes of these problems or issues (see Section A3 in Additional file 1), which will help to focus the assessment on the appropriate issues.

The draft tool has the desirable characteristics highlighted by the literature review. It recognises that HRH impacts and implications may be determined by a broad range of factors, by not restricting users to focusing only on factors within the health sector. It encourages the collation of supporting evidence and proper documentation of the process as well as simply reporting the findings and conclusions (see, for example, part (b) of Section B1). It is systematic yet practical, using a method that is proportionate to the relevance of HRH to the initiative being assessed (the screening Module (part A) is relatively short and simple to complete so that initiatives with few or no health workforce impacts or implications will not need to conduct a full assessment). It considers how impacts might be different for different groups of people or organisations (see Section B3). It considers both short- and long-term impacts and implications (see, for example, the preamble to Section A5). It includes a stakeholder mapping exercise (see Section B4). There are detailed instructions about the process (see the first two pages of the tool), accountability is promoted by the requirement to record the names of those carrying out the HWIA in Section A2, and those completing a full review are prompted to start an action plan in Section B6. Finally, the tool is aligned with relevant strategies and policies such as the SDGs and the Global Strategy on HRH.

As an ex ante IA, the draft tool also takes advantage of the opportunity to consider the processes by which impacts are achieved as well as the impacts themselves. It does this by instructing the user to consider how and why the anticipated impacts and implications will occur (see, for example, the instruction for part (b) of Section B2).

The tool aims to capture all types of impact, whether direct or indirect, intended or unintended, positive or negative (see the preamble to Section A5). It also aims to capture impacts that may occur at different levels, which in the case of HRH may include individual health workers, groups of health workers (e.g. a cadre) and employing organisations. Sections A5 and B2 of the draft HWIA tool aim to capture impacts of all relevant types and levels.

The draft HWIA tool aims to strike an appropriate balance between practicality and rigour and to facilitate a process that organisations involved in global health initiatives will find helpful, without being excessively bureaucratic and cumbersome. It therefore avoids complicated matrices and analysis techniques, instead using a mix of simple checklists in the screening module and open questions in the main module to obtain the relevant detail. This should encourage organisations involved in global health initiatives to build into each new initiative actions that will maximise the potential for positive HRH impacts and minimise the potential for negative ones. The tool does not require the user to provide quantitative data such as numbers of workers affected, although there is space for users to include these data if they are available and considered relevant. This is in recognition of the fact that the tool should be applicable to a wide range of types of initiative, and that there will usually be as much—if not more—interest in how and why the anticipated impacts will occur as in estimating the magnitude of the impact.

Requiring a full HWIA for every single grant, investment or resolution would be a heavy administrative burden, which would be a disincentive to carrying out an IA even when one would be helpful. The two-stage approach involves a relatively short and simple screening module (part A) to be completed for all grants, investments and resolutions, and a requirement to complete a full assessment (part B) only for those expected to have HRH implications or impacts.

Sometimes it will be clear that an initiative has HRH implications or impacts, but there will also be situations when the relevance of an initiative to the health workforce is less obvious, and situations in which there is a risk of unforeseen negative consequences for the health workforce. In such cases, IAs will help organisations to ensure that the impacts on and implications for HRH are properly taken into account [38], and may assist with ‘mainstreaming’ something that may otherwise be systematically overlooked [16, 37].

If carried out thoroughly, a full HWIA will be a time-consuming exercise. This brings with it a risk that people or organisations will be tempted to avoid a full assessment by indicating in the screening module that no HRH implications or impacts are anticipated. Similarly, there may be a temptation to manipulate the assessment to validate a personal preference, or a reluctance to give an honest opinion due to a negative organisational culture or fear of offending a donor. To mitigate against this, it is important to make the process and findings available for public scrutiny and debate [33‐35]. Thus, if an assessment concludes that there will be no significant impact on or implications for HRH, readers and users of the completed tool or assessment report will either be able to satisfy themselves that this is a reasonable conclusion, or to challenge it in an informed way [15].

Because materials relevant to this analysis are not systematically captured in the peer-reviewed literature, but are rather scattered across a variety of different sources, and probably in some cases not publicly available at all, a key limitation of this method is that it may not have located all relevant examples of documentation relating to ex ante IA tools and techniques. However, as the identification of tools and other documents was designed to be illustrative rather than systematic, the approach taken was adequate to generate insights to inform the design of an HWIA tool. Due to the search being conducted in English, relevant examples in other languages were not captured, except for one item from Switzerland which was in French [22].

To fulfil the WHA resolution relating to the Global Strategy on HRH, organisations involved in developing resolutions, strategic documents, grants and investments in support of global health objectives need to be able to carry out high-quality HWIAs. Based on an established HRH framework and good practice principles from other fields, a draft HWIA tool has been developed and is annexed to this paper. Following finalisation and adoption of the tool, it will be important to assess its use and any measurable effects it might have on improved design and/or impact on HRH.