Mapping of multiple criteria for priority setting of health interventions: an aid for decision makers

Concerns on the costs, effectiveness and cost-effectiveness of health interventions have dominated the debate on health rationing in a wide range of countries since long [1‐3]. More recently, the explicit use of a number of equity-related criteria have been put forward, like severity of disease, socio-economic status, or gender, reflecting the increased attention for distribution of health in a population, as summarized by Johri and Norheim [4]. Still other criteria, like ease of implementation [5] or political acceptability [6] are presently finding their way in the prioritization of health interventions. The recognition that not a single but multiple criteria should explicitly be considered has led to the development of multi-criteria decision analysis (MCDA). This method sets programme priorities by referring to a comprehensive set of explicit criteria and guides decision makers in understanding the trade-offs between values that may be conflicting. For example while mobile clinics for HIV testing may be costly and therefore inefficient, they may deserve priority because they reach out to remote areas and therefore contribute to equity in service delivery. A core component in any MCDA is the identification of criteria that decision-makers consider important in their specific contexts. As a next step, MCDA scores the performance of health interventions on these criteria [7, 8].

At the same time, surprisingly little work has been done to develop a meaningful conceptual mapping of criteria that can help to identify the relevant set of criteria. A recent report that advices the UK’s National Institute for Health and Clinical Excellence (NICE) for the use of MCDA reviewed the literature and one of the findings was that most applications have a fixed set of criteria and lack explanation of the rationale behind the selected criteria and the categories used. In addition, it was concluded that the applications give neither flexibility nor assistance to decision makers to select an unique set of criteria in their decision context [8]. More specifically, in 1999, Musgrove presented the ‘nine criteria for public spending on health care’ in a spider-web like diagram, however, without classification of criteria [9]. Baltussen and Niessen presented in 2006 the ‘cloud of criteria’, suggesting that criteria cannot be systematically categorized [7]. Another framework, introduced by EVIDEM in 2010, does not explain an underlying rationale for the choice of categories that are used [10, 11]. Furthermore, various reviews simply list priority setting criteria [4, 8, 11, 12]. Only the list of criteria reported in the review of Golan et al., categorizes criteria according to the principles of allocative justice for rationing health care, i.e. need, maximizing and egalitarian principles, but is therefore limited in scope [13]. Our paper aims to develop a conceptual mapping of a comprehensive set of criteria, including efficiency, equity and feasibility concerns.

Categorization of criteria is important for two main reasons. Firstly, for decision makers, the grouping of a large and diffuse set of criteria into categories may ease their interpretation and facilitate decision-making. Second, such a categorization may be an aid to well-define criteria, and to avoid overlap and double-counting of criteria. Criteria, especially those related to health distribution (i.e. equity) concerns are often difficult to define and a proper mapping sets boundaries to facilitate this.

The conceptual mapping of criteria for priority setting is provided in Figure 2. The definitions of the categories are similar to those in the original WHO health system frameworks and are presented in Table 1. Our literature review resulted in the identification of a large set of criteria, which are often similar in concept but different in the ways they are described. In the Additional file 1, Overview of criteria considered for inclusion in criteria map, we list all criteria we considered for inclusion in our framework, and present a rationale for their inclusion or exclusion. The included thirty-one criteria are all single not-overlapping arguments to prioritize health interventions and are defined in Table 2. Here, we will give the rationale used for a selection of the criteria considered.

On the right panel of the map we distinguish five categories of criteria related to intervention’s goals. The first category is ‘health level’, and includes criteria ‘effectiveness on individual level’, ‘effectiveness on population level’ ‘patient reported health status’ and ‘safety’. Whereas reviews include ‘quality of evidence on effectiveness’, we excluded this from our map, as we consider quality of evidence to be relevant to all criteria, e.g. how costly or complex an intervention is. Rather, we propose to capture quality of evidence in uncertainty analysis. The second category is ‘health distribution’, and included criteria that relate to the core concept of ‘equal life time health’, which means that all people - independent of their background, their disease status or the availability of treatment - should have a fair chance to live a full healthy life [18]. This concept encompasses both horizontal and vertical equity. We define horizontal equity as the provision of equal treatment for people with equal health needs. Horizontal equity is non-discriminative towards certain groups in society to give them equal access to care as other groups with the same needs. We define vertical equity as the provision of unequal but equitable treatment for people with unequal health needs – this implies giving priority to certain groups in society, on the basis of their background (‘socioeconomic status’, ‘area of living’, ‘sex and gender’, ‘age’, ‘ethnicity’, ‘sexual orientation’), disease status (‘severity of disease’) or the ‘availability of treatment’ [19]. ‘Responsibility for health’ is included as interventions that focus on people that have bad luck may deserve more priority. The precise identification and definition of equity-related criteria is topic of a recent collaboration between different experts (ethicists, public health experts, economist, etc.) from academic institutes and the WHO [4, 19]. Therefore we only provisionally list and define these criteria here.

The third category is ‘responsiveness’, and includes ‘patient perceived quality of care’. Interventions should be responsive to people’s expectations in regard to non-health matters and reflect the importance of people’s dignity and autonomy, and the confidentiality of information. Although this seems a general, and therefore system-wide concern, some interventions do better than other interventions in satisfying perceived quality of care. We include ‘burden of disease’ to represent the wish of society to target high burden diseases. The fourth category is ‘social and financial risk protection’ and includes ‘catastrophic health expenditure’, ‘economic productivity and care for others’ and ‘rare diseases’. Regarding the latter, interventions targeting rare diseases may deserve priority because they may be very costly (as intervention (especially drug) development costs are only shared by a small number of patients) and could push people into poverty. The fifth category is ‘improved efficiency’ and reflects the economies of scale that can be obtained when reaching large number of people. Therefore we have included the criterion ‘size of target group’. We decided to exclude ‘cost-effectiveness’ as a criterion on itself, as theoretically costs (as a feasibility constraint) and effectiveness (as a goal) are both implicitly included in the mapping as individual criteria. However, regarding the importance of the criterion of cost-effectiveness in decision-making, decision-makers may nevertheless consider it as a separate criterion.

On the left panel of the map we distinguish one category of criteria related to the feasibility of implementation of interventions. This category is divided in six subcategories, based on the building blocks, i.e. ‘service delivery’, ‘health workforce’, ‘information’, ‘medical products, vaccine and technologies’, ‘financing’, and ‘leadership/governance’. In contrast to the criteria related to intervention goals (discussed above), little work has been done on these criteria (except on the criteria ‘cost’) and the criteria we put forward are first propositions. The first four subcategories relate to the current capacity of the health system and criteria reflect the requirements the for implementation of an intervention. The fifth subcategory on financing encompasses ‘unit costs’, ‘budget impact’ and ‘financing party’. The ‘unit costs’ are the total costs per patient from a health systems perspective whereas 'budget impact' incorporates the scale of an intervention. The ‘financing party’ criterion captures who is paying for a health intervention and reflects notions on its financial sustainability. The sixth subcategory represents the feasibility in terms of leadership/governance and includes ‘congruency previous priority setting’, ‘cultural acceptability’, ‘political acceptability’, ‘stakeholder acceptability’, and ‘legal barriers’.

Our map should not be regarded as a top-down expert advice on a fixed set of criteria that should always be considered in all prioritization decisions, but rather as an aid to decision-makers in their selection of relevant criteria. We see two broad applications of priority setting, and therefore of our mapping of criteria. First, it can inform decision makers who work in a specific context, e.g. on the reimbursement decision of a single intervention. These decisions are taken in the presence of a known budget and are likely limited by factors such as the currently available physical infrastructure, human resources or political consideration, at least in the short- to medium-term. This is labeled ‘context-specific priority setting’. The second application is to guide decisions on a wide range of interventions, to provide general information on their relative rank order to arrive a more informed debate on resource allocation priorities. Because priority setting in this application is not meant to provide a solution to a specific resource allocation question, it need not be highly contextualized in terms of e.g. physical infrastructure and/or human resources constraints. This is labeled ‘generalized priority setting’ [20]. The set of criteria for ‘context-specific priority setting’ is likely to be much more specific than those for ‘generalized priority setting’, but stem from the same conceptual mapping of criteria as presented above. That our mapping of criteria should not be considered as a fixed set of criteria is especially clear when setting priorities in a specific disease area. For example, an important criterion in HIV/AIDS health rationing decisions is whether the intervention reduces stigma. To include these disease-specific criteria in a generic list would increase its total number of criteria and make it unmanageable.

As our mapping is based on the WHO health systems frameworks, it has a certain credibility among decision makers. However, the choice of a different underlying framework might lead to another mapping of criteria.

In our framework, we included criteria related to the health system inputs and health system goals, but no criteria related to intermediate outcome measures as access and utilization. These measures are instrumental to reach health system goals, and are as such no goals in themselves. However, decision-makers can use them to monitor and evaluate progress towards the realization of the health system goals.

Another important step in MCDA is to define indicators to operationalize the criteria. For example, the severity of disease that an intervention targets can be measured in terms of health state valuations [21] and health gains in terms of disability adjusted life years averted. The operationalization of criteria would complement our mapping of criteria, and would allow the construction of a performance matrix that systematically demonstrates the performance of an intervention on all criteria [7, 8]. Such a matrix can consequently be the basis for rationing decisions on (a set of) health interventions. Such decisions should eventually also account for non-quantifiable criteria related to e.g. complicated ethical judgments. These criteria are not reflected in our framework, and further research should be carried out on how these can best be accounted for, e.g. through a process of elaboration [8, 20, 22].

We consider the presented mapping of priority setting criteria as preliminary only and not as a final map. We welcome discussions to further develop it, to improve the use of MCDA for setting priorities in health.

This conceptual mapping of criteria, based on well-established health system frameworks, will further develop the field of priority setting by assisting decision makers in the identification of multiple criteria for selection of health interventions.