HTA and policy decisions are usually complex due to the multiple aspects considered and the extensive amount of evidence. Frequent gaps in the evidence and associated uncertainty also contribute to the challenges faced by decision makers. Complex interventions complicate this problem further. The implementation aspects of complex health interventions are an essential link to the desired health outcomes [
2,
3]. In turn, the success or failure in improving health outcomes is not always attributed to the complex intervention itself but to context- and implementation-related issues. In order to achieve a comprehensive assessment of complex technologies, a variety of different issues have to be assessed such as effectiveness, ethical, context, and implementation issues [
2]. However, the different aspects of HTA are not systematically taken into account for health policymaking. The results are mainly presented side-by-side and decision-makers are struggling to evaluate contradicting outcomes of complex HTA (e.g., better medical outcome but worse social outcome) [
4].
In addition, the current approaches in HTA and healthcare decision-making have some limitations regarding the integration of the diversity of stakeholders’ preferences and perspectives in their processes. On the one hand, patient and public involvement is gaining more and more attention from health policymakers [
5]. On the other hand, health economic tools are not able to identify and address the multiple voices of healthcare stakeholders [
6]. Daniels’ ethical framework of ‘accountability for reasonableness’ provides the foundation for fair evaluation of healthcare interventions and fair decision-making [
7]. According to this framework, all reasons and criteria for funding healthcare have to be accessible to all stakeholders. The reasons must be based on principles that ‘fair-minded’ people would agree upon. The criteria should reflect a society’s value [
8]. These issues are all of a fundamental democratic nature and thus constitute the basis for acceptability of decisions. An optimal scenario would be to have a societal consensus on a collective solution for society and all important stakeholders to address rationing issues and the decision-making process associated with these. This could be achieved by engaging all stakeholders and ensure consideration of all stakeholders perspectives, preferences, and constraints.
Comprehensive multi-criteria decision analysis (MCDA) provides a tool in this direction. Its methodological basis enables the exploration of stakeholders’ preferences and perspectives and to explicitly structure the broad range of criteria on which real life evaluations and decisions are based [
9]. MCDA provides insights into the rationale behind decision-making processes [
10]. The MCDA process is democratic by nature and consists of several steps. Firstly, the decision problem needs to be defined and structured, i.e., the identification of valuable healthcare interventions from a holistic perspective. Secondly, a set of mutually independent criteria is defined and weighted based on their importance to individual stakeholders involved in the process. Thirdly, the appraised interventions are assigned scores based on their performance for each criterion; this is performed based on data available, hence the importance of aligning data development with decision criteria. Finally, a value estimate is calculated by combining weights and scores. A number of MCDA methods are available [
11], with various degrees of complexity, including direct methods, such as 5- or 10-point weighting scales (Kepner Tregoe [
12]), ranking, point allocation, analytic hierarchy process (AHP) [
13], or indirect methods such discrete choice experiments (DCEs) [
14-
17]. DCEs have been successfully employed when the number of outcomes is small, while AHP is cognitively demanding for participants. The hierarchical structure of AHP in addition to the high number of evaluated alternatives can appear too complex for participants [
18]. In DCE studies, the number of criteria levels is an important issue. Scoring of criteria with two levels is mostly not sufficient to illustrate the real world. However, the addition of criteria levels would have increased the complexity of discrete choices for respondents [
14,
17,
19,
20]. For this study, we selected an existing, open source, comprehensive MCDA framework, developed collaboratively through input of various stakeholders and which meets the methodological requirements of completeness, redundancy, and mutual independence [
21-
23]. This pragmatic framework, tested, adapted, and used by several HTA agencies [
24-
26], provides several weight elicitation methods (
www.evidem.org), and includes a set of relevant criteria to explore stakeholders perspectives and preferences regarding evaluation and decision making for healthcare interventions. The framework consists of a core quantitative MCDA model and a qualitative contextual tool, with a comprehensive range of criteria and sub-criteria, which allows for adaptation to context.
The objectives of this study were to explore perspectives and preferences, in the German context and across different types of stakeholders, when appraising healthcare interventions using multi-criteria assessment of a heart pulmonary sensor as a case study.