The scope of the work in Work Package 6 (WP6) of the European network for Health Technology Assessment (EUnetHTA) Project was to analyze the relationship between HTA and health policy in Europe and to exchange views, expectations, and feedback on HTA with stakeholders (Reference Kristensen, Mäkelä and Allgurin Neikter8;Reference Kristensen, Lampe and Chase9). The partners in WP6 are presented in Kristensen et al. (Reference Kristensen, Mäkelä and Allgurin Neikter8). Analyses of the relations between HTA and health policy in Europe have been published as a separate book (Reference Velasco Garrido, Kristensen, Palmhøj Nielsen and Busse18). This article focuses on the stakeholder activities that were part of the project.
Stakeholders are important groups to involve in HTA processes because they have a legitimate interest in the outcome of HTA projects and the decisions made with HTA as an input (Reference Brugha and Varvasovszky1). Hence, the need for EUnetHTA to initiate activities ensuring stakeholder involvement was obvious. The main challenge was to initiate appropriate activities reflecting that EUnetHTA is a European network aimed at producing practical tools—not a network that produces HTAs directly for utilization in decision making. Therefore, it was not possible to imitate stakeholder involvement strategies known from national HTA agencies because their positions and relationships with stakeholders are of a different kind. Activities had to be tailored to fit the EUnetHTA aims and the position at a European level. The resulting strategy is reflected in the activities described in this article.
The international literature typically defines stakeholders as:
“. . .individuals, groups, or organizations who have an interest (stake) and the potential to influence the actions or aims of an organization, project or policy direction” (Reference Crosby2;Reference Mason and Mitroff10;Reference Walt19).
This definition was used as a starting point for the discussions on EUnetHTA stakeholder activities, and was later adapted to fit EUnetHTA's aims.
In general, stakeholder involvement in HTA is not an easy task because various interests and perspectives must be balanced in a process involving various stakeholder groups in an HTA. One possible perspective is that an HTA should be a strictly scientific product, which ought not to include stakeholder perspectives. However, it is helpful to view HTA production as part of a larger societal process that includes perspectives from various kinds of stakeholders (Reference Gibbons, Limoges and Nowotny5). Also the usefulness and utilization of HTAs depend on successful interaction with different kinds of stakeholders (Reference Merino, Lema, Velasco Garrido, Kristensen, Palmhøj Nielsen and Busse11;Reference Palmhøj Nielsen, Sarriá Santamera, Vondeling, Velasco Garrido, Kristensen, Palmhøj Nielsen and Busse13;Reference Sorensen, Drummond, Kristensen and Busse17). Based on these considerations, basic questions in relation to involvement of stakeholders in HTA are (i) who to involve, (ii) how to involve, and (iii) what to involve different stakeholder groups in.
HTA institutions have worked for years establishing systems to address these questions and to involve different stakeholders in their national/regional processes. These systems vary and take national/regional needs and traditions into account. In relation to the project and a sustainable EUnetHTA Collaboration, we found it relevant to consider how the HTA community should engage with stakeholders at a European level to develop a dialogue and opportunities to exchange opinions on the development of HTA in Europe.
AIMS AND OBJECTIVES
This article describes the stakeholder perspective and how the EUnetHTA Project addressed it. The overall aim was to develop an open forum for stakeholders for exchange of views, expectations, and feedback on HTA. During the project we considered how this forum should be structured and developed on a long-term basis. Hence, stakeholder activities in the project were aimed particularly at outlining a structure and process for stakeholder involvement in a future sustainable EUnetHTA Collaboration.
STRUCTURE AND METHODS
This article presents EUnetHTA's definition of the term stakeholder as it was used in relation to the stakeholder activities. We also explain how the stakeholder groups included in EUnetHTA stakeholder activities were identified as a foundation for developing a forum for stakeholders. The basic theoretical considerations regarding the necessity of involving stakeholders in HTA to ensure legitimacy and representation of interests in policy making in health care are briefly described, as are the specific activities carried out in the project. Finally, we discuss the result of the work and the implications for future stakeholder involvement in HTA at a European level and present several recommendations.
Who Are the EUnetHTA Stakeholders?
The literature has presented various definitions of stakeholder since the term first appeared in the business/management literature during the 1930s (Reference Brugha and Varvasovszky1). Building on the general definition presented above, we developed the following definition of stakeholders to fit the specific purpose of the Project:
“Stakeholders are groups or organizations which potentially will be affected by, or have an interest in and may in a consultative role influence on the actions or aims of an organization, project, or policy direction” (3).
This definition excludes individuals as stakeholders because it was decided early in the project to focus stakeholder activities on groups and organizations, not individuals, with a stake in HTA at the European level. Organizations are accountable to their members and can be viewed as stakeholder groups with reliable positions. In addition, the definition reflects discussions within EUnetHTA. The issue of stakeholder involvement was a sensitive one among the EUnetHTA Partners as several were hesitant to allow much stakeholder influence. Hence, the definition is a result of a process that balanced the views of EUnetHTA Partners. This resulted in a definition stating that the stakeholders are given a consultative role and thus may influence the actions or aims of EUnetHTA.
While identifying specific stakeholder groups to be included in EUnetHTA stakeholder activities it was necessary to delimit the possible groups. Not all possible stakeholder groups were included in the activities, as it was decided to focus on the stakeholder groups that are not EUnetHTA Partners. This shifted the focus toward groups outside EUnetHTA that are funding and/or using HTA and groups providing health technologies. The following stakeholder categories were chosen: Policy makers at the national/regional level; Policy makers at the hospital level, in statutory health insurance, or in health maintenance organizations; Patient organizations; Healthcare professional organizations; and Industry (3).
Why Stakeholder Activities and Involvement?
Before going into depth about concrete stakeholder activities, we briefly discuss the underlying theoretical rationale for focusing on stakeholder activities and developing structures for future stakeholder involvement in the EUnetHTA Collaboration. The main objective of stakeholder involvement is to ensure representation of interests in, and legitimacy of, HTA and thereby enhance the possibility of utilizing HTA in policy making.
Strengthening the Representation of Interests and Legitimacy of HTA through Stakeholder Involvement. Representation of interests and legitimacy is usually investigated in relation to democracy and political processes. Hence, most of the general theoretical considerations presented in the international literature concern political decision-making processes. However, it is also relevant to consider these issues in relation to the foundations underlying decisions, e.g., HTAs. Even if nearly all HTA agencies are not political or regulatory bodies—and, therefore, do not hold decision-making authority—they resemble political institutions in the sense that they are (often) public, tax-financed institutions seeking to improve the healthcare system by their activities. Thus, theoretical considerations on interest representation and legitimacy can also contribute important insights regarding HTA production.
In studies concerning interest representation, legitimacy, and democratic, political institutions, it is suggested that legitimacy rests on “reason giving” (Reference Humphrey6;Reference King7;Reference Parkinson14). This means that legitimacy concerns the ability to give “reasons for policy decisions that are understood and perceived as sound by those who are to submit to that authority” (Reference Humphrey6). The idea is that decisions building on understandable and sound arguments are more easily accepted even if there is some disagreement on the decision content. Furthermore, it is argued that a decision can be perceived as legitimate even if the reasons are rejected, provided the reasons were revealed through processes perceived as open and fair (Reference Humphrey6). Hence, the validity of political decisions, or foundations for decisions, does not rely solely on content, but also on the process from which they stem.
Viewing HTA production in light of these considerations, one could argue that HTA products are more likely to be accepted, and may thus impact on policy making, if stakeholders accept the scientific methodology upon which the results rest (corresponds to finding the arguments sound and understandable) and/or consider the production processes open and fair. This reveals that, with regard to HTA, legitimacy must be understood in relation to: (i) the scientific quality of HTA products and (ii) the fairness and transparency of the production processes (Figure 1).
Figure 1. Simple relation between health technology assessment (HTA) product, stakeholder involvement, and legitimacy.
Which Processes Should Be Established to Obtain Legitimacy?. The research objects/objectives of HTAs are potentially of major concern for stakeholders and may involve conflicting interests. Consequently, broad agreements on HTA results cannot always be expected. If stakeholders are to accept HTA results as legitimate, then predefined processes must address potentially conflicting interests in a manner that all parties accept as fair. Transparent processes may be the only way to “prove” that no interests were inappropriately favored. Transparency appears to be especially crucial in regard to stating interests, selecting topics for HTA, formulating research questions, including literature, and selecting and interpreting indicators and effect scores as these processes determine the conclusions of a report and interpret the “value” of the health technology in question. However, transparency is also highly relevant when it comes to EUnetHTA's production of practical tools, which sets the frames for HTA production in the future.
What constitutes transparency and fairness with regard to HTA processes is not well defined. However, several publications address which processes should be established to obtain legitimacy. A survey from OECD reveals that the perception of a “fair” decision-making process relates to (i) the relevance of the problem, (ii) the transparency of the process, (iii) the existence of an appeal mechanism, and (iv) public regulation mechanisms ensuring fulfillment of the before mentioned (12). Sorensen et al. emphasize that more stakeholder involvement (including patients/consumers and industry) is needed to improve HTA processes and to implement decisions and related policy—and EUnetHTA is mentioned as a network that can enhance stakeholder involvement (Reference Sorensen, Drummond and Kanavos16). The same authors also underline that HTA processes in general lack transparency, and that more transparency is necessary to ensure an open, systematic, and unbiased decision-making process (Reference Sorensen, Drummond and Kanavos16). Likewise, some have suggested that transparency and accountability of involvement should be improved, focusing especially on the extent to which a broad range of stakeholders (e.g., healthcare professionals, patients, and industry) are included and represented (Reference Sorensen, Drummond, Kristensen and Busse17). Finally, relevant stakeholders should be involved to facilitate the acceptance and implementation of decisions. A transparent, well-communicated, decision-making process should give legitimacy to subsequent recommendations (Reference Sorensen, Kanavos and Drummond15).
These statements are all recent assessments of stakeholder involvement to improve legitimacy and representation of interest in HTA processes. The statements are not very specific, but all call for more stakeholder involvement and more transparent and fair processes in this involvement.
For EUnetHTA and its products to be perceived as legitimate they must: (i) build on arguments considered scientifically sound by stakeholders, and (ii) have been revealed through processes that stakeholders perceive to be transparent and fair. Hence, the political relevance of HTA products, transparency in HTA processes, evidence requirements, and balanced stakeholder involvement are important topics to consider. Legitimacy is important for HTA products to have the best chances to impact policy making because stakeholders with potentially conflicting interests may try to block the decision-making process if they do not find the foundation for a decision valid.
In the long-term, legitimacy affects not only the use of HTA products in national/regional HTA institutions, but also the use of EUnetHTA practical tools and the production processes undertaken by EUnetHTA Partners.
What Did EUnetHTA Do?
Given the above considerations as a starting point, the challenge was to begin to develop the framework for future stakeholder involvement that could contribute to (i) the legitimacy of EUnetHTA and its products; (ii) the inclusion of stakeholder interests, knowledge, experience, and perspectives in the development of EUnetHTA and its results; and (iii) the recognition of the EUnetHTA Partners as producers of scientifically sound products who do not uncritically adopt stakeholder opinions.
At the outset of the EUnetHTA Project each Work Package (WP) formulated its own “guiding principles on stakeholder involvement” for the project, which described the policy for involving stakeholders in the development of their products (4). These principles included the opportunity for stakeholders to give input to the draft products developed in EUnetHTA. Corresponding general stakeholder activities focused on informing stakeholders about developments and activities in EUnetHTA, identifying relevant stakeholder groups to interact with, analyzing stakeholder input to EUnetHTA, producing a draft stakeholder policy, and organizing a stakeholder meeting.
General Information to Stakeholders. During the project, a general priority was to inform about EUnetHTA and to present its results, including future plans for a sustainable collaboration. The European Health Forum in Gastein, October 2006, dedicated a half-day forum to presenting EUnetHTA to stakeholders, giving them an opportunity to be informed about the Project already in its early stages. At the Gastein session, several stakeholders were invited to present their views on HTA and EUnetHTA. From 2006 through 2008, EUnetHTA was presented at various meetings and conferences. Such wide-ranging dissemination of information about the Project and its results was given to broad audiences, but did not necessarily reach the targeted stakeholder categories.
Web Site. A Stakeholder Open Forum page was established on EUnetHTA's Web site, www.EUnetHTA.net, to give stakeholders targeted information about developments in EUnetHTA (3) and establish a platform for virtual communication. Typical information included ongoing activities, for example, meetings and conference where it was possible to hear about and discuss EUnetHTA. The Web site also contained documents about HTA that could be of general interest to stakeholders (position papers, publications, etc.) and syntheses of stakeholder opinions on different HTA issues. In addition, the Web site was used to inform stakeholders about opportunities to comment on the draft products from EUnetHTA.
Identification of Stakeholders. Taking the five selected stakeholder groups as a starting point, specific stakeholders to be included in the EUnetHTA stakeholder activities were identified. The Partners decided to focus on European umbrella organizations operating at the European level. This decision was intended to ensure that EUnetHTA did not interfere with national/regional stakeholder processes—the aim was to focus on EUnetHTA and HTA at a European level. It was also decided to focus on generic rather than disease-specific groups (patient organizations) and healthcare professionals in the areas of medicine, nursing, and dentistry. Furthermore, it was decided that national policy makers should be reached through the EUnetHTA Partners and not through umbrella organizations.
Because there was no tradition of formalized stakeholder activities in relation to HTA at a European level, it was necessary to identify specific European umbrella organizations to communicate with. Although it was easy to identify relevant umbrella organizations in some of the stakeholder categories (e.g., industry), organizations in other categories were more difficult to identify (e.g., regional government). They were identified through personal knowledge or contacts among the involved EUnetHTA Partners, Internet searches, lists of stakeholder organizations from other European organizations, and declarations of interest from umbrella organizations. Based on this work, twenty-nine umbrella organizations were included. Box 1 lists the organizations that were invited to submit comments on the proposal for a sustainable EUnetHTA Collaboration and to participate in a EUnetHTA stakeholder meeting.
Box 1. Invited Stakeholder Organizations
Discussion Topic Catalog. In preparation for the EUnetHTA Stakeholder meeting (June 13, 2008), a working group in WP6 developed a discussion topic catalog aiming at synthesizing the issues raised by stakeholders concerning the EUnetHTA Collaboration and HTA processes in Europe. The discussion topic catalog addressed several themes that could be discussed by stakeholders and the EUnetHTA Collaboration. The catalog built mainly on stakeholder comments to the EUnetHTA draft proposal of November 2007 for a permanent collaboration on HTA across Europe. Further information sources included position papers on HTA from industry; an article discussing key conceptual and policy issues related to Coverage with Evidence Development (CED) raised at a Health Technology Assessment International (HTAi) Policy Forum Meeting; the summary report of a workshop on rare diseases held by the European Platform for Patients’ Organizations, Science, and Industry (EPPOSI); and a visionary book on future health care from Health First Europe. The sources of information were obtained by means of direct communication with stakeholders and Web site searches, for example, for position papers.
The discussion topic catalog was part of the background information sent to stakeholders before the stakeholder meeting. Its content was reflected upon at the meeting and stakeholders generally found that the catalog reflected their essential concerns, and they encouraged us to develop answers to the questions set up in the catalog. Some of the questions were already addressed in the final public proposal for a permanent European Collaboration on HTA of June 2008, some were reflected upon in a subsection of the EUnetHTA stakeholder Web site (FAQ). The discussion topic catalog and answers are available on the EUnetHTA Stakeholder Web site (3).
Draft Stakeholder Policy. Before the stakeholder meeting, and in parallel with developing the EUnetHTA proposal, a WP6 working group developed a draft stakeholder policy for the EUnetHTA Collaboration. The aim was to clarify the definition of EUnetHTA stakeholders and the relation between the future EUnetHTA Collaboration and stakeholders. The draft stakeholder policy included considerations of transparency in stakeholder involvement, in financing of EUnetHTA, and in working methods.
Because the policy was a draft, its content could be further refined. It was developed in line with thirty-four EUnetHTA Steering Committee members reaching consensus on certain issues for the final public version of the EUnetHTA Collaboration proposal. This process specifically addressed the issue of stakeholder involvement. The draft stakeholder policy of June 2008 was sent to stakeholders as part of the background information for the stakeholder meeting. This draft was presented at the meeting along with the options for giving input. The main avenues for stakeholder involvement included the proposed establishment an advisory forum in the future EUnetHTA Collaboration and participation of individual experts in working groups. The draft stakeholder policy is available on the EUnetHTA stakeholder Web site (3).
Stakeholder Meeting. The general aim of the stakeholder meeting was to exchange views and voice expectations on HTA processes and the future development of EUnetHTA, and more specifically to present the latest developments in EUnetHTA and discuss the discussion topic catalog and the draft stakeholder policy. All twenty-nine identified stakeholder organizations (Box 1) were invited to the meeting, and the following organizations attended: Hope – European Hospital and Healthcare Federation, HFE – Health First Europe, EFPIA – European Federation of Pharmaceutical Industries and Associations, EBE – European Biopharmaceutical Enterprises, and EUCOMED
Other invited organizations were either unable to participate on that specific date, or not used to working with HTA at a European level and, therefore, uncertain as to whether HTA would become part of their task as European umbrella organizations in the future. All organizations indicated an interest in following the development of a future EUnetHTA Collaboration. Stakeholder groups from industry were better represented at the meeting than other groups and, therefore, gave more input into the process.
Meeting notes summarized the discussions. The stakeholders present agreed to forward the draft stakeholder policy in its present form, the meeting notes, and the discussion topic catalog to the founders of the EUnetHTA Collaboration. These reflect the issues that the stakeholders considered important, problematic, or vaguely described.
RESULTS
Results of the work with stakeholder activities in relation to EUnetHTA show first and foremost that an initiative was taken to start exchanging views and voicing expectations on HTA processes and the future development of EUnetHTA with stakeholders. The first steps were taken to ensure representation of interests in relation to EUnetHTA. The group also addressed ensuring legitimacy of EUnetHTA and its products to promote its potential for facilitating the use of HTA in making national/regional policy. Tangible results were the EUnetHTA Stakeholder Web site, discussion topic catalog, information service, and draft stakeholder policy. The discussion topic catalog is a platform for further discussions with stakeholders because it represents some of the issues that stakeholders find important, problematic, or vaguely described.
DISCUSSION
All activities of WP6 were directed at ensuring involvement of stakeholder interests, legitimacy, and transparency in the EUnetHTA processes. Hence, EUnetHTA has taken solid steps toward honoring the requirement described in recent analyses of the status of, and criteria for, stakeholder involvement in HTA (Reference Merino, Lema, Velasco Garrido, Kristensen, Palmhøj Nielsen and Busse11;Reference Palmhøj Nielsen, Sarriá Santamera, Vondeling, Velasco Garrido, Kristensen, Palmhøj Nielsen and Busse13;Reference Sorensen, Drummond and Kanavos16;Reference Sorensen, Drummond, Kristensen and Busse17). However, the EUnetHTA stakeholder definition and the draft stakeholder policy indicate some reluctance among EUnetHTA Partners concerning the extent of stakeholder involvement, due to fear of letting stakeholder interests influence the content of EUnetHTA production in an “unscientific” direction. This is reflected in the advisory role that stakeholders have been given in the proposed interaction between stakeholders and the EUnetHTA Collaboration. On the other hand, stakeholders have been given a well-defined role in the EUnetHTA Collaboration, and this role should be developed further with participation of stakeholders and EUnetHTA Partners.
Industry was heavily overrepresented in the stakeholder meeting. The goal was to obtain balanced representation of the different stakeholder categories at the meeting and in the continued stakeholder involvement. However, it was obvious that the industry umbrella organizations had a tradition of working with HTA and were comfortable about being involved. Some of the other stakeholder categories were interested, but were either not used to working with HTA at a European level, and, therefore, hesitant to participate, or they were unable to participate at the specific date of the meeting. Hence, we face an ongoing challenge to ensure balanced stakeholder representation in relation to EUnetHTA and HTA in Europe.
CONCLUSIONS AND RECOMMENDATIONS
Stakeholder involvement in EUnetHTA is necessary to ensure transparency of interests and processes, legitimacy, and utilization of EUnetHTA and its products. The described activities create the foundation for a dialogue with and involvement of stakeholders. The EUnetHTA stakeholder meeting can be considered as a successful experience of dialogue with stakeholders, which should be continued. However, the experiences during the project show that it has been difficult to obtain balanced stakeholder representation across the identified stakeholder groups. It is recommended that continued attention be given to acquiring wide stakeholder representation to ensure balance. It is necessary that all stakeholders find the content and process fair and transparent, and broader representation of stakeholder groups will promote the legitimacy and utilization of the EUnetHTA Collaboration.
It is important to recognize that the processes for involvement described in the draft stakeholder policy need to be specified in greater detail, and this work should take place in dialogue with a EUnetHTA Collaboration stakeholder forum.
CONTACT INFORMATION
Camilla Palmhøj Nielsen, MA, PhD Student (cpn@sst.dk), Special Advisor, Danish Centre for Health Technology Assessment, National Board of Health, Islands Brygge 67, DK-2300 Copenhagen S, Denmark; PhD Student, Department of Political Science, University of Copenhagen, Østre Farimagsgade 5, DK-1353 Copenhagen K, Denmark
Sarah Wadmann Lauritsen, B.scient.san.publ (s.wadmann@pubhealth.ku.dk), Trainee, Danish Centre for Health Technology Assessment, National Board of Health, Islands Brygge 67, DK-2300 Copenhagen S, Denmark; Research Assistant, Institute of Public Health, Department of Health Services Research, University of Copenhagen, Østre Farimagsgade 5, DK-1014 Copenhagen K, Denmark
Finn Børlum Kristensen, MD, PhD (fbk@sst.dk), Director, European network for Health Technology, National Board of Health, Islands Brygge 67, DK-2300 Copenhagen S, Denmark; Adjunct Professor, Faculty of Health Science, University of Southern Denmark, Winsløwparken 19, 3, Odense C, DK 5000, Denmark
Marie Louise Bistrup, cand.arch., MPH (mlb@sst.dk), Research Assistant, Danish Centre for Health Technology Assessment, National Board of Health, Islands Brygge 67, DK-2300 Copenhagen S, Denmark
Americo Cecchetti, PhD (acicchetti@rm.unicatt.it), Director of Research, Health Technology Assessment Unit, “A. Gemelli” University Hospital, 8 Largo A. Gemelli, Rome, Italy, 00168; Professor of Management, Department of Management, Università Cattolica del Sacro Cuore, 1 Largo F. Vito, Rome, Italy 00168
Eva Turk, MA, MBA (eva.turk@ivz-rs.si), Senior Researcher, Center for Health Economics and Organization in Health Care, Institute of Public Health of the Republic of Slovenia, Trubarjeva 2, Ljubljana, 1000, Slovenia
