Revealed preferences towards the appraisal of orphan drugs in Poland - multi criteria decision analysis

There are up to 36 million individuals suffering from as many as 5000–8000 different types of rare diseases in the EU. Although many of them significantly impact life expectancy, available data suggests that there is still considerable number of patients that remains without the treatment [1].

Given the growing challenges with limited budgets being spent on innovative orphan medicine products (OMPs), more transparent and objective pricing & reimbursement (P&R) decision-making process is sought. Among new initiatives, the Transparent Value Framework (TVF) developed by the Working Group on Mechanism for Coordinated Access to OMPs (MoCA) could be pointed out [2]. The TVF directs special attention towards the inclusion of non-economic criteria, such as unmet medical needs and availability of alternative treatment options in the P&R process of OMPs. It attempts to evaluate feasibility of the inclusion of a broad scope of decision-making criteria in the efforts towards more transparent and equitable assessment of the value of OMPs.

Despite limited number of examples of specific P&R frameworks being established for the evaluation of OMPs, majority of jurisdictions have not separated reimbursement processes for OMPs and non-OMPs [3‐5]. With that, it is interesting to ask what kind of criteria is utilized for the assessment of OMPs in the P&R system without any special arrangements for the evaluation of treatments for rare diseases. It is particularly valuable to provide a relevant insight into whether the broader set of criteria as envisaged by the MoCA project is at least implicitly taken into consideration in the appraisal processes of OMPs.

Central and Eastern European (CEE) settings can serve as an interesting research field for that purpose. On the one hand, criteria of clinical and cost effectiveness analysis alongside budget impact analysis are commonly established in the format of standard HTA, and there are some instances where the fixed cost effectiveness threshold is explicitly set amongst P&R rules for both orphan and non-orphan drug technologies [6, 7]. On the other hand, CEE countries lag behind the rest of the EU with regard to accessibility to treatment for patients with rare diseases. For instance, it was revealed that only in 22 out of registered 61, and 28 out of registered 72 OMPs were available in Bulgaria and Latvia respectively [8‐10]. According to the EURORDIS published study, as many as 90% of 60 OMPs were covered by public funds in the Netherlands, whilst only 30% were covered in Romania [10].

The Polish jurisdiction was chosen for the purpose of this study. It is the biggest country in the CEE Region with a well-established HTA process. The proceedings from the P&R negotiations are not available in the public domain. In contrast to other countries in the CEE Region, the Polish HTA Agency does, however, publish its recommendations.

The objective of our study was to reveal the value criteria appraised in the HTA appraisal process in Poland. The Polish Appraisal Body consists of the President of the HTA agency and ten members of the Appraisal Committee. Among them, there are representatives of the Ministry of Health, the National Health Fund, regulatory body, and patients’ ombudsman [11]. The HTA recommendations are not mandatory in Poland. The pricing negotiations are conducted at the discretion of the Ministry of Health.

The Multi Criteria Decision Analysis (MCDA) framework was chosen for the purpose of the study. There is an increasing interest in the MCDA’s implementation into the P&R process as it allows for a trade-off between different decision-making criteria [12]. In the particular case of our research, it provides an opportunity to reveal the preferences towards value criteria, and to assign weights to each of them representing their importance in the HTA appraisal process. It was hypothesized that findings from our research can help elicit recommendations on how to achieve more transparency in the pricing and reimbursement process for OMPs.

The analysis was divided into five steps following the ISPOR checklist designed for MCDA studies [15]:

The MCDA analysis was performed utilizing validated ZRx MCDA tool [13].

As the differences in access to OMPs across EU settings become more and more visible, there is a growing understanding that new approaches should be implemented to ensure a more transparent and fair allocation of funds across all individuals who suffer from rare diseases. Despite some international initiatives, new P&R pathways tailored specifically to meet peculiarities of the value assessment of OMPs are still really scarce. The currently available HTA guidelines are mainly limited to the assessment of clinical and cost effectiveness as well as safety. In the field of rare diseases, the importance of the consideration of non-economic criteria in the P&R decision-making is especially raised. Therefore, it was interesting to investigate what kind of value attributes are considered in the HTA process when there are no guidelines designed for the assessment of OMPs.

The study proved that both clinical evidence and economic considerations (CEA, BIA, the cost of therapy) played an important role in the assessment of OMPs in Poland. The results were consistent across different MCDA methods, and both the AHP weights and the SLAM rankings lead to similar conclusions.

The Polish Appraisal Body tended to rank the clinical evidence as the most important factor in the decision-making process. Interestingly, the notion of cost of therapy constituted the second largest contribution to the HTA recommendation process followed by the safety aspects. The importance of these two criteria, however, was similar, as could be seen in the comparison of AHP weights and the results of the sensitivity analysis. Nevertheless, it should be noted that there were statistically significant differences between positive and negative HTA recommendations in terms of the deliberation of safety aspects, which received more attention in the latter group.

According to both SLAM and AHP methods, the existence of a comparator was the fourth most impactful factor in the recommendation process at the Polish HTA agency. This was more important than disease rarity and severity. Interestingly, the results indicated that the availability of a therapeutic alternative was also valued more than economic evaluations. The weights of both CEA and BIA were set to 9%. The sensitivity analysis provided more insight into the importance of economic evaluation. The maximum VIP results for the CEA were set to one, which are only the second criteria with such a high value. if a lower regret is set with CEA versus BIA, then the former was more influential in the recommendation process than the latter.

Even though the cost of therapy played such a significant role in the HTA appraisal process, the results of the budget impact analysis (BIA) did not contribute that heavily towards the HTA outcome. BIA’s maximal regret set above 0.5 indicates that there is a significant opportunity cost for its inclusion in the list of recommendation making criteria.

Oddly enough, HTA recommendations issued by other HTA agencies also played an important role in the Appraisal Body’s judgment. Their weight was almost as high as the one established for the economic criteria. In addition, it should be mentioned that the influence of other HTA agencies was greater for the negative rather than the positive HTA recommendations of the Polish Appraisal Body in a statistically significant manner.

Although the study produced some interesting results, it is not free from limitations, some of which are inherited within the MCDA methodology itself. Firstly, the consequences of the adaptation of the SLAM bring certain simplifications to the analysis that are not without an impact on the results. The SLAM assumes mutual preference independence, which means that preference for an outcome measured by one criterion is independent from the outcome related to another criterion. It can be envisaged, however, that the real value judgment of the HTA Appraisal Body may not be independent across different criteria in the same manner in which value attributes correlate with each other. SLAM introduces a complete compensatory rule that involves an offset mechanism where a bad performance on one criterion can be compensated by a good performance on others. In real life settings, it is, however, difficult to expect that the HTA Appraisal Body will be willing to trade the value of a given OMP between two different attributes. Secondly, the adaptation of AHP methods was applied without transforming the scale to Saaty’s integer values 1–9. There are a number of approaches, which allow for this kind of switch with the adoption of the geometric scale [19], balanced scale [20], power [21], logarithmic [22], and finally root square [21]. It has to be underscored, however, that the analytic hierarchy process (AHP) method was selected following great consideration as it allows a pair-wise comparison of attributes. The application of AHP in this setting allowed the identification of intransitivity of collected criteria in the recommendation making process as well.

In addition to the limitations inherited in the MCDA methodology itself, there are also some additional risks of bias in our results related to the study design. Firstly, although we have taken into consideration all drug indication pairs assessed by the HTA agency between 2011 and 2015, we have not accounted for any potential changes in the Appraisal Body’s preferences across the study period. Secondly, two reviewers subjectively conducted the assessment of attitudes towards different MCDA criteria. Thirdly, the study was limited to the HTA recommendations only. Some of the Appraisal Body’s considerations failed to be presented in these documents, and the assessment of other P&R decision makers were left outside of the scope of our analysis.

Despite these limitations, our study contributes to the growing body of literature focused on the adaptation of MCDA to the OMPs assessment. In contrast to other publications, it presents the application of MCDA to the analysis of preferences of decision makers towards the value attributes of OMPs. To our knowledge, none of previous studies had a similar objective. For instance, Schey et al. [23] adopted MCDA framework to the value OMPs from the UK perspective. Iskrov [24] developed an MCDA value model for the assessment of OMPs from the perspective of the Bulgarian payer. In the manuscript of Sussex J [25], the objective was to pilot the use of MCDA as a framework for a value assessment of OMPs.

Still, the current study could be regarded as a continuation of earlier research that verified the potential impact of the implementation of an MCDA approach on the value assessment of OMPs in the Polish context [14]. Despite different objectives and the scope of data analysed, there is some resemblance in the conclusions between both studies. Firstly, it was revealed that the economic arguments had an immense importance in the Polish HTA process. Secondly, it was indicated that other criteria beyond the set of value attributes defined in HTA guidelines, such as the cost of therapy, HTA outcomes from other jurisdictions, played a role in the appraisal process also in Poland.

In addition to MCDA studies, there are some publications that provide insight into the value attributes used by manufacturers in price setting for OMPs as well. Therefore, some comparisons can be made between the preferences of decision makers and producers. In this regard, at least three interesting observations should be noted. Firstly, the opposite valuation was found with respect to the criteria of the clinical effectiveness. Although the highest importance of data regarding efficacy in the recommendation process was clearly expressed, it has not been found among the predictors of the price of orphan drugs [26, 27]. Secondly, some similarities could be drawn between the preferences of manufacturers and decision makers with respect to the availability of comparator. While it was valued as the fourth most relevant criteria in the HTA appraisal process, some studies indicated its importance in the manufacturers’ setting of a pricing policy. For instance, the Belgian comparative analysis of prices of reimbursed OMPs revealed that total annual costs were lower for drugs with a comparator compared to those without alternative treatment options [28]. Finally, a contradiction could be noted with respect to the notion of disease rarity. As this was not listed amongst the key attributes of the HTA appraisal process, the opposite could be said with respect to pricing studies. This review of 75 OMPs, which were granted marketing authorization by the EMA until 2014, revealed a significant inverse relationship between disease prevalence and the annual cost of drug treatment [29]. However, analysis of 45 OMPs in the five biggest EU jurisdictions revealed that the number of available alternatives as well as the prevalence of disease was correlated with the price of the drug [30].

In order to ensure the contribution of our findings towards further improvements of access to treatments for rare diseases, some recommendations for future development of value framework of OMPs can be elicited. The first recommendation is dedicated to further enhancement of the broader engagement of different stakeholders’ groups. Given the nature and peculiarities of rare diseases, uncertainty regarding both clinical and economic consequences of the implementation of a given health technology to the routine practice is inevitable. Therefore, it is not surprising that the Polish Appraisal Body attributed the highest value to the clinical evidence and noted the issue with credibility of data so many times among the reasons for negative HTA recommendation. To address such challenges, the development of a value framework should take views of both all decision makers and patient advocacy groups (PAGs) into consideration. With respect to the first group, the need of collaboration between the HTA agencies and EMA must be highlighted. Given the scarcity of clinical and safety data for OMPs, early dialog can be of great value especially when it comes to common understanding of unmet medical needs and scientific approaches to real life data collection. In addition to the wide range of incentives designed for OMPs available for the regular approval process, there are a number of special pathways for the marketing authorization such as accelerated, conditional, and exceptional pathways that enable the most needed OMPs to reach the patients in timely fashion [31]. Early engagement of HTA bodies in such processes could potentially facilitate a generation of P&R outcomes as well. As far as the contribution of the second group is concerned, it is again the lack of sufficient clinical and safety data that makes the PAG’s role in the P&R decision making so valuable. Given limited evidence regarding particular health states, patient experience can provide a needed insight into the real burden of disease. Such individual stories cannot compete with the breadth of evidence available for common diseases. Nevertheless, as long as it is ensured that patients’ stories are collected in a systematic and transparent manner, it can certainly successfully address some of the uncertainties of the HTA process. The second recommendation calls for the introduction of new innovative pricing & reimbursement arrangements. The results of our study indicated that the notion of the cost of therapy was a key consideration during the recommendation process of OMPs. At the same time, the price of a drug and an unfavorable cost effectiveness ratio were the most frequent reasons for the negative HTA recommendations. Therefore, a new set of P&R rules should enable the implementation of a special form of innovative managed entry agreements. Alternatively, a financing mechanism that links reimbursement with drug performance can be introduced to validate cost-effectiveness claims. Following the preferences of expert payers, the second one should especially be taken into consideration in the CEE Region [32]. Temporary access to OMPs could be secured as long as an HTA agency envisages the opportunity for further real life data collection before a final recommendation has been completed. For example, this is a common approach in the Netherlands and Sweden where temporary coverage is granted conditional upon the conduct of an observational study.

In summary, it can be concluded that our study provides an interesting insight into the value judgment of the Polish HTA process. It also indicates that the decision-making preference of the HTA Appraisal Body goes beyond a standard set of HTA criteria. In addition to clinical and economic evidence, such criteria as the cost of therapy and the rationalization analysis, along with HTA recommendations from other HTA agencies, played a significant role without being included in the HTA guidelines. Therefore, there is a need for a new approach to the value assessment of OMPs that will even take into consideration those criteria that are not yet formally included in the HTA framework. Unless this happens, transparency and objectivity of the decision-making process cannot be fully ensured, and equity with regard to access to OMPs cannot be improved.