The quality of economic evaluations of ultra-orphan drugs in Europe – a systematic review

In the European Union (EU), a disease is considered ‘orphan’ if it is a life-threatening or seriously debilitating disorder that affects fewer than 1 per 2 000 (or less than 0.05 %) of the population [1]. To date, 7 000 rare diseases have been identified, affecting 30–40 million patients in the EU [2]. Although no legal definition of ‘ultra-orphan’ diseases has been established, this subcategory was introduced by the National Institute for Health and Care Excellence (NICE). It is suggested to be applied to diseases with an estimated prevalence of <1:50 000 [3]. Initially, the small consumers’ market of orphan drugs restrained the pharmaceutical industry from developing medicines for rare diseases. For this reason, European legislation was introduced in 2000 to stimulate the development of orphan drugs, following the example of the United States who introduced the Orphan Drug Act in 1983. This legislation implies that the pharmaceutical industry has a right to i) obtain protocol assistance at a reduced charge, ii) gain access to the centralized authorization procedure, iii) get reduction of registration costs, and iv) benefit from 10 years of market exclusivity after registration [4]. This has led to the authorization of 124 new orphan drugs in the EU from 2000 until 2015 by the European Medicines Agency (EMA), of which about one-third is indicated for an ultra-orphan disease (http://​www.​ema.​europa.​eu/​ema/​). Authorization of these ultra-orphan drugs is often based on studies in small patient groups not representing the entire patient population. Moreover, biomarkers and intermediate endpoints are often used [5]. Because of this, longer term effectiveness on clinically relevant outcome measures is often unclear at the time of authorization. If, at time of authorization, insufficient evidence on efficacy and safety exists, the regulatory authorities can register an orphan drug by approving it ‘under exceptional circumstances’. This applies to drugs for which the applicant can demonstrate that comprehensive data cannot be provided (due to specific reasons foreseen in the legislation). At the same time, costs of orphan drugs have increased over the past years, in particular costs of ultra-orphan drugs. This rise has to do with the rapid technical advances including unravelling of molecular mechanisms that underlie some diseases and require a more personalised approach [6]. About 250 new rare diseases are described annually [7]. For member states to make decisions on reimbursement, it is important to gain a better insight into the balance between expenses and health gains for a specific drug in order to determine the ‘value for money’ for orphan drugs. Consequently, economic evaluations play an important role in current decision making about the reimbursement of drugs. In some countries in the EU, reimbursement of drugs depends on the incremental costs per quality-adjusted life year (QALY). In England and Wales, incremental costs exceeding £20 000 to £30 000 per QALY gained are in principle not reimbursed [8‐10]. Conversely, other countries in the EU recommend against any pre-set threshold. It is, however, a scientific challenge to evaluate the cost-effectiveness or cost-utility of orphan drugs. With this review we aim to assess the economic evaluations ultra-orphan drugs marketed in the EU that have been performed so far with specific focus on the methods used and the quality of the studies.

We conducted a systematic review to identify cost-effectiveness and cost-utility studies for ultra-orphan drugs marketed in the EU and applied the CHEC-list to assess the quality of studies included. The challenges to perform adequate economic evaluations in the very small populations of ultra-orphan drugs are expected to be similar to the challenges in the broader group of orphan drugs and for drugs that are used for rare indications but have not received orphan drug designation. Only a limited number (N = 16) of studies on the cost-effectiveness or cost-utility of ultra-orphan drugs have been published, most of which were performed in the USA, the UK and the Netherlands. The large representation of studies from the UK and the Netherlands can be explained by the obligation to perform post-marketing economic evaluations in order to gain reimbursement. In the USA health technology assessments are not systematically conducted, but to an increasing extent orphans are part of the debate on pricing and cost-effectiveness [29].

Five of the 16 studies did not succeed to perform a real economic evaluation, or used an approach that was too simplistic to lead to a realistic estimate of the ICER/ICUR [14, 16, 21‐23]. The latter either assumed the maximum possible benefit from therapy (immediate death in case of no treatment versus survival with perfect quality of life with treatment) [14, 21, 23], or based their estimate of effectiveness on only a very limited number of studies which probably led to an overestimation of treatment effect [16]. These studies highlight the fact that economic evaluations of ultra-orphan drugs are faced with many difficulties and limitations. The low number of patients and often slowly progressive, chronic character of the disease make a randomized clinical trial to estimate cost-effectiveness or cost-utility unfeasible. Pharmacoeconomic models using data from different sources such as literature, own patient cohorts and clinical experience are therefore more suitable [30]. Indeed, the majority of studies (N = 11) used models to estimate the cost-effectiveness or cost-utility ratio, including Markov-state-transition models in 7 studies, patient-level simulation models in 3 studies, and a decision tree in 1 study. A decision tree is the most simple model of disease progression. This type of model is primarily useful for short time horizon decisions where the likelihood of an event's occurrence is constant over time. It is in general not suitable for slowly progressive, chronic diseases although it has been used in the assessment of mercaptamin for the treatment of cystinosis [17]. The authors assumed that treatment would delay renal failure for 5 years. After renal failure all patients were assumed to follow the same disease course with dialysis and transplants, irrespective of previous treatment. Such a simplification of the disease course can be analysed with a decision tree, but the modelling choice for a chronic disease usually comes down to a choice between a cohort-level modelling approach such as a Markov-state-transition model, and a patient-level simulation model such as discrete event simulation (DES), which both allow for a more detailed representation of the disease course.

In patient-level simulation models, each patient is represented individually and the final outcomes are calculated by aggregating over all individuals [31]. This type of model has greater flexibility and allows for a better representation of heterogeneous patient cohorts than a Markov-state-transition model, since it allows for irregular time intervals between events. Also, the possibility to make the time to future events dependent on patient attributes including their history of previous events may result in a better reflection of the actual disease course. However, this type of model is in general more complex and needs more patient data to be able to reliably estimate specific effects and interactions of symptoms and complications on the progression of the disease. Kanters et al. used such a model to calculate costs and effects of alglucosidase alfa for the treatment of the infantile form of Pompe disease [24]. Only survival was modeled as event, making it a relatively simple model without the need for large patient numbers. Cost-effectiveness of bosentan has also been studied with patient-level simulation models [20, 27]. Again, only a limited number of events (clinical deterioration and/or death) were modelled. All 3 studies scored between 14 and 17 points. The question is whether a model with a very limited number of events accurately reflects the actual disease course. Complications that arise during the course of the disease were not taken into account while they probably result in a decrease in quality of life and an increase in costs. The model would become more complicated if these events were to be incorporated. The very small patient numbers in the case of ultra-orphan diseases may then limit the use of a patient-level simulation model, since it will be hard to gather sufficient information for reliable estimates of effects and interactions of symptoms and complications. Therefore, economic evaluations of ultra-orphan drugs will often use a cohort-simulation model, more specifically a Markov-state-transition model. This type of model has been used to estimate costs and effects of ERT for Gaucher, Fabry and Pompe disease [13, 19, 25], as well as drugs indicated for paroxysmal nocturnal haemoglobinuria [28] and PAH [15, 18, 26]. All studies scored between 14 and 19 points. In contrast to the patient-level simulation model, the Markov model represents the study population as a homogeneous cohort. It is particularly suitable to reflect the continuous risk of a disease over a long time period [30]. However, the suitability might be limited by the Markovian assumption of ‘no memory’, which means that the probability of transitioning from one state to the other entirely depends on the state the patient is in, regardless of the states the patient passed through the past.

In summary, the choice of which type of economic model to use is dependent on the type of disease that is described, the amount of data available, but also on the time and experience of the researcher. A Markov-state-transition model seems to be most suitable model for economic evaluations of ultra-orphan drugs considering the need for the incorporation of several disease states and the small patient groups. It should be noted here that we found a wide range of ICURs for the same drug even if the same type of model was used. This is at least partly caused by the many assumptions that had to be made due to limited data sets. To be able to build a model that comes as close as possible to the actual course of the disease under study, and to reduce as much as possible the uncertainty of the ICER/ICUR, it is of utmost importance to gather sufficient data. A crucial step in generating data is to establish disease-specific registries which include longitudinal data on all affected patients in the EU. Ideally, registries would start before drugs are marketed, to be able to generate data about the natural history of the disease. For this purpose, EU countries need to work in close collaboration. Existence of reliable data on the natural course and effectiveness of therapy enables researchers to conduct sound economic evaluations.

On the other hand, it has been argued that cost-effectiveness or cost-utility studies are inappropriate in the case of (ultra) orphan drugs since conventional criteria for cost-effectiveness will never be met. Indeed, considering studies using either a Markov-state-transition or patient-level simulation model, ICURs for different types of ERT ranged from €432 540 (Gaucher disease) to €3 282 252 (Fabry disease). Wyatt et al. calculated the number of QALYs to be generated each year in order for different types of ERT to be cost-effective which resulted in the impossible amount of 3.6-17.9 QALYs [22]. Consequently, most ultra-orphan drugs are not ‘cost-effective’ if standard health technology assessment procedures were to be applied to them, which was already stated in a previous paper by Drummond et al. [32]. It is highly unlikely that the ICUR could be decreased by the orders of magnitude required to make ultra-orphan drugs cost-efficient by current standards in the future. Although there is no clear consensus on what constitutes an acceptable cost-utility ratio, in the Netherlands a value of €80 000 per QALY for illnesses associated with a considerable burden has been suggested. However, so far, no drugs have been withdrawn from being reimbursed because of their unfavourable ICUR in the Netherlands. The National Institute for Health and Clinical Excellence (NICE) in England and Wales uses a threshold ICUR of £20 000 - £30 000 per QALY for non-orphan drugs [33]. The value of ultra-orphan drugs is assessed in a ‘Highly specialised technology (HST)’ programme, which is similar to the Single Technology Appraisal process [34]. The ICUR of £20 000 - £30 000 per QALY is not strictly adhered to when evaluating orphan drugs, and greater emphasis is applied to other factors such as ethics and lack of alternative treatments. In other countries in the EU (such as Ireland) there is no threshold ICUR value [35]. The lack of consensus on the threshold of the ICUR and absence of consequences for reimbursement in some countries may imply that cost-utility is not the only principle that has to be taken into account when assessing the value of orphan drugs. From a societal perspective, other (ethical) considerations may matter, such as the rarity and seriousness of the disease, the availability of alternative therapies and the cost to the patient if the drug would not be reimbursed [36]. These may be reasons to accept higher costs per QALY. Possibilities to deal with this are to apply variable cost-utility thresholds with a higher threshold for orphan drugs, or to value the health gain in a patient with a rare disease with a higher weight [5]. It is questionable, however, whether the QALY in itself is a suitable outcome measure in the case of ultra-orphan diseases. In most studies the EQ-5D (EuroQoL five dimensions) quality of life questionnaire is used to determine health status. Each observed health score profile on this questionnaire can be converted to a utility score ranging from - 0.594 (i.e., serious health problems with mobility, self-care, usual activities, pain/discomfort, and mood) to 1 (i.e., no problems at all). Patients with a slowly progressive, chronic disease will not be cured from one day to the other. In other words, the utility score will mostly not increase to perfect quality of life upon therapy. Instead, patients may feel less tired and better capable of working which is less well represented in the EQ-5D questionnaire. Consequently, the use of the QALY in its current form for the assessment of cost-utility of ultra-orphan drugs has been constantly discussed (see for example: http://​www.​rarediseaseblogs​.​net/​author/​cees-smit/​). Others have argued that the QALY measure is even invalid [37]. Beresniak et al. asked more than 1300 subjects to express their preferences regarding combinations of different health states and time durations, which were subsequently compared to the results of the QALY formula (quality of life x number of life-years). They showed that observed and calculated utility values were significantly different. This might explain why costs/QALY estimates vary greatly. The authors argued that other approaches for health care decisions should be considered. Indeed, the costs/QALY approach has been rejected by the USA and Germany for ethical and methodological reasons [38, 39]. However, to remove an imperfect measure without replacing it with another might be not advisable [40].

Aside from using more suitable, representative ways to determine health status, bringing down costs would also lead to more acceptable ICERs/ICURs. The extremely high prices charged for new orphan drugs may cause an unsustainable pricing market that threatens the health care structure [41].

Altogether, economic evaluations of ultra-orphan drugs are feasible if pharmacoeconomic modelling is used. The most suitable type of model seems to be a Markov-state-transition model. It should be realised, however, that most ultra-orphan drugs will not meet conventional criteria for cost-effectiveness. Still, ultra-orphan drugs are often reimbursed. Further discussion on the application of economic evaluations and their consequences in case of ultra-orphan drugs is therefore called for. The challenges described in this study may also apply to the broader group of (non-ultra) orphan drugs. However, this needs further study.