Methodological Challenges in the Economic Evaluation of a Gene Therapy for RPE65-Mediated Inherited Retinal Disease: The Value of Vision

Gene therapies have long been a promise, potentially bringing durable or even curative clinical benefit. Recently, several therapies have started delivering on this promise, although long-term effectiveness remains unclear [1]. Currently available gene therapies are targeted to small patient populations with a specific gene mutation and provided in one single dose with high upfront costs, while the benefits theoretically should last a long time (sometimes even a lifetime). These characteristics have led to a discussion among health economists about whether a specific methodological reference case is required for economic evaluation of gene therapies [2]. Drummond et al. argued that a new methodological reference case for economic evaluation of gene therapies is not required because their characteristics are not unique to gene therapies [2]. However, they emphasized that “the confluence of various characteristics can lead to specific methodological challenges when evaluating the cost-effectiveness of a gene therapy” [2]. Therefore, they proposed a checklist for economic evaluations of gene therapies to inform audiences whether and to what extent key elements affecting gene therapies have been identified and considered in the analyses [2].

The objective of this study was to illustrate the methodological challenges when evaluating a gene therapy and the implications on cost-effectiveness results by discussing the most recent gene therapy for which economic evaluations have been conducted: voretigene neparvovec (VN) for the treatment of RPE65-mediated inherited retinal dystrophy (IRD). VN (brand name Luxturna) is one of the first gene therapies approved by the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA) (in 2017 and 2018, respectively) [3, 4]. VN is an adeno-associated viral vector-based gene therapy for the treatment of patients with vision loss due to IRD caused by mutations in the RPE65 gene [3, 5, 6]. During a surgical procedure called vitrectomy, a healthy copy of the defective RPE65 gene is introduced into the RPE cells in the retina. The healthy copy of the gene enables patients to produce functional RPE65 protein, which may prevent further deterioration of the patient’s visual abilities that would eventually lead to blindness in untreated patients. Four of the first economic evaluations of VN were selected for review in this study. First, the economic evaluation of the US Institute for Clinical and Economic Review [7, 8], who were the first to evaluate the cost effectiveness of VN compared with best supportive care. They reported an incremental cost-effectiveness ratio (ICER) of US$643,813 per QALY from the US health care system perspective [7]. Since then, the marketing authorization holder of VN has submitted health technology assessment (HTA) dossiers to national HTA agencies in several countries using a commissioned model (from now on referred to as the ‘company model’). Two of the first HTA submissions of VN submitted in 2019 (to the ‘Zorginsitituut Nederland’ [ZIN] in the Netherlands and to the ‘National Institute for Care and Health Excellence’ [NICE] in the United Kingdom) will also be reviewed in this study [9‐11]. Using the list price of VN, the ICER in the UK HTA submission from an NHS perspective was £86,635/QALY (US$115,513/QALY) and in the Dutch HTA submission the ICER from a societal perspective was €80,994/QALY (US$95,625/QALY). Finally, an economic evaluation applying a similar model as the company model to the United States’ setting by Johnson et al. (2019) was included in this study [12]. Johnson and colleagues’ results were vastly different from the US Institute for Clinical and Economic Review results; they found an ICER of US$79,618/QALY when applying a healthcare payer perspective. These examples show that there can be substantial differences in the outcomes of economic evaluations of the same gene therapy.

In the following, Drummonds’ checklist for assessing gene therapies will be applied to the four economic evaluations of VN discussed above [2]. Subsequently, the impact of methodological choices on cost-effectiveness outcomes will be explored. Finally, the necessity of special methods to assess the cost effectiveness of gene therapies will be discussed.

Four economic evaluations of VN were evaluated performed by US Institute for Clinical and Economic Review [7, 8], the HTA submission to NICE in the UK [9, 10], the HTA submission to ZIN in the Netherlands [11] and Johnson et al. [12]. The HTA submissions in the UK and the Netherlands both used the company model but made different methodological choices and are therefore reviewed separately. The cost-effectiveness model developed by US Institute for Clinical and Economic Review, the company model (used in the submission in the UK and the Netherlands) and the model used in Johnson et al. had comparable settings. All models used a lifetime time horizon, cycle length of 1 year and starting age of 15 in the base case. However, the US Institute for Clinical and Economic Review model structure differed from the company and Johnson et al. model. The US Institute for Clinical and Economic Review model used two health states (alive and dead). Within the alive state, visual acuity (VA, the ability of the eyes to discriminate detail or ‘clarity of vision’) and visual field (VF, the total area in which objects can be seen while focusing the eyes on a central point) were modelled using an exponential and linear functional form, respectively. Efficacy of VN was based on the change in VA and VF observed in the clinical trial and extrapolated over 10 years followed by a 10-year waning period (Table 1) [5]. In contrast, the company and Johnson et al. models used VA and VF observed in the clinical trial to populate a Markov model with five health states (Fig. 1). The extrapolation of the treatment effect observed in the clinical trial varied between 20 and 70 years in these economic evaluations (Table 1). Furthermore, different list prices, assumptions for the duration and waning of treatment effect and sources for utilities were used in the four economic evaluations. The model characteristics and settings of the four studies are summarized in Table 1. Full descriptions of the economic evaluations are provided elsewhere [7‐9, 11, 12]. The HTA submissions in the UK and the Netherlands were sponsored by Novartis Pharma and the study of Johnson et al. was sponsored by Spark Therapeutics, the economic evaluation of US Institute for Clinical and Economic Review did not have external sponsors.

Table 2 shows the completed checklist for assessing gene therapies [2] for the four economic evaluations performed for VN. Individual items that varied between the four economic evaluations or required more explanation than provided in Table 2 are discussed in more detail below.

Four economic evaluations of VN were discussed in this paper. Properties of the clinical evidence, such as sample size, duration of the clinical trial and the use of surrogate outcomes, required a number of assumptions to model long-term cost-effectiveness. Different reimbursement authorities preferred different assumptions and this resulted in large differences in the cost-effectiveness estimates, mainly due to different assumptions on duration of treatment effect of VN, discount rates, sources of utility values and model structures. However, the difficulty in translating available clinical evidence to long-term cost-effectiveness estimates is not unique to gene therapies. Also, the impact of assumptions can be assessed with standard methods for sensitivity analyses that can also be applied to other types of therapies. Therefore, this study underlines the statement of Drummond et al. that gene therapies do not require a new methodological reference case for economic evaluation. However, as Angelis et al. rightly expressed, gene therapies are “pushing against the boundaries of the methodological and budgetary capacity available” [35]. Our study showed that guidelines on the acceptable extrapolation of treatment effect of gene therapies and on the reporting of results when important input parameters are highly uncertain are necessary to aid HTA research and align submissions of new gene therapies. In addition, alternative payment models, such as MEAs between manufacturers and payers, can reduce the financial risk of the payer associated with the uncertainty of (cost-) effectiveness due to the lack of long-term data at launch.

In the case of VN, uncertainty was mostly related to the extrapolation of the treatment effect. According to Gonçalves, the willingness to consider economic models that include extrapolating longer-term benefit from limited existing data varies between HTA agencies [27]. In the case of VN, both NICE and ZIN did not have objections about the extrapolation of treatment effect proposed by the manufacturer for the base-case analysis, but in the UK, this was set to 40 years, while in the Netherlands a more conservative assumption of 20 years was applied. Uncertainty about the long-term benefits is not unique to VN but will likely be an important question in other gene therapies as well due to the unavoidable lack of long-term data supporting lifetime benefits at market entry [36]. One may even suggest that it is impossible to define a base case based on this limited data, but in practice HTA agencies will probably request or define the most plausible scenario anyway. Therefore, the methodological reference case may not need to be changed for this issue, but it does call for guidance on the acceptable extrapolation of treatment effect and the required evidence base (e.g. duration of trial or expert opinion) to support this assumption. Furthermore, the uncertainty around the assumed duration of treatment effect can be handled in different ways. In the economic evaluations of VN, the uncertainty around the treatment effect duration assumption was addressed by performing extensive scenario analyses assuming different treatment effect durations that can be presented in a table or plot (e.g. Fig. 4 in this paper). This is in line with the new approach of the US Institute for Clinical and Economic Review, who decided to perform ‘optimistic’ and ‘conservative’ scenario analyses in which the clinical benefit is varied in their analyses of high-impact “single and short-term therapies” [37], and recommendations by Angelis et al. [35]. Although these scenario analyses can show the importance of this assumption on the cost-effectiveness results, it is still up to the discretion of the decision maker how this uncertainty is weighted in the final reimbursement decision. One way to potentially improve the plausibility of the treatment effect duration assumption is to include feedback from clinical experts about the expected long-term benefits in a formal manner using the Sheffield Elicitation Framework (SHELF) [38]; for example, like Cope et al. did for CAR-T therapy [39]. Experts could be asked to estimate upper and lower plausible limits and then the most likely duration of the treatment effect of VN based on their clinical experience and knowledge, for example. Based on the uncertainty around the most likely estimates of the experts provided by the upper and lower plausible limits, a probability distribution of this input parameter could be included in the PSA. This way the uncertainty around the treatment effect duration would be formally included in the cost-effectiveness plane and acceptability curve.

Another important issue was the small patient population, which, among other things, complicated collecting utility values. Utilities in the HTA submissions in the UK and the Netherlands as well as Johnson et al. were based on a vignette-based study where utilities were estimated based on expert valuation of health state descriptions given by patients and caregivers [18]. The main critique of this study by HTA agencies was that the resulting utilities are based on proxy utility assessment [9, 11]. Ideally, utility data should have been collected directly from (the parents of) patients using a quality-of-life questionnaire (i.e. HUI or EQ-5D-5L) during the RCT. The main limitation of a direct valuation study would be that it yields data from a small number of patients. However, the transition probabilities used in the health economic model are also based on this small number of patients and the patient-reported utilities could at least have been used as validation of the utilities derived by expert elicitation. In the specific case of vision problems, additional problems arise in using EQ-5D to collect utility data as it does not include a vision component [19]. In contrast, the HUI-3 does include a vision component and indeed showed better performance in patients with vision disorders and the majority of studies on the convergent validity of the HUI-3 demonstrated good validity [19, 20]. Another alternative is using a mapping algorithm to convert VA to EQ-5D-3L published recently by Pennington et al. [40]. However, an important limitation of using this algorithm in the case of VN would be that it is only based on VA and not on the primary outcome of the clinical trial (i.e. MLMT) or any of the other important secondary outcomes (i.e. VF and FST). In contrast to extrapolation of treatment effect, limited quality of available data on HRQoL is not necessarily an issue in all gene therapies. Therefore, this issue also does not call for changing the methodological reference case for gene therapies. The specific (and often small) patient populations with certain gene mutations that gene therapies are often targeted to, however, bring the same challenges as with evaluating interventions for orphan diseases. Therefore, guidance and regulations for rare diseases may also be applicable to gene therapies.

Although sensitivity analyses can provide more insight on the impact of certain uncertain parameters, the capability of these analyses to help deal with the decision uncertainty is limited [2]. In practice, MEAs are often arranged to share the financial risk between the healthcare payer and the manufacturer [41]. This often entails that funding and use of the intervention is restricted to certain centres, and real-world patient outcomes must be recorded [42]. Outcomes-based arrangements have their limitations, mostly related to the difficulty in defining appropriate outcomes and thresholds to define response, and the significant clinical, administrative and patient burden. This has historically limited their implementation and favoured confidential discounts instead [42]. In the case of VN, however, the patient population is limited and durable outcomes that are sensitive to change are well defined [15, 43]. The feasibility of such an agreement has been shown in the US, where the manufacturer of VN (Spark Therapeutics) is offering outcome-based agreements that include rebates to payers if established efficacy goals (in terms of full-field light sensitivity threshold scores against baseline measurements) are not reached at 30–90 days and 30 months [44].

As the first gene therapy is approved by the FDA and EMA for retinal diseases, clinicians are gaining their first experiences with gene therapies that may increasingly become standard care in the future. At the same time, VN also provided a valuable learning experience for health economists by highlighting the challenges in evaluating the cost effectiveness of gene therapies, which can help in improving the economic evaluation of future gene therapies. But in general, this study showed that standard methods can be used for the economic evaluation of gene therapies, supporting Drummond et al. statement. However, due to uncertainty about the (long-term) outcomes of gene therapies, health economists evaluating the same intervention can end up making different methodological choices resulting in large variations in cost-effectiveness results. This study showed that the main challenge in evaluating gene therapies is handling the promise of long-term benefits that are not observed in clinical trials yet. This calls for guidance on the acceptable extrapolation of treatment effect of gene therapies in the base-case analysis and on how to handle the uncertainty around this extrapolation in scenario and sensitivity analyses to aid HTA research and align submissions of future gene therapies. As a single assumption on extrapolation of treatment effect is likely to be incorrect, exploration of the impact of the main assumptions that drive model results in a table or figure as presented in this paper (Fig. 4) is recommended. In addition, alternative payment models can reduce the financial risk caused by uncertainty of long-term outcomes due to lack of long-term data at launch.