Randomized controlled trials and real-world evidence for market access and surveillance of high-risk products—The example of paclitaxel

In recent years the legal framework of the European Union (EU) has been subject to far-reaching reforms, which have also had an impact on care for patients suffering from cardiovascular diseases as well as research in this field. Pursuant to the EU General Data Protection Regulation (EU GDPR), European Directive 93/42/EEC (Medical Devices Directive, MDD) will be replaced by European Regulation 2017/745 (Medical Device Regulation, MDR) on 26 May 2021. Following a 3-year transition phase the MDR will constitute applicable law and can then theoretically be subjected to judicial review without national implementation via the German Medical Devices Act (Medizinproduktegesetz, MPG) or supplementary regulations. Existing certification for medical devices pursuant to MDD will remain valid for 5 years (from the date of issue). The Conformité Européenne (CE) marking has been central to the market regulation of medical devices in Europe for around 35 years. Manufacturers indicate with this marking that their products comply with the requirements applicable in the European Economic Area (EEA) trade area. In recent decades, critics of the CE marking have found fault with the excessively liberal market access and lack of surveillance over the entire life cycle of medical devices, especially high-risk medical devices (class III, e.g. implants). Catheters, stents and other cardiovascular implants are included in this category, which means that countless overlaps exist between the highest risk class and the fields of surgical interventional vascular medicine, radiology and angiology.

An interesting example illustrating the importance of market access and the surveillance of high-risk medical devices is the use of paclitaxel-coated stents and balloons to treat atherosclerotic peripheral vascular diseases. A controversial international debate on the safety of medical devices has raged since the end of 2018 when a meta-analysis of cumulative findings in randomized controlled trials (RCT) reported an association between the use of paclitaxel-coated products in the femoropopliteal artery and increased all-cause mortality after 2 years [18, 25]. Findings published in journals and presentations had to be corrected following the publication and the recruitment for ongoing studies was temporarily suspended. Several authorities including the German Federal Institute for Drugs and Medical Devices (Bundesinstitut für Arzneimittel und Medizinprodukte, BfArM) and the US Food and Drug Administration (FDA) issued corresponding warnings for the use of affected medical devices [2, 44]. The findings of the first meta-analysis by Katsanos et al. were gradually confirmed in investigations by the FDA as well as in further meta-analyses that used patient-level data along with additional data [32]. Meanwhile, an inverse association was observed in a number of real-world studies that used nonrandomized registry data and routinely collected data [5, 7, 20, 37]. The two sides have responded with countless arguments in articles and at congresses without a conclusive outcome being reached [4]. A second meta-analysis by the same authors on the use of paclitaxel-coated balloons in infrapopliteal arteries rekindled this debate [19]. The findings confirmed the initial indications of higher complication and amputation rates previously suggested in the IN.PACT DEEP randomized trial [42].

This review examines the market introduction and rapid spread of paclitaxel-coated stents and balloons in vascular medicine. It draws on the divergent findings from randomized and nonrandomized studies to debate the strengths and limitations of complementary study designs for market access and surveillance.

Around 130 interventional paclitaxel studies were conducted between 2001 and 2020. These included 45 on specific stents or balloons, a total of 58,617 patients and substantial input from 6 manufacturers. Of the studies 21 are still underway.

The following clinical trials were the key to the development of the current market situation and the two meta-analyses by Katsanos et al.: THUNDER [39], ZILVER PTX™ [2, 11] and IN.PACT SFA [36] for treatment on the femoropopliteal artery, and LUTONIX BTK, [28] BIOLUX PII, [43] DEBATE-BTK, [24] ACOART II/BTK, IN.PACT DEEP [42] and SINGA-PACLI for vascular lesions below the knee.

Following the discovery and isolation of the paclitaxel (Taxol) active ingredient from the bark of the Pacific yew tree in the 1960s and the preliminary description of its anticarcinogenic effects in the late 1970s, market access was granted for its use in coronary arteries in the early 2000s [25, 41]. The rationale for this was the inhibition of cytoskeleton metabolism by paclitaxel, which was already used in the treatment of countless types of malignant tumors. Due to the superiority of sirolimus-based medical devices, paclitaxel failed to prevail in the cardiological sector, with just a handful of exceptions [16, 23, 31, 34, 40]. While paclitaxel belongs to the group of cytostatic drugs, sirolimus (rapamycin) is an immunosuppressant (mammalian target of rapamycin, mTOR inhibitor) [27]. Various working groups and manufacturers have since investigated the effect in peripheral arteries of the lower extremities and developed corresponding products to market maturity. Since 2009, around 15 paclitaxel-coated balloons and 2 stents have been granted market access in Europe for the treatment of peripheral arteries. Almost 3 years later, 3 balloons and 1 stent were also approved in the USA for use in peripheral arteries [44]. Following their market introduction and dissemination in Europe and the USA, countless successive guidelines have included recommendations or considerations (Table 1).

The preliminary indications of improved patency and reintervention rates following the use of paclitaxel-coated stents and balloons are discussed in the German S3 guidelines on the diagnosis, treatment and aftercare of peripheral arterial occlusive disease (PAOD) published by the German Association of the Scientific Medical Societies in Germany (Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften e. V., AWMF) in 2015 [3]. The authors emphasise that the evidence is insufficient, especially with respect to patient-relevant outcomes (e.g. walking distance, morbidity, mortality, quality of life, leg preservation). Therefore, it was not possible to assess the clinical value of coated stents sufficiently. Hence only a weak recommendation was issued for coated balloons in the femoropopliteal artery if the rate of restenosis or reintervention rate is considered significant (consensus recommendation). There was not enough evidence to recommend usage in infrapopliteal arteries [3].

The American College of Cardiology/American Heart Association (ACC/AHA) published guidelines in 2016 [15]. Although the authors discuss a potential benefit of coated medical devices in patients with intermittent claudication and critical limb ischemia, they do not expand on the effectiveness and efficiency of specific endovascular techniques. Rather, the guidelines also emphasize that no patient-oriented outcomes are available [15]. The guidelines of the European Society for Cardiology/European Society for Vascular Surgery (ESC/ESVS) published in 2017 review the available study data, which suggests better long-term patency when using coated devices in femoropopliteal arteries. In the corresponding (weak) recommendations, it is suggested that coated balloons or stents should therefore be considered for short (<25 cm) lesions of femoropopliteal arteries and coated balloons considered for in-stent restenosis (recommendation class IIB). To date, no study has found coated balloons to be superior to uncoated balloons in below the knee treatment [1].

In 2018, the Society for Cardiac Angiography and Interventions (SCAI) published consensus guidelines in which the importance of specific endovascular techniques, including paclitaxel-coated balloons and stents, is described. These guidelines contain a strong recommendation for use in femoropopliteal arteries based on high-quality evidence (recommendation class I) [13].

Two further guidelines containing recommendations or statements on the use of paclitaxel-coated balloons and stents in the lower extremities have been published since the first meta-analysis by Katsanos et al. [18]. In the global vascular guidelines (GVG) published in 2019 on the treatment of patients with critical limb ischemia, the three participating professional societies (European Society for Vascular Surgery, Society for Vascular Surgery, World Federation of Vascular Societies) included a statement on the safety of paclitaxel-coated medical devices in response to the international discussion. The authors emphasized the importance of further studies and stated that caution should be exercised when using coated balloons and stents outside of clinical trials [10].

Most recently, the 2019 guidelines published by the European Society for Vascular Medicine (ESVM) take up the ongoing discussion on the meta-analysis by Katsanos et al. [18]. Based on a secondary analysis of patient-level data from the LEVANT trial [29] and one of several available observational studies with routinely collected data, the authors concluded that the treatment of femoropopliteal lesions with coated balloons can be recommended as a standard of care (recommendation class II) [14].

Countless RCTs and meta-analyses thereof can be drawn on in the international debate and as the basis for guideline recommendations on the use of coated medical devices in the lower extremities. While these primarily focus on technical outcomes (patency and reintervention rates), various systematic reviews and meta-analyses have been published since the end of 2018 on the long-term safety outcomes.

In 2018 Katsanos et al. included the published trial findings of 28 RCTs and 4663 patients with endovascular treatment of femoropopliteal arteries (approximately 90% with intermittent claudication) in the analyses for the first time. All-cause mortality was significantly higher in the paclitaxel group (vs. uncoated balloons or stents) at 2 years (12 RCTs with 2316 patients; 7.2% vs. 3.8%, risk ratio: 1.68) and at 5 years (3 RCTs with 863 patients; 14.7% vs. 8.1%, risk ratio: 1.93) [18]. In a further meta-analysis of individual patient data, Rocha-Singh et al. were able to examine the long-term data of a total of 2185 patients based on an intention-to-treat analysis and further sensitivity analyses. A total of 386 deaths (17.7%) were registered within the 4‑year follow-up period. The authors identified an absolute increase in all-cause mortality risk of 4.6% in the paclitaxel group; however, they could not find any evidence of a drug dose-mortality association [32]. Schneider et al. performed a patient-level meta-analysis of 2 prospective clinical trials and 2 RCTs on coated balloons with a total of 1980 patients and a 5-year follow-up period. In the analyses stratified by dose, no association could be discerned between paclitaxel and increased all-cause mortality [35].

Katsanos et al. also published a meta-analysis on the use of paclitaxel-coated balloons in arteries below the knee. Data from 8 RCTs encompassing 1420 patients (97% with critical limb ischemia) was included in the analysis. After the 1‑year follow-up period, higher rates of amputations and deaths were observed in the paclitaxel group (13.7% vs. 9.4%; hazard ratio: 1.52), whereby this association was only significant in the group of balloons coated with a high dose [19]. Kuno et al. identified a total of 57 RCTs with 9362 patients. A comparison of seven different revascularization strategies returned no evidence of increased short-term mortality after the use of coated balloons or stents or of increased long-term mortality after the use of coated stents; however, the use of paclitaxel-coated balloons was associated with increased long-term mortality [22]. Dinh et al. also investigated the association between mortality and paclitaxel-coated medical devices in patients with critical limb ischemia. A total of 11 RCTs with 1450 patients were identified, with a mean follow-up period of 25.6 months. The authors found no differences in the short-term and medium-term outcomes [12].

In comprehensive longitudinal analyses of routinely collected data, a significant increase could be observed in the number of coated medical devices since their market access in Germany (2009). In the BARMER cohort, which included almost 10% of insured persons in Germany and enables a patient-level evaluation, just 111 patients had received a coated stent and 138 patients a corresponding balloon by 2009. In 2018, the proportion of coated devices thus increased from 3% for the treatment of critical limb ischemia and 4% for the treatment of intermittent claudication (2010) to 39% and 48%, respectively [5, 17]. A similar situation can be observed in practice patterns in the USA where around 50% of all procedures are today performed using coated stents or balloons [26]. In a Germany-wide propensity score-matched and adjusted analysis of the primary outcomes of 1) overall survival, 2) amputation-free survival and 3) freedom from major cardiovascular events, no evidence was found of increased complications in the femoropopliteal or crural artery in the paclitaxel group (37,914 and 14,738 patients, respectively) after a 5-year follow-up period. Rather, an inverse association was discerned with improved outcomes following the index procedure, which could be confirmed in numerous sensitivity analyses [5, 17].

Comparable findings were discerned in international real-world data. In a weighted analysis of the administrative US Medicare database on 16,560 patients and 1883 hospitals, for which extended long-term data with US OPTUM routinely collected data has since also been presented at the Vascular Interventional Advances (VIVA) conference, the authors could not find any evidence of increased all-cause mortality in the follow-up period (median of 389 days or 2.65 years) [37]. In a further propensity score-matched registry data evaluation by the Society for Vascular Surgery (SVS) Vascular Quality Initiative (VQI) on 8376 patients with endovascular treatment of the femoropopliteal artery, it was discerned after the 1‑year follow-up period that patients with critical limb ischemia had comparable mortality and patients with intermittent claudication had lower mortality [7]. A total of 2071 patients could be analysed in a multicenter registry analysis at 3 hospitals in Great Britain and Greece. After a median follow-up period of 24 months, no higher mortality was observed in the group of patients treated with coated stents or balloons [33]. A retrospective single-center analysis is also available from Germany of balloon-supported angioplasties of the femoropopliteal artery from a high-volume centre. A total of 624 patients with critical limb ischemia were revascularized (68% with a coated balloon). No disadvantages to survival were found in the paclitaxel group over the 3‑year follow-up period [8].

In the midst of a comprehensive reform of the European Medical Device Regulation (MDR) and the related discussions on the long-term safety and surveillance of high-risk products, two independent meta-analyses ignited an intensive and controversial debate among the international vascular medicine community [18, 19]. The reported association between the use of paclitaxel-coated medical devices in femoropopliteal and crural arteries and increased all-cause mortality and amputation rates led to uncertainty among users and authorities alike. This was further exacerbated by the publication of errata and corrections.

The two sides countered in countless scientific journals and presentations with a multitude of arguments for or against an existing safety indication. Indeed, a consensus is still not foreseeable today [4]. The fact is that the association observed between paclitaxel and mortality could also be confirmed in further patient-level analyses, while various independent observational studies of a high methodological quality were able to prove an inverse association [5, 17, 32, 33, 37].

Both researchers working in the medical device industry and users have been left wondering how to assess the discordant studies. A central argument against the Katsanos et al. meta-analyses is the lack of evidence of a drug dose–mortality association [32]. Further arguments concern the selection of patients for study arms or subsequent changes to the study design. Exposure to paclitaxel through its use in other arteries or even in other therapeutic indications is also suggested as a possible cause of the excess mortality, underlining the extreme complexity of the target population. International networks for healthcare research and quality development such as the Medical Device Epidemiology Network (MDEpiNet) and its associated task forces are currently investigating the possible methodological aspects underlying the inverse associations in randomized versus non-randomized study designs. Essentially, it appears that the causal or predictive factors have not been adequately measured. The question therefore remains of the extent to which the cohorts of both study types differ. In the comprehensive sensitivity analyses of a large propensity score-matched routinely collected data analysis of the treatment of arterial lesions below the knee with paclitaxel-coated balloons or stents, it could be proven that the paclitaxel–mortality association is only marginally affected by the use of paclitaxel-coated products in other arteries, by the total case volume, by the postoperative drug supply or by the paclitaxel case volume (Fig. 1; [17]). Against this background, the influence of an observational study’s design is particularly interesting. Minor changes to the inclusion and exclusion criteria, stratification of different subgroups, adjustments and methods to take confounders into account, cause substantial shifts in the findings observed and conclusions reached. It could be shown in previous studies that a gradual approximation to the findings of RCTs can be achieved with growing complexity of the methodology and robustness of the model [4, 9].

The swift spread of new techniques and medical devices means that high-risk patients with complex comorbidities and lesions for whom treatment was not readily accessible in the past can today be treated. The rapid increase in the number of endovascular procedures (+61% between 2010 and 2016) for the treatment of peripheral arterial occlusive disease (PAOD) and the overall rise in the number of treatment cases in Germany (+23%) is impressive proof of this [21]. According to the latest VASCUNET report on the treatment of symptomatic PAOD in which almost 1.2 million treatments from 11 countries were included, Germany has slightly above-average proportions of elective treatments (60%) and endovascular procedures (66%) compared to its international counterparts [6]. That being said, in light of the inhomogeneous and changing guideline recommendations, this development also underlines the need for well-planned RCTs to provide the treating physicians with reliable bases for evidence and recommendation. The ongoing discussion and resulting uncertainty show that this has not been the case to date.

At the same time, although there is no way around sufficiently powered RCTs to prove causalities, observational studies without substantial selection bias remain the method of choice for quality assurance. While well-planned RCTs can adequately eliminate the influence of confounders on the procedure-outcome association, the generalizability and detection of rare occurrences is limited. The Idea Development, Exploration, Assessment, Long-term follow-up (IDEAL) statement for innovation in surgery highlights the complementary function of routinely collected data in the verification of the efficacy of procedures [38]. Against this background, the global paclitaxel debate provides a good illustration of the complementary benefits of different study designs. The extreme complexity of PAOD patient risk profiles and diverse treatments over their entire patient history mean the methodology must be carefully planned. It has been proven that around 25% of procedures performed on PAOD patients were reinterventions. This limits the use of case-related or procedure-related datasets (e.g. data from the Institute for the Hospital Remuneration System (Institut für das Entgeldsystem im Krankenhaus, InEK) and intersectoral quality assurance pursuant to §137a of Book V of the German Social Code (SGB V)) or at least introduces substantial bias. As such, longitudinal research data from social insurance institutions can provide a valuable database to enable valid quality development over an appropriate follow-up period [21, 30].

It remains to be seen to what extent the European Medical Device Regulation (MDR) due to be introduced in May 2021 can improve the quality of the evidence base for the market access and surveillance of medical devices. The significance of high-quality real-world data from registers or routinely collected data has thus explicitly increased. Notified bodies have a particular responsibility in this respect during the review of scientific standards, transparency and data validity.