A systematic literature review of evidence-based clinical practice for rare diseases: what are the perceived and real barriers for improving the evidence and how can they be overcome?

Clinical trials on RDs are characterised by an intrinsic difficulty to identify patients. This problem resides in the difficulty to diagnose RDs, to record diseases, and to trace RD patients [28]. This is due in part to the scarce knowledge about these diseases, and to the fact that far from all countries have efficient processes for referral [29], resulting in significant delays in diagnosis. RD patients often remain undiagnosed even in the best conditions of expertise due to lack of knowledge about natural history or clinical signs and symptoms. However, the most recent technological developments such as lower-cost, next-generation gene sequencing is increasing the diagnostic capacity for monogenic diseases, thereby contributing to increased knowledge of potentially actionable ethiopathogenic mechanisms [30].

In all cases, RDs are poorly represented in medical nomenclatures used in health information systems [28], making it difficult to identify participants for clinical research from medical records. Most countries use the International Classification of Diseases (ICD-10) to record patients, where around 500 RDs have a specific code [31]. In countries using Systematized Nomenclature of Medicine (SNOMED), the situation is not much better because only around 40% of RDs are listed here (Ana Rath, personal communication on the Orphanet-SNOMED CT mapping exercise, August 2015).

Another source for identification of RD patients is disease-specific patient registries. There are 690 such registries in Europe, covering 984 RDs [32]. Most are national (482 registries), or regional (75 registries), with some being European (59 registries) or international (74 registries). However, quality, scope, and capacity of many registries are limited [28].

The geographical dispersion of patients requires multicentric, multinational collaboration, introducing additional regulatory and funding barriers. For severe RDs, travel to research centres may pose an insurmountable barrier to research participation. Solutions include the leveraging of technology to monitor patients remotely, and setting up community centres to better deliver these trials to patients who otherwise would be unable to access them [33]. Effective recruitment is also supported through partnership with patient organisations when they exist, but also with patient registries and centres of expertise.

These barriers hamper recruitment into clinical trials. In Europe, a voluntary policy has been undertaken in order to improve diagnostic rates, i.e. by enhancing the expertise of specialised centres, and to establish European Reference Networks (ERNs) expected to spread expertise and share best practice. ERNs are expected to catalyse the international cooperation and patient engagement needed for clinical research. In parallel, in order to increase the visibility of RD patients in health information systems, a specific standard nomenclature for RDs – the Orphanet nomenclature [34] – is promoted in the European member states [35]. The implementation of the Orphanet nomenclature of RDs (the Orphacode) which is linked to other nomenclatures and resources used both in the clinical setting (ICD-10, SNOMED Clinical Terms) and in the research setting (Online Mendelian Inheritance in Man (OMIM); Human Gene Nomenclature Committee (HGNC); Universal Protein Resource (UniProtKB); among others [31]) will make it possible to more easily identify patients from health records for clinical research. The Orphanet nomenclature should also enable data exploitation with the aim of improving knowledge of the natural history of RDs.

The natural history of most RDs is often difficult to document, yet it is a necessary step to inform to the trial design for the disease. Few relevant epidemiological studies are published due to the difficulty of identifying and documenting patients widely spread geographically, not always diagnosed properly, and rarely followed-up by academic centres in a systematic way. Most attempts to collect good quality data are supported by short-term grants, which do not allow continuity in the effort. The high cost of high-quality natural history studies has been a significant obstacle to their conduct. This is well identified as a barrier requiring solutions and has been the target of recommendations of the EU Committee of Experts on Rare Diseases (Commission Expert Groups on Rare Diseases (CEGRD), formerly EUCERD) [36] and of the International Rare Diseases Research Consortium [37]. The lack of natural history information provides little insight into how to choose outcomes or how to design and power a clinical trial. When disease-specific registries meet quality standards, their relevance for contributing to high-quality clinical trials is demonstrated [38]. Structure and design of natural history studies are pivotal to capture clinical information efficiently in order to be used in safety and efficacy determination. Knowledge of natural history is one of the first crucial steps for building evidence as it allows for a better choice of clinically relevant outcomes as well as of the duration needed to monitor for them to occur [33]. There is a need to capture clinical information more cost-efficiently and to help inform the optimal approach to treatment development. Data collection in the framework of European Reference Networks should be encouraged and facilitated by common interoperability standards and tools to address this issue.

RCTs are the goal standard for producing evidence on the efficacy of an intervention because they have a strong internal validity by minimising bias and confounder factors [1, 39]. Systematic reviews of RCTs provide the highest level of evidence assessing the benefits and harms of interventions [1]. However, randomisation can prove to be difficult with RDs, mostly because of the small size of the patient population.

The European Medicines Agency (EMA), in a guideline on trials in RD populations, stated that there are no methods relevant for small trials that are not also applicable to large studies [40]. The problem for trials in RD populations is that the reverse would lead to requests for sample sizes that are not practicable, or simply impossible to reach.

The traditional RCT designs are difficult to conduct in small populations because it is very difficult to create homogeneous groups and to adequately assess changes between variable groups. Alternative methods have been proposed and could be applicable under certain conditions. We will briefly discuss some of them here. For in-depth analyses and comparisons between some of these different trial designs, see references [39‐57].

The traditional fixed error rates (alpha = 0.05; beta = 0.20) cannot capture all desirable inferences in different clinical research settings. Therefore, Ioannidis and colleagues have developed models that optimise the selection of type I and type II errors according to available sample size and a plausible intervention effect [50].

Controlled rigorous designs that allow within-patient comparisons and treat all participants would assess therapies more accurately if feasible. Such study designs comprise n-of-1 designs and crossover trials. Both assess efficacy of a treatment based on short-term outcomes and mitigate the effects of clinical heterogeneity in a patient population [39, 40].

Pragmatic RCTs could represent an alternative to early phase RCTs while keeping most of their methodological advantages. These pragmatic trials are intended to inform decisions in common practice, so eligibility criteria are more inclusive, comparisons are done against standards of care instead of placebo, and follow-up tends to evaluate longer-term effects than early phase RCTs [39].

Vickers and Scardino and Potter et al. argue for a wide adoption of what they call ‘clinically integrated’ or ‘hybrid design’ RCTs [39, 51]. These trials incorporate aspects of observational and interventional trials (for instance, cohort multiple randomised controlled trials – cmRCTs), thus allowing for a more efficient knowledge transfer into real-world clinical practice. The designs promote longitudinal observational data collection (registries and cohorts). Such investment would be, in our view, the most efficient use of resources in the long term, as it allows for a better understanding of diseases and for the assessment of different interventions over time in a controlled way while knowledge progresses.

Other study designs more focused on proving the efficacy of interventions, more often drugs that are expected to transform the disease course, include adaptive designs. Response-adaptive methods change allocation ratios depending on which treatment appears to be best. Adaptive methods are defined by the EMA as a ‘statistical methodology (that) allows the modification of a design element (e.g. sample size, randomisation ratio, number of treatment arms) at an interim analysis with full control of type I error’ [52]. Adaptive trials are complex and need even stronger measures to prevent biasing adaptive decisions in the course of the trial [53]. Mauer and EORTC collaborators, for instance, point out the fact that regulatory and financial management need to be adaptive as well, so such trials increase the organisational and economical burdens [50]. Adaptive methods rely on real-time data, which may be easier in RD trials because recruitment tends to be slower. Some adaptive designs are now used for rare cancers [53]. Other sequential adaptive methods are proposed for testing different therapeutic possibilities in a small population [54, 55]. Regulators accept or recommend some of these designs [56]. For an in extenso review of the different designs available and their acceptance by regulatory bodies (FDA, EMA) please see Billingham et al. [57].

The RD field needs the development of cost-efficient, novel, rigorous controlled trial designs and relevant analyses that are effective in studying efficacy in heterogeneous, small populations. Recently, the European Commission funded three projects in this area [58‐60]. In addition, the IRDiRC consortium has established a task force to address the question and produce recommendations [61].

As for any other disease, the laws of probability and statistics apply to RDs. Therefore, valid evidence on interventions requires valid clinical research in the form of large, well-conducted RCTs [1, 7, 39, 45].

Patients with most RDs are not so few as to prevent conducting large RCTs. In the EU, a prevalence of 1/100,000 with a RD (i.e. well below the threshold of 5/10,000) results in an availability of 5000 potential trial participants [62]. It requires, however, multinational cooperation, which introduces a new line of barriers in the form of comprehensive organisational, regulatory, and economical requirements. The identification of partners having both the expertise and the capacity to conduct international RCTs, the organisation of the collaboration, and also of the monitoring and follow-up are challenging. The collection and maintenance of high-quality data among all parties involved is a major issue, and specific measures should be put in place to ensure the best, easy-to-use quality. These challenges are greater in RDs, as they often need a multi-disciplinary management team as well as professionals from diverse hospital departments, which makes monitoring and organisation more complex.

Different legal frameworks in different countries contribute to the regulatory barriers of conducting multicentre international RCTs. Heterogeneity can involve all the following: ethics committee submission, patient information and consent, insurance acquisition, activation of the clinical centre, data protection rules, and investigator reimbursement. The need for harmonised procedures has been addressed by setting the Voluntary Harmonisation Procedure (VHP) in the European Union in order to organise the assessment of multinational RCTs, which is the responsibility of the Clinical Trials Facilitation Group of the Heads of Medicines Agencies (HMA) [63]. From 2009 to 2015, 22% of European clinical trials underwent that procedure, the numbers having increased impressively over time [64]. When the European clinical trials regulation is implemented in 2018 [65], the need for VHP is expected to decrease or completely disappear.

Maybe more than in other fields, RDs are often characterised by important clinical variability, including age of onset, severity, speed of evolution, responsiveness to treatment, global impact in health status, and functional consequences. This situation leads to a very large range at baseline for many measures of efficacy, making it hard to detect important changes of an intervention. In fact, traditional RCTs assess average treatment impact in selected patients, and thus do not accommodate clinical heterogeneity very well [39]. Researchers often use surrogate outcomes to measure the effects of an intervention [66]. Such surrogates must be correlated to a clinically meaningful outcome. However, correlation alone does not make a surrogate valid [66]. Intensive analyses linking the intervention effect on the surrogate to patient-centred outcomes are needed [66‐69]. Biomarker development is one source for potential surrogate outcomes. FDA and EMA orphan drug regulations contemplate approval of drugs for which the benefits for patients with unmet medical needs are based on reasonable evidence, often based on surrogate outcomes, that should demonstrate their clinical benefit during post-marketed studies [29]. Such ‘adaptive’ pathways and procedures seem to have their special problems making them less attractive or outright dangerous to patients [21, 70]. Drug or medical device companies could base their marketing authorisation applications on uncontrolled or controlled observational studies rather than pivotal RCTs. Such applications could lead to marketing approval of interventions that are without effect or that are even harmful. Such interventions are difficult or impossible to remove from the market.

However, both patients and decision-makers will seek more patient-centred, clinically relevant benefits [39]. These patient-centred outcomes can be reported by clinicians (clinician-reported outcome measures) or other observers (observer-reported outcome measures), or by the patients themselves (patient-reported outcome measures, PROM) [71].

On the other hand, the frequent complexity of disease manifestations in multiple body systems may require more than one clinical outcome for one domain to adequately assess a clinically effective treatment. That puts extra burden on the statistical assessment of outcomes [72]. As single clinical outcomes may not adequately cover the multiple expression of a disease, novel approaches to combine independent clinical outcomes in multi-domain analyses could potentially help assessing the clinical efficacy of an intervention. However, such analyses are statistically complex, the weight of each clinical variable could not be adequately measured, and results could be difficult to compare from trial to trial [73]. Nevertheless, the development of multiple domain outcome strategies in smaller populations offers important advantages over single primary outcomes [74]. Well-chosen and designed multiple domain outcomes would capture broader therapeutic data, provide greater insight into overall treatment effects, and allow successful small trials with compelling new treatments when the benefit might be varying between individual patients.

The development of agreed standardised sets of outcomes, the core outcome sets (COS) would result in better comparability between clinical studies, by defining the minimum outcomes that should be assessed when evaluating a new intervention. Initiatives like COMET develop both COS and a consensus core outcome sets database in which several RDs are represented [75]. A task force on patient-centred outcome measures have been set up by the IRDiRC, and a first overview and recommendations document has been open to public consultation [67]. The landscape of initiatives on the matter, including those concerning RDs, is depicted [71]. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR)[76] has set up a task force on Rare Disease Trials Clinical Outcome Assessment (COA) measurement [77]. This task force aims to provide recommendations on the development of patient-centred outcome measures (PCOMs) in conformity to the regulatory guidance for the evaluation and proof of treatment benefits for medicinal products approval. The recent recommendations of the ISPOR Pediatric PRO Task Force [78] provide good research practices in developing and implementing paediatric patient-reported outcomes instruments and, therefore, is of interest for RDs, because most of them are paediatric.

Understanding the clinical meaningfulness of clinical changes in a patient is difficult without significant prior clinical experience. A systematic approach using natural history and comparable disease information has to be developed. The construction of the future evidence starts by the collection of natural history data in a systematic way (registries, cohorts) and by the capture of data from clinical records in a structured way. Patient engagement should be sought and encouraged from this early stage.

The design and specific methodological aspects of a clinical trial need to be carefully discussed with all relevant partners. As stated in Potter et al. [39] ‘ideally, evaluative research should incorporate outcomes that are of greatest importance with respect to treatment goals, based on a consensus among patients, clinical providers, researchers, and policy decision-makers’. The final goal is to translate knowledge into clinical practice, based on the best evidence. Doing this requires effective interaction among stakeholders from the earliest phase, i.e. data collection to increase the knowledge of each RD. Thus, databases and registries should incorporate patients’ views. Some experiences exist already in which patients contribute directly to data collection [79].

Usually, a significant proportion of those with an RD must be enrolled in trials to reach the required sample size. The relationship between the clinician and the patient needs to be based on mutual trust for the patients to agree to take part and once in the trial, to stay in and provide outcome data. These data must answer a question that is important for patients, clinicians, and policy-makers, and data must be collected in such a way that taking part in the trial leaves a participant willing to take part in more.

A trial involving a RD population must, in short, be compellingly efficient and involve all stakeholders, especially patients, in its design. The regulators as well should be included in the discussion about the most appropriate design for a specific trial, as early as possible in the research process. Protocol assistance and scientific advice from regulatory bodies have been shown to play a key role in guiding the conduct of studies to address the benefit/risk analysis for marketing authorisation and approval [80]. However, the scientific advice provided by regulatory authorities poses serious concerns about conflict of interests as it is delivered by the very same agency that will grant the marketing approval at a later time point.

Centres with expertise in RDs should play a major role in fostering clinical research networks and infrastructures and in disseminating and sharing study outcomes. Training of investigators and patients’ representatives will ensure a better understanding of regulatory, methodological, and ethical requirements. The development of European reference networks in the coming months offers an opportunity to put this statement into practice. In addition, support from organisations, such as ECRIN, can greatly enhance the organisation and management of multinational clinical trials on RDs. In effect, ECRIN brings together national networks of clinical trial units across Europe, making it possible to accelerate patient recruitment and trial implementation, while ensuring the appropriate management services for smooth trial conduct. By using ECRIN (or a similar infrastructure), there is an opportunity to develop common and harmonised practices for the submission, monitoring and reporting of multicentre and multinational RD clinical trials.