Adaptive designs in clinical trials: from scientific advice to marketing authorisation to the European Medicine Agency

In recent years, increasing experience on the application of adaptive designs in confirmatory clinical trials has accumulated. Their popularity is owed in part to their flexibility to adapt the design based on information arising while the trial progresses but also to their potential to reduce costs and the number of patients enrolled. However, the additional planning and operational complexity, including the potential impact of interim analysis on the trial integrity and on the type I error rate¹ and other operating characteristics (such as power and impact on estimates [1, 2], have been perceived as drawbacks [3‐10]. In order to provide regulatory guidance, the European Medicines Agency (EMA) released the “Reflection paper on methodological issues in confirmatory clinical trials planned with an adaptive design” in 2007 [11] and co-organised two workshops on ”adaptive design in confirmatory trials” with the European Federation of Pharmaceutical Industries and Associations (EFPIA) in 2008 and 2009. The US Food and Drug Administration (FDA) also drafted regulatory documents on the same topic in 2010 for medicines and biologics [12] and has recently published another guideline for medical devices [13].

As confirmatory adaptive designs are now routinely used in clinical trials, in addition to regulatory guidance (which focus mainly on planning aspects), reporting guidance in the form of a Consolidated Standards of Reporting Trials (CONSORT) extension is under development [14].

The EMA “Reflection paper on methodological issues in confirmatory clinical trials planned with an adaptive design” (CHMP/EWP/2459/02) [11] defines a study design as adaptive “if the statistical methodology allows the modification of a design element (for example, sample-size, randomization ratio, number of treatment arms) at an interim analysis with full control of the type I error”.

This is the common theme in all three regulatory guidance documents — for a confirmatory (pivotal) clinical trial strict control of the type I error rate is a regulatory pre-requisite for its acceptance. The EMA reflection paper describes the relevant statistical principles but does not propose specific statistical approaches for clinical trials with adaptive elements. A broad range of statistical methods have been developed allowing design modifications at an interim analysis based on interim data without compromising the type I error rate. The first steps towards flexibility were fully pre-specified adaptation rules, e.g. as in group sequential designs [15] or for blinded sample size reassessment (SSR) (e.g. [16‐21]). A milestone for confirmatory adaptive design — almost 30 years ago — was the development of fully adaptive designs, which ensure strict control of type I error rate without fully specifying the adaptation rule [22‐24]. The EMA does not require a full pre-specification of the adaptation rule; however, it explicitly requires a pre-specification of the adaptive interim analysis and the statistical methods to control the type I error accordingly. A full specification of the adaptive rule would be required to explore operating characteristics such as power or bias but not necessarily to demonstrate type I error rate control. In the beginning focus was given on sample size adaptations, which are now well established if properly implemented. Further developments included research on multiple hypotheses testing with selection of hypothesis at interim; e.g. see [25, 26]. Initially focus was given on adaptive designs with treatment selection at interim (“adaptive seamless phase II/III designs”) [2, 27‐29]. Recently adaptive enrichment designs allowing subgroup selection have attracted more and more interest [30‐35], which is also related to the advances made in personalised medicines and targeted therapies. Challenging from a methodological point are still adaptive trials based on time-to-event endpoints; e.g. see [36‐38]. For a more detailed review on the development of confirmatory trials with adaptive design over the last 25 years, we refer to [3] and its references.

In parallel to methodological developments, sponsors of clinical trials have regularly sought scientific advice (SA) from the EMA to obtain regulatory feedback on adaptive design proposals. A paper published in 2014 by Elsäßer et al. [39] reports on a review of SA letters issued between 1 January 2007 and 8 May 2012 at the EMA discussing medicines whose development was planned using an adaptive trial. Fifty-nine procedures corresponding mainly to phase II and/or III trials were identified and the objective, design and statistical analysis of the corresponding trials discussed. More information about the general aspects of SA procedures can be found on the EMA website (http://​www.​ema.​europa.​eu/​ema/​index.​jsp?​curl=​pages/​regulation/​general/​general_​content_​000049.​jsp&​mid=​WC0b01ac05800229​b9) and in the original survey of Elsäßer et al. [39].

The aim of our article is to follow up these trials and to investigate whether they have actually been conducted and if they concluded. We also retrieved the marketing authorisation application (MAA) supported by the results of the completed trials. Our motivation was to investigate if these trials were still planned using adaptive elements as initially proposed at the time of SA and to evaluate the impact of the adaptive elements on the trial success. Furthermore, we analysed the time elapsed between SA and trial start. Some regulatory perspectives as regards the implementation of adaptive designs in confirmatory clinical trials are proposed before we finish with a discussion and a conclusion.

Numerous descriptive statistics about the sample of 59 products discussed during SA can be found in the original survey of Elsäßer et al. [39] and are therefore not reported again in this paper. Of note, all the trials were sponsored by pharmaceutical companies.

In this section we discuss the most frequent regulatory comments on the use of adaptive elements raised by the CHMP during SA or the assessment of the centralised procedure, as illustrated with the four medicines having reached MAA. Some items are not specific to adaptive designs but can apply to standard group sequential designs as well. In the context of drug regulation, the wording “applicants” is often used in this section, referring, e.g. to the body submitting an MAA or requesting an SA procedure to the EMA. This word could easily by substituted by “researcher” or “trialist”, referring to the person designing the protocol and the methodology of a clinical trial.

In our sample more than half (58%) of the retrieved trials were reported in the different trial registries as planned with adaptive elements after the SA letter was given. Since the planning of adaptive elements is not systematically recorded in the AdisInsight database or in the related trial registries, this figure must thus be considered as a lower bound. Trial registries, indeed, fail to offer convenient reporting options of the trial design elements, which makes it difficult to disentangle trials for which adaptations were modified or abandoned after the SA letter from underreporting. Dose selection, SSR and SFF are now part of the regulatory landscape and were planned with variable levels of complexity in this survey. On the contrary, less standard adaptations such as changing the primary analysis, population enrichment and Bayesian adaptive randomisation are still less often encountered. Of note, the latter is generally not encouraged, because if by random chance the first patients enrolled in the placebo arm have a high response, most of the subsequent patients would also be likely to be randomised in the placebo arm. In the second case study discussed in this paper, the controversial use of this methodology was mitigated by the fact that there were several other phase III trials submitted in the package.

The findings elicited in this article have to be tempered by the still limited number of trials, under-reporting and missing information about trial design in the public trial databases. Overall it was a challenging exercise to track down the 59 original adaptive trials discussed by the EMA Scientific Advice Working Party (SAWP) since sponsors are not obliged to notify EMA when a trial starts or to report its outcome if the related medicine has not been submitted for approval. Moreover, for retrieved trials the statistical significance of the treatment effect on the primary endpoint was not always reported in AdisInsight or related trial registries. Also, when the corresponding p value was reported, it was in general unclear if it was accounting for the multiplicity due to the adaptations. In addition, reporting on the intended use of adaptive elements is not mandated by public databases, and there is no field available for sponsors to record adaptive elements of the design. In a survey of the adaptive designs registered in ClinicalTrials.gov, other authors recently highlighted the inappropriateness of this database to report adaptive characteristics included in some trials (e.g. SSR, stopping rules) and thus to retrieve the adaptive designs [60]. In the 31 retrieved trials the median time to trial start after SA was 5 months. In order to put this figure in context and compare it to trials which were planned without adaptations, it would be interesting to plan a prospective dedicated case-control study based on SA letter submissions aiming at comparing time to trial start between adaptive designs and non-adaptive ones. Of note, this analysis would have to be adjusted for confounders such as therapeutic area for example.

The main regulatory checkpoints for adaptive designs include the justification for choosing complex adaptive rather than standard designs, the control of the type I error rate, data integrity and results consistency across stages of the adaptive design. Others, including a recent retrospective survey of adaptive designs from the US FDA, encouraged similar practices [4, 61]: the rationale of planning an adaptive trial rather than a simpler design, thorough explanation of the adaptations planned (with detailed statistical analysis plan), control of the type I error rate and minimisation of operational bias to maintain data integrity during interim analysis.

Although only a limited number of trials could be retrieved for this analysis, we confirm that properly pre-planned adaptive designs are not a barrier to regulatory approval. Provided that the adaptive elements are properly planned, justified, conducted and reported with diligence, adaptive trials can be considered by sponsors to support the demonstration of benefit/risk balance [60]. Type I error rate control, trial integrity and results consistency should especially be discussed thoroughly, and it is particularly important to provide reassurance that the integrity of the trial has not been damaged. Adaptive designs can be beneficial to the trial success and interim analyses. They can also enable sponsors to stop a disappointing trial for toxicity or futility, but it is definitely clear that a commitment to higher investments at the planning stage is needed. As also recommended by the US FDA [61], early dialogue with regulators through SA is necessary so that potential issues can be solved before the start of the trial, and in order to facilitate approval later.