Background
Methods
Study design and setting
Sample size
Selection of participants
Analysis and reporting
Results
Description of participants
Variable | Scoring | Total |
---|---|---|
(N = 27) | ||
Sex | Male | 17(63 %) |
Female | 10(37 %) | |
Age group (years) | >30 to 35 | 4(15 %) |
>35 to 40 | 2(7 %) | |
>40 to 45 | 8(30 %) | |
>45 to 50 | 4(15 %) | |
>50 to 55 | 4(15 %) | |
>55 | 5(19 %) | |
Academic qualifications | MSc/MA or equivalent | 3(11 %) |
PhD/DPhil/DSc or equivalent | 21(78 %) | |
Other | 3(11 %) | |
Trials experience (years) | 0 to 2 | 1(4 %) |
>2 to 5 | 1(4 %) | |
>5 to 10 | 2(7 %) | |
>10 to 15 | 6(22 %) | |
>15 | 17(63 %) | |
Current employment sector | Private | 4(15 %) |
Publica | 22(81 %) | |
Both private and public | 1(4 %) | |
Mode of interview | Face-to-face | 7(26 %) |
Telephone | 17(63 %) | |
Skype telephone call | 1(4 %) | |
Skype video call | 2(7 %) | |
Location | UK | 23(85 %) |
International | 4(15 %) |
Perceived value of ADs and opportunity for use in confirmatory trials
Perceived advantages of ADs
Thematic area | Characterisation of potential benefits | Some supporting quotations |
---|---|---|
Ethics: patient benefit | Early stopping of trials as soon as there is sufficient evidence to answer the research question(s) means: | ‘It really depends on the type of AD so if you have a GSD (Group Sequential Design) then of course you can stop early for futility or overwhelming effect and this clearly has many ethical and financial advantages. So for futility stopping –if it doesn’t work you can stop early on and the patients don’t get exposed to a drug that does not work or if you have overwhelming effect that is also very positive you can move on with the development of your drug and you don’t have to finish the whole trial’. (QL11 Statistician) |
▪ Minimisation of unnecessary over recruitment of research patients | ||
▪ Acceleration of the evaluation of interventions, approval and commissioning into practice. Thus, patients receive effective interventions quicker | ||
‘…from a patient point of view the sooner that if there is a new intervention that is really effective then we want to get that into NHS (National Health Service) practice, equally if it is dangerous or if there is anything that we shouldn’t be using then we would want to get that out and into guidelines and NHS practice as much as possible’. (QL01 CTU Deputy Director, Proposal Developer) | ||
▪ Minimisation of the exposure of patients to potentially ineffective and/or unsafe interventions | ||
▪ Patients are likely to be allocated to interventions to which they have a higher chance to respond better; vital in critically ill patients | ‘When you certainly have a limited patient pool like orphan disease implications where you know you’re not going to be able to actually recruit sufficient patients for a full Phase 2/3 traditional development programme and we accept and understand that, as regulators and industry, you’re offered appropriate incentives under orphan designation in the EU (European Union) and US (United States). So, there’s undoubtedly a challenge - an opportunity to maximise the best use of patients. I've once actually seen a combined Phase 1/2/3 study, all in one go’. (QL16 Regulator) | |
Efficiency in trial design | ▪ Mitigating the risk of making wrong design assumptions | |
‘One of the things that I always tend to tell people is that ADs will make you address the objectives of interest enabling you to make the right decisions earlier rather than later, for example, the biggest opportunity is stopping poor drugs early, most of our drugs fail, 90 % of the drugs that we start developing in phase 1 never get to the full registration, we should be killing those drugs as early as possible and ADs allow you to do that, whether it is in phase 2 or 3 there is always that opportunity to stop early for futility’. (QL15 Statistician) | ||
▪ Allows simultaneous testing of multiple interventions from a competing list with option for dropping inferior interventions in a single trial rather than multiple series of two-arm trials [46] | ||
▪ Allows the answering of research questions quicker to expedite decision making | ||
‘The main advantages of ADs accrue to funders because funders are not paying for essentially redundant data and ethically I think there is a benefit to patients because clearly we don’t want to be recruiting patients to trials when there is no significant potential of that trial and additional data giving you any new information’. (QL04 Chief Investigator, Vice Chair - Public Funder) | ||
▪ Efficient use of a limited patient pool, particularly in rare or orphan disease implications | ||
▪ Efficient use of a finite pool of clinical chief investigators | ||
Value for money for funders/sponsors | ▪ Avoiding pursuing the lost cause when trials are stopped early for futility | |
▪ Efficient use of available research resources. Stopping trials early means resources are reallocated to other promising or priority areas |
Perceived therapeutic areas of opportunity to use ADs
Perceived types of ADs with potential in confirmatory setting
Public funders’ perceived change in attitudes towards ADs
‘I think generally speaking we are receptive to those ideas (of ADs) and in fact we, at [organisation] have held our own workshops on ADs last year or the year before in order to try and promote more use of ADs providing they are appropriate of course. So I think 10 years ago our attitudes were more towards traditional parallel group, it was a sort of traditional well-known pathway but I think now our modern thinking is that we welcome ADs when appropriate and it is very much for the applicants to make the case for why they want maybe four arms with interim analyses for dropping arms …’ (QL35 Chair - Public Funder)
Regulatory receptiveness and improving awareness and experiences
‘I haven’t got the figures in front of me and I wouldn’t know how to get them but you see a lot more of them at the moment in the scientific advice arena, when people are coming saying ‘this is what we are going to do, what do you think?’ … I get a lot of them starting and not so many of them finished yet’. (QL19 Regulator, regulatory assessment experience)
Cross-disciplinary interest and positive clinical will
‘I guess there are a lot of concerns about them and so that’s perhaps why they’re not taken up so much. But it is interesting to see that there’s a lot more interest in the past few years and so maybe that is changing’. (QL26 Statistician, design and conduct experience)
‘… influential bodies like the FDA are now embracing ADs and there is probably an increasing number of ADs that are being utilised and will come through and report over the next 2/3/4 years …’ (QL21 Chief Investigator, design experience)
‘We definitely have an interest in advancing new methods in the field of sepsis and in particular there is probably room for improving clinical trial design and that is the focus of our group (ADs methods)’. (QL22 Chief Investigator, design experience)
‘I think generally once you have explained it (AD concept), and you have said that it will be a very big expensive trial if we did it fully powered for as long as it would take, but say that it can be broken down to give different options to the funder for shorter time periods, and less cost, then they can see the advantages to it. … if we are happy to do it and design it and write that section up for them they will take it on’. (QL01 CTU Deputy Director, Proposal Developer, design experience)
‘Sometimes you need to sell it to them to get them to see its positives and advantages and in terms of the extra complication it takes to implement them’. (QL07 Statistician, design and conduct experience)
‘… (ADs) makes a lot of sense from my point of view and in terms of optimising the design and feasibility of the study to address the particular research question. I think it is important that statisticians and clinicians discuss thoroughly the options that are available in clinical trial design to agree the best proposal because each will have particular insights with regard to how to address a research question and so communication is really essential’. (QL24 Chief Investigator, design experience)
‘…there is a lot of interest in them from a funder’s point of view, in that particularly difficulties in recruitment, when it has taken a long time to recruit for trials when recruitment is not up to its expected levels, it is very helpful to be able to have a design that allows you to have multiple looks at the data and to potentially stop early’. (QL04 Chief Investigator, Vice Chair - Public Funder, design experience)
Perceived potential barriers to the use of ADs in confirmatory trials
Cross-disciplinary lack of awareness and understanding
Confusion over what is meant by an AD
‘I would say, over the last three years, I’ve become aware of (the) detail of ADs. Prior to that, it was a sort of loosely bandied term … I could be in a room and everybody thinks they’re talking about the same thing and they’re talking about very different things’. (QL8 CTU Director, no experience)
‘So I am generally in favour (of ADs); however, convincing the community of that takes some work, so a big threat for ADs is just that it’s a cutesy word that means different things to different people, there’s misinformation about it and there are some existing biases in the community and so there really needs to be a lot of education’. (QL22 Chief Investigator, design experience)
Cross-disciplinary conservatism
Stakeholder | Secondary theme associated with conservatism | Contributors linked to secondary theme |
---|---|---|
Cross-disciplinary | ▪ Unfamiliarity and lack of understanding | |
▪ Fear of introducing operational bias during conduct and compromising the trial | ||
▪ Concern about the robustness of ADs in decision-making | ▪ Fear of making wrong decisions | |
▪ Concerns about premature early stopping of trials | ||
▪ Concern that the research community may struggle or be reluctant to accept the findings from an adaptive trial | ||
▪ Contrived general perception by journal editors and reviewers that early trial stopping is a failure | ||
▪ Impact of early trial stopping on other secondary but important objectives | ||
▪ Research teams being more comfortable with traditional fixed designs than ADs | ▪ Sticking to what we know best and fear of venturing into the unknown | |
▪ Lack of knowledge and experience | ||
▪ Generation effect - more senior trialists being sceptical of change from what they know best and perceive as standard | ||
▪ Perceived operational and statistical complexities during planning and implementation | ||
Regulators | ▪ Buy in reluctance in confirmatory setting | ▪ Lack of understanding of the inferential and regulatory price to pay by using an AD |
▪ Fear of lowering the level of evidence | ||
▪ Fear of making wrong decisions that may taint their reputation in the future (for instance, approving a drug that will subsequently be proved to be unsafe or ineffective) | ||
▪ Limited experiences in the assessment and approval of ADs | ||
Statisticians | ▪ Negative attitude towards ADs among some influential statistical community | ▪ Generation effect - more senior researchers being sceptical of change from what they know best and perceive as standard |
Private and public funders | ▪ Reluctant to fund potential high risk high value research projects with huge uncertainty | ▪ Uncertainty around the actual sample size, duration and actual cost of the trial |
▪ Inadequate description of variable costs, decision-making criteria and time frames on grant applications (public funders) | ||
▪ Limited commissioning and funding experiences, especially among public funders | ||
▪ Difficulties in drawing up flexible employment contracts (public funders) | ||
▪ Limited number of AD grant proposals being submitted by researchers for consideration (public funders) | ||
▪ Negative attitudes towards ADs among some public funding panel members | ▪ Lack of familiarity | |
IDMC and TSC members | ▪ Perceived negative attitudes towards multiple examinations of the trial data | ▪ Lack of familiarity and understanding |
▪ Reluctant to stop trials early unless for safety reasons |
‘… people should be cautious I guess in trying to do too much and having too many adaptations … We must still make sure we have that body of confirmatory evidence, so I think there might be a place in phase 3 for ADs, but only sort of minimal adaptations. We should sort of keep things under control in that particular setting …’ (QL19 Statistician, regulatory assessment experience)
‘In terms of the multi-arm trials I’m much more comfortable now with the idea of maybe setting out, even on a phase 3 trial, with 4 or 5 potential interventions and dropping the ones that look least promising’. (QL14 Statistician, no experience)
Lack of knowledge and experience
‘… the main challenge … I think it is a bit broader - is the lack of experience and knowledge within the bio-statistics community. There is a lack of understanding of adaptive methods, a lack of understanding of the opportunities, you know and a lack of familiarity’. (QL12 Clinical Research Leader, Trial Methodologist, design and conduct experience)
Degree of statistical and operational complexity
Amount of work and effort required and marketing of ADs to key stakeholders
Statistical simulation work at the design stage
Robust data management infrastructure
Confidentiality and implications of ADs on IDMC duties and responsibilities
Additional statistical considerations
Concerns around trial credibility and integrity
‘I think it (ADs) will always raise an element of suspicion if there have been some decisions made along the way that have been data driven. And the key thing is just to have all the documentation in place; it has to be set out precisely in the protocol how it will be done and you need the right mechanisms in terms of the monitoring committee or steering committee makes the decision and make sure you comply with all the mechanisms. I mean it’s like GCP (Good Clinical Practice); it’s not enough to do the right thing, you’ve actually got to be able to prove you’ve done the right thing… with adaptive trials it’s that much harder to prove that you’ve done it legitimately. So you’ve got to be very careful about the process and got to be able to demonstrate through documentation that you have followed true process’. (QL14 Statistician, no experience)
Concerns around trial validity
Public sector perspective
Worry about impact of ADs on research staff employment contracts
‘… Because of the size of the trials unit there are many trials that are taking place so we look very closely at people’s contracts and what studies are taking place, it’s not just based on 1 study. We have a lot of different trials at the trials unit so the infrastructure allows for -if the trial stops early then they would be able to work on another trial. So it is not driven by the fact that the contracts or by whether or not it would stop early on this particular trial because of the other trials taking place requiring statistical, trial management, data management support’. (QL27 Statistician, design and conduct)
Lack of capacity within UK CTUs and time limitation
‘One is just the lack of expertise within the unit, so it is easier when you are very busy to put forward a design you know rather than one you don’t. It is also easier because if you put forward a design that does not look the same to clinicians who expect straightforward designs you have to be very confident in that design to be able to convince them to some extent’. (QL9 CTU Director, Statistician, design experience)
Lack of bridge funding for UK CTUs to support planning
‘I think for some of the really complex ADs it would be good if there was availability to go for some small trial development grants so that you could say ‘look this is a convincing clinical question, we think it should be approximately this sort of design but actually we need £20,000 or whatever to properly work it up and design it’ and that type of trial development grant I think would help unlock some of that’. (QL09 CTU Director, Statistician, design experience)
‘Typically for complex ADs then you have to do quite a lot of modelling -that could take 12 or 18 months. Ideally, there should be grants to cover that early development work. Yes, I have sympathy to the idea that there needs to be additional funding but on the other hand I suppose all work that CTUs do prior to a trial application is done at risk. When I was CTU director, typically you are talking about 2 years work before you applied to do a definitive trial. I could say there ought to be more grants to help with all of that and the reality is that we in [organisation] in a sense do pay that upfront because we have a scheme whereby we support CTUs, we give them £250,000 per year if you like, like a front loaded loan, which they use to buy core staff in order to develop new projects. So in a way I think we are doing it already’. (QL25 Chair - Public Funder)
Limitations of the grant application process
‘From a practical point of view when you are designing adaptive trials there is more work involved for the application in planning the trial and working out the timelines … you have to do it for a number of different scenarios. So the work involved in that is more from the trialist and statistician’s point of view, the statistician has to do various modelling and look at different scenarios and we have to do all of the different planning and you are usually on a fairly tight deadline for applications because of the way that NIHR funding works so if you only have 6 weeks to work with the team, trying to fit in time to do lots of different scenarios can be quite tricky and can make it more difficult’. (QL01 CTU Deputy Director, Proposal Developer, design experience)
‘There is not an existing section in grant submissions that says ‘if you are doing an adaptive trial design please provide the following information’, so I just don’t know that it’s well organised yet and that could be a good thing or a bad thing …’ (QL22 Chief Investigator, design experience)