Introduction
Methods
Study management and group composition
Prior work to inform the need for a CONSORT extension
Scoping literature review
Checklist drafting process
What do we consider an adaptive design trial?
A clinical trial design that offers pre-planned opportunities to use accumulating trial data to modify aspects of an ongoing trial while preserving the validity and integrity of that trial
What are the concerns for adaptive design trials?
Drafting of the first extension checklist
The sampling frame for the Delphi surveys
The Delphi process
Number of survey rounds
Scoring system
Delphi round 1
Delphi round 2
Consensus decision-making criteria
Analysis methods
-
Self-selected key stakeholder group (clinical trial user, clinical trialist, or methodologist);
-
Current employment sector (public sector or industry);
-
Self-reported regulatory assessment experience (yes or no); and
-
Primary role in clinical trials research as a statistician (yes or no).
Decision-making process
Feedback-based adaptation process
Consensus meeting and onwards
Results
Response rates across rounds
Characteristics of registered participants and responders
Delphi round 1
Perceptions of proposed items
Open-ended feedback from participants and Steering Committee decisions
-
Open registration of new participants prior to round 2 specifically targeting journal editors to improve their participation;
-
Exclude the rating of unchanged items in round 2 to shorten completion time but include these items in the survey only to gather any qualitative feedback;
-
Terminate the Delphi survey after round 2 because the ratings suggested it was unlikely that additional valuable feedback would be gathered after this stage;
-
Submit an ethics amendment to increase the number of survey reminders sent out to non-responders to six and the completion period by 1 to 2 weeks in order to improve the response rate;
-
Provide general and itemised feedback to responders summarising their feedback and the SC’s response (what you said and what we did/will do);
Delphi round 2
Perceptions of proposed items
Open-ended feedback from participants
Consensus meeting discussions
Main checklist item | Summary of the discussion and advisory decisions and suggestions made |
---|---|
3a Description of the trial design | Queried the need for the modification. However, it was noted that the allocation ratio for some ADs can change over time and therefore needs greater prominence or some alternative language in the E&E document to indicate that the initial allocation ratio can be fixed or change during the course of the trial Decision: 16 (61.5%) voted to drop the modification and keep the original item |
3b Rationale for an AD | Noted the importance of the rationale especially when interacting with key stakeholders particularly at the planning stages. In addition, from a regulatory perspective, a well-explained rationale is important in the evaluation process. However, the need for a standalone item covering the rationale for the AD was questioned, especially given that no justification for fixed sample size designs is required. Some suggested to drop 3b as a standalone item but make it clear in the E&E document that 2a ‘scientific background and explanation of the rationale’ should also be about ‘scientific rationale for choosing an AD’ Decision: 17 (65.4%) voted to drop the standalone item and find a compromise solution |
3c Specification of pre-planned adaptations | Described as the essential part of the trial design. The importance of separating planned adaptations from unplanned changes was highlighted. It was suggested that the new text from 3a could be added to 3c to ensure that all material is adequately covered Decision: 21 (80.8%) voted to keep the new item as standalone |
3d Unplanned changes to the trial design or methods | Importance of covering both planned adaptations and unplanned changes adequately, as this is key to ADs. Decision: 21 (80.8%) voted to keep the modified item as standalone and address ordering issues |
3e Adaptive design properties | Importance of AD properties from a regulatory perspective was highlighted. Noted that statistical and operational properties of the ADs are broad and include sample size. There was a discussion about whether the AD properties should be covered here or under sample size (7a) since the aspects covered by 3e and 7a overlap but not identical. Some suggested this should be under the section heading ‘Sample size and operating characteristics’. Some felt that 3e, 7a, and 7c overlap, so some further work is needed to clarify this. Decision: 13 (50.0%) voted to keep the item and address structural issues |
6a Pre-specified outcomes | Discussion reflected that a trial could be adapted based on an ‘intermediate’ outcome that may or may not be a secondary outcome. The need for ‘clinical rationale’ was questioned and clarification given as it relates to the rationale for selecting an ‘intermediate’ outcome to adapt the trial or help make interim decisions. Some suggested rewording to ensure that pre-specified primary and secondary outcomes, together with additional ‘intermediate’ outcomes are all adequately covered. The complexity of material covered by this item was acknowledged. Decision: 23 (88.5%) voted to keep the modified item and address wording issues |
6b Unplanned changes to outcomes | Mixed views on the necessity of the modification. Some were concerned that this part of studies is often poorly reported. Some concern that modifying the item may obscure the original meaning. Reflecting on the discussion, we failed to clarify that some AD trials can change the outcome in a planned manner reflected under 3c. To retain the intention of the original item, this modification a clarification to capture unintended changes to outcomes (outside the scope of the planned adaptations) Decision: 12 (46.2%) voted to keep the modified item. Steering Committee to review |
7a Sample size | Mixed views on whether the modification was necessary or proportionate. Suggestions that the wording could be simplified or shortened so that content on sample size is not obscured. For example, by adding AD properties, as per discussion of 3e above, modifying section heading, and further details would be better added to the E&E document Decision: 17 (65.4%) voted to keep the modified item and address raised issues |
7b Decision-making criteria to guide trial adaptation | Importance acknowledged and suggestions to simplify the wording and discuss details in the E&E document Decision: 23 (88.5%) voted to keep the modified item and address wording issues |
8c Randomisation updates after trial commencement | Importance reflected in the discussion and suggestions to merge the material of items 8b and 8c Decision: 16 (61.5%) voted to discard this item as a standalone, but keep the content of the suggested new item by merging with item 8b |
11c Confidentiality and minimisation of operational bias | General agreement that the information included was essential Decision: 26 (100.0%) voted to keep the new item as a standalone |
12c Methods for statistical inference; 12d Methods to combine data across stages; 12e Dealing with over-run participants; 12f Methods for dealing with multiple treatment comparisons/outcomes; 12g Prior selection | Extended discussions about whether the material covered by 12c–12g should be addressed via individual checklist items or a merger. Some concern that the use of a long, compound item would not improve the quality of reporting, as authors retreat to the bare minimum to complete the checklist. Cross-referencing the protocol or the statistical analysis plan was suggested to capture the complexity of statistical inferential methods in the AD. Some suggested discussing the potential sub-items in the E&E document. Discussed whether the approach to methods used for futility analyses should be covered here; some suggested rewording 3c such that it also captures methods used to derive statistical information used to adapt a trial. Acknowledged the need to reword the material such that key aspects of the dropped items 12d–12g are reflected in some way Decisions:
25 (96.2%) consensus to keep 12c but address rewording and reflection of dropped items
21 (80.0%) consensus to drop 12d as a standalone item
22 (84.6%) consensus to drop 12e as a standalone item
20 (76.9%) consensus to drop 12f as a standalone item
21 (80.8%) consensus to drop 12g as a standalone item
|
13a Randomised, received intended treatment… | Extended discussions about the definition of ‘intermediate’ and interim outcomes/analysis and need for simplification. Discussion on the meaning of ‘subpopulations’ and its limited applicability to population enrichment designs, which suggests it should be removed and discussed in the E&E document. Mixed views expressed on coverage of reporting and whether this can be differentiated for all adaptations; graphical complexities highlighted for some trial adaptations Decision: 20 (76.9%) consensus to keep the modified item and address rewording |
14a Dates defining recruitment periods | Important to ensure the meaning of the original 14a is not lost with respect to study dates Decision: 14 (53.8%) voted to keep the modified item |
14b Unexpected termination | Decision: 17 (65.4%) voted to drop the modification and keep the original item |
14c Adaptation decisions | Acknowledged that adaptation dates and decisions should be included, but query on whether items 14a and 14c are the correct place and need for rewording. Discussion on the need for implementation resources to help with reporting. Some confusion about details/coverage of item 14c evident in discussions Decision: 12 (46.2%) voted to keep the item. Steering Committee to review |
15a Appropriate baseline data for comparability | The necessity of modifying 15a was queried and the need to reword ‘subpopulation’. Query over whether using baseline is correct, but acknowledged that changing this would require a change to CONSORT 2010. Suggestion to drop extended 15a but include an explanation of the need to present information differently for some AD trials such as population enrichment, with an example in the E&E document Decision: 13 (50.0%) voted to drop an item and consider expanding the explanatory text. Steering Committee to review |
15b Representativeness of patient population | No specific issues raised
Decision: 20 (76.9%) consensus to keep the new item
|
16 Numbers analysed at interim and final analysis | The distinction for AD trials with examples could be adequately covered in the E&E document without the need to modify the original item wording Decision: 19 (73.1%) voted to drop the item and expand the explanatory text |
17a Primary outcome results 17c Suitable representation of interim outcome results | Suggestion that it is unnecessary to modify 17a, but rather ensure that new material is all captured under 17c. Queries about whether CONSORT 2010 already covers the relevant content, and whether additional content is just required for the E&E document (similar to items 15a and 16). Highlighted the importance of understanding important changes relative to the feasibility of covering all adaptation aspects within a standard journal article. Importance of transparency about the location of more detailed analyses was suggested as a compromise—it is more about access to key information. Suggestion that reporting of treatment arms that have been dropped should be mandatory. Decisions:
16 (61.5%) voted to drop item 17a and expand the E&E text
21 (80.8%) consensus to retain 17c but needs rewording
|
20 Limitations, sources of bias, imprecision, and deviations | Questioned the necessity of the modification given that the original item is already broad. Making reporting more stringent for ADs relative to fixed sample size designs was questioned. Clarification of AD-related issues could be given in the E&E document without the need to reword the item Decision: 25 (96.2%) voted to drop the item and expand the explanatory text |
21 Generalisability (external validity and applicability) | The necessity of modification queried on the ground that this is too specific to only a small type of ADs such as population enrichment. Again, clarification of AD-related issues could be given in the E&E document without the need to reword the item Decision: 25 (96.2%) voted to drop the item and expand the explanatory text |
22b Contribution to future-related research | Consensus group appreciated the sentiment of this new item, but the necessity was strongly questioned Decision: 23 (88.5%) voted to drop the item |
24b Intentionally withheld information 24c Statistical analysis plan 24d Simulation protocol and report 24e Data Monitoring Committee Charter 24f Statistical code | Discussed the feasibility and necessity of including all proposed individual items, and whether these could be merged into one larger item covering additional trial information. Some delegates felt that 24b and 24c should be kept as standalone items and merge 24d to 24f under the heading ‘Availability of other trial documents, if available’ Decisions:
17 (65.3%) voted to keep item 24b as a standalone and merge with 24d, 24e, and 24f
14 (53.8%) voted to keep item 24c as a standalone and address merging issues
|
Abstract item | Summary of the discussion and advisory decisions and suggestions |
---|---|
1b Description of trial design | Query about feasibility including detailed AD features in limited word count for abstracts. Debated the use of term ‘adaptive’ in the abstract to help identify these trials; care should be taken as there is a grey area around the classification of some group sequential designs as ADs in some quotas. Some suggested making a distinction between trials where the only adaptation is to stop the whole study and other ADs that must use the term adaptive in the abstract. The E&E could address the scope by highlighting the type of ADs. Decision: 21 (80.8%) consensus to keep the modified item but reword |
1c Clearly defined outcome for this report | Importance of describing adaptive outcome used to aid credibility of results and help with locating AD trials. Replace the term ‘intermediate’ outcomes consistent with earlier discussions Decision: 19 (73.1%) consensus to keep the modified item but reword |
1d Result for each group | Concern expressed about the feasibility of including results for each outcome in the abstract. Discussion around the necessity of including results for primary and intermediate results, particularly where the latter are used as the basis for adaptation decisions. Concerns about confusion in terminology (‘interim’ and ‘intermediate’) Decision: 21 (80.8%) consensus to drop the modified item and keep the original |
1e Adaptive decisions made | Several participants acknowledged the importance of this item but queried coverage of reporting. Helpful for literature searching to identify specific trials. Important to ensure that authors indicate where no changes or adaptations made. Suggestion to cover the checklist earlier before outcomes. Noted results inconsistencies between items 1e and 14c although it is the same item—perhaps due to the confusion highlighted under item 14c Decision: 23 (88.5%) consensus to keep the new item |
1f Conclusions | For consistency with earlier items (21 and 22), the group acknowledged that this item should not be extended Decision: 22 (84.6%) consensus to drop the modified item and keep the original |
Finalisation of the checklist
Section/topic by item no | Abstract extension for adaptive design randomised trials | |
Title and abstract | ||
Trial design | Description of the trial design (for example, parallel, cluster, non-inferiority) | Description of the trial design (for example, parallel, cluster, non-inferiority); include the word ‘adaptive’ in the content or at least as a keyword |
Outcome | Clearly defined primary outcome for this report | [expand E&E text for clarification] |
Adaptation decisions made | Specify what trial adaptation decisions were made in light of the pre-planned decision-making criteria and observed accrued data | |
Section/topic by item no | Main report extension for adaptive design randomised trials | |
Trial design | ||
3b « pre-planned adaptive design features | Type of adaptive design used, with details of the pre-planned trial adaptations and the statistical information informing the adaptations | |
3c « 3b | Important changes to methods after trial commencement (such as eligibility criteria), with reasons | Important changes to the design or methods after trial commencement (such as eligibility criteria) outside the scope of the pre-planned adaptive design features, with reasons |
Outcomes | ||
6a | Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed | Completely define pre-specified primary and secondary outcome measures, including how and when they were assessed. Any other outcome measures used to inform pre-planned adaptations should be described with the rationale |
6b | Any changes to trial outcomes after the trial commenced, with reasons | Any unplanned changes to trial outcomes after the trial commenced, with reasons |
Sample size and operating characteristics | ||
7a | How sample size was determined | How sample size and operating characteristics were determined |
7b | When applicable, explanation of any interim analyses and stopping guidelines | Pre-planned interim decision-making criteria to guide the trial adaptation process; whether decision-making criteria were binding or nonbinding; pre-planned and actual timing and frequency of interim data looks to inform trial adaptations |
Sequence generation | ||
8b | Type of randomisation; details of any restriction (such as blocking and block size) | Type of randomisation; details of any restriction (such as blocking and block size); any changes to the allocation rule after trial adaptation decisions; any pre-planned allocation rule or algorithm to update randomisation with timing and frequency of updates |
Blinding | ||
11c Confidentiality and minimisation of operational bias | Measures to safeguard the confidentiality of interim information and minimise potential operational bias during the trial | |
Statistical methods | ||
12a | Statistical methods used to compare groups for primary and secondary outcomes | Statistical methods used to compare groups for primary and secondary outcomes, and any other outcomes used to make pre-planned adaptations |
12b « Estimation and inference methods | For the implemented adaptive design features, statistical methods used to estimate treatment effects for key endpoints and to make inferences | |
Participant flow (a diagram is strongly recommended) | ||
13a | For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome | For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome and any other outcomes used to inform pre-planned adaptations, if applicable |
Recruitment and adaptations | ||
14a | Dates defining the periods of recruitment and follow-up | Dates defining the periods of recruitment and follow-up, for each group |
14b | Why the trial ended or was stopped | [expand E&E text for clarification] |
14c Adaptation decisions | Specify what trial adaptation decisions were made in light of the pre-planned decision-making criteria and observed accrued data | |
Baseline data | ||
15a « 15 | A table showing baseline demographic and clinical characteristics for each group | [expand E&E text for clarification] |
15b Similarity between stages | Summary of data to enable the assessment of similarity in the trial population between interim stages | |
Numbers analysed | ||
16 | For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups | [expand E&E text for clarification] |
Outcomes and estimation | ||
17a | For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval) | [expand E&E text for clarification] |
17c Interim results | Report interim results used to inform interim decision-making | |
20 Limitations | Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses | [expand E&E text for clarification] |
21 Generalisability | Generalisability (external validity, applicability) of the trial findings | [expand E&E text for clarification] |
Statistical analysis plan and other relevant trial documents | ||
24b | Where the full statistical analysis plan and other relevant trial documents can be accessed |