Design, analysis, and presentation of crossover trials

Because they reduce bias associated with imbalance in known and unknown confounding variables, randomized clinical trials (RCTs) represent the 'gold standard' for evaluating therapeutic effectiveness.[1] Unlike the parallel group trial, crossover trials provide each participant with two or more sequential treatments in a random order usually separated by a washout period [2]. Within a trial, each participant is able to act as his or her own control and permits between and within group comparisons [3, 4].

For the study of new and developmental drugs, crossover studies are extremely popular [4, 5], particularly when the new treatment may only be a slight modification to the standard. In this case, there is likely to be a positive correlation in the responses to the new and old treatments making the crossover design ideal [6]. Crossover studies are most appropriate in studies where the effects of the treatment(s) are short-lived and reversible and are best suited to trials related to symptomatic but chronic conditions or diseases [3, 7]. It is generally agreed that the crossover design should not be used when the condition of interest is unstable and may change regardless of interventions [3]. In spite of criticism [8], however, the crossover design appears to be used commonly in inappropriate circumstances [3, 9].

Despite their popularity, little is known about the quality or prevalence of randomized crossover trials. We aimed to review key methodological issues in the reporting of these trials in a representative sample of published trials.

We found that important design issues are often under-reported in randomized crossover trials. Given their popularity – representing almost a quarter of trials published in December 2000 [10] – few reported important methodological issues such as allocation concealment, issues of carryover effects, and within-participant effects. Transparency and interpretation can be improved by creating standard reporting guidelines for authors and journals reporting the cross-over trial design. As yet the CONSORT reporting guidelines [12] have not been extended specifically for crossover trials.

There are several important strengths and limitations to be considered in our analysis. Strengths include our rigorous searching of PubMed during the study period, ensuring that adequate time had passed to allow all potential trials to be filed on the database. We extracted data in duplicate to reduce abstraction errors and resolved discrepancies by consensus. There are also limitations to consider. We searched only PubMed, the largest and most accessed database of medical articles. Other databases may have included additional articles. While every randomized trial published in December 2000 was read and appraised, it is possible that we missed some trials originally designed as crossover trials that were reported as parallel trials, reporting on only the first or second period of the trial. The methodological issues that we examined are a matter of debate. While evidence of bias exists for methodological issues such as blinding, sequence generation and allocation concealment [26], such evidence is lacking for other details such as flow diagrams, patient preference, and importantly, carryover effects. It is possible that if we had identified other methodological issues, we would have found different results. However, we developed these criteria based on studies in which we have participated and widespread consensus on methodological criteria, as reported in the CONSORT Statements [27]. Our data abstraction focused on prespecified criteria. During peer-review, a reviewer noted the important issue of differing analysis issues according to whether the main outcome measure in a trial is continuous, categorical, ordinal or binary, issues we had not considered. Finally, our analysis is based on the assumption that the reporting of methods and results in a published article reflects what was actually done. It is possible that some authors did conduct the methodological item, but failed to report it [28].

The crossover design has numerous advantages that investigators may wish to use for early stage trials. The particular strength of this design is that the interventions under investigation are evaluated within the same patients and so eliminates between-subject variability [4]. Further, this trial design permits opportunities of head-to-head trials and patients receiving multiple treatments can express preferences for or against particular treatments.

However, even when properly applied, crossover trials may have certain weaknesses. Patients may drop out after the first intervention period and thus not receive a second or third treatment. This makes within-subject comparison impossible [3] and is particularly important if withdrawal is related to side-effects [2, 7]. This further complicates the concept of intent-to-treat analysis as patients randomized may complete the first period, but randomization typically does not occur at the second period. Also, there may be a residual [5] or carry-over of effect of treatments across study periods, which could potentially distort the results obtained during the second treatment or subsequent periods [7, 29], although examples of this are few [30]. Thus, the observed treatment effects will depend upon the order in which they were received.

Some have argued against consistent testing for carryover effects of interventions across periods as carry-over effects are rare and statistical manipulation after the fact cannot address the impact of a carry-over effect.[30] Senn, in particular, has argued for a common sense approach to crossover trials, where no carry-over is assumed and thus, not tested for.[31] He specifically argues that tests for carry-over are generally underpowered even with an appreciable carry-over effect. He recommends instead that the wash-out period between periods be sufficient to prevent carryover effects. This paper does not aim to solve this issue, but rather displays the incongruence across crossover trials on the issue of carry-over and other design issues.

Another major potential threat to the validity of the crossover design involves the use of inappropriate statistical analysis [2]. Given that subjects act as their own controls, the analyses could be based on paired data (using an unpaired test) [5, 6] and the within-subject variability in outcomes could be considered in sample size calculations [32]. Essentially, the use of a paired design is much more efficient than a parallel group design when researchers expect a high correlation between patients' responses to the different treatments.

We found large heterogeneity in the reporting of crossover trials, possibly reflecting a lack of standards within the field. There is a clear need for minimum standards for transparent reporting of crossover trials.

Features used to assess reporting of methodological details in published crossover studies