Assessing potential sources of clustering in individually randomised trials

In the context of a RCT, we define clustering as when observations are grouped together based upon common attributes. This includes standard examples of clustering such as multicentre trials (where patients are grouped together within centres), crossover trials (where observations are grouped within patients), and trials where the intervention is a type of surgery or therapy and patients are grouped together by surgeon or therapist.

Our definition of clustering also includes some non-standard situations (for example when patients are grouped according to baseline factors, for example age and sex, and then randomised within these strata). Although this type of scenario would not generally be regarded as clustering in the typical sense, it affects the analysis in exactly the same way as standard clustering, and therefore we have included it in our definition for completeness.

Clustering can also be defined as either pre- or post-randomisation, and determining whether clustering is non-ignorable will depend on when it occurs. Pre-randomisation clustering occurs when patients are grouped into clusters and then randomised, for example when patients present to different centres and are randomised upon presentation. Post-randomisation clustering occurs when patients are randomised and then assigned to clusters, for example when they are randomised to a type of surgery and then assigned to a specific surgeon.

It should be noted that whether clustering is considered to be pre- or post-randomisation largely depends on the timing of the randomisation. For example, if patients are randomised and then are assigned to a therapist, therapist would be considered post-randomisation clustering. However, if therapist is used as a stratification factor in the randomisation, and patients are assigned to a therapist and then randomised, therapist would be considered pre-randomisation clustering.

Based on results from Parzen et al. [7], it can be shown under what circumstances clustering will be non-ignorable. In the presence of clustering, the true variance of the treatment effect for a continuous outcome can be written as:

where β ^ is the treatment effect, V ₀ is the usual (asymptotic) variance of the treatment effect when clustering is not present, and V _E is an additional factor based on the clustering, which can be either positive or negative. For further mathematical details of this expression, please see Parzen et al. [7] (it should be noted that our notation differs slightly to theirs).

When V _E =0, the estimate of variance ignoring clustering will be unbiased, and clustering will be ignorable. However, when V _E ≠0 the estimate of variance ignoring clustering will be either biased upwards (if V _E <0) or downwards (if V _E >0). When V _E ≠0 the clustering is therefore non-ignorable, and needs to be accounted for in the analysis in order to obtain valid results.

V _E is a function of the correlation between outcomes for patients in the same cluster (generally referred to as the intraclass correlation coefficient or ICC), and the correlation between treatment assignments for patients in the same cluster. If either of these correlations are 0, V _E will also be 0 and the clustering will not need to be included in the analysis in order to obtain valid type I error rates (although it still may be preferable to adjust for this type of clustering as it could increase power or precision). However, if both of these correlations are non-zero then V _E ≠0, and the clustering will be non-ignorable, and must be accounted for in the analysis. Parzen et al. showed similar results for binary and time-to-event outcomes [7].

Assuming that the ICC is positive, then V _E >0 when the correlation between treatment assignments is positive (leading to the SE for treatment being biased downwards), and V _E <0 when the correlation between treatment assignments is negative (leading to the SE being biased upwards).

Correlation between patient outcomes within a cluster may occur for two primary reasons. The first is that patients with similar characteristics may be more likely to present to the same cluster (e.g. patients with similar socio-economic status may be share the same hospital). The second possibility is that the clusters themselves exert some influence on outcome (e.g. patients within a certain hospital may be more likely to have a positive outcome due to different processes of care or quality of hospital staff).

Treatment assignments between patients in the same cluster will be correlated if patients in certain clusters are more likely to be in a certain treatment group [7]. A simple example of this is a cluster randomised trial, in which all patients in a cluster receive the same treatment. If we know the treatment group of one patient, we then know the treatment group of all patients in that cluster, leading to a correlation between treatment assignments of 1. Conversely, in a 2×2 crossover trial where each patient receives one treatment in the first period and the other treatment in the second period, if we know which of the two treatments they received in the first period than we also know which treatment they received in the second period, indicating the correlation between treatment assignments is -1. Stratified permuted blocks within clusters leads to negative correlation between treatment assignments (for each patient assigned to a specific treatment, it makes it less likely that future patients will be assigned to that same treatment). The exact correlation for stratified permuted blocks is - 1 n - 1, where n is the block size. This indicates that the correlation will always be between -1 and 0. Simple randomisation (where all patients are randomised independently) leads to a correlation of 0.

As discussed in the previous section, non-ignorable clustering occurs when both the ICC and the correlation between treatment assignments within a cluster is non-zero. However, if either the ICC or the correlation between treatment assignments within a cluster is 0, the clustering is ignorable, and valid SEs and type I error rates can be obtained regardless of whether the clustering is accounted for in the analysis (although not adjusting for clustering may lead to a loss of power).

It is important to identify any sources of non-ignorable clustering during the planning stages of the trial to ensure they can be adequately adjusted for in the analysis. In order to determine whether clustering is non-ignorable, we first need to determine whether the ICC and the correlation between treatment assignments is non-zero.

The ICC will not generally be known prior to the trial commencing (unless previous data is available). It is possible to estimate the ICC based on the trial data, however this type of data-dependent model selection has been shown to give poor results, and could potentially inflate the type I error [8]. Therefore, in order to avoid erroneously excluding non-ignorable clustering from the analysis, we suggest that the ICC should be assumed to be non-zero unless there is evidence to the contrary or strong reasons for suspecting it is 0.

Determining whether the correlation between patient assignments within clusters is non-zero depends on whether the clustering is pre or post-randomisation. If it is pre-randomisation clustering (i.e. patients are grouped into clusters and then randomised) then this correlation will be non-zero if the clustering is used in the randomisation process. Examples of clusters being used in the randomisation process include cluster randomised trials (where the clusters themselves are randomised), randomisation that balances on patient factors (such as recruiting centre or baseline prognostic factors), or any trials where patient outcomes are measured and analysed at several time points (as happens in crossover trials and longitudinal studies). If the cluster is not used in the randomisation process then the clustering is ignorable, and type I error rates will be correct regardless of whether the cluster is accounted for in the analysis. For example, an analysis of a multicentre trial ignoring centre effects will give unbiased SEs and correct type I error rates, provided that centre was not balanced on in the randomisation.

For post-randomisation clustering, the correlation between treatment assignments will be 0 if patients in both treatment groups have an equal chance of being assigned to the same clusters (for example, if ward nurses are responsible for the care of patients, but have an equal chance of treating patients from each treatment arm). If the treatment groups are assigned to clusters with different probabilities, then the correlation will be non-zero. Examples of this include therapy or surgery trials when therapists and surgeons only treat patients in one arm. If therapists or surgeons treat patients in both arms, but are more likely to treat patients from a specific arm, this will still result in correlation between treatment assignments within clusters.

We use the FASTER trial (Function After Spinal Treatment: Exercise and Rehabilitation) [9] as a case study for assessing sources of clustering in a real RCT. FASTER was a 2×2 factorial trial designed to assess the impact of a series of rehabilitation classes or an educational booklet on outcomes following back surgery. Although two treatments were assessed in this trial, we focus here on rehabilitation. The primary outcome was the Oswestry disability index, which assesses how much impact a patient’s back pain has on their functional ability. Table 1 shows the structure of the clustering for FASTER.

338 patients were recruited from seven centres, and operations were performed by one of 23 surgeons involved in the trial. The majority of surgeons performed operations across multiple centres. Randomisation was stratified by operating surgeon and the type of surgery (discectomy or decompression), but not by recruiting centre.

Six weeks after randomisation, patients assigned to the rehabilitation group were expected to attend rehabilitation classes for six weeks, with two classes per week. Patients generally attended classes at the centre where the surgery was performed, but could choose to attend at another centre if they wished.

During the planning stages of the trial, the following sources of clustering would need to be assessed to determine whether they are non-ignorable:

We performed a simulation study to assess the impact of clustering on study results. Simulations imitated a trial of therapeutic intervention, where patients were randomised, and then assigned to a therapist (post-randomisation clustering).

Data were generated from the following model (which can be used to describe both pre- and post-randomisation clustering):

where y _ij is a continuous outcome for patient i from therapist j, α is an intercept, β _treat is the treatment effect and x _ij a binary variable indicating which treatment arm the patient was in, u _j the therapist effect for therapist j, and e _ij a random error term for patient i from therapist j. We generated e _ij from a normal distribution with mean 0 and variance 1, and u _j from a normal distribution with mean 0, and σ ² (where σ ² was set to give the desired ICC). We generated e _ij and u _j independently. β _treat was set to 0 for all simulations. It should be noted that the choice of variance for e _ij is arbitrary and has no effect on results.

We varied the following parameters:

We used 5000 replications for each of the 36 simulated scenarios. For each scenario we performed two analyses; the first adjusted for therapist effects, whereas the second did not. For each analysis method we assessed the type I error rate. Unadjusted analyses were performed using a linear regression model with the treatment assignment as the only covariate. When therapists treated patients in both arms (regardless of whether they treated both arms equally), adjusted analyses were performed using a linear regression model with treatment as a covariate, and therapist included as a fixed effect, using indicator variables.

When therapists treated only patients in one treatment arm, adjusted analyses were performed using cluster level summaries [2]; briefly, this involves calculating the mean outcome for each therapist, and fitting a linear regression model with these summaries as the outcome, and which treatment arm the therapist saw as a covariate. We used cluster-level summaries rather than a mixed-effects model in this scenario as there is evidence that mixed-effects models may not perform well in scenarios when clustering occurs within treatment arms, and there is a small number of clusters [2].

Simulation results are shown in Figures 1, 2, 3 and 4. Results were similar across all cluster and sample size combinations. When clustering was ignorable (that is, when either the ICC was 0, or therapists treated patients from both arms equally), type I error rates were correct regardless of whether an adjusted or unadjusted analysis was used (mean 5.1 and 5.0% for unadjusted and adjusted analyses respectively).

When clustering was non-ignorable (ICC > 0 and patients were assigned to therapists with different probabilities depending on the treatment group they were in), unadjusted analyses led to inflated type I error rates. This was most pronounced when therapists only treated patients in one arm (mean 13.6 and 21.5% for ICC values of 0.05 and 0.10 respectively). Even when therapists treated patients in both arms, but were more likely to treat patients in one arm, type I error rates were inflated (mean 7.8 and 11.1% for ICC values of 0.05 and 0.10 respectively).

Conversely, adjusted analyses gave correct type I error rates in all scenarios with non-ignorable clustering (mean 5.1%, range across 24 scenarios 4.1 to 5.6%).