Step 2: Framework
2(a) Assumptions In this subsection, we recommend describing the study layout, the statistical model, the types of bias and the metric to measure these biases in detail. Usually, there are many design aspects which have to be taken into account, such as the result of the sample size calculation including the defined effect size, the desired allocation ratio, the type of endpoint, the layout of the study, the number of treatment arms, stratification factors, and the number and the timing of interim inspections.
2(b) Options In this section, we propose to specify the randomization procedures under evaluation while taking into account their parameterization and specific properties. A comprehensive review of the randomization procedures is given in Rosenberger and Lachin (2016) [
2], so we do not repeat the details of their computation here.
In order to choose the randomization procedure which best mitigates bias, we recommend including a variety of procedures in the evaluation, covering the whole spectrum of available procedures. We identify three partly overlapping classes of randomization procedures that arise from different types of restrictions imposed on the randomization process. We now introduce the classes proceeding from the weakest to the strongest restrictions.
We start with the class of randomization procedures where weakest restrictions are imposed.
Complete randomization is within this class and is characterized by unrestricted treatment assignments without any control of the imbalance neither during nor at the end of the trial. The procedure is accomplished by tossing a fair coin, so the probability that patient
i will receive treatment
E is always 1/2, and may be considered as the “gold standard" with respect to unpredictability. Most clinical researchers avoid complete randomization because it can lead to large imbalances on the number of patients on each treatment either at the end or during the course of the trial, especially in small samples. Another candidate is
Efron’s biased coin design [
22] (EBC(
p)) which consists of flipping a biased coin with probability
p≥0.5 in favor of the treatment which has been allocated less frequently, and a fair coin in case of equality. Note that this class includes complete randomization (CR) when
p=0.5. With Efron’s biased coin more unbalanced allocation sequences become less probable. The third candidate in this group is
Wei’s urn design [
24] (UD(
α,
β), where
α and
β are user specified nonnegative integer parameters. The procedure tends to balance treatment assignments by adaptively modifying the next allocation probabilities based on the current degree of imbalance. It can be regarded as an adaptively biased coin design.
One restriction implemented in randomization procedures is to control the imbalance during the treatment assignment process. Randomization procedures which ensure that the difference in the number of treatment assignments does not exceed a certain value either exact or by probability during the allocation are designed to control a given
maximum tolerated imbalance [
23]. A procedure which controls the imbalance strictly is the
big stick design [
25] (BSD(
a)), which can be implemented via complete randomization with a forced deterministic assignment when a maximal imbalance
a is reached during the enrollment. Another candidate related to Efron’s biased coin is
Chen’s design (Chen(
a),
p) [
26], where a maximum tolerated imbalance is applied to Efron’ biased coin. A broader class of designs results from the
accelerated biased coin design [
27]. The
maximal procedure of Berger (MP(
a)) is another candidate, which, in the most recent version, controls the maximal tolerated imbalance, but does not force balance at the end of the allocation process [
28].
The next type of restriction is characterized by controlling the total imbalance after completion of the assignment process. Randomization procedures which ensure that the difference in the number of treatment assignments does not exceed a certain value at the end of the allocation process control the final imbalance. One candidate is Random allocation rule (RAR), which assigns half the patients to E and C randomly. Permuted block randomization (PBR(b)) with block size b uses RAR within blocks of b patients, and therefore controls the maximum tolerated imbalance as well as terminal balance.
For the evaluation and comparison of randomization procedures, some candidates are natural choices, such as CR which is considered gold standard for unpredictability, and RAR and PBR for a strict control of the imbalance during and at the end of a trial. Note that the permuted block design is the most frequently used procedure [
28]. It is the investigator’s decision which procedures to include in the comparison study. However, due to the different properties we strongly recommend including at least one representative from each class in the evaluation study. For small trials, the use of complete randomization is not suitable as it does not control any imbalances, and can therefore lead to a loss in power. For example, for total sample size
N=50, the probability for an imbalance of 25% that leads to a loss in power of 5% (reduction from 80 to 75%) is larger than 3%.
2(c) Metric The application of the ERDO requires a suitable metric for the target criterion that reflects the objective of the evaluation. A large number of different metrics have been defined in the literature, such as the expected number of correct guesses [
11], the loss in power [
2], or the balancing behavior [
29]. Less work has been done to combine the different metrics. For instance, Atkinson [
29] investigated the loss in power by imbalance and the impact of bias by the the average number of correct guesses. Schindler [
30] proposes a unified linked assessment criterion to combine various standardized metrics. In the case study, we considered the value of the new metric
P
RP
(
ω≤0.05) as well as for comparative reasons the mean type I error probability.
Step 3. Evaluation Method
For this step, we recommend a concise description of the method used for the evaluation of the randomization procedures. Usually it will comprise a comprehensive simulation study rather than analytical results. Different randomization procedures should be considered with varying parameter settings (e.g., different block sizes, in case of the permuted block design or different values of p in Efron’s biased coin design). As mentioned in step 2(b), the set of randomization procedures under evaluation should be large and diverse with respect their properties.
The estimates of the selection bias effect η and the time trend effect θ should be derived from the literature or preceding clinical trials. One should vary η and θ to determine the sensitivity of the comparison to changes in the assumptions. It should be noted that it may be unrealistic to assume no bias in a clinical trial. However more experience through re-analysis of existing data is necessary to derive well-founded estimates for η and θ.
Step 6. Discussion and Conclusion
This step concerns the discussion of the results and their interpretation with particular regard to the trial setting.
The above argument may imply that one should relax the terminal balance requirement by using the big stick design to achieve better results. Indeed, if the investigator is willing to accept imbalance in the data, say by 40 patients, it results an acceptable loss of power. However, in the case of BSD(10) 53% of allocation sequences under selection bias (0.09) and time trend (0.26) still preserve the type I error probability of 0.05. So the maximum tolerated imbalance can be restricted to 10 for BSD.
It should be taken into account that the evaluation above uses small selection and chronological bias effects in the example above. This may be different in other clinical settings. However, the evaluation shows that ignoring the influence of selection bias as well as chronological bias may affect the test decision by means of type I error rate probability. The effect may be conservative or anti-conservative test decisions.