Comparison of three tests of homogeneity of odds ratios in multicenter trials with unequal sample sizes within and among centers

We studied the statistical properties of the three mentioned-above homogeneity tests, in terms of empirical power and type I error rate, by simulation in SAS statistical package v 9.1. Our simulation design was based on the mixed logistic model 1; we set the fixed effect parameters for all simulation scenarios arbitrarily as: α = -1 and β = 1 . The random components u _k and b _ik are generated independently from normal distribution with mean zero and variance and . In order to generate the response variable, the binomial distribution with parameters n _ik and was used. The influence of within-center and among-centers inequality, different values of K, , and also number of observations in each centers, n _k , on the statistical properties of all the homogeneity tests were evaluated. We generated 1000 random data sets for each simulation scenario. Namely, we chose three values for K (4, 6 and 8), two values for (0.1 and 0.5) and five values for (0, 0.2, 0.4, 0.6 and 0.8). In the equal sample size design, in which equal numbers of patients were allocated between two treatment arms and also among all centers, the number of individuals per center, n _k , was set at 40, 100 and 200. In addition, in this study two forms of unequal sample size designs were considered. In the first one, ie within-center inequality, the same sample size, which can be allocated equally between two treatment arms within a center, is allocated unequally in the ratio of 3:1. In this case, the sample sizes were considered equal in all centers. In the latter one, ie among-centers inequality, the same total sample size, which can be allocated equally among centers, is allocated unequally. In this situation, one center has a much larger sample size in comparison with the other ones. The exact details of different sample size configurations are described in Table 2. In addition, SAS simulation code for performing the three homogeneity tests is presented in the additional file 1.

Tables 3 shows empirical type I error rate (when ) and power (when ) of the three homogeneity tests under the equal sample size design and various combinations of n _k , K, , and . In terms of power, the tests are ordered BD≥DL≥LR in all cases and they perform similarly when the total sample size in each center is as large as 200 and . In addition, our findings reveal that substantial gain in the power of all homogeneity tests occurred with increasing number of centers, sample size per center and degree of heterogeneity of odds ratios, . However, increasing degree of heterogeneity of center, , from 0.1 to 0.5 resulted in a reduction of approximately 4.93% and 3.76% in the power of DL and BD, respectively, while no regular trend was detected for the change in LR.

Table 4 shows type I error rate and the power of the homogeneity tests under within-center inequality for different values of n _k , K, , and . Indeed, when = 0.1, as compared with the equal sample size design, there were decreases of approximately 16.35%, 9.93% and 11.62% in the power of LR test for within-center inequality, when K was equal to 4, 6 and 8, respectively. In addition, the power of the BD statistic decreased approximately by 10.67%, 7.79% and 7.15% and the power of DL was reduced approximately by 14.46%, 11.24% and 9.76% for K = 4, 6 and 8, respectively. It should be noted that, for a given value of K, the reported amount of reduction was the average reduction for all values of .

Type I error rate and the power of the homogeneity tests under among-centers inequality are illustrated in Table 5. In fact, when = 0.1 as compared with the equal sample size design, there were decreases of approximately 17.15%, 11.78% and 11.89% in the power of LR test and 16.63%, 13.62% and 12.57% in the power of the BD statistic for among-centers inequality, when K was equal to 4, 6 and 8, respectively. In addition, this amount of reduction in the power of DL test was more critical and approximately equal to 21.56%, 18.4% and 16.61% for the same sequence of K.

It should be pointed out that the amount of reduction in the power of the three homogeneity tests for = 0.5 under two forms of unequal sample size designs was approximately similar to those of = 0.1.

Tables 3, 4 and 5, when = 0, represent empirical type I error rate of the three homogeneity tests at the nominal significance level of 0.05. As indicated, LR test showed conservative behavior when K = 4, otherwise, the type I error rate was close to the nominal level. In addition, BD performed adequately in terms of type I error rate in almost all cases. On the other hand, type I error rate of the DL statistic was close or below the nominal level.

Breslow and Day [9] proposed the test statistic:

Where OR _MH is the Mantel-Haenszel estimator of common odds ratio. E(y _1k |OR _MH ) and V(y _1k |OR _MH ) are the expected value and variance of y _1k under the null hypothesis of homogeneity of odds ratios. Under the assumption of large sample size in each 2 × 2 table, BD has approximately chi-square distribution with k - 1 degree of freedom [9].

The Dersimonian and laird statistic is based on random effects model, which obtained by:

When log-odds ratio is used as a summary measure ie, common treatment effect, y _k = ln(OR _k ) where ln(OR _k ) is the log-odds ratio in kth center and where . DL statistic has approximately chi-square distribution with k - 1 degree of freedom [10].

To test homogeneity of odds ratios based on mixed logistic models, we compare the model

to the simpler model not containing the random effect term of the center-by-treatment interaction, namely:

which is equivalent to test the null hypothesis of versus [8]. As the null hypothesis is on the boundary of the parameter space, the asymptotic null distribution for the likelihood ratio test is a mixture of a and a with equal probability 1/2, rather than classical single chi-square distribution [11, 12].

Based on Model * which was presented above, the odds of success in treatment 1 in kth center is derived as:

and in a similar manner the odds of success in treatment 2 is derived as: therefore, the odds ratio for treatment 1 versus 2 in kth center is exp(β+b _1k - b _2k ).

data binomial;

cards;

1 1 30 100

1 2 50 100

2 1 45 100

2 2 75 100

;

run;

proc nlmixed data = binomial qpoints = 15; *Mixed logistic model with one random effect, no interaction;

bounds su>=0;

expz=exp(z);

pi=expz/(1+expz);

model y~binomial(n,pi);

random u~ normal(0,su*su) subject=center;

ods output FitStatistics=test1;

ods listing select test1;

run;

bounds su>0; bounds sb>0;

expz=exp(z);

pi=expz/(1+expz);

model y~binomial(n,pi);

random a b~ normal([0,0],[su*su,0,sb*sb]) subject=center;

ods output FitStatistics=test2;

ods listing select test2;

run;

merge test1(rename=(value=d1)) test2(rename=(value=d2));

if descr='-2 Log Likelihood';

run;

data combmix; set mixed;

delta=d1-d2;

run;

data lr; set combmix;

run;

data rejectmixed;

set lr;

rej1 = 0.5*(1-x);

a=(rej1<0.05);

run;