Background
Methods
General model for estimating across-study treatment effect
General model for comparing single treatment against multiple control types
Basic model for ordinal outcome
Extended models for ordinal outcome
Extended model for multi-site RCTs
Extended model for assessing heterogeneity of treatment effect
Criteria for selecting prior distribution assumptions
Goodness-of-fit using posterior predictive checking
Interim monitoring for efficacy
Basic model for binary outcome
Interim monitoring
Results
Evaluating and choosing prior distribution assumptions for the basic model
Versions | 1 | 2 | 3 | final |
---|---|---|---|---|
\(\alpha\) | 0 | 0 | 0 | Normal (\(\mu = 0,\ \sigma = 0.1\)) |
\(\tau _{yk}\) | Normal (\(\mu = 0,\ \sigma = 100\)) | Normal (\(\mu = 0,\ \sigma = 100\)) | Normal (\(\mu = 0,\ \sigma = 100\)) | \(t_{\text {student}} (\mathrm {df} = 3,\ \mu = 0,\ \sigma = 8)\) |
\(\boldsymbol{\beta}\) | Normal (\(\boldsymbol\mu=\mathbf0,\mathrm\Sigma=100^2I_{p\times p}\)) | Normal (\(\boldsymbol\mu=\mathbf0,\mathrm\Sigma=100^2I_{p\times p}\)) | Normal (\(\boldsymbol\mu=\mathbf0,\mathrm\Sigma=100^2I_{p\times p}\)) | Normal (\(\boldsymbol\mu=\mathbf0,\mathrm\Sigma=2.5^2I_{p\times p}\)) |
\(\delta _{k_{c}}\) | Normal (\(\mu = \delta _{c},\ \sigma = \eta\)) | Normal (\(\mu = \delta _{c},\ \sigma = \eta\)) | Normal (\(\mu = \delta _{c},\ \sigma = \eta\)) | Normal (\(\mu = \delta _{c},\ \sigma = \eta\)) |
\(\eta\) | Cauchy (\(\mu = 0,\ \sigma = 100\)) | \(t_{\text {student}} (\mu = 0,\ \sigma = 100)\) | \(t_{\text {student}} (\mu = 0,\ \sigma = 100)\) | \(t_{\text {student}} (\mathrm {df} = 3, \mu = 0,\ \sigma = 0.25)\) |
\(\delta _{c}\) | Normal (\(\mu=-\triangle_{co},\;\sigma=\eta_0\)) | Normal (\(\mu=-\triangle_{co},\;\sigma=\eta_0\)) | Normal (\(\mu=-\triangle_{co},\;\sigma=\eta_0\)) | Normal (\(\mu=-\triangle_{co},\;\sigma=\eta_0\)) |
\(\eta_0\) | Cauchy (\(\mu = 0,\ \sigma = 100\)) | \(t_{\text {student}} (\mu = 0,\ \sigma = 100)\) | \(t_{\text {student}} (\mu = 0,\ \sigma = 100)\) | 0.1 |
\(-\triangle _{co}\) | Normal (\(\mu = 0,\ \sigma = 100\)) | Normal (\(\mu = 0,\ \sigma = 100\)) | Normal (\(\mu = 0,\ \sigma = 0.354\)) | Normal (\(\mu = 0,\ \sigma = 0.354\)) |
Simulation setup - basic model
-
We assumed different effect sizes for the three different control types. The overall effect \(\Delta _{co}\) was set at the simple negative average of the three \(\delta _c\)’s :
-
\(\delta _1 = 0.3\)
-
\(\delta _2 = 0.4\)
-
\(\delta _3 = 0.5\)
-
\(\Delta _{co} = -0.4\)
-
-
The between study and within control type variation was set at \(\sigma = 0.1\)
-
We assumed three RCTs within each control type, with the size of the RCTs being
-
1 large RCT with \(n=150\)
-
2 small RCTs, each with \(n=75\)
-
-
We started with a total sample size 900 as this was our initial aspiration for the COMPILE study
Models Version 1 and 1(a)
Model Version 2
Model Version 3
Final Version
Evaluating and choosing priors for the extended model
Extended model for multi-site RCTs
-
3 control types with effect sizes: \(\delta _1 = 0.3\), \(\delta _2\) = 0.4, \(\delta _3\) = 0.5
-
Between study (within control type) variation \(\sigma = 0.1\)
-
3 RCTs within each control type
-
1 large RCT with \(n=150\): 1 large site with \(n=110\) and 2 small sites with \(n=20\)
-
2 small RCTs, each with \(n=75\): 1 large site with \(n=55\) and 2 small sites with \(n=10\)
-
-
Between sites (within RCT) variation \(\sigma = 0.1\)
Extended model for assessing heterogeneity of treatment effect
Simulation results for goodness-of-fit
Assumption | ||||||
---|---|---|---|---|---|---|
Treatment | CCP | Control | ||||
Test quantity: % subjects | \(T(D^{original})\) | 95% int. for \(T(D^{rep})\) | Bayesian P value | \(T(D^{original})\) | 95% int. for \(T(D^{rep})\) | Bayesian P value |
a Proportional cumulative odds | ||||||
WHO \(\le\) 0 | 5.32 | [2.91, 7.99] | 0.48 | 5.35 | [2.18, 6.80] | 0.14 |
WHO \(\le\) 1 | 14.19 | [8.96, 16.95] | 0.24 | 11.36 | [7.28, 14.56] | 0.34 |
WHO \(\le\) 2 | 21.06 | [14.53, 24.21] | 0.23 | 17.15 | [11.65, 20.63] | 0.32 |
WHO \(\le\) 3 | 27.94 | [21.07, 32.20] | 0.30 | 23.83 | [17.48, 27.91] | 0.29 |
WHO \(\le\) 4 | 35.25 | [29.06, 41.16] | 0.45 | 31.40 | [24.76, 36.17] | 0.34 |
WHO \(\le\) 5 | 46.12 | [40.19, 53.03] | 0.53 | 41.87 | [34.95, 47.57] | 0.42 |
WHO \(\le\) 6 | 58.31 | [53.27, 65.86] | 0.67 | 55.23 | [48.06, 60.68] | 0.40 |
WHO \(\le\) 7 | 66.74 | [60.77, 72.64] | 0.52 | 61.02 | [55.58, 67.96] | 0.61 |
WHO \(\le\) 8 | 82.04 | [75.30, 85.23] | 0.28 | 75.50 | [71.36, 82.04] | 0.68 |
WHO \(\le\) 9 | 92.90 | [88.86, 95.16] | 0.35 | 89.31 | [86.41, 93.69] | 0.74 |
b Non-proportional cumulative odds (Case I) | ||||||
WHO \(\le\) 0 | 7.57 | [2.91, 8.50] | 0.07 | 0.67 | [0.24, 2.66] | 0.88 |
WHO \(\le\) 1 | 15.37 | [8.25, 16.50] | 0.07 | 1.55 | [1.45, 5.08] | 0.95 |
WHO \(\le\) 2 | 24.05 | [16.75, 27.43] | 0.21 | 5.99 | [3.63, 8.96] | 0.54 |
WHO \(\le\) 3 | 30.96 | [24.03, 36.17] | 0.37 | 9.31 | [5.81, 12.59] | 0.42 |
WHO \(\le\) 4 | 40.98 | [35.19, 48.06] | 0.55 | 15.30 | [10.17, 18.89] | 0.30 |
WHO \(\le\) 5 | 48.55 | [41.75, 54.85] | 0.44 | 16.85 | [13.32, 22.76] | 0.65 |
WHO \(\le\) 6 | 61.25 | [53.40, 66.02] | 0.32 | 21.51 | [20.10, 31.23] | 0.93 |
WHO \(\le\) 7 | 72.61 | [69.90, 80.58] | 0.84 | 43.02 | [35.35, 47.70] | 0.32 |
WHO \(\le\) 8 | 80.18 | [76.21, 85.92] | 0.65 | 48.56 | [43.34, 55.93] | 0.63 |
WHO \(\le\) 9 | 88.86 | [84.47, 92.23] | 0.42 | 61.86 | [56.90, 69.01] | 0.64 |
c Non-proportional cumulative odds (Case II) | ||||||
WHO \(\le\) 0 | 9.11 | [0.49, 3.16] | 0.00 | 0.22 | [3.15, 9.20] | 1.00 |
WHO \(\le\) 1 | 15.33 | [2.43, 7.28] | 0.00 | 10.22 | [11.62, 21.07] | 1.00 |
WHO \(\le\) 2 | 23.78 | [9.95, 18.45] | 0.00 | 38.00 | [32.69, 45.04] | 0.61 |
WHO \(\le\) 3 | 33.11 | [24.03, 36.17] | 0.16 | 69.56 | [57.14, 69.25] | 0.02 |
WHO \(\le\) 4 | 38.22 | [33.01, 46.12] | 0.65 | 80.89 | [66.83, 77.97] | 0.00 |
WHO \(\le\) 5 | 48.89 | [46.84, 59.95] | 0.90 | 90.67 | [77.72, 87.17] | 0.00 |
WHO \(\le\) 6 | 60.44 | [58.50, 71.12] | 0.92 | 95.56 | [84.75, 92.25] | 0.00 |
WHO \(\le\) 7 | 75.78 | [74.03, 84.47] | 0.92 | 98.22 | [91.53, 96.85] | 0.00 |
WHO \(\le\) 8 | 82.89 | [81.80, 90.53] | 0.94 | 99.56 | [94.19, 98.31] | 0.00 |
WHO \(\le\) 9 | 92.22 | [91.26, 97.09] | 0.92 | 99.78 | [97.34, 99.76] | 0.01 |
Bayesian stopping rules for efficacy
-
The Bayesian paradigm includes nine data looks at 20%, 33%, 40%, 50%, 60%, 67%, 80%, 90% and 100% of the data. Three sets of control-specific treatment effects as measured by \(\log \mathrm {OR}\) \((\delta _1, \delta _2, \delta _3)\) are considered:
-
(0,0,0), pooled control effect is 0
-
(0.1, 0.2, 0.3), pooled control effect is 0.2
-
(0.4, 0.5, 0.6), pooled control effect is 0.5
-
-
No covariate adjustment in both data generation and analysis