Background
Methods
Upstrap algorithm
Simulation settings
Method validation
Method calibration
Method application
Results
Performance of upstrapping across sample sizes and information fractions
Sequential monitoring results
Null scenario | Alternative scenario | ||||
---|---|---|---|---|---|
Method | N | Difference in TIE from fixed sample | Ratio of ESS to fixed sample | Difference in power from fixed sample | Ratio of ESS to fixed sample |
AU | 40 | −0.003 | 0.58 | −0.052 | 0.90 |
AU | 160 | −0.008 | 0.56 | −0.072 | 0.86 |
AU | 600 | −0.005 | 0.61 | −0.064 | 0.89 |
AU | 2000 | −0.005 | 0.62 | −0.066 | 0.91 |
CU | 40 | −0.011 | 0.48 | −0.122 | 0.80 |
CU | 160 | −0.009 | 0.46 | −0.135 | 0.80 |
CU | 600 | −0.013 | 0.45 | −0.147 | 0.79 |
CU | 2000 | −0.015 | 0.45 | −0.183 | 0.79 |
GU | 40 | −0.002 | 0.60 | 0.001 | 0.90 |
GU | 160 | 0.002 | 0.61 | −0.032 | 0.91 |
GU | 600 | −0.004 | 0.59 | −0.076 | 0.89 |
GU | 2000 | −0.008 | 0.59 | −0.083 | 0.90 |
OBF | 40 | −0.002 | 0.62 | −0.001 | 0.90 |
OBF | 160 | 0.004 | 0.64 | −0.017 | 0.92 |
OBF | 600 | 0.004 | 0.66 | −0.035 | 0.93 |
OBF | 2000 | −0.005 | 0.67 | −0.026 | 0.94 |
PO | 40 | −0.010 | 0.42 | −0.126 | 0.75 |
PO | 160 | 0.000 | 0.49 | −0.093 | 0.82 |
PO | 600 | 0.000 | 0.53 | −0.097 | 0.84 |
PO | 2000 | −0.001 | 0.54 | −0.081 | 0.87 |
CP 1% | 40 | −0.013 | 0.52 | −0.008 | 0.98 |
CP 1% | 160 | −0.021 | 0.50 | −0.026 | 0.92 |
CP 1% | 600 | −0.012 | 0.50 | −0.033 | 0.93 |
CP 1% | 2000 | −0.019 | 0.49 | −0.047 | 0.93 |
CP 5% | 40 | −0.021 | 0.42 | −0.056 | 0.90 |
CP 5% | 160 | −0.026 | 0.39 | −0.098 | 0.83 |
CP 5% | 600 | −0.016 | 0.38 | −0.117 | 0.83 |
CP 5% | 2000 | −0.025 | 0.39 | −0.122 | 0.85 |
CP 10% | 40 | −0.027 | 0.38 | −0.126 | 0.85 |
CP 10% | 160 | −0.028 | 0.34 | −0.128 | 0.80 |
CP 10% | 600 | −0.022 | 0.34 | −0.164 | 0.78 |
CP 10% | 2000 | −0.031 | 0.34 | −0.203 | 0.78 |
CP 20% | 40 | −0.029 | 0.32 | −0.219 | 0.72 |
CP 20% | 160 | −0.033 | 0.31 | −0.211 | 0.72 |
CP 20% | 600 | −0.024 | 0.31 | −0.275 | 0.69 |
CP 20% | 2000 | −0.035 | 0.31 | −0.296 | 0.71 |
Main results
Results for under- and over-powered scenarios
Case study: TREAT NOW data application
Method | Stage | Calculated futility metric | Reference futility threshold | Futility decision |
---|---|---|---|---|
AU | 25% | P = 0.515 | P < 0.05 | Continue |
AU | 50% | P = 0.065 | P < 0.05 | Continue |
AU | 75% | P = 0.091 | P < 0.05 | Continue |
CU | 25% | P = 0.549 | P < 0.20 | Continue |
CU | 50% | P = 0.040 | P < 0.20 | Stop |
CU | 75% | P = 0.046 | P < 0.20 | Stop |
GU | 25% | NA | NA | Continue |
GU | 50% | P = 0.691 | P < 0.90 | Stop |
GU | 75% | P = 0.501 | P < 0.80 | Stop |
OBF | 25% | NA | NA | Continue |
OBF | 50% | Z = 0.566 | Z < 0.543 | Continue |
OBF | 75% | Z = 1.074 | Z < 1.301 | Stop |
PO | 25% | Z = 0.858 | Z < 0.280 | Continue |
PO | 50% | Z = 0.566 | Z < 0.711 | Stop |
PO | 75% | Z = 1.074 | Z < 1.319 | Stop |
CP 1% | 25% | CP = 0.194 | CP < 1% | Continue |
CP 1% | 50% | CP = 0.143 | CP < 1% | Continue |
CP 1% | 75% | CP = 0.446 | CP < 1% | Continue |
CP 5% | 25% | CP = 0.194 | CP < 5% | Continue |
CP 5% | 50% | CP = 0.143 | CP < 5% | Continue |
CP 5% | 75% | CP = 0.446 | CP < 5% | Continue |
CP 10% | 25% | CP = 0.194 | CP < 10% | Continue |
CP 10% | 50% | CP = 0.143 | CP < 10% | Continue |
CP 10% | 75% | CP = 0.446 | CP < 10% | Continue |
CP 20% | 25% | CP = 0.194 | CP < 20% | Stop |
CP 20% | 50% | CP = 0.143 | CP < 20% | Stop |
CP 20% | 75% | CP = 0.446 | CP < 20% | Continue |