The online version of this article (doi:10.1186/1471-2288-14-41) contains supplementary material, which is available to authorized users.
Ellen C Lee, Amy L Whitehead, Richard M Jacques and Steven A Julious contributed equally to this work.
The authors declare that they have no competing interests.
All authors contributed equally to the work in this paper. All authors read and approved the final manuscript.
In an evaluation of a new health technology, a pilot trial may be undertaken prior to a trial that makes a definitive assessment of benefit. The objective of pilot studies is to provide sufficient evidence that a larger definitive trial can be undertaken and, at times, to provide a preliminary assessment of benefit.
We describe significance thresholds, confidence intervals and surrogate markers in the context of pilot studies and how Bayesian methods can be used in pilot trials. We use a worked example to illustrate the issues raised.
We show how significance levels other than the traditional 5% should be considered to provide preliminary evidence for efficacy and how estimation and confidence intervals should be the focus to provide an estimated range of possible treatment effects. We also illustrate how Bayesian methods could also assist in the early assessment of a health technology.
We recommend that in pilot trials the focus should be on descriptive statistics and estimation, using confidence intervals, rather than formal hypothesis testing and that confidence intervals other than 95% confidence intervals, such as 85% or 75%, be used for the estimation. The confidence interval should then be interpreted with regards to the minimum clinically important difference. We also recommend that Bayesian methods be used to assist in the interpretation of pilot trials. Surrogate endpoints can also be used in pilot trials but they must reliably predict the overall effect on the clinical outcome.
Wood J, Lambert M: Sample size calculations for trials in health services research. J Health Serv Res Policy. 1999, 4 (4): 226-229. PubMed
Julious SA, Patterson SD: Sample sizes for estimation in clinical research. Pharm Stat. 2004, 3 (3): 213-215. 10.1002/pst.125. CrossRef
Biomarkers Definitions Working Group: Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin Pharmacol Ther. 2001, 69 (3): 89-95. CrossRef
Papadakis S, Aitken D, Gocan S, Riley D, Laplante MA, Bhatnagar-Bost A, Cousineau D, Simpson D, Edjoc R, Pipe AL, Sharma M, Reid RD: A randomised controlled pilot study of standardised counselling and cost-free pharmacotherapy for smoking cessation among stroke and TIA patients. BMJ Open. 2011, 1 (2): e000366- CrossRefPubMedPubMedCentral
Legault C, Jennings JM, Katula JA, Dagenbach D, Gaussoin SA, Sink KM, Rapp SR, Rejeski WJ, Shumaker SA, Espeland MA: Designing clinical trials for assessing the effects of cognitive training and physical activity interventions on cognitive outcomes: the Seniors Health and Activity Research Program Pilot (SHARP-P) study, a randomized controlled trial. BMC Geriatr. 2011, 11: 27-10.1186/1471-2318-11-27. CrossRefPubMedPubMedCentral
Morrell CJ, Walters SJ, Dixon S, Collins KA, Brereton LML, Peters J, Brooker CGD: Cost effectiveness of community leg ulcer clinics: randomised controlled trial. Br Med J. 1998, 316 (7143): 1487-1491. 10.1136/bmj.316.7143.1487. CrossRef
Collins K, Morrell J, Peters J, Walters S, Brooker C, Brereton L: Problems associated with patient satisfaction surveys. Bri J Commun Health Nurs. 2007, 2 (3): 156-163. CrossRef
Carpenter JR, Kenward MG: Multiple Imputation and its Application. 2013, Chichester: Wiley CrossRef
De Gruttola VG, Clax P, DeMets DL, Downing GJ, Ellenberg SS, Friedman L, Gail MH, Prentice R, Wittes J, Zeger SL: Considerations in the evaluation of surrogate endpoints in clinical trials: Summary of a National Institutes of Health Workshop. Control Clin Trials. 2001, 22 (5): 485-502. 10.1016/S0197-2456(01)00153-2. CrossRefPubMed
International Conference on Harmonisation: ICH E9 statistical principals for clinical trials. 1998, http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf,
Temple R: Are surrogate markers adequate to assess cardiovascular disease drugs?. J Am Med Assoc. 1999, 282 (8): 790-795. 10.1001/jama.282.8.790. CrossRef
Julious SA, Machin D, Tan SB: An Introduction to Statistics in Early Phase Trials. 2010, Oxford: Wiley-Blackwell CrossRef
Julious SA, Swank DJ: Moving statistics beyond the individual clinical trial: applying decision science to optimize a clinical development plan. Pharm Stat. 2005, 4 (1): 37-46. 10.1002/pst.149. CrossRef
Chuang-Stein C, Kirby S, French J, Kowalski K, Marshall S, Smith MK, Bycott P, Beltangady M: A quantitative approach for making go/no-go decisions in drug development. Drug Inform J. 2011, 45 (2): 187-202.
O’Hagan A, Stevens JW, Campbell MJ: Assurance in clinical trial design. Pharm Stat. 2005, 4 (3): 187-201. 10.1002/pst.175. CrossRef
Parmar MKB, Ungerleider RS, Simon R: Assessing whether to perform a confirmatory randomized clinical trial. J Natl Canc Inst. 1996, 88 (22): 1645-1651. 10.1093/jnci/88.22.1645. CrossRef
Spiegelhalter DJ, Abrams KR, Myles JP: Bayesian Approaches to Clinical Trials and Health-Care Evaluation. 2004, Chichester: John Wiley & Sons
Lee PM: Bayesian Statistics: An Introduction. 1989, New York: Oxford University Press; Edward Arnold
- The statistical interpretation of pilot trials: should significance thresholds be reconsidered?
Ellen C Lee
Amy L Whitehead
Richard M Jacques
Steven A Julious
- BioMed Central
Neu im Fachgebiet AINS
Meistgelesene Bücher aus dem Fachgebiet AINS
Mail Icon II