The online version of this article (doi:10.1186/s12916-017-0821-9) contains supplementary material, which is available to authorized users.
While journals and reporting guidelines recommend the presentation of confidence intervals, many authors adhere strictly to statistically significant testing. Our objective was to determine what proportions of not statistically significant (NSS) cardiovascular trials include potentially clinically meaningful effects in primary outcomes and if these are associated with authors’ conclusions.
Cardiovascular studies published in six high-impact journals between 1 January 2010 and 31 December 2014 were identified via PubMed. Two independent reviewers selected trials with major adverse cardiovascular events (stroke, myocardial infarction, or cardiovascular death) as primary outcomes and extracted data on trial characteristics, quality, and primary outcome. Potentially clinically meaningful effects were defined broadly as a relative risk point estimate ≤0.94 (based on the effects of ezetimibe) and/or a lower confidence interval ≤0.75 (based on the effects of statins).
We identified 127 randomized trial comparisons from 3200 articles. The primary outcomes were statistically significant (SS) favoring treatment in 21% (27/127), NSS in 72% (92/127), and SS favoring control in 6% (8/127). In 61% of NSS trials (56/92), the point estimate and/or lower confidence interval included potentially meaningful effects. Both point estimate and confidence interval included potentially meaningful effects in 67% of trials (12/18) in which authors’ concluded that treatment was superior, in 28% (16/58) with a neutral conclusion, and in 6% (1/16) in which authors’ concluded that control was superior. In a sensitivity analysis, 26% of NSS trials would include potential meaningful effects with relative risk thresholds of point estimate ≤0.85 and/or a lower confidence interval ≤0.65.
Point estimates and/or confidence intervals included potentially clinically meaningful effects in up to 61% of NSS cardiovascular trials. Authors’ conclusions often reflect potentially meaningful results of NSS cardiovascular trials. Given the frequency of potentially clinical meaningful effects in NSS trials, authors should be encouraged to continue to look beyond significance testing to a broader interpretation of trial results.
Additional file 1: Supplementary material: included studies. (DOCX 61 kb)12916_2017_821_MOESM1_ESM.docx
Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. Br Med J (Clin Res Ed). 1986;292:746–50. CrossRef
Cummings P, Koepsell TD. P values vs estimates of association with confidence intervals. Arch Pediatr Adolesc Med. 2010;164:193–6. PubMed
Cannon CP, Blazing MA, Giugliano RP, McCagg A, White JA, Theroux P, Darius H, Lewis BS, Ophuis TO, Jukema JW, De Ferrari GM, Ruzyllo W, De Lucca P, Im K, Bohula EA, Reist C, Wiviott SD, Tershakovec AM, Musliner TA, Braunwald E, Califf RM, IMPROVE-IT Investigators. ezetimibe added to statin therapy after acute coronary syndromes. N Engl J Med. 2015;372(25):2387–97. CrossRefPubMed
Taylor F, Huffman MD, Macedo AF, Moore TH, Burke M, Davey Smith G, Ward K, Ebrahim S. Statins for the primary prevention of cardiovascular disease. Cochrane Database Syst Rev. 2013;1, CD004816.
- Are potentially clinically meaningful benefits misinterpreted in cardiovascular randomized trials? A systematic examination of statistical significance, clinical significance, and authors’ conclusions
G. Michael Allan
Caitlin R. Finley
Michael R. Kolber
Adriennne J. Lindblad
- BioMed Central
Neu im Fachgebiet Allgemeinmedizin
Meistgelesene Bücher aus dem Fachgebiet
Mail Icon II