Skip to main content
Erschienen in: Journal of General Internal Medicine 1/2022

06.08.2021 | Research and Reporting Methods

The Fragility Index for Assessing the Robustness of the Statistically Significant Results of Experimental Clinical Studies

verfasst von: Adrienne K. Ho, MBBS

Erschienen in: Journal of General Internal Medicine | Ausgabe 1/2022

Einloggen, um Zugang zu erhalten

Excerpt

In a randomized controlled trial (RCT) comparing treatments A and B, a null hypothesis (H0) of no difference in a primary outcome of interest is defined. Whether any observed difference is statistically significant has traditionally been based on the P value and the confidence interval (CI). For a century, an arbitrary threshold of 0.05 (1/20) has been used to define statistical significance.1 Because this probability is quite low, we conclude that P ≤ 0.05 suggests that the observed difference between A and B is incompatible with H0, and, with a ≥ 95% degree of certainty, that A and B are different. The 95% CI is a lower-upper limit in which the true effect estimate lies within a 95% certainty. A 95% CI that does not include the null effect size indicates the observed difference has reached statistical significance, and H0 is rejected. The 95% CI provides information about the precision of the estimate and complements the P value. Reporting both is recommended. …
Literatur
1.
Zurück zum Zitat Fisher RA. The arrangement of field experiments. J Min Agriculture of G Britain 1926; 33:503-13. Fisher RA. The arrangement of field experiments. J Min Agriculture of G Britain 1926; 33:503-13.
2.
Zurück zum Zitat Kennedy-Shaffer L. Before p < 0.05 to beyond p < 0.05: using history to contextualize p values and significance testing. Am Stat 2019; 73:sup 1: 82-90.CrossRef Kennedy-Shaffer L. Before p < 0.05 to beyond p < 0.05: using history to contextualize p values and significance testing. Am Stat 2019; 73:sup 1: 82-90.CrossRef
3.
Zurück zum Zitat Chavalarias D, Wallach JD, Li AHT, Ioannidis JPA. Evolution of reporting P values in the biomedical literature 1990-2015. JAMA 2016; 315:1141-8.CrossRef Chavalarias D, Wallach JD, Li AHT, Ioannidis JPA. Evolution of reporting P values in the biomedical literature 1990-2015. JAMA 2016; 315:1141-8.CrossRef
4.
Zurück zum Zitat Scherer RW, Meerpohl JJ, Pfeifer N, et al. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev 2018; 11: MR000005. Scherer RW, Meerpohl JJ, Pfeifer N, et al. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev 2018; 11: MR000005.
5.
Zurück zum Zitat Colquhoun D. The reproducibility of research and the misinterpretation of p values. R Soc Open Sc 2017; 4:171085.CrossRef Colquhoun D. The reproducibility of research and the misinterpretation of p values. R Soc Open Sc 2017; 4:171085.CrossRef
6.
Zurück zum Zitat Ioannidis JPA. The mass production of redundant, misleading, and conflected systematic reviews and meta-analyses. Milbank Q 2016; 94:485-514.CrossRef Ioannidis JPA. The mass production of redundant, misleading, and conflected systematic reviews and meta-analyses. Milbank Q 2016; 94:485-514.CrossRef
7.
Zurück zum Zitat Hacke C, Nunan D. Discrepancies in meta-analyses answering the same clinical question were hard to explain: a meta-epidemiological study J Clin Epidemiol 2020; 119:47-56.CrossRef Hacke C, Nunan D. Discrepancies in meta-analyses answering the same clinical question were hard to explain: a meta-epidemiological study J Clin Epidemiol 2020; 119:47-56.CrossRef
8.
Zurück zum Zitat Walsh M, Srinathan SK, McAuley DF, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index. J Clin Epidemiol 2014; 67:622-8.CrossRef Walsh M, Srinathan SK, McAuley DF, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index. J Clin Epidemiol 2014; 67:622-8.CrossRef
9.
Zurück zum Zitat Caldwell JE, Youssefzadeh K, Limpisvasti O. A method for calculating the fragility index of continuous outcomes. J Clin Epidemiol 2021; 136:20-5.CrossRef Caldwell JE, Youssefzadeh K, Limpisvasti O. A method for calculating the fragility index of continuous outcomes. J Clin Epidemiol 2021; 136:20-5.CrossRef
10.
Zurück zum Zitat Atal I, Porcher R, Boutron I, Ravaud P. The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses. J Clin Epidemiol 2019; 111:32-40.CrossRef Atal I, Porcher R, Boutron I, Ravaud P. The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses. J Clin Epidemiol 2019; 111:32-40.CrossRef
11.
Zurück zum Zitat Bomze D, Asher N, Ali OH, et al. Survival-inferred fragility index of phase 2 clinical trials evaluating immune checkpoint inhibitors. JAMA Netw Open 2020; 3:e2017675CrossRef Bomze D, Asher N, Ali OH, et al. Survival-inferred fragility index of phase 2 clinical trials evaluating immune checkpoint inhibitors. JAMA Netw Open 2020; 3:e2017675CrossRef
12.
Zurück zum Zitat Feinstein AR. The unit fragility index: an additional appraisal of “statistical significance” for a contrast of two proportions. J Clin Epidemiol 1990; 43:201-9.CrossRef Feinstein AR. The unit fragility index: an additional appraisal of “statistical significance” for a contrast of two proportions. J Clin Epidemiol 1990; 43:201-9.CrossRef
13.
Zurück zum Zitat Oviedo-Joekes E, Brissette S, Marsh D, et al. Diacetylmorphine versus methadone for the treatment of opioid addiction. N Engl J Med 2009; 361:777-86.CrossRef Oviedo-Joekes E, Brissette S, Marsh D, et al. Diacetylmorphine versus methadone for the treatment of opioid addiction. N Engl J Med 2009; 361:777-86.CrossRef
14.
Zurück zum Zitat Majeed M, Agrwal R, Attar BM, et al. Fragility index: how fragile is the data that support the American College of Gastroenterology guidelines for the management of Crohn’s disease? Eur J Gastroenterol Hepatol 2020; 32:193-8.CrossRef Majeed M, Agrwal R, Attar BM, et al. Fragility index: how fragile is the data that support the American College of Gastroenterology guidelines for the management of Crohn’s disease? Eur J Gastroenterol Hepatol 2020; 32:193-8.CrossRef
15.
Zurück zum Zitat Khan M, Evaniew N, Gichuru M, et al. The fragility of statistically significant findings from randomized trials in sports surgery: a systematic survey. Am J Sports Med 2017; 45:2164-70.CrossRef Khan M, Evaniew N, Gichuru M, et al. The fragility of statistically significant findings from randomized trials in sports surgery: a systematic survey. Am J Sports Med 2017; 45:2164-70.CrossRef
16.
Zurück zum Zitat Shen CS, Shamsudeen I, Farrokhyar F, Sabri K. Fragility of results in ophthalmology randomized controlled trials: a systematic review. Ophthalmology 2018; 125:642-8.CrossRef Shen CS, Shamsudeen I, Farrokhyar F, Sabri K. Fragility of results in ophthalmology randomized controlled trials: a systematic review. Ophthalmology 2018; 125:642-8.CrossRef
17.
Zurück zum Zitat Noel CW, McMullen C, Yao C, et al. The fragility of statistically significant findings from randomized trials in head and neck surgery. Laryngoscope 2018; 128:2094-100.CrossRef Noel CW, McMullen C, Yao C, et al. The fragility of statistically significant findings from randomized trials in head and neck surgery. Laryngoscope 2018; 128:2094-100.CrossRef
18.
Zurück zum Zitat Edwards E, Wayant C, Besas J, et al. How fragile are clinical trial outcomes that support the CHEST clinical practice guidelines for VTE? Chest 2018; 154:512-20.CrossRef Edwards E, Wayant C, Besas J, et al. How fragile are clinical trial outcomes that support the CHEST clinical practice guidelines for VTE? Chest 2018; 154:512-20.CrossRef
19.
Zurück zum Zitat Narayan VM, Gandhi S, Chrouser K, et al. The fragility of statistically significant findings from randomised controlled trials in the urological literature. BJU Int 2018; 122:160-6.CrossRef Narayan VM, Gandhi S, Chrouser K, et al. The fragility of statistically significant findings from randomised controlled trials in the urological literature. BJU Int 2018; 122:160-6.CrossRef
20.
Zurück zum Zitat Ridgeon EE, Young PJ, Bellomo R, et al. The fragility index in multicenter randomized controlled critical care trials. Crit Care Med 2016; 44:1278-84.CrossRef Ridgeon EE, Young PJ, Bellomo R, et al. The fragility index in multicenter randomized controlled critical care trials. Crit Care Med 2016; 44:1278-84.CrossRef
21.
Zurück zum Zitat Bertaggia L, Baiardo Redaelli M, Lembo R, et al. The fragility index in peri-operative randomized controlled trials that reported significant mortality effects in adults. Anaesthesia 2019; 74:1057-60.CrossRef Bertaggia L, Baiardo Redaelli M, Lembo R, et al. The fragility index in peri-operative randomized controlled trials that reported significant mortality effects in adults. Anaesthesia 2019; 74:1057-60.CrossRef
22.
Zurück zum Zitat Khan MS, Ochani RK, Shaikh A, et al. Fragility index in cardiovascular randomized controlled trials. Circ Cardiovasc Qual Outcomes 2019; 12:e005755.PubMedPubMedCentral Khan MS, Ochani RK, Shaikh A, et al. Fragility index in cardiovascular randomized controlled trials. Circ Cardiovasc Qual Outcomes 2019; 12:e005755.PubMedPubMedCentral
23.
Zurück zum Zitat Evaniew N, Files C, Smith C, et al. The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J 2015; 15:2188–2197.CrossRef Evaniew N, Files C, Smith C, et al. The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J 2015; 15:2188–2197.CrossRef
24.
Zurück zum Zitat Brown J, Lane A, Cooper C, Vassar M. The results of randomized controlled trials in emergency medicine are frequently fragile. Ann Emerg Med 2019; 73:564-76. Brown J, Lane A, Cooper C, Vassar M. The results of randomized controlled trials in emergency medicine are frequently fragile. Ann Emerg Med 2019; 73:564-76.
25.
Zurück zum Zitat Dawson J, Liu CY, Francisco GE, et al. Vagus nerve stimulation paired with rehabilitation for upper limb motor function after ischaemic stroke (VNS-REHAB): a randomised, blinded, pivotal, device trial. Lancet 2021; 397:1545-53.CrossRef Dawson J, Liu CY, Francisco GE, et al. Vagus nerve stimulation paired with rehabilitation for upper limb motor function after ischaemic stroke (VNS-REHAB): a randomised, blinded, pivotal, device trial. Lancet 2021; 397:1545-53.CrossRef
26.
Zurück zum Zitat Andò G, Capodanno D. Radial versus femoral access in invasively managed patients with acute coronary syndrome. A systematic review and meta-analysis. Ann Intern Med 2015; 163:932-40.CrossRef Andò G, Capodanno D. Radial versus femoral access in invasively managed patients with acute coronary syndrome. A systematic review and meta-analysis. Ann Intern Med 2015; 163:932-40.CrossRef
27.
Zurück zum Zitat Tignanelli CJ, Napoliano LM. The fragility index in randomized clinical trials as a means of optimizing patient care. JAMA Surg 2019; 154:74-9.CrossRef Tignanelli CJ, Napoliano LM. The fragility index in randomized clinical trials as a means of optimizing patient care. JAMA Surg 2019; 154:74-9.CrossRef
28.
Zurück zum Zitat Khan MS, Fonarow GC, Friede T, et al. Application of the reverse fragility index to statistically nonsignificant randomized clinical trial results. JAMA Netw Open 2020; 3:e2012469.CrossRef Khan MS, Fonarow GC, Friede T, et al. Application of the reverse fragility index to statistically nonsignificant randomized clinical trial results. JAMA Netw Open 2020; 3:e2012469.CrossRef
29.
Zurück zum Zitat Carter RE, McKie PM, Storlie CB. The fragility index: a P value in sheep’s clothing. Eur Heart J; 2017; 38:346-8.PubMed Carter RE, McKie PM, Storlie CB. The fragility index: a P value in sheep’s clothing. Eur Heart J; 2017; 38:346-8.PubMed
30.
Zurück zum Zitat Niforatos JD, Zheutlin AR, Chaitoff A, Pescatore RM. The fragility index of practice changing clinical trials is low and highly correlated with P values. J Clin Epidemiol 2020; 119:140-5.CrossRef Niforatos JD, Zheutlin AR, Chaitoff A, Pescatore RM. The fragility index of practice changing clinical trials is low and highly correlated with P values. J Clin Epidemiol 2020; 119:140-5.CrossRef
31.
Zurück zum Zitat Condon TM, Sexton RW, Wells AJ, To M-S. The weakness of fragility index exposed in an analysis of the traumatic brain injury management guidelines: a meta-epidemiolgical and simulation study. PLoS ONE 2020; 15:e0237879.CrossRef Condon TM, Sexton RW, Wells AJ, To M-S. The weakness of fragility index exposed in an analysis of the traumatic brain injury management guidelines: a meta-epidemiolgical and simulation study. PLoS ONE 2020; 15:e0237879.CrossRef
32.
Zurück zum Zitat Das S, Xaviar S. Calculation of the fragility index of randomized controlled trials in epilepsy published in twelve major journals. Epilepsy Res 2020; 159:106258.CrossRef Das S, Xaviar S. Calculation of the fragility index of randomized controlled trials in epilepsy published in twelve major journals. Epilepsy Res 2020; 159:106258.CrossRef
33.
Zurück zum Zitat Holek M, Bdair F, Khan M, et al. Fragility of clinical trials across research fields: a synthesis of methodological reviews. Contemp Clin Trials 2020; 97:106151.CrossRef Holek M, Bdair F, Khan M, et al. Fragility of clinical trials across research fields: a synthesis of methodological reviews. Contemp Clin Trials 2020; 97:106151.CrossRef
34.
Zurück zum Zitat The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med 1995; 333:1581-8.CrossRef The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med 1995; 333:1581-8.CrossRef
37.
Zurück zum Zitat Potter GE. Dismantling the fragility index: a demonstration of statistical reasoning. Stat Med 2020; 39:3720-31.CrossRef Potter GE. Dismantling the fragility index: a demonstration of statistical reasoning. Stat Med 2020; 39:3720-31.CrossRef
38.
Zurück zum Zitat Woods KL, Fletcher S, roffe C, Haider Y. Intravenous magnesium sulphate in suspected acute myocardial infarction: results of the second Leicester Intravenous Magnesium Intervention Trial (LIMIT-2). Lancet 1992; 339:1553e8.CrossRef Woods KL, Fletcher S, roffe C, Haider Y. Intravenous magnesium sulphate in suspected acute myocardial infarction: results of the second Leicester Intravenous Magnesium Intervention Trial (LIMIT-2). Lancet 1992; 339:1553e8.CrossRef
39.
Zurück zum Zitat ISIS-4: a randomized factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58,050 patients with suspected acute myocardial infarction. ISIS-4 (Fourth International Study of Infarct Survival) Collaborative Group. Lancet 1995; 345:669e85. ISIS-4: a randomized factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58,050 patients with suspected acute myocardial infarction. ISIS-4 (Fourth International Study of Infarct Survival) Collaborative Group. Lancet 1995; 345:669e85.
40.
Zurück zum Zitat Harrington D, D’Agostino Sr RB, Gatsonis C, et al. New guidelines for statistical reporting in the journal. N Engl J Med 2019; 381:285-6.CrossRef Harrington D, D’Agostino Sr RB, Gatsonis C, et al. New guidelines for statistical reporting in the journal. N Engl J Med 2019; 381:285-6.CrossRef
Metadaten
Titel
The Fragility Index for Assessing the Robustness of the Statistically Significant Results of Experimental Clinical Studies
verfasst von
Adrienne K. Ho, MBBS
Publikationsdatum
06.08.2021
Verlag
Springer International Publishing
Erschienen in
Journal of General Internal Medicine / Ausgabe 1/2022
Print ISSN: 0884-8734
Elektronische ISSN: 1525-1497
DOI
https://doi.org/10.1007/s11606-021-06999-9

Weitere Artikel der Ausgabe 1/2022

Journal of General Internal Medicine 1/2022 Zur Ausgabe

Leitlinien kompakt für die Innere Medizin

Mit medbee Pocketcards sicher entscheiden.

Seit 2022 gehört die medbee GmbH zum Springer Medizin Verlag

Echinokokkose medikamentös behandeln oder operieren?

06.05.2024 DCK 2024 Kongressbericht

Die Therapie von Echinokokkosen sollte immer in spezialisierten Zentren erfolgen. Eine symptomlose Echinokokkose kann – egal ob von Hunde- oder Fuchsbandwurm ausgelöst – konservativ erfolgen. Wenn eine Op. nötig ist, kann es sinnvoll sein, vorher Zysten zu leeren und zu desinfizieren. 

Umsetzung der POMGAT-Leitlinie läuft

03.05.2024 DCK 2024 Kongressbericht

Seit November 2023 gibt es evidenzbasierte Empfehlungen zum perioperativen Management bei gastrointestinalen Tumoren (POMGAT) auf S3-Niveau. Vieles wird schon entsprechend der Empfehlungen durchgeführt. Wo es im Alltag noch hapert, zeigt eine Umfrage in einem Klinikverbund.

Proximale Humerusfraktur: Auch 100-Jährige operieren?

01.05.2024 DCK 2024 Kongressbericht

Mit dem demographischen Wandel versorgt auch die Chirurgie immer mehr betagte Menschen. Von Entwicklungen wie Fast-Track können auch ältere Menschen profitieren und bei proximaler Humerusfraktur können selbst manche 100-Jährige noch sicher operiert werden.

Die „Zehn Gebote“ des Endokarditis-Managements

30.04.2024 Endokarditis Leitlinie kompakt

Worauf kommt es beim Management von Personen mit infektiöser Endokarditis an? Eine Kardiologin und ein Kardiologe fassen die zehn wichtigsten Punkte der neuen ESC-Leitlinie zusammen.

Update Innere Medizin

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.