Skip to main content
Erschienen in: European Radiology 4/2019

15.11.2018 | Experimental

Citation bias in imaging research: are studies with higher diagnostic accuracy estimates cited more often?

verfasst von: Robert A. Frank, Anahita Dehmoobad Sharifabadi, Jean-Paul Salameh, Trevor A. McGrath, Noémie Kraaijpoel, Wilfred Dang, Nicole Li, Isabelle D. Gauthier, Mark Z. Wu, Patrick M. Bossuyt, Deborah Levine, Matthew D. F. McInnes

Erschienen in: European Radiology | Ausgabe 4/2019

Einloggen, um Zugang zu erhalten

Abstract

Objectives

To assess the risk of citation bias in imaging diagnostic accuracy research by evaluating whether studies with higher accuracy estimates are cited more frequently than those with lower accuracy estimates.

Methods

We searched Medline for diagnostic accuracy meta-analyses published in imaging journals from January 2005 to April 2016. Primary studies from the meta-analyses were screened; those assessing the diagnostic accuracy of an imaging test and reporting sensitivity and specificity were eligible for inclusion. Studies not indexed in Web of Science, duplicates, and inaccessible articles were excluded. Topic (modality/subspecialty), study design, sample size, journal impact factor, publication date, times cited, sensitivity, and specificity were extracted for each study. Negative binomial regression was performed to evaluate the association of citation rate (times cited per month since publication) with Youden’s index (sensitivity + specificity -1), highest sensitivity, and highest specificity, controlling for the potential confounding effects of modality, subspecialty, impact factor, study design, sample size, and source meta-analysis.

Results

There were 1016 primary studies included. A positive association between Youden’s index and citation rate was present, with a regression coefficient of 0.33 (p = 0.016). The regression coefficient for sensitivity was 0.41 (p = 0.034), and for specificity, 0.32 (p = 0.15).

Conclusion

A positive association exists between diagnostic accuracy estimates and citation rates, indicating that there is evidence of citation bias in imaging diagnostic accuracy literature. Overestimation of imaging test accuracy may contribute to patient harm from incorrect interpretation of test results.

Key Points

• Studies with higher accuracy estimates may be cited more frequently than those with lower accuracy estimates.
This citation bias could lead clinicians, reviews, and clinical practice guidelines to overestimate the accuracy of imaging tests, contributing to patient harm from incorrect interpretation of test results.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Duyx B, Urlings MJE, Swaen GMH, Bouter LM, Zeegers MP (2017) Scientific citations favor positive results: a systematic review and meta-analysis. J Clin Epidemiol 88:92–101CrossRefPubMed Duyx B, Urlings MJE, Swaen GMH, Bouter LM, Zeegers MP (2017) Scientific citations favor positive results: a systematic review and meta-analysis. J Clin Epidemiol 88:92–101CrossRefPubMed
2.
Zurück zum Zitat Jannot AS, Agoritsas T, Gayet-Ageron A, Perneger TV (2013) Citation bias favoring statistically significant studies was present in medical research. J Clin Epidemiol 66:296–301CrossRefPubMed Jannot AS, Agoritsas T, Gayet-Ageron A, Perneger TV (2013) Citation bias favoring statistically significant studies was present in medical research. J Clin Epidemiol 66:296–301CrossRefPubMed
4.
Zurück zum Zitat Seglen PO (1997) Citations and journal impact factors: questionable indicators of research quality. Allergy 52:1050–1056CrossRefPubMed Seglen PO (1997) Citations and journal impact factors: questionable indicators of research quality. Allergy 52:1050–1056CrossRefPubMed
5.
Zurück zum Zitat Steneck NH (2006) Fostering integrity in research: definitions, current knowledge, and future directions. Sci Eng Ethics 12:53–74CrossRefPubMed Steneck NH (2006) Fostering integrity in research: definitions, current knowledge, and future directions. Sci Eng Ethics 12:53–74CrossRefPubMed
6.
Zurück zum Zitat Hutchison BG, Oxman AD, Lloyd S (1995) Comprehensiveness and bias in reporting clinical trials. Study of reviews of pneumococcal vaccine effectiveness. Can Fam Physician 41:1356–1360PubMedPubMedCentral Hutchison BG, Oxman AD, Lloyd S (1995) Comprehensiveness and bias in reporting clinical trials. Study of reviews of pneumococcal vaccine effectiveness. Can Fam Physician 41:1356–1360PubMedPubMedCentral
8.
Zurück zum Zitat Misemer BS, Platts-Mills TF, Jones CW (2016) Citation bias favoring positive clinical trials of thrombolytics for acute ischemic stroke: a cross-sectional analysis. Trials 17:473CrossRefPubMedPubMedCentral Misemer BS, Platts-Mills TF, Jones CW (2016) Citation bias favoring positive clinical trials of thrombolytics for acute ischemic stroke: a cross-sectional analysis. Trials 17:473CrossRefPubMedPubMedCentral
9.
Zurück zum Zitat Kjaergard LL, Gluud C (2002) Citation bias of hepato-biliary randomized clinical trials. J Clin Epidemiol 55:407–410CrossRefPubMed Kjaergard LL, Gluud C (2002) Citation bias of hepato-biliary randomized clinical trials. J Clin Epidemiol 55:407–410CrossRefPubMed
10.
Zurück zum Zitat Korevaar DA, Cohen JF, Spijker R et al (2016) Reported estimates of diagnostic accuracy in ophthalmology conference abstracts were not associated with full-text publication. J Clin Epidemiol 79:96–103CrossRefPubMedPubMedCentral Korevaar DA, Cohen JF, Spijker R et al (2016) Reported estimates of diagnostic accuracy in ophthalmology conference abstracts were not associated with full-text publication. J Clin Epidemiol 79:96–103CrossRefPubMedPubMedCentral
11.
Zurück zum Zitat Korevaar DA, van Es N, Zwinderman AH, Cohen JF, Bossuyt PM (2016) Time to publication among completed diagnostic accuracy studies: associated with reported accuracy estimates. BMC Med Res Methodol 16:68CrossRefPubMedPubMedCentral Korevaar DA, van Es N, Zwinderman AH, Cohen JF, Bossuyt PM (2016) Time to publication among completed diagnostic accuracy studies: associated with reported accuracy estimates. BMC Med Res Methodol 16:68CrossRefPubMedPubMedCentral
12.
Zurück zum Zitat Sharifabadi AD, Korevaar DA, McGrath TM, et al (2018) Reporting bias in imaging: higher accuracy is linked to faster publication. Eur Radiol 28:3632−3639 Sharifabadi AD, Korevaar DA, McGrath TM, et al (2018) Reporting bias in imaging: higher accuracy is linked to faster publication. Eur Radiol 28:3632−3639
13.
Zurück zum Zitat Frank RA, McInnes MDF (2018) Citation bias in imaging research: are studies with higher diagnostic accuracy estimates cited more often? Study protocol. Retrieved from osf.io/j964x Frank RA, McInnes MDF (2018) Citation bias in imaging research: are studies with higher diagnostic accuracy estimates cited more often? Study protocol. Retrieved from osf.io/j964x
14.
Zurück zum Zitat Frank RA, McInnes MDF, Levine D et al (2017) Are study and journal characteristics reliable indicators of ‘truth’ in imaging research? Radiology 287:215–223CrossRefPubMed Frank RA, McInnes MDF, Levine D et al (2017) Are study and journal characteristics reliable indicators of ‘truth’ in imaging research? Radiology 287:215–223CrossRefPubMed
15.
Zurück zum Zitat McGrath TA, McInnes MD, Korevaar DA, Bossuyt PM (2016) Meta-analyses of diagnostic accuracy in imaging journals: analysis of pooling techniques and their effect on summary estimates of diagnostic accuracy. Radiology 281:78–85CrossRefPubMed McGrath TA, McInnes MD, Korevaar DA, Bossuyt PM (2016) Meta-analyses of diagnostic accuracy in imaging journals: analysis of pooling techniques and their effect on summary estimates of diagnostic accuracy. Radiology 281:78–85CrossRefPubMed
16.
Zurück zum Zitat van Enst WA, Ochodo E, Scholten RJ, Hooft L, Leeflang MM (2014) Investigation of publication bias in meta-analyses of diagnostic test accuracy: a meta-epidemiological study. BMC Med Res Methodol 14:70CrossRefPubMedPubMedCentral van Enst WA, Ochodo E, Scholten RJ, Hooft L, Leeflang MM (2014) Investigation of publication bias in meta-analyses of diagnostic test accuracy: a meta-epidemiological study. BMC Med Res Methodol 14:70CrossRefPubMedPubMedCentral
17.
Zurück zum Zitat Devillé WL, Bezemer PD, Bouter LM (2000) Publications on diagnostic test evaluation in family medicine journals: an optimal search strategy. J Clin Epidemiol 53:65–69CrossRefPubMed Devillé WL, Bezemer PD, Bouter LM (2000) Publications on diagnostic test evaluation in family medicine journals: an optimal search strategy. J Clin Epidemiol 53:65–69CrossRefPubMed
19.
Zurück zum Zitat R Core Team (2017) R: A language and environment forstatistical computing. R Foundation for Statistical Computing, V., Austria. Available via https://www.R-project.org/. Accessed 1 Aug 2018 R Core Team (2017) R: A language and environment forstatistical computing. R Foundation for Statistical Computing, V., Austria. Available via https://​www.​R-project.​org/. Accessed 1 Aug 2018
20.
Zurück zum Zitat Nieminen P, Carpenter J, Rucker G, Schumacher M (2006) The relationship between quality of research and citation frequency. BMC Med Res Methodol 6:42CrossRefPubMedPubMedCentral Nieminen P, Carpenter J, Rucker G, Schumacher M (2006) The relationship between quality of research and citation frequency. BMC Med Res Methodol 6:42CrossRefPubMedPubMedCentral
21.
Zurück zum Zitat Kulkarni AV, Busse JW, Shams I (2007) Characteristics associated with citation rate of the medical literature. PLoS One 2:403CrossRef Kulkarni AV, Busse JW, Shams I (2007) Characteristics associated with citation rate of the medical literature. PLoS One 2:403CrossRef
22.
Zurück zum Zitat Sawin VI, Robinson KA (2016) Biased and inadequate citation of prior research in reports of cardiovascular trials is a continuing source of waste in research. J Clin Epidemiol 69:174–178CrossRefPubMed Sawin VI, Robinson KA (2016) Biased and inadequate citation of prior research in reports of cardiovascular trials is a continuing source of waste in research. J Clin Epidemiol 69:174–178CrossRefPubMed
23.
Zurück zum Zitat Sheth U, Simunovic N, Tornetta P 3rd, Einhorn TA, Bhandari M (2011) Poor citation of prior evidence in hip fracture trials. J Bone Joint Surg Am 93:2079–2086 Sheth U, Simunovic N, Tornetta P 3rd, Einhorn TA, Bhandari M (2011) Poor citation of prior evidence in hip fracture trials. J Bone Joint Surg Am 93:2079–2086
24.
Zurück zum Zitat Etter JF, Stapleton J (2009) Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. J Clin Epidemiol 62:831–837CrossRefPubMed Etter JF, Stapleton J (2009) Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. J Clin Epidemiol 62:831–837CrossRefPubMed
25.
Zurück zum Zitat Nieminen P, Rucker G, Miettunen J, Carpenter J, Schumacher M (2007) Statistically significant papers in psychiatry were cited more often than others. J Clin Epidemiol 60:939–946CrossRefPubMed Nieminen P, Rucker G, Miettunen J, Carpenter J, Schumacher M (2007) Statistically significant papers in psychiatry were cited more often than others. J Clin Epidemiol 60:939–946CrossRefPubMed
26.
Zurück zum Zitat Saha S, Saint A, Christakis DA (2003) Impact factor: a valid measure of journal quality? J Med Libr Assoc 9:42–46 Saha S, Saint A, Christakis DA (2003) Impact factor: a valid measure of journal quality? J Med Libr Assoc 9:42–46
27.
Zurück zum Zitat Dilauro M, McInnes MD, Korevaar DA et al (2016) Is there an association between STARD statement adherence and citation rate? Radiology 280:62–67 Dilauro M, McInnes MD, Korevaar DA et al (2016) Is there an association between STARD statement adherence and citation rate? Radiology 280:62–67
28.
Zurück zum Zitat Choi YJ, Chung MS, Koo HJ, Park JE, Yoon HM, Park SH (2016) Does the reporting quality of diagnostic test accuracy studies, as defined by STARD 2015, affect citation? Korean J Radiol 17:706–714CrossRefPubMedPubMedCentral Choi YJ, Chung MS, Koo HJ, Park JE, Yoon HM, Park SH (2016) Does the reporting quality of diagnostic test accuracy studies, as defined by STARD 2015, affect citation? Korean J Radiol 17:706–714CrossRefPubMedPubMedCentral
29.
Zurück zum Zitat McInnes MDF, Moher D, Thombs BD et al (2018) Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA 319:388–396CrossRef McInnes MDF, Moher D, Thombs BD et al (2018) Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA 319:388–396CrossRef
30.
Zurück zum Zitat McGrath TA, McInnes MDF, van Es N, Leeflang MMG, Korevaar DA, Bossuyt PMM (2017) Overinterpretation of research findings: evidence of “spin” in systematic reviews of diagnostic accuracy studies. Clin Chem 63:1353–1362CrossRefPubMed McGrath TA, McInnes MDF, van Es N, Leeflang MMG, Korevaar DA, Bossuyt PMM (2017) Overinterpretation of research findings: evidence of “spin” in systematic reviews of diagnostic accuracy studies. Clin Chem 63:1353–1362CrossRefPubMed
31.
Zurück zum Zitat Ochodo EA, de Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM (2013) Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”. Radiology 267:581–588CrossRefPubMed Ochodo EA, de Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM (2013) Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”. Radiology 267:581–588CrossRefPubMed
Metadaten
Titel
Citation bias in imaging research: are studies with higher diagnostic accuracy estimates cited more often?
verfasst von
Robert A. Frank
Anahita Dehmoobad Sharifabadi
Jean-Paul Salameh
Trevor A. McGrath
Noémie Kraaijpoel
Wilfred Dang
Nicole Li
Isabelle D. Gauthier
Mark Z. Wu
Patrick M. Bossuyt
Deborah Levine
Matthew D. F. McInnes
Publikationsdatum
15.11.2018
Verlag
Springer Berlin Heidelberg
Erschienen in
European Radiology / Ausgabe 4/2019
Print ISSN: 0938-7994
Elektronische ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-018-5801-8

Weitere Artikel der Ausgabe 4/2019

European Radiology 4/2019 Zur Ausgabe

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.