Skip to main content
Erschienen in: European Radiology 3/2010

01.03.2010 | Chest

Computer-aided detection (CAD) of lung nodules in CT scans: radiologist performance and reading time with incremental CAD assistance

verfasst von: Justus E. Roos, David Paik, David Olsen, Emily G. Liu, Lawrence C. Chow, Ann N. Leung, Robert Mindelzun, Kingshuk R. Choudhury, David P. Naidich, Sandy Napel, Geoffrey D. Rubin

Erschienen in: European Radiology | Ausgabe 3/2010

Einloggen, um Zugang zu erhalten

Abstract

Objective

The diagnostic performance of radiologists using incremental CAD assistance for lung nodule detection on CT and their temporal variation in performance during CAD evaluation was assessed.

Methods

CAD was applied to 20 chest multidetector-row computed tomography (MDCT) scans containing 190 non-calcified ≥3-mm nodules. After free search, three radiologists independently evaluated a maximum of up to 50 CAD detections/patient. Multiple free-response ROC curves were generated for free search and successive CAD evaluation, by incrementally adding CAD detections one at a time to the radiologists’ performance.

Results

The sensitivity for free search was 53% (range, 44%–59%) at 1.15 false positives (FP)/patient and increased with CAD to 69% (range, 59–82%) at 1.45 FP/patient. CAD evaluation initially resulted in a sharp rise in sensitivity of 14% with a minimal increase in FP over a time period of 100 s, followed by flattening of the sensitivity increase to only 2%. This transition resulted from a greater prevalence of true positive (TP) versus FP detections at early CAD evaluation and not by a temporal change in readers’ performance. The time spent for TP (9.5 s ± 4.5 s) and false negative (FN) (8.4 s ± 6.7 s) detections was similar; FP decisions took two- to three-times longer (14.4 s ± 8.7 s) than true negative (TN) decisions (4.7 s ± 1.3 s).

Conclusions

When CAD output is ordered by CAD score, an initial period of rapid performance improvement slows significantly over time because of non-uniformity in the distribution of TP CAD output and not to a changing reader performance over time.
Literatur
1.
Zurück zum Zitat Beigelman-Aubry C, Raffy P, Yang W, Castellino RA, Grenier PA (2007) Computer-aided detection of solid lung nodules on follow-up MDCT screening: evaluation of detection, tracking, and reading time. AJR Am J Roentgenol 189:948–955CrossRefPubMed Beigelman-Aubry C, Raffy P, Yang W, Castellino RA, Grenier PA (2007) Computer-aided detection of solid lung nodules on follow-up MDCT screening: evaluation of detection, tracking, and reading time. AJR Am J Roentgenol 189:948–955CrossRefPubMed
2.
Zurück zum Zitat Beyer F, Zierott L, Fallenberg EM, Juergens KU, Stoeckel J, Heindel W, Wormanns D (2007) Comparison of sensitivity and reading time for the use of computer-aided detection (CAD) of pulmonary nodules at MDCT as concurrent or second reader. Eur Radiol 17:2941–2947CrossRefPubMed Beyer F, Zierott L, Fallenberg EM, Juergens KU, Stoeckel J, Heindel W, Wormanns D (2007) Comparison of sensitivity and reading time for the use of computer-aided detection (CAD) of pulmonary nodules at MDCT as concurrent or second reader. Eur Radiol 17:2941–2947CrossRefPubMed
3.
Zurück zum Zitat Godoy MC, Cooperberg PL, Maizlin ZV, Yuan R, McWilliams A, Lam S, Mayo JR (2008) Detection sensitivity of a commercial lung nodule CAD system in a series of pathologically proven lung cancers. J Thorac Imaging 23:1–6CrossRefPubMed Godoy MC, Cooperberg PL, Maizlin ZV, Yuan R, McWilliams A, Lam S, Mayo JR (2008) Detection sensitivity of a commercial lung nodule CAD system in a series of pathologically proven lung cancers. J Thorac Imaging 23:1–6CrossRefPubMed
4.
Zurück zum Zitat Jankowski A, Martinelli T, Timsit JF, Brambilla C, Thony F, Coulomb M, Ferretti G (2007) Pulmonary nodule detection on MDCT images: evaluation of diagnostic performance using thin axial images, maximum intensity projections, and computer-assisted detection. Eur Radiol 17:3148–3156CrossRefPubMed Jankowski A, Martinelli T, Timsit JF, Brambilla C, Thony F, Coulomb M, Ferretti G (2007) Pulmonary nodule detection on MDCT images: evaluation of diagnostic performance using thin axial images, maximum intensity projections, and computer-assisted detection. Eur Radiol 17:3148–3156CrossRefPubMed
5.
Zurück zum Zitat Marten K, Engelke C (2007) Computer-aided detection and automated CT volumetry of pulmonary nodules. Eur Radiol 17:888–901CrossRefPubMed Marten K, Engelke C (2007) Computer-aided detection and automated CT volumetry of pulmonary nodules. Eur Radiol 17:888–901CrossRefPubMed
6.
Zurück zum Zitat Rubin GD, Lyo JK, Paik DS, Sherbondy AJ, Chow LC, Leung AN, Mindelzun R, Schraedley-Desmond PK, Zinck SE, Naidich DP, Napel S (2005) Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology 234:274–283CrossRefPubMed Rubin GD, Lyo JK, Paik DS, Sherbondy AJ, Chow LC, Leung AN, Mindelzun R, Schraedley-Desmond PK, Zinck SE, Naidich DP, Napel S (2005) Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology 234:274–283CrossRefPubMed
7.
Zurück zum Zitat Saba L, Caddeo G, Mallarini G (2007) Computer-aided detection of pulmonary nodules in computed tomography: analysis and review of the literature. J Comput Assist Tomogr 31:611–619CrossRefPubMed Saba L, Caddeo G, Mallarini G (2007) Computer-aided detection of pulmonary nodules in computed tomography: analysis and review of the literature. J Comput Assist Tomogr 31:611–619CrossRefPubMed
8.
Zurück zum Zitat White CS, Pugatch R, Koonce T, Rust SW, Dharaiya E (2008) Lung nodule CAD software as a second reader: a multicenter study. Acad Radiol 15:326–333CrossRefPubMed White CS, Pugatch R, Koonce T, Rust SW, Dharaiya E (2008) Lung nodule CAD software as a second reader: a multicenter study. Acad Radiol 15:326–333CrossRefPubMed
9.
Zurück zum Zitat Yuan R, Vos PM, Cooperberg PL (2006) Computer-aided detection in screening CT for pulmonary nodules. AJR Am J Roentgenol 186:1280–1287CrossRefPubMed Yuan R, Vos PM, Cooperberg PL (2006) Computer-aided detection in screening CT for pulmonary nodules. AJR Am J Roentgenol 186:1280–1287CrossRefPubMed
10.
Zurück zum Zitat Das M, Muhlenbruch G, Heinen S, Mahnken AH, Salganicoff M, Stanzel S, Gunther RW, Wildberger JE (2008) Performance evaluation of a computer-aided detection algorithm for solid pulmonary nodules in low-dose and standard-dose MDCT chest examinations and its influence on radiologists. Br J Radiol 81:841–847CrossRefPubMed Das M, Muhlenbruch G, Heinen S, Mahnken AH, Salganicoff M, Stanzel S, Gunther RW, Wildberger JE (2008) Performance evaluation of a computer-aided detection algorithm for solid pulmonary nodules in low-dose and standard-dose MDCT chest examinations and its influence on radiologists. Br J Radiol 81:841–847CrossRefPubMed
11.
Zurück zum Zitat Lee JY, Chung MJ, Yi CA, Lee KS (2008) Ultra-low-dose MDCT of the chest: influence on automated lung nodule detection. Korean J Radiol 9:95–101CrossRefPubMed Lee JY, Chung MJ, Yi CA, Lee KS (2008) Ultra-low-dose MDCT of the chest: influence on automated lung nodule detection. Korean J Radiol 9:95–101CrossRefPubMed
12.
Zurück zum Zitat Das M, Muhlenbruch G, Mahnken AH, Flohr TG, Gundel L, Stanzel S, Kraus T, Gunther RW, Wildberger JE (2006) Small pulmonary nodules: effect of two computer-aided detection systems on radiologist performance. Radiology 241:564–571CrossRefPubMed Das M, Muhlenbruch G, Mahnken AH, Flohr TG, Gundel L, Stanzel S, Kraus T, Gunther RW, Wildberger JE (2006) Small pulmonary nodules: effect of two computer-aided detection systems on radiologist performance. Radiology 241:564–571CrossRefPubMed
13.
Zurück zum Zitat Zheng B, Leader JK, Abrams G, Shindel B, Catullo V, Good WF, Gur D (2004) Computer-aided detection schemes: the effect of limiting the number of cued regions in each case. AJR Am J Roentgenol 182:579–583PubMed Zheng B, Leader JK, Abrams G, Shindel B, Catullo V, Good WF, Gur D (2004) Computer-aided detection schemes: the effect of limiting the number of cued regions in each case. AJR Am J Roentgenol 182:579–583PubMed
14.
Zurück zum Zitat Paik DS, Beaulieu CF, Rubin GD, Acar B, Jeffrey RB Jr, Yee J, Dey J, Napel S (2004) Surface normal overlap: a computer-aided detection algorithm with application to colonic polyps and lung nodules in helical CT. IEEE Trans Med Imaging 23:661–675CrossRefPubMed Paik DS, Beaulieu CF, Rubin GD, Acar B, Jeffrey RB Jr, Yee J, Dey J, Napel S (2004) Surface normal overlap: a computer-aided detection algorithm with application to colonic polyps and lung nodules in helical CT. IEEE Trans Med Imaging 23:661–675CrossRefPubMed
15.
Zurück zum Zitat Cheng H, Macaluso M (1997) Comparison of the accuracy of two tests with a confirmatory procedure limited to positive results. Epidemiology 8:104–106CrossRefPubMed Cheng H, Macaluso M (1997) Comparison of the accuracy of two tests with a confirmatory procedure limited to positive results. Epidemiology 8:104–106CrossRefPubMed
16.
Zurück zum Zitat Eliasziw M, Donner A (1991) Application of the McNemar test to non-independent matched pair data. Stat Med 10:1981–1991CrossRefPubMed Eliasziw M, Donner A (1991) Application of the McNemar test to non-independent matched pair data. Stat Med 10:1981–1991CrossRefPubMed
17.
Zurück zum Zitat Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174CrossRefPubMed Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174CrossRefPubMed
18.
Zurück zum Zitat Brochu B, Beigelman-Aubry C, Goldmard JL, Raffy P, Grenier PA, Lucidarme O (2007) Computer-aided detection of lung nodules on thin collimation MDCT: impact on radiologists’ performance. J Radiol 88:573–578CrossRefPubMed Brochu B, Beigelman-Aubry C, Goldmard JL, Raffy P, Grenier PA, Lucidarme O (2007) Computer-aided detection of lung nodules on thin collimation MDCT: impact on radiologists’ performance. J Radiol 88:573–578CrossRefPubMed
19.
Zurück zum Zitat Suzuki K, Armato SG 3rd, Li F, Sone S, Doi K (2003) Massive training artificial neural network (MTANN) for reduction of false positives in computerized detection of lung nodules in low-dose computed tomography. Med Phys 30:1602–1617CrossRefPubMed Suzuki K, Armato SG 3rd, Li F, Sone S, Doi K (2003) Massive training artificial neural network (MTANN) for reduction of false positives in computerized detection of lung nodules in low-dose computed tomography. Med Phys 30:1602–1617CrossRefPubMed
20.
Zurück zum Zitat Ge Z, Sahiner B, Chan HP, Hadjiiski LM, Cascade PN, Bogot N, Kazerooni EA, Wei J, Zhou C (2005) Computer-aided detection of lung nodules: false positive reduction using a 3D gradient field method and 3D ellipsoid fitting. Med Phys 32:2443–2454CrossRefPubMed Ge Z, Sahiner B, Chan HP, Hadjiiski LM, Cascade PN, Bogot N, Kazerooni EA, Wei J, Zhou C (2005) Computer-aided detection of lung nodules: false positive reduction using a 3D gradient field method and 3D ellipsoid fitting. Med Phys 32:2443–2454CrossRefPubMed
21.
Zurück zum Zitat Roy AS, Armato SG 3rd, Wilson A, Drukker K (2006) Automated detection of lung nodules in CT scans: false-positive reduction with the radial-gradient index. Med Phys 33:1133–1140CrossRefPubMed Roy AS, Armato SG 3rd, Wilson A, Drukker K (2006) Automated detection of lung nodules in CT scans: false-positive reduction with the radial-gradient index. Med Phys 33:1133–1140CrossRefPubMed
22.
Zurück zum Zitat Armato SG 3rd, McNitt-Gray MF, Reeves AP, Meyer CR, McLennan G, Aberle DR, Kazerooni EA, MacMahon H, van Beek EJ, Yankelevitz D, Hoffman EA, Henschke CI, Roberts RY, Brown MS, Engelmann RM, Pais RC, Piker CW, Qing D, Kocherginsky M, Croft BY, Clarke LP (2007) The Lung Image Database Consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans. Acad Radiol 14:1409–1421CrossRefPubMed Armato SG 3rd, McNitt-Gray MF, Reeves AP, Meyer CR, McLennan G, Aberle DR, Kazerooni EA, MacMahon H, van Beek EJ, Yankelevitz D, Hoffman EA, Henschke CI, Roberts RY, Brown MS, Engelmann RM, Pais RC, Piker CW, Qing D, Kocherginsky M, Croft BY, Clarke LP (2007) The Lung Image Database Consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans. Acad Radiol 14:1409–1421CrossRefPubMed
23.
Zurück zum Zitat MacMahon H, Austin JH, Gamsu G, Herold CJ, Jett JR, Naidich DP, Patz EF Jr, Swensen SJ (2005) Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner Society. Radiology 237:395–400CrossRefPubMed MacMahon H, Austin JH, Gamsu G, Herold CJ, Jett JR, Naidich DP, Patz EF Jr, Swensen SJ (2005) Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner Society. Radiology 237:395–400CrossRefPubMed
24.
Zurück zum Zitat Brown MS, Goldin JG, Rogers S, Kim HJ, Suh RD, McNitt-Gray MF, Shah SK, Truong D, Brown K, Sayre JW, Gjertson DW, Batra P, Aberle DR (2005) Computer-aided lung nodule detection in CT: results of large-scale observer test. Acad Radiol 12:681–686CrossRefPubMed Brown MS, Goldin JG, Rogers S, Kim HJ, Suh RD, McNitt-Gray MF, Shah SK, Truong D, Brown K, Sayre JW, Gjertson DW, Batra P, Aberle DR (2005) Computer-aided lung nodule detection in CT: results of large-scale observer test. Acad Radiol 12:681–686CrossRefPubMed
Metadaten
Titel
Computer-aided detection (CAD) of lung nodules in CT scans: radiologist performance and reading time with incremental CAD assistance
verfasst von
Justus E. Roos
David Paik
David Olsen
Emily G. Liu
Lawrence C. Chow
Ann N. Leung
Robert Mindelzun
Kingshuk R. Choudhury
David P. Naidich
Sandy Napel
Geoffrey D. Rubin
Publikationsdatum
01.03.2010
Verlag
Springer-Verlag
Erschienen in
European Radiology / Ausgabe 3/2010
Print ISSN: 0938-7994
Elektronische ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-009-1596-y

Weitere Artikel der Ausgabe 3/2010

European Radiology 3/2010 Zur Ausgabe

Screening-Mammografie offenbart erhöhtes Herz-Kreislauf-Risiko

26.04.2024 Mammografie Nachrichten

Routinemäßige Mammografien helfen, Brustkrebs frühzeitig zu erkennen. Anhand der Röntgenuntersuchung lassen sich aber auch kardiovaskuläre Risikopatientinnen identifizieren. Als zuverlässiger Anhaltspunkt gilt die Verkalkung der Brustarterien.

S3-Leitlinie zu Pankreaskrebs aktualisiert

23.04.2024 Pankreaskarzinom Nachrichten

Die Empfehlungen zur Therapie des Pankreaskarzinoms wurden um zwei Off-Label-Anwendungen erweitert. Und auch im Bereich der Früherkennung gibt es Aktualisierungen.

Fünf Dinge, die im Kindernotfall besser zu unterlassen sind

18.04.2024 Pädiatrische Notfallmedizin Nachrichten

Im Choosing-Wisely-Programm, das für die deutsche Initiative „Klug entscheiden“ Pate gestanden hat, sind erstmals Empfehlungen zum Umgang mit Notfällen von Kindern erschienen. Fünf Dinge gilt es demnach zu vermeiden.

„Nur wer sich gut aufgehoben fühlt, kann auch für Patientensicherheit sorgen“

13.04.2024 Klinik aktuell Kongressbericht

Die Teilnehmer eines Forums beim DGIM-Kongress waren sich einig: Fehler in der Medizin sind häufig in ungeeigneten Prozessen und mangelnder Kommunikation begründet. Gespräche mit Patienten und im Team können helfen.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.