Skip to main content
Erschienen in: Japanese Journal of Radiology 11/2020

26.06.2020 | Original Article

Efficiency of a computer-aided diagnosis (CAD) system with deep learning in detection of pulmonary nodules on 1-mm-thick images of computed tomography

verfasst von: Takenori Kozuka, Yuko Matsukubo, Tomoya Kadoba, Teruyoshi Oda, Ayako Suzuki, Tomoko Hyodo, SungWoon Im, Hayato Kaida, Yukinobu Yagyu, Masakatsu Tsurusaki, Mitsuru Matsuki, Kazunari Ishii

Erschienen in: Japanese Journal of Radiology | Ausgabe 11/2020

Einloggen, um Zugang zu erhalten

Abstract

Purpose

To evaluate the performance of a deep learning-based computer-aided diagnosis (CAD) system at detecting pulmonary nodules on CT by comparing radiologists’ readings with and without CAD.

Materials and methods

A total of 120 chest CT images were randomly selected from patients with suspected lung cancer. The gold standard of nodules ≥ 3 mm was established by a panel of three expert radiologists. Two less experienced radiologists read the images without and afterward with CAD system. Their reading times were recorded.

Results

The radiologists’ sensitivity increased from 20.9% to 38.0% with the introduction of CAD. The positive predictive value (PPV) decreased from 70.5% to 61.8%, and the F1-score increased from 32.2% to 47.0%. The sensitivity significantly increased from 13.7% to 32.4% for small nodules (3–6 mm) and from 33.3% to 47.6% for medium nodules (6–10 mm). CAD alone showed a sensitivity of 70.3%, a PPV of 57.9%, and an F1-score of 63.5%. Reading time decreased by 11.3% with the use of CAD.

Conclusion

CAD improved the less experienced radiologists’ sensitivity in detecting pulmonary nodules of all sizes, especially including a significant improvement in the detection of clinically important-sized medium nodules (6–10 mm) as well as small nodules (3–6 mm) and reduced their reading time.
Literatur
1.
Zurück zum Zitat Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global Cancer Statistics 2018. CA Cancer J Clin. 2018;68:394–424.CrossRefPubMed Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global Cancer Statistics 2018. CA Cancer J Clin. 2018;68:394–424.CrossRefPubMed
2.
Zurück zum Zitat El-Baz A, Suri J. Lung imaging and computer aided diagnosis. 1st ed. Abingdon: Taylor and Francis; 2011. El-Baz A, Suri J. Lung imaging and computer aided diagnosis. 1st ed. Abingdon: Taylor and Francis; 2011.
3.
Zurück zum Zitat Li Q. Recent progress in computer-aided diagnosis of lung nodules on thin-section CT. Computed Imaging Graph. 2007;31(4–5):248–57.CrossRef Li Q. Recent progress in computer-aided diagnosis of lung nodules on thin-section CT. Computed Imaging Graph. 2007;31(4–5):248–57.CrossRef
5.
Zurück zum Zitat Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, National Lung Screening Trial Research Team. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365(5):395–409.CrossRefPubMed Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, National Lung Screening Trial Research Team. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365(5):395–409.CrossRefPubMed
6.
Zurück zum Zitat Aberle DR, DeMello S, Berg CD, Black WC, Brewer B, Church TR, et al. Results of the two incidence screenings in the national lung screening trial. N Engl J Med. 2013;369(10):920–31.CrossRefPubMedPubMedCentral Aberle DR, DeMello S, Berg CD, Black WC, Brewer B, Church TR, et al. Results of the two incidence screenings in the national lung screening trial. N Engl J Med. 2013;369(10):920–31.CrossRefPubMedPubMedCentral
7.
Zurück zum Zitat Lo SCB, Freedman MT, Gillis LB, White CS, Mun SK. Computer-aided detection of lung nodules on CT with a computerized pulmonary vessel suppressed function. Am J Roentgenol. 2018;210:480–8.CrossRef Lo SCB, Freedman MT, Gillis LB, White CS, Mun SK. Computer-aided detection of lung nodules on CT with a computerized pulmonary vessel suppressed function. Am J Roentgenol. 2018;210:480–8.CrossRef
8.
Zurück zum Zitat Al Mohammad B, Brennan PC, Mello-Thoms C. A review of lung cancer screening and the role of computer-aided detection. Clin Radiol. 2017;72(6):433–42.CrossRefPubMed Al Mohammad B, Brennan PC, Mello-Thoms C. A review of lung cancer screening and the role of computer-aided detection. Clin Radiol. 2017;72(6):433–42.CrossRefPubMed
9.
Zurück zum Zitat Li F, Sone S, Abe H, MacMahon H, Armato SG, Doi K. Lung cancers missed at low-dose helical CT screening in a general population: comparison of clinical, histopathologic, and imaging findings. Radiology. 2002;225:673–83.CrossRefPubMed Li F, Sone S, Abe H, MacMahon H, Armato SG, Doi K. Lung cancers missed at low-dose helical CT screening in a general population: comparison of clinical, histopathologic, and imaging findings. Radiology. 2002;225:673–83.CrossRefPubMed
10.
Zurück zum Zitat Torres EL, Fiorina E, Pennazio F, Peroni C, Saletta M, Camarlinghi N, et al. Large scale validation of the M5L lung CAD on heterogeneous CT datasets. Med Phys. 2015;42:1477–89.CrossRefPubMed Torres EL, Fiorina E, Pennazio F, Peroni C, Saletta M, Camarlinghi N, et al. Large scale validation of the M5L lung CAD on heterogeneous CT datasets. Med Phys. 2015;42:1477–89.CrossRefPubMed
11.
Zurück zum Zitat Goo JM, Kim HY, Lee JW, Lee HJ, Lee CH, Lee KW, et al. Is the computer-aided detection scheme for lung nodule also useful in detecting lung cancer? J Comput Assist Tomogr. 2008;32(4):570–5.CrossRefPubMed Goo JM, Kim HY, Lee JW, Lee HJ, Lee CH, Lee KW, et al. Is the computer-aided detection scheme for lung nodule also useful in detecting lung cancer? J Comput Assist Tomogr. 2008;32(4):570–5.CrossRefPubMed
12.
Zurück zum Zitat Marten K, Seyfarth T, Auer F, Wiener E, Grillhösl A, Obenauer S, et al. Computer-assisted detection of pulmonary nodules: performance evaluation of an expert knowledge-based detection system in consensus reading with experienced and inexperienced chest radiologists. Eur Radiol. 2004;14:1930–8.CrossRefPubMed Marten K, Seyfarth T, Auer F, Wiener E, Grillhösl A, Obenauer S, et al. Computer-assisted detection of pulmonary nodules: performance evaluation of an expert knowledge-based detection system in consensus reading with experienced and inexperienced chest radiologists. Eur Radiol. 2004;14:1930–8.CrossRefPubMed
13.
Zurück zum Zitat Sahiner B, Pezeshk A, Hadjiiski LM, Wang X, Drukker K, Cha KH, et al. Deep learning in medical imaging and radiation therapy. Med Phys. 2019;46(1):e1–36.CrossRefPubMed Sahiner B, Pezeshk A, Hadjiiski LM, Wang X, Drukker K, Cha KH, et al. Deep learning in medical imaging and radiation therapy. Med Phys. 2019;46(1):e1–36.CrossRefPubMed
15.
16.
Zurück zum Zitat da Silva GLF, Valente TLA, Silva AC, de Paiva AC, Gattass M. Convolutional neural network-based PSO for lung nodule false positive reduction on CT images. Comput Methods Programs Biomed. 2018;162:109–18.CrossRefPubMed da Silva GLF, Valente TLA, Silva AC, de Paiva AC, Gattass M. Convolutional neural network-based PSO for lung nodule false positive reduction on CT images. Comput Methods Programs Biomed. 2018;162:109–18.CrossRefPubMed
17.
Zurück zum Zitat Dou Q, Chen H, Yu L, Qin J, Heng PA. Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans Biomed Eng. 2017;64:1558–677.CrossRefPubMed Dou Q, Chen H, Yu L, Qin J, Heng PA. Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans Biomed Eng. 2017;64:1558–677.CrossRefPubMed
18.
Zurück zum Zitat Li W, Cao P, Zhao D, Wang J. Pulmonary nodule classification with deep convolutional neural networks on computed tomography images. Comput Math Methods Med. 2016;2016:7 (6215085). Li W, Cao P, Zhao D, Wang J. Pulmonary nodule classification with deep convolutional neural networks on computed tomography images. Comput Math Methods Med. 2016;2016:7 (6215085).
19.
Zurück zum Zitat Ren S, He K, Girshick R, Sun J. Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence. Faster R-CNN. 2017;39(6):1137–49. Ren S, He K, Girshick R, Sun J. Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence. Faster R-CNN. 2017;39(6):1137–49.
20.
Zurück zum Zitat MacMahon H, Bankier AA, Naidich DP. Lung cancer screening: what is the effect of using a larger nodule threshold size to determine who is assigned to short-term CT follow-up? Radiology. 2014;273:326–7.CrossRefPubMed MacMahon H, Bankier AA, Naidich DP. Lung cancer screening: what is the effect of using a larger nodule threshold size to determine who is assigned to short-term CT follow-up? Radiology. 2014;273:326–7.CrossRefPubMed
21.
Zurück zum Zitat Vassallo L, Traverso A, Agnello M, Bracco C, Campanella D, Chiara G, et al. A cloud-based computer-aided detection system improves identification of lung nodules on computed tomography scans of patients with extra-thoracic malignancies. Eur Radiol. 2019;29(1):144–52.CrossRefPubMed Vassallo L, Traverso A, Agnello M, Bracco C, Campanella D, Chiara G, et al. A cloud-based computer-aided detection system improves identification of lung nodules on computed tomography scans of patients with extra-thoracic malignancies. Eur Radiol. 2019;29(1):144–52.CrossRefPubMed
22.
Zurück zum Zitat Li L, Liu Z, Huang H, Lin M, Luo D. Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: comparison with the performance of double reading by radiologists. Thoracic Cancer. 2019;10:183–92.CrossRefPubMed Li L, Liu Z, Huang H, Lin M, Luo D. Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: comparison with the performance of double reading by radiologists. Thoracic Cancer. 2019;10:183–92.CrossRefPubMed
23.
Zurück zum Zitat Armato SG III, Li F, Giger ML, MacMahon H, Sone S, Doi K. Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology. 2002;225:685–92.CrossRefPubMed Armato SG III, Li F, Giger ML, MacMahon H, Sone S, Doi K. Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology. 2002;225:685–92.CrossRefPubMed
24.
Zurück zum Zitat Yuan R, Vos PM, Cooperberg PL. Computer-aided detection in screening CT for pulmonary nodules. Am J Roentgenol. 2006;186:1280–7.CrossRef Yuan R, Vos PM, Cooperberg PL. Computer-aided detection in screening CT for pulmonary nodules. Am J Roentgenol. 2006;186:1280–7.CrossRef
25.
Zurück zum Zitat Lee IJ, Gamsu G, Czum J, Wu N, Johnson R, Chakrapani S. Lung nodule detection on chest CT: evaluation of a computer-aided detection (CAD) system. Korean J Radiol. 2005;6:89–93.CrossRefPubMedPubMedCentral Lee IJ, Gamsu G, Czum J, Wu N, Johnson R, Chakrapani S. Lung nodule detection on chest CT: evaluation of a computer-aided detection (CAD) system. Korean J Radiol. 2005;6:89–93.CrossRefPubMedPubMedCentral
26.
Zurück zum Zitat Jacobs C, van Rikxoort EM, Murphy K, Prokop M, Schaefer-Prokop CM, van Ginneken B. Computer-aided detection of pulmonary nodules: a comparative study using the public LIDC/IDRI database. Eur Radiol. 2016;26:2139–47.CrossRefPubMed Jacobs C, van Rikxoort EM, Murphy K, Prokop M, Schaefer-Prokop CM, van Ginneken B. Computer-aided detection of pulmonary nodules: a comparative study using the public LIDC/IDRI database. Eur Radiol. 2016;26:2139–47.CrossRefPubMed
27.
Zurück zum Zitat Zhao Y, de Bock GH, Vliegenthart R, van Klaveren RJ, Wang Y, Bogoni L, et al. Performance of computer-aided detection of pulmonary nodules in low-dose CT: comparison with double reading by nodule volume. Eur Radiol. 2012;22:2076–84.CrossRefPubMedPubMedCentral Zhao Y, de Bock GH, Vliegenthart R, van Klaveren RJ, Wang Y, Bogoni L, et al. Performance of computer-aided detection of pulmonary nodules in low-dose CT: comparison with double reading by nodule volume. Eur Radiol. 2012;22:2076–84.CrossRefPubMedPubMedCentral
28.
Zurück zum Zitat Rubin GD, Lyo JK, Paik DS, Sherbondy AJ, Chow LC, Leung AN, et al. Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology. 2005;234(1):274–83.CrossRefPubMed Rubin GD, Lyo JK, Paik DS, Sherbondy AJ, Chow LC, Leung AN, et al. Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology. 2005;234(1):274–83.CrossRefPubMed
29.
Zurück zum Zitat Retico A. Computer-aided detection for pulmonary nodule identification: improving the radiologist's performance? Imaging Med. 2013;5:249–63.CrossRef Retico A. Computer-aided detection for pulmonary nodule identification: improving the radiologist's performance? Imaging Med. 2013;5:249–63.CrossRef
30.
Zurück zum Zitat Beigelman-Aubry C, Raffy P, Yang W, Castellino RA, Grenier PA. Computed-aided detection of solid lung nodules on followup MDCTscreening: evaluation of detection, tracking, and reading time. Am J Roentgenol. 2007;189:948–55.CrossRef Beigelman-Aubry C, Raffy P, Yang W, Castellino RA, Grenier PA. Computed-aided detection of solid lung nodules on followup MDCTscreening: evaluation of detection, tracking, and reading time. Am J Roentgenol. 2007;189:948–55.CrossRef
31.
Zurück zum Zitat Godoy MCB, Kim TJ, White CS, Bogoni L, de Groot P, Florin C, et al. Benefit of computer-aided detection analysis for the detection of subsolid and solid lung nodules on thin- and thick-section CT. Am J Roentgenol. 2013;200:74–83.CrossRef Godoy MCB, Kim TJ, White CS, Bogoni L, de Groot P, Florin C, et al. Benefit of computer-aided detection analysis for the detection of subsolid and solid lung nodules on thin- and thick-section CT. Am J Roentgenol. 2013;200:74–83.CrossRef
32.
Zurück zum Zitat Wahidi MM, Govert JA, Goudar RK, Gould MK, McCrory DC. Evidence for the treatment of patients with pulmonary nodules: when is it lung cancer? ACCP evidence-based clinical practice guidelines (2nd edition). Chest. 2007;132:94S–107S.CrossRefPubMed Wahidi MM, Govert JA, Goudar RK, Gould MK, McCrory DC. Evidence for the treatment of patients with pulmonary nodules: when is it lung cancer? ACCP evidence-based clinical practice guidelines (2nd edition). Chest. 2007;132:94S–107S.CrossRefPubMed
33.
Zurück zum Zitat Scholten ET, Horeweg N, de Koning HJ, Vliegenthart R, Oudkerk M, Mali WP, et al. Computed tomographic characteristics of interval and post screen carcinomas in lung cancer screening. Eur Radiol. 2015;25:81–8.CrossRefPubMed Scholten ET, Horeweg N, de Koning HJ, Vliegenthart R, Oudkerk M, Mali WP, et al. Computed tomographic characteristics of interval and post screen carcinomas in lung cancer screening. Eur Radiol. 2015;25:81–8.CrossRefPubMed
34.
Zurück zum Zitat Henschke CI, Yankelevitz DF, Yip R, Reeves AP, Farooqi A, Xu D, et al. Lung cancers diagnosed at annual CT screening: volume doubling times. Radiology. 2012;263:578–83.CrossRefPubMedPubMedCentral Henschke CI, Yankelevitz DF, Yip R, Reeves AP, Farooqi A, Xu D, et al. Lung cancers diagnosed at annual CT screening: volume doubling times. Radiology. 2012;263:578–83.CrossRefPubMedPubMedCentral
35.
Zurück zum Zitat Henschke CI, Yankelevitz DF, Mirtcheva R, McGuinness G, McCauley D, Miettinen OS. CT screening for lung cancer: frequency and significance of part-solid and nonsolid nodules. Am J Roentgenol. 2002;178(5):1053–7.CrossRef Henschke CI, Yankelevitz DF, Mirtcheva R, McGuinness G, McCauley D, Miettinen OS. CT screening for lung cancer: frequency and significance of part-solid and nonsolid nodules. Am J Roentgenol. 2002;178(5):1053–7.CrossRef
36.
Zurück zum Zitat Armato SG 3rd, McNitt-Gray MF, Reeves AP, Meyer CR, McLennan G, Aberle DR, et al. The lung image database consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans. Acad Radiol. 2007;14:1409–21.CrossRefPubMedPubMedCentral Armato SG 3rd, McNitt-Gray MF, Reeves AP, Meyer CR, McLennan G, Aberle DR, et al. The lung image database consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans. Acad Radiol. 2007;14:1409–21.CrossRefPubMedPubMedCentral
Metadaten
Titel
Efficiency of a computer-aided diagnosis (CAD) system with deep learning in detection of pulmonary nodules on 1-mm-thick images of computed tomography
verfasst von
Takenori Kozuka
Yuko Matsukubo
Tomoya Kadoba
Teruyoshi Oda
Ayako Suzuki
Tomoko Hyodo
SungWoon Im
Hayato Kaida
Yukinobu Yagyu
Masakatsu Tsurusaki
Mitsuru Matsuki
Kazunari Ishii
Publikationsdatum
26.06.2020
Verlag
Springer Japan
Erschienen in
Japanese Journal of Radiology / Ausgabe 11/2020
Print ISSN: 1867-1071
Elektronische ISSN: 1867-108X
DOI
https://doi.org/10.1007/s11604-020-01009-0

Weitere Artikel der Ausgabe 11/2020

Japanese Journal of Radiology 11/2020 Zur Ausgabe

Letter to the Editor

In reply

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.