Skip to main content
Erschienen in: European Radiology 11/2022

08.04.2022 | Imaging Informatics and Artificial Intelligence

Deep learning–based algorithm improved radiologists’ performance in bone metastases detection on CT

verfasst von: Shunjiro Noguchi, Mizuho Nishio, Ryo Sakamoto, Masahiro Yakami, Koji Fujimoto, Yutaka Emoto, Takeshi Kubo, Yoshio Iizuka, Keita Nakagomi, Kazuhiro Miyasa, Kiyohide Satoh, Yuji Nakamoto

Erschienen in: European Radiology | Ausgabe 11/2022

Einloggen, um Zugang zu erhalten

Abstract

Objectives

To develop and evaluate a deep learning–based algorithm (DLA) for automatic detection of bone metastases on CT.

Methods

This retrospective study included CT scans acquired at a single institution between 2009 and 2019. Positive scans with bone metastases and negative scans without bone metastasis were collected to train the DLA. Another 50 positive and 50 negative scans were collected separately from the training dataset and were divided into validation and test datasets at a 2:3 ratio. The clinical efficacy of the DLA was evaluated in an observer study with board-certified radiologists. Jackknife alternative free-response receiver operating characteristic analysis was used to evaluate observer performance.

Results

A total of 269 positive scans including 1375 bone metastases and 463 negative scans were collected for the training dataset. The number of lesions identified in the validation and test datasets was 49 and 75, respectively. The DLA achieved a sensitivity of 89.8% (44 of 49) with 0.775 false positives per case for the validation dataset and 82.7% (62 of 75) with 0.617 false positives per case for the test dataset. With the DLA, the overall performance of nine radiologists with reference to the weighted alternative free-response receiver operating characteristic figure of merit improved from 0.746 to 0.899 (p < .001). Furthermore, the mean interpretation time per case decreased from 168 to 85 s (p = .004).

Conclusion

With the aid of the algorithm, the overall performance of radiologists in bone metastases detection improved, and the interpretation time decreased at the same time.

Key Points

A deep learning–based algorithm for automatic detection of bone metastases on CT was developed.
In the observer study, overall performance of radiologists in bone metastases detection improved significantly with the aid of the algorithm.
Radiologists’ interpretation time decreased at the same time.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Coleman RE (2001) Metastatic bone disease: clinical features, pathophysiology and treatment strategies. Cancer Treat Rev 27:165–176CrossRef Coleman RE (2001) Metastatic bone disease: clinical features, pathophysiology and treatment strategies. Cancer Treat Rev 27:165–176CrossRef
2.
Zurück zum Zitat Macedo F, Ladeira K, Pinho F et al (2017) Bone metastases: an overview. Oncol Rev 11:321 Macedo F, Ladeira K, Pinho F et al (2017) Bone metastases: an overview. Oncol Rev 11:321
3.
Zurück zum Zitat D’Oronzo S, Coleman R, Brown J, Silvestris F (2019) Metastatic bone disease: pathogenesis and therapeutic options: up-date on bone metastasis management. J Bone Oncol 15:100205CrossRef D’Oronzo S, Coleman R, Brown J, Silvestris F (2019) Metastatic bone disease: pathogenesis and therapeutic options: up-date on bone metastasis management. J Bone Oncol 15:100205CrossRef
4.
Zurück zum Zitat O’Sullivan GJ, Carty FL, Cronin CG (2015) Imaging of bone metastasis: an update. World J Radiol 7:202–211CrossRef O’Sullivan GJ, Carty FL, Cronin CG (2015) Imaging of bone metastasis: an update. World J Radiol 7:202–211CrossRef
5.
Zurück zum Zitat Heindel W, Gübitz R, Vieth V, Weckesser M, Schober O, Schäfers M (2014) The diagnostic imaging of bone metastases. Dtsch Arztebl Int 111:741–747 Heindel W, Gübitz R, Vieth V, Weckesser M, Schober O, Schäfers M (2014) The diagnostic imaging of bone metastases. Dtsch Arztebl Int 111:741–747
6.
Zurück zum Zitat Kalogeropoulou C, Karachaliou A, Zampakis P (2009) Radiologic evaluation of skeletal metastases: role of plain radiographs and computed tomography. In: Cancer metastasis – biology and treatment, 12:119–136. Springer, Dordrecht Kalogeropoulou C, Karachaliou A, Zampakis P (2009) Radiologic evaluation of skeletal metastases: role of plain radiographs and computed tomography. In: Cancer metastasis – biology and treatment, 12:119–136. Springer, Dordrecht
7.
Zurück zum Zitat Groves AM, Beadsmoore CJ, Cheow HK et al (2006) Can 16-detector multislice CT exclude skeletal lesions during tumour staging? Implications for the cancer patient. Eur Radiol 16:1066–1073CrossRef Groves AM, Beadsmoore CJ, Cheow HK et al (2006) Can 16-detector multislice CT exclude skeletal lesions during tumour staging? Implications for the cancer patient. Eur Radiol 16:1066–1073CrossRef
8.
Zurück zum Zitat Chmelik J, Jakubicek R, Walek P et al (2018) Deep convolutional neural network-based segmentation and classification of difficult to define metastatic spinal lesions in 3D CT data. Med Image Anal 49:76–88CrossRef Chmelik J, Jakubicek R, Walek P et al (2018) Deep convolutional neural network-based segmentation and classification of difficult to define metastatic spinal lesions in 3D CT data. Med Image Anal 49:76–88CrossRef
9.
Zurück zum Zitat Hammon M, Dankerl P, Tsymbal A et al (2013) Automatic detection of lytic and blastic thoracolumbar spine metastases on computed tomography. Eur Radiol 23:1862–1870CrossRef Hammon M, Dankerl P, Tsymbal A et al (2013) Automatic detection of lytic and blastic thoracolumbar spine metastases on computed tomography. Eur Radiol 23:1862–1870CrossRef
10.
Zurück zum Zitat Vandemark RM, Shpall EJ, Lou AM (1992) Bone metastases from breast cancer: value of CT bone windows. J Comput Assist Tomogr 16:608–614CrossRef Vandemark RM, Shpall EJ, Lou AM (1992) Bone metastases from breast cancer: value of CT bone windows. J Comput Assist Tomogr 16:608–614CrossRef
11.
Zurück zum Zitat Pomerantz SM, White CS, Krebs TL et al (2000) Liver and bone window settings for soft-copy interpretation of chest and abdominal CT. AJR Am J Roentgenol 174:311–314 Pomerantz SM, White CS, Krebs TL et al (2000) Liver and bone window settings for soft-copy interpretation of chest and abdominal CT. AJR Am J Roentgenol 174:311–314
12.
Zurück zum Zitat Burns JE, Yao J, Wiese TS, Muñoz HE, Jones EC, Summers RM (2013) Automated detection of sclerotic metastases in the thoracolumbar spine at CT. Radiology 268:69–78CrossRef Burns JE, Yao J, Wiese TS, Muñoz HE, Jones EC, Summers RM (2013) Automated detection of sclerotic metastases in the thoracolumbar spine at CT. Radiology 268:69–78CrossRef
13.
Zurück zum Zitat Choy G, Khalilzadeh O, Michalski M et al (2018) Current applications and future impact of machine learning in radiology. Radiology 288:318–328CrossRef Choy G, Khalilzadeh O, Michalski M et al (2018) Current applications and future impact of machine learning in radiology. Radiology 288:318–328CrossRef
14.
Zurück zum Zitat Roth HR, Lu L, Liu J et al (2016) Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging 35:1170–1181CrossRef Roth HR, Lu L, Liu J et al (2016) Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging 35:1170–1181CrossRef
15.
Zurück zum Zitat Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: MICCAI 2015. Lecture Notes in Computer Science, 9351:234–241. Springer, Cham Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: MICCAI 2015. Lecture Notes in Computer Science, 9351:234–241. Springer, Cham
16.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) p770–778. Las Vegas He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) p770–778. Las Vegas
17.
Zurück zum Zitat Noguchi S, Nishio M, Yakami M, Nakagomi K, Togashi K (2020) Bone segmentation on whole-body CT using convolutional neural network with novel data augmentation techniques. Comput Biol Med 121:103767CrossRef Noguchi S, Nishio M, Yakami M, Nakagomi K, Togashi K (2020) Bone segmentation on whole-body CT using convolutional neural network with novel data augmentation techniques. Comput Biol Med 121:103767CrossRef
18.
Zurück zum Zitat Zou KH, Warfield SK, Bharatha A et al (2004) Statistical validation of image segmentation quality based on a spatial overlap index. Acad Radiol 11:178–189CrossRef Zou KH, Warfield SK, Bharatha A et al (2004) Statistical validation of image segmentation quality based on a spatial overlap index. Acad Radiol 11:178–189CrossRef
19.
Zurück zum Zitat Chakraborty DP, Zhai X (2016) On the meaning of the weighted alternative free-response operating characteristic figure of merit. Med Phys 43:2548–2557CrossRef Chakraborty DP, Zhai X (2016) On the meaning of the weighted alternative free-response operating characteristic figure of merit. Med Phys 43:2548–2557CrossRef
20.
Zurück zum Zitat Chakraborty DP (2017) Observer performance methods for diagnostic imaging: foundations, modeling, and applications with R-based examples. CRC Press, Boca RatonCrossRef Chakraborty DP (2017) Observer performance methods for diagnostic imaging: foundations, modeling, and applications with R-based examples. CRC Press, Boca RatonCrossRef
22.
Zurück zum Zitat Sakamoto R, Yakami M, Fujimoto K et al (2017) Temporal subtraction of serial CT images with large deformation diffeomorphic metric mapping in the identification of bone metastases. Radiology 285:629–639CrossRef Sakamoto R, Yakami M, Fujimoto K et al (2017) Temporal subtraction of serial CT images with large deformation diffeomorphic metric mapping in the identification of bone metastases. Radiology 285:629–639CrossRef
23.
Zurück zum Zitat Nakamoto Y, Osman M, Wahl RL (2003) Prevalence and patterns of bone metastases detected with positron emission tomography using F-18 FDG. Clin Nucl Med 28:302–307CrossRef Nakamoto Y, Osman M, Wahl RL (2003) Prevalence and patterns of bone metastases detected with positron emission tomography using F-18 FDG. Clin Nucl Med 28:302–307CrossRef
24.
Zurück zum Zitat Kakhki VRD, Anvari K, Sadeghi R, Mahmoudian AS, Torabian-Kakhki M (2013) Pattern and distribution of bone metastases in common malignant tumors. Nucl Med Rev 16:66–69CrossRef Kakhki VRD, Anvari K, Sadeghi R, Mahmoudian AS, Torabian-Kakhki M (2013) Pattern and distribution of bone metastases in common malignant tumors. Nucl Med Rev 16:66–69CrossRef
25.
Zurück zum Zitat Kobatake H (2007) Future CAD in multi-dimensional medical images: - project on multi-organ, multi-disease CAD system -. Comput Med Imaging Graph 31:258–266 Kobatake H (2007) Future CAD in multi-dimensional medical images: - project on multi-organ, multi-disease CAD system -. Comput Med Imaging Graph 31:258–266
26.
Zurück zum Zitat Liu K, Li Q, Ma J et al (2019) Evaluating a fully automated pulmonary nodule detection approach and its impact on radiologist performance. Radiol Artif Intell 1:e180084CrossRef Liu K, Li Q, Ma J et al (2019) Evaluating a fully automated pulmonary nodule detection approach and its impact on radiologist performance. Radiol Artif Intell 1:e180084CrossRef
27.
Zurück zum Zitat Xie H, Yang D, Sun N, Chen Z, Zhang Y (2019) Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pattern Recognit 85:109–119CrossRef Xie H, Yang D, Sun N, Chen Z, Zhang Y (2019) Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pattern Recognit 85:109–119CrossRef
28.
Zurück zum Zitat Pehrson LM, Nielsen MB, Lauridsen CA (2019) Automatic pulmonary nodule detection applying deep learning or machine learning algorithms to the LIDC-IDRI database: a systematic review. Diagnostics 9:29CrossRef Pehrson LM, Nielsen MB, Lauridsen CA (2019) Automatic pulmonary nodule detection applying deep learning or machine learning algorithms to the LIDC-IDRI database: a systematic review. Diagnostics 9:29CrossRef
29.
Zurück zum Zitat Chlebus G, Schenk A, Moltz JH, van Ginneken B, Hahn HK, Meine H (2018) Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing. Sci Rep 8:15497CrossRef Chlebus G, Schenk A, Moltz JH, van Ginneken B, Hahn HK, Meine H (2018) Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing. Sci Rep 8:15497CrossRef
30.
Zurück zum Zitat Vorontsov E, Cerny M, Régnier P et al (2019) Deep learning for automated segmentation of liver lesions at ct in patients with colorectal cancer liver metastases. Radiol Artif Intell 1:e180014CrossRef Vorontsov E, Cerny M, Régnier P et al (2019) Deep learning for automated segmentation of liver lesions at ct in patients with colorectal cancer liver metastases. Radiol Artif Intell 1:e180014CrossRef
31.
Zurück zum Zitat Azer SA (2019) Deep learning with convolutional neural networks for identification of liver masses and hepatocellular carcinoma: a systematic review. World J Gastrointest Oncol 11:1218–1230CrossRef Azer SA (2019) Deep learning with convolutional neural networks for identification of liver masses and hepatocellular carcinoma: a systematic review. World J Gastrointest Oncol 11:1218–1230CrossRef
32.
Zurück zum Zitat van Leeuwen KG, Schalekamp S, Rutten MJCM, van Ginneken B, de Rooij M (2021) Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur Radiol 31:3797–3804CrossRef van Leeuwen KG, Schalekamp S, Rutten MJCM, van Ginneken B, de Rooij M (2021) Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur Radiol 31:3797–3804CrossRef
33.
Zurück zum Zitat Çiray I, Åström G, Sundström C, Hagberg H, Ahlström H (1997) Assessment of suspected bone metastases: CT with and without clinical information compared to CT-guided bone biopsy. Acta Radiol 38:890–895 Çiray I, Åström G, Sundström C, Hagberg H, Ahlström H (1997) Assessment of suspected bone metastases: CT with and without clinical information compared to CT-guided bone biopsy. Acta Radiol 38:890–895
Metadaten
Titel
Deep learning–based algorithm improved radiologists’ performance in bone metastases detection on CT
verfasst von
Shunjiro Noguchi
Mizuho Nishio
Ryo Sakamoto
Masahiro Yakami
Koji Fujimoto
Yutaka Emoto
Takeshi Kubo
Yoshio Iizuka
Keita Nakagomi
Kazuhiro Miyasa
Kiyohide Satoh
Yuji Nakamoto
Publikationsdatum
08.04.2022
Verlag
Springer Berlin Heidelberg
Erschienen in
European Radiology / Ausgabe 11/2022
Print ISSN: 0938-7994
Elektronische ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-022-08741-3

Weitere Artikel der Ausgabe 11/2022

European Radiology 11/2022 Zur Ausgabe

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.