Artificial Intelligence-powered automatic volume calculation in medical images – available tools, performance and challenges for nuclear medicine

 

 Permissions and Reprints

Abstract

Volumetry is crucial in oncology and endocrinology, for diagnosis, treatment planning, and evaluating response to therapy for several diseases. The integration of Artificial Intelligence (AI) and Deep Learning (DL) has significantly accelerated the automatization of volumetric calculations, enhancing accuracy and reducing variability and labor. In this review, we show that a high correlation has been observed between Machine Learning (ML) methods and expert assessments in tumor volumetry; Yet, it is recognized as more challenging than organ volumetry. Liver volumetry has shown progression in accuracy with a decrease in error. If a relative error below 10 % is acceptable, ML-based liver volumetry can be considered reliable for standardized imaging protocols if used in patients without major anomalies. Similarly, ML-supported automatic kidney volumetry has also shown consistency and reliability in volumetric calculations. In contrast, AI-supported thyroid volumetry has not been extensively developed, despite initial works in 3D ultrasound showing promising results in terms of accuracy and reproducibility. Despite the advancements presented in the reviewed literature, the lack of standardization limits the generalizability of ML methods across diverse scenarios. The domain gap, i. e., the difference in probability distribution of training and inference data, is of paramount importance before clinical deployment of AI, to maintain accuracy and reliability in patient care. The increasing availability of improved segmentation tools is expected to further incorporate AI methods into routine workflows where volumetry will play a more prominent role in radionuclide therapy planning and quantitative follow-up of disease evolution.

Publication History

Received: 16 October 2023

Accepted: 26 October 2023

Article published online:
23 November 2023

© 2023. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial-License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

• References

• 1 Riveira-Martin M, Akhavanallaf A, Mansouri Z. et al. Predictive value of 99mTc-MAA-based dosimetry in personalized 90Y-SIRT planning for liver malignancies. EJNMMI Res 2023; 13 (01) 63
  CrossrefPubMedGoogle Scholar
• 2 Krönke M, Eilers C, Dimova D. et al. Tracked 3D ultrasound and deep neural network-based thyroid segmentation reduce interobserver variability in thyroid volumetry. PLOS ONE 2022; 17 (07) e0268550
  PubMedGoogle Scholar
• 3 Sunoqrot MRS, Selnæs KM, Sandsmark E. et al. The Reproducibility of Deep Learning-Based Segmentation of the Prostate Gland and Zones on T2-Weighted MR Images. Diagn Basel Switz 2021; 11 (09) 1690
  Google Scholar
• 4 Ding J, Cao P, Chang HC. et al. Deep learning-based thigh muscle segmentation for reproducible fat fraction quantification using fat–water decomposition MRI. Insights Imaging 2020; 11 (01) 128
  CrossrefPubMedGoogle Scholar
• 5 Seifert P, Ullrich SL, Kühnel C. et al. Optimization of Thyroid Volume Determination by Stitched 3D-Ultrasound Data Sets in Patients with Structural Thyroid Disease. Biomedicines 2023; 11 (02) 381
  CrossrefPubMedGoogle Scholar
• 6 Brunn J, Block U, Ruf G. et al. [Volumetric analysis of thyroid lobes by real-time ultrasound (author’s transl)]. Dtsch Med Wochenschr 1946 1981; 106 (41) 1338-1340
  Google Scholar
• 7 Campennì A, Avram AM, Verburg FA. et al. The EANM guideline on radioiodine therapy of benign thyroid disease. Eur J Nucl Med Mol Imaging [Internet] 2023; DOI: 10.1007/s00259-023-06274-5.
  CrossrefPubMedGoogle Scholar
• 8 Muggli D, Müller MA, Karlo C. et al. A simple method to approximate liver size on cross-sectional images using living liver models. Clin Radiol 2009; 64 (07) 682-689
  CrossrefPubMedGoogle Scholar
• 9 Hussain MA, Hamarneh G, Garbi R. Cascaded Regression Neural Nets for Kidney Localization and Segmentation-free Volume Estimation. IEEE Trans Med Imaging 2021; 40 (06) 1555-1567
  CrossrefPubMedGoogle Scholar
• 10 Weber M, Telli T, Kersting D. et al. Prognostic Implications of PET-Derived Tumor Volume and Uptake in Patients with Neuroendocrine Tumors. Cancers 2023; 15 (14) 3581
  CrossrefPubMedGoogle Scholar
• 11 Mikhaeel NG, Heymans MW, Eertink JJ. et al. Proposed New Dynamic Prognostic Index for Diffuse Large B-Cell Lymphoma: International Metabolic Prognostic Index. J Clin Oncol Off J Am Soc Clin Oncol 2022; 40 (21) 2352-2360
  CrossrefPubMedGoogle Scholar
• 12 Gafita A, Calais J, Grogan TR. et al. Nomograms to predict outcomes after 177Lu-PSMA therapy in men with metastatic castration-resistant prostate cancer: an international, multicentre, retrospective study. Lancet Oncol 2021; 22 (08) 1115-1125
  CrossrefPubMedGoogle Scholar
• 13 Seifert R, Herrmann K, Kleesiek J. et al. Semiautomatically Quantified Tumor Volume Using 68Ga-PSMA-11 PET as a Biomarker for Survival in Patients with Advanced Prostate Cancer. J Nucl Med Off Publ Soc Nucl Med 2020; 61 (12) 1786-1792
  Google Scholar
• 14 Hutchings M, Barrington SF. PET/CT for therapy response assessment in lymphoma. J Nucl Med Off Publ Soc Nucl Med 2009; 50 (Suppl. 01) 21S-30S
  Google Scholar
• 15 Isensee F, Jaeger PF, Kohl SAA. et al. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 2021; 18 (02) 203-211
  CrossrefPubMedGoogle Scholar
• 16 Gutsche R, Lowis C, Ziemons K. et al. Automated Brain Tumor Detection and Segmentation for Treatment Response Assessment Using Amino Acid PET. J Nucl Med Off Publ Soc Nucl Med 2023; DOI: 10.2967/jnumed.123.265725.
  CrossrefPubMedGoogle Scholar
• 17 Vivanti R, Szeskin A, Lev-Cohain N. et al. Automatic detection of new tumors and tumor burden evaluation in longitudinal liver CT scan studies. Int J Comput Assist Radiol Surg 2017; 12 (11) 1945-1957
  CrossrefPubMedGoogle Scholar
• 18 Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM. et al. Medical Image Computing and Computer-Assisted Intervention – MICCAI. Cham: Springer International Publishing; 2015: 234-241
  Google Scholar
• 19 Shapey J, Wang G, Dorent R. et al. An artificial intelligence framework for automatic segmentation and volumetry of vestibular schwannomas from contrast-enhanced T1-weighted and high-resolution T2-weighted MRI. J Neurosurg 2019; 134 (01) 171-179
  CrossrefPubMedGoogle Scholar
• 20 Kang H, Witanto JN, Pratama K. et al. Fully Automated MRI Segmentation and Volumetric Measurement of Intracranial Meningioma Using Deep Learning. J Magn Reson Imaging JMRI 2023; 57 (03) 871-881
  CrossrefPubMedGoogle Scholar
• 21 Oktay O, Schlemper J, Folgoc LL. Attention U-Net: Learning Where to Look for the Pancreas. In: Conference Book [Internet]. Amsterdam: Radboud University Medical Center; 2018 [cited 2023 Sep 21]. 15 Available from: https://openreview.net/pdf?id=Skft7cijM
  Google Scholar
• 22 Toyonaga T, Shao D, Shi L. et al. Deep learning-based attenuation correction for whole-body PET – a multi-tracer study with 18F-FDG, 68 Ga-DOTATATE, and 18F-Fluciclovine. Eur J Nucl Med Mol Imaging 2022; 49 (09) 3086-3097
  CrossrefPubMedGoogle Scholar
• 23 Hirata K, Furuya S, Huang SC. et al. A semi-automated method to separate tumor from physiological uptakes on FDG PET-CT for efficient generation of training data targeting deep learning. J Nucl Med 2019; 60 (Suppl. 01) 1213-1213
  PubMedGoogle Scholar
• 24 Van Ginneken B, Heimann T, Styner M. 3D segmentation in the clinic: A grand challenge. In: MICCAI workshop on 3D segmentation in the clinic: a grand challenge. 2007: 7-15
  PubMedGoogle Scholar
• 25 Soler L, Hostettler A, Agnus V. et al 3D image reconstruction for comparison of algorithm database. 2010 Available from: https://www.ircad.fr/research/data-sets/liver-segmentation-3d-ircadb-01
  PubMedGoogle Scholar
• 26 CodaLab – Competition [Internet]. [cited 2023 Sep 25]. Available from: https://competitions.codalab.org/competitions/17094#results
  Google Scholar
• 27 Takamoto T, Ban D, Nara S. et al. Automated Three-Dimensional Liver Reconstruction with Artificial Intelligence for Virtual Hepatectomy. J Gastrointest Surg Off J Soc Surg Aliment Tract 2022; 26 (10) 2119-2127
  CrossrefPubMedGoogle Scholar
• 28 Lu F, Wu F, Hu P. et al. Automatic 3D liver location and segmentation via convolutional neural network and graph cut. Int J Comput Assist Radiol Surg 2017; 12 (02) 171-182
  CrossrefPubMedGoogle Scholar
• 29 Marinelli B, Kang M, Martini M. et al. Combination of Active Transfer Learning and Natural Language Processing to Improve Liver Volumetry Using Surrogate Metrics with Deep Learning. Radiol Artif Intell 2019; 1 (01) e180019
  CrossrefPubMedGoogle Scholar
• 30 Ibtehaz N, Rahman MS. MultiResUNet : Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw Off J Int Neural Netw Soc 2020; 121: 74-87
  CrossrefPubMedGoogle Scholar
• 31 Koitka S, Gudlin P, Theysohn JM. et al. Fully automated preoperative liver volumetry incorporating the anatomical location of the central hepatic vein. Sci Rep 2022; 12 (01) 16479
  CrossrefPubMedGoogle Scholar
• 32 Shin TY, Kim H, Lee JH. et al. Expert-level segmentation using deep learning for volumetry of polycystic kidney and liver. Investig Clin Urol 2020; 61 (06) 555-564
  CrossrefPubMedGoogle Scholar
• 33 Milletari F, Navab N, Ahmadi SA. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In IEEE;. 2016: 565-571
  PubMedGoogle Scholar
• 34 Cayot B, Milot L, Nempont O. et al. Polycystic liver: automatic segmentation using deep learning on CT is faster and as accurate compared to manual segmentation. Eur Radiol 2022; 32 (07) 4780-4790
  CrossrefPubMedGoogle Scholar
• 35 Ng YS, Xi Y, Qian Y. et al. Use of Spectral Detector Computed Tomography to Improve Liver Segmentation and Volumetry. J Comput Assist Tomogr 2020; 44 (02) 197-203
  CrossrefPubMedGoogle Scholar
• 36 Chlebus G, Meine H, Thoduka S. et al. Reducing inter-observer variability and interaction time of MR liver volumetry by combining automatic CNN-based liver segmentation and manual corrections. PloS One 2019; 14 (05) e0217228
  CrossrefPubMedGoogle Scholar
• 37 Saunders SL, Clark JM, Rudser K. et al. Comparison of automatic liver volumetry performance using different types of magnetic resonance images. Magn Reson Imaging 2022; 91: 16-23
  CrossrefPubMedGoogle Scholar
• 38 Wang K, Mamidipalli A, Retson T. et al. Automated CT and MRI Liver Segmentation and Biometry Using a Generalized Convolutional Neural Network. Radiol Artif Intell 2019; 1 (02) 180022
  CrossrefPubMedGoogle Scholar
• 39 Bilic P, Christ P, Li HB. et al. The Liver Tumor Segmentation Benchmark (LiTS). Med Image Anal 2023; 84: 102680
  CrossrefPubMedGoogle Scholar
• 40 Sundlöv A, Sjögreen-Gleisner K, Svensson J. et al. Individualised 177Lu-DOTATATE treatment of neuroendocrine tumours based on kidney dosimetry. Eur J Nucl Med Mol Imaging 2017; 44 (09) 1480-1489
  CrossrefPubMedGoogle Scholar
• 41 Hsiao CH, Sun TL, Lin PC. et al. A deep learning-based precision volume calculation approach for kidney and tumor segmentation on computed tomography images. Comput Methods Programs Biomed 2022; 221: 106861
  CrossrefPubMedGoogle Scholar
• 42 Heller N, Sathianathen N, Kalapara A. et al The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes [Internet]. arXiv; 2020 [cited 2023 Sep 21]. Available from: http://arxiv.org/abs/1904.00445
  Google Scholar
• 43 Licht K, Darr A, Opfermann T. et al. 3D ultrasonography is as accurate as low-dose CT in thyroid volumetry. Nukl Nucl Med 2014; 53 (03) 99-104
  Article in Thieme ConnectPubMedGoogle Scholar
• 44 Reinartz P, Sabri O, Zimny M. et al. Thyroid volume measurement in patients prior to radioiodine therapy: comparison between three-dimensional magnetic resonance imaging and ultrasonography. Thyroid Off J Am Thyroid Assoc 2002; 12 (08) 713-717
  CrossrefPubMedGoogle Scholar
• 45 Chang CY, Lei YF, Tseng CH. et al. Thyroid segmentation and volume estimation in ultrasound images. IEEE Trans Biomed Eng 2010; 57 (06) 1348-1357
  CrossrefPubMedGoogle Scholar
• 46 Ham FM, Kostanic I. Principles of Neurocomputing for Science and Engineering. McGraw Hill 2000; 680
  Google Scholar
• 47 Kennedy J, Eberhart R. Particle swarm optimization. In: Proceedings of ICNN’95 – International Conference on Neural Networks [Internet]. Perth, WA, Australia: IEEE; 1995 [cited 2023 Sep 25]. 1942-1948 Available from: http://ieeexplore.ieee.org/document/488968/
  Google Scholar
• 48 Eberhart R, Kennedy J. A new optimizer using particle swarm theory. MHS95 Proc Sixth Int Symp Micro Mach Hum Sci [Internet]. 1995: 39-43 Available from: http://ieeexplore.ieee.org/document/494215/
  PubMedGoogle Scholar
• 49 Kumar V, Webb J, Gregory A. et al. Automated Segmentation of Thyroid Nodule, Gland, and Cystic Components From Ultrasound Images Using Deep Learning. IEEE Access Pract Innov Open Solut 2020; 8: 63482-63496
  Google Scholar
• 50 Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions [Internet]. arXiv; 2016 [cited 2023 Sep 25]. Available from: http://arxiv.org/abs/1511.07122
  Google Scholar
• 51 Chan TF, Vese LA. Active contours without edges. IEEE Trans Image Process [Internet]. 2001; 10 (02) 266-277 http://ieeexplore.ieee.org/document/902291/
  PubMedGoogle Scholar
• 52 Rother C, Kolmogorov V, Blake A. ‘GrabCut’: interactive foreground extraction using iterated graph cuts. ACM Trans Graph [Internet]. 2004; 23 (03) 309-314 DOI: 10.1145/1015706.1015720.
  CrossrefGoogle Scholar
• 53 Breiman L. Random Forests. Mach Learn [Internet]. 2001; 45 (01) 5-32 DOI: 10.1023/A:1010933404324.
  CrossrefGoogle Scholar
• 54 Criminisi A, Shotton J. Decision Forests for Computer Vision and Medical Image Analysis. Springer Science & Business Media 2013; 367
  Google Scholar
• 55 Krönke M, Eilers C, Dimova D. et al. Tracked 3D ultrasound and deep neural network-based thyroid segmentation reduce interobserver variability in thyroid volumetry. PloS One 2022; 17 (07) e0268550
  CrossrefPubMedGoogle Scholar
• 56 Dietlein M, Grünwald F, Schmidt M. et al. Radioiodtherapie bei benignen Schilddrüsenerkrankungen (Version 5)*. Nukl – Nucl [Internet]. 2016; 55 (06) 213-220 DOI: 10.3413/Nukmed-0823-16-04.
  Article in Thieme ConnectPubMedGoogle Scholar
• 57 Guha RoyA, Conjeti S, Navab N. et al. QuickNAT: A fully convolutional network for quick and accurate segmentation of neuroanatomy. NeuroImage [Internet] 2019; 186: 713-727 https://www.sciencedirect.com/science/article/pii/S1053811918321232
  CrossrefPubMedGoogle Scholar
• 58 Zielke J, Eilers C, Busam B. et al. RSV: Robotic Sonography for Thyroid Volumetry. IEEE Robot Autom Lett 2022; 7 (02) 3342-3348
  CrossrefGoogle Scholar
• 59 Yu SH, Choi SJ, Noh H. et al. Comparison of CT Volumetry and RECIST to Predict the Treatment Response and Overall Survival in Gastric Cancer Liver Metastases. Taehan Yongsang Uihakhoe Chi 2021; 82 (04) 876-888
  PubMedGoogle Scholar
• 60 Hofmann FO, Heinemann V, D’Anastasi M. et al. Standard diametric versus volumetric early tumor shrinkage as a predictor of survival in metastatic colorectal cancer: subgroup findings of the randomized, open-label phase III trial FIRE-3/AIO KRK-0306. Eur Radiol 2023; 33 (02) 1174-1184
  CrossrefPubMedGoogle Scholar
• 61 Siegel MJ, Ippolito JE, Wahl RL. et al. Discrepant Assessments of Progressive Disease in Clinical Trials between Routine Clinical Reads and Formal RECIST 1.1 Interpretations. Radiol Imaging Cancer 2023; 5 (05) e230001
  CrossrefPubMedGoogle Scholar
• 62 Garin E, Tselikas L, Guiu B. et al. Personalised versus standard dosimetry approach of selective internal radiation therapy in patients with locally advanced hepatocellular carcinoma (DOSISPHERE-01): a randomised, multicentre, open-label phase 2 trial. Lancet Gastroenterol Hepatol 2021; 6 (01) 17-29
  CrossrefPubMedGoogle Scholar
• 63 Pacilio M, Conte M, Frantellizzi V. et al. Personalized Dosimetry in the Context of Radioiodine Therapy for Differentiated Thyroid Cancer. Diagn Basel Switz 2022; 12 (07) 1763
  Google Scholar
• 64 Mariani G, Tonacchera M, Grosso M. et al. The Role of Nuclear Medicine in the Clinical Management of Benign Thyroid Disorders, Part 1: Hyperthyroidism. J Nucl Med Off Publ Soc Nucl Med 2021; 62 (03) 304-312
  Google Scholar
• 65 van Heek L, Weindler J, Gorniak C. et al. Prognostic value of baseline metabolic tumor volume (MTV) for forecasting chemotherapy outcome in early-stage unfavorable Hodgkin lymphoma: Data from the phase III HD17 trial. Eur J Haematol 2023; 111 (06) 881-887
  CrossrefPubMedGoogle Scholar