Skip to main content
Erschienen in: European Journal of Nuclear Medicine and Molecular Imaging 13/2019

01.07.2019 | Original Article

Novel adversarial semantic structure deep learning for MRI-guided attenuation correction in brain PET/MRI

verfasst von: Hossein Arabi, Guodong Zeng, Guoyan Zheng, Habib Zaidi

Erschienen in: European Journal of Nuclear Medicine and Molecular Imaging | Ausgabe 13/2019

Einloggen, um Zugang zu erhalten

Abstract

Objective

Quantitative PET/MR imaging is challenged by the accuracy of synthetic CT (sCT) generation from MR images. Deep learning-based algorithms have recently gained momentum for a number of medical image analysis applications. In this work, a novel sCT generation algorithm based on deep learning adversarial semantic structure (DL-AdvSS) is proposed for MRI-guided attenuation correction in brain PET/MRI.

Materials and methods

The proposed DL-AdvSS algorithm exploits the ASS learning framework to constrain the synthetic CT generation process to comply with the extracted structural features from CT images. The proposed technique was evaluated through comparison to an atlas-based sCT generation method (Atlas), previously developed for MRI-only or PET/MRI-guided radiation planning. Moreover, the commercial segmentation-based approach (Segm) implemented on the Philips TF PET/MRI system was included in the evaluation. Clinical brain studies of 40 patients who underwent PET/CT and MR imaging were used for the evaluation of the proposed method under a two-fold cross validation scheme.

Results

The accuracy of cortical bone extraction and CT value estimation were investigated for the three different methods. Atlas and DL-AdvSS exhibited similar cortical bone extraction accuracy resulting in a Dice coefficient of 0.78 ± 0.07 and 0.77 ± 0.07, respectively. Likewise, DL-AdvSS and Atlas techniques performed similarly in terms of CT value estimation in the cortical bone region where a mean error (ME) of less than −11 HU was obtained. The Segm approach led to a ME of −1025 HU. Furthermore, the quantitative analysis of corresponding PET images using the three approaches assuming the CT-based attenuation corrected PET (PETCTAC) as reference demonstrated comparative performance of DL-AdvSS and Atlas techniques with a mean standardized uptake value (SUV) bias less than 4% in 63 brain regions. In addition, less that 2% SUV bias was observed in the cortical bone when using Atlas and DL-AdvSS approaches. However, Segm resulted in 14.7 ± 8.9% SUV underestimation in the cortical bone.

Conclusion

The proposed DL-AdvSS approach demonstrated competitive performance with respect to the state-of-the-art atlas-based technique achieving clinically tolerable errors, thus outperforming the commercial segmentation approach used in the clinic.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Zaidi H, Becker M. The promise of hybrid PET/MRI: technical advances and clinical applications. IEEE Signal Process Mag. 2016;33:67–85.CrossRef Zaidi H, Becker M. The promise of hybrid PET/MRI: technical advances and clinical applications. IEEE Signal Process Mag. 2016;33:67–85.CrossRef
2.
Zurück zum Zitat Mehranian A, Arabi H, Zaidi H. Vision 20/20: magnetic resonance imaging-guided attenuation correction in PET/MRI: challenges, solutions, and opportunities. Med Phys. 2016;43:1130–55.CrossRef Mehranian A, Arabi H, Zaidi H. Vision 20/20: magnetic resonance imaging-guided attenuation correction in PET/MRI: challenges, solutions, and opportunities. Med Phys. 2016;43:1130–55.CrossRef
3.
Zurück zum Zitat Erlandsson K, Dickson J, Arridge S, Atkinson D, Ourselin S, Hutton BF. MR imaging-guided partial volume correction of PET data in PET/MR imaging. PET Clinics. 2016;11:161–77.CrossRef Erlandsson K, Dickson J, Arridge S, Atkinson D, Ourselin S, Hutton BF. MR imaging-guided partial volume correction of PET data in PET/MR imaging. PET Clinics. 2016;11:161–77.CrossRef
4.
Zurück zum Zitat Catana C. Motion correction options in PET/MRI. Semin Nucl Med. 2015;45:212–23.CrossRef Catana C. Motion correction options in PET/MRI. Semin Nucl Med. 2015;45:212–23.CrossRef
5.
Zurück zum Zitat Mehranian A, Belzunce M, Prieto C, Hammers A, Reader A. Synergistic PET and SENSE MR image reconstruction using joint sparsity regularization. IEEE Trans Med Imaging. 2018;37:20–34.CrossRef Mehranian A, Belzunce M, Prieto C, Hammers A, Reader A. Synergistic PET and SENSE MR image reconstruction using joint sparsity regularization. IEEE Trans Med Imaging. 2018;37:20–34.CrossRef
6.
Zurück zum Zitat Gong K, Cheng-Liao J, Wang G, Chen KT, Catana C, Qi J. Direct patlak reconstruction from dynamic PET data using the kernel method with MRI information based on structural similarity. IEEE Trans Med Imaging. 2018;37:955–65.CrossRef Gong K, Cheng-Liao J, Wang G, Chen KT, Catana C, Qi J. Direct patlak reconstruction from dynamic PET data using the kernel method with MRI information based on structural similarity. IEEE Trans Med Imaging. 2018;37:955–65.CrossRef
7.
Zurück zum Zitat Zaidi H, Montandon M-L, Slosman DO. Magnetic resonance imaging-guided attenuation and scatter corrections in three-dimensional brain positron emission tomography. Med Phys. 2003;30:937–48.CrossRef Zaidi H, Montandon M-L, Slosman DO. Magnetic resonance imaging-guided attenuation and scatter corrections in three-dimensional brain positron emission tomography. Med Phys. 2003;30:937–48.CrossRef
8.
Zurück zum Zitat Martinez-Moller A, Souvatzoglou M, Delso G, Bundschuh RA, Chefd’hotel C, Ziegler SI, et al. Tissue classification as a potential approach for attenuation correction in whole-body PET/MRI: evaluation with PET/CT data. J Nucl Med. 2009;50:520–6.CrossRef Martinez-Moller A, Souvatzoglou M, Delso G, Bundschuh RA, Chefd’hotel C, Ziegler SI, et al. Tissue classification as a potential approach for attenuation correction in whole-body PET/MRI: evaluation with PET/CT data. J Nucl Med. 2009;50:520–6.CrossRef
9.
Zurück zum Zitat Keereman V, Fierens Y, Broux T, De Deene Y, Lonneux M, Vandenberghe S. MRI-based attenuation correction for PET/MRI using ultrashort echo time sequences. J Nucl Med. 2010;51:812–8.CrossRef Keereman V, Fierens Y, Broux T, De Deene Y, Lonneux M, Vandenberghe S. MRI-based attenuation correction for PET/MRI using ultrashort echo time sequences. J Nucl Med. 2010;51:812–8.CrossRef
10.
Zurück zum Zitat Arabi H, Rager O, Alem A, Varoquaux A, Becker M, Zaidi H. Clinical assessment of MR-guided 3-class and 4-class attenuation correction in PET/MR. Mol Imaging Biol. 2015;17:264–76.CrossRef Arabi H, Rager O, Alem A, Varoquaux A, Becker M, Zaidi H. Clinical assessment of MR-guided 3-class and 4-class attenuation correction in PET/MR. Mol Imaging Biol. 2015;17:264–76.CrossRef
11.
Zurück zum Zitat Wollenweber SD, Ambwani S, Delso G, Lonn AHR, Mullick R, Wiesinger F, et al. Evaluation of an atlas-based PET head attenuation correction using PET/CT & MR patient data. IEEE Trans Nucl Sci. 2013;60:3383–90.CrossRef Wollenweber SD, Ambwani S, Delso G, Lonn AHR, Mullick R, Wiesinger F, et al. Evaluation of an atlas-based PET head attenuation correction using PET/CT & MR patient data. IEEE Trans Nucl Sci. 2013;60:3383–90.CrossRef
12.
Zurück zum Zitat Arabi H, Zaidi H. One registration multi-atlas-based pseudo-CT generation for attenuation correction in PET/MRI. Eur J Nucl Med Mol Imaging. 2016;43:2021–35.CrossRef Arabi H, Zaidi H. One registration multi-atlas-based pseudo-CT generation for attenuation correction in PET/MRI. Eur J Nucl Med Mol Imaging. 2016;43:2021–35.CrossRef
13.
Zurück zum Zitat Burgos N, Cardoso MJ, Guerreiro F, Veiga C, Modat M, McClelland J, et al. Robust CT synthesis for radiotherapy planning: application to the head and neck region. In: International conference on medical image computing and computer-assisted intervention: Springer; 2015. p. 476–84. Burgos N, Cardoso MJ, Guerreiro F, Veiga C, Modat M, McClelland J, et al. Robust CT synthesis for radiotherapy planning: application to the head and neck region. In: International conference on medical image computing and computer-assisted intervention: Springer; 2015. p. 476–84.
14.
Zurück zum Zitat Defrise M, Rezaei A, Nuyts J. Time-of-flight PET data determine the attenuation sinogram up to a constant. Phys Med Biol. 2012;57:885–99.CrossRef Defrise M, Rezaei A, Nuyts J. Time-of-flight PET data determine the attenuation sinogram up to a constant. Phys Med Biol. 2012;57:885–99.CrossRef
15.
Zurück zum Zitat Mehranian A, Zaidi H, Reader AJ. MR-guided joint reconstruction of activity and attenuation in brain PET-MR. NeuroImage. 2017;162:276–88.CrossRef Mehranian A, Zaidi H, Reader AJ. MR-guided joint reconstruction of activity and attenuation in brain PET-MR. NeuroImage. 2017;162:276–88.CrossRef
16.
Zurück zum Zitat Rezaei A, Deroose CM, Vahle T, Boada F, Nuyts J. Joint reconstruction of activity and attenuation in fime-of-flight PET: a quantitative analysis. J Nucl Med. 2018;59:1624–9.CrossRef Rezaei A, Deroose CM, Vahle T, Boada F, Nuyts J. Joint reconstruction of activity and attenuation in fime-of-flight PET: a quantitative analysis. J Nucl Med. 2018;59:1624–9.CrossRef
17.
Zurück zum Zitat Huynh T, Gao Y, Kang J, Wang L, Zhang P, Lian J, et al. Estimating CT image from MRI data using structured random forest and auto-context model. IEEE Trans Med Imaging. 2016;35:174–83.CrossRef Huynh T, Gao Y, Kang J, Wang L, Zhang P, Lian J, et al. Estimating CT image from MRI data using structured random forest and auto-context model. IEEE Trans Med Imaging. 2016;35:174–83.CrossRef
18.
Zurück zum Zitat Han X. MR-based synthetic CT generation using a deep convolutional neural network method. Med Phys. 2017;44:1408–19.CrossRef Han X. MR-based synthetic CT generation using a deep convolutional neural network method. Med Phys. 2017;44:1408–19.CrossRef
19.
Zurück zum Zitat Leynes AP, Yang J, Wiesinger F, Kaushik SS, Shanbhag DD, Seo Y, et al. Zero-echo-time and Dixon deep pseudo-CT (ZeDD CT): direct generation of pseudo-CT images for pelvic PET/MRI attenuation correction using deep convolutional neural networks with multiparametric MRI. J Nucl Med. 2018;59:852–8.CrossRef Leynes AP, Yang J, Wiesinger F, Kaushik SS, Shanbhag DD, Seo Y, et al. Zero-echo-time and Dixon deep pseudo-CT (ZeDD CT): direct generation of pseudo-CT images for pelvic PET/MRI attenuation correction using deep convolutional neural networks with multiparametric MRI. J Nucl Med. 2018;59:852–8.CrossRef
20.
Zurück zum Zitat Nie D, Trullo R, Lian J, Petitjean C, Ruan S, Wang Q, et al. Medical image synthesis with context-aware generative adversarial network. In: Medical image computing and computer-assisted intervention − MICCAI 2017, Quebec, Canada; Springer, 2017. pp 417–25. Nie D, Trullo R, Lian J, Petitjean C, Ruan S, Wang Q, et al. Medical image synthesis with context-aware generative adversarial network. In: Medical image computing and computer-assisted intervention − MICCAI 2017, Quebec, Canada; Springer, 2017. pp 417–25.
21.
Zurück zum Zitat Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology. 2018;286:676–84.CrossRef Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology. 2018;286:676–84.CrossRef
22.
Zurück zum Zitat Greenspan H, van Ginneken B, Summers RM. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging. 2016;35:1153–9.CrossRef Greenspan H, van Ginneken B, Summers RM. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging. 2016;35:1153–9.CrossRef
23.
Zurück zum Zitat Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88.CrossRef Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88.CrossRef
24.
Zurück zum Zitat Emami H, Dong M, Nejad-Davarani SP, Glide-Hurst CK. Generating synthetic CTs from magnetic resonance images using generative adversarial networks. Med Phys. 2018;45:3627–36.CrossRef Emami H, Dong M, Nejad-Davarani SP, Glide-Hurst CK. Generating synthetic CTs from magnetic resonance images using generative adversarial networks. Med Phys. 2018;45:3627–36.CrossRef
25.
Zurück zum Zitat Dinkla AM, Wolterink JM, Maspero M, Savenije MHF, Verhoeff JJC, Seravalli E, et al. MR-only brain radiation therapy: dosimetric evaluation of synthetic CTs generated by a dilated convolutional neural network. Int J Radiat Oncol Biol Phys. 2018;102:801–12.CrossRef Dinkla AM, Wolterink JM, Maspero M, Savenije MHF, Verhoeff JJC, Seravalli E, et al. MR-only brain radiation therapy: dosimetric evaluation of synthetic CTs generated by a dilated convolutional neural network. Int J Radiat Oncol Biol Phys. 2018;102:801–12.CrossRef
26.
Zurück zum Zitat Hwang D, Kim KY, Kang SK, Seo S, Paeng JC, Lee DS, et al. Improving the accuracy of simultaneously reconstructed activity and attenuation maps using deep learning. J Nucl Med. 2018;59:1624–9.CrossRef Hwang D, Kim KY, Kang SK, Seo S, Paeng JC, Lee DS, et al. Improving the accuracy of simultaneously reconstructed activity and attenuation maps using deep learning. J Nucl Med. 2018;59:1624–9.CrossRef
27.
Zurück zum Zitat Chen S, Qin A, Zhou D, Yan D. Technical note: U-net-generated synthetic CT images for magnetic resonance imaging-only prostate intensity-modulated radiation therapy treatment planning. Med Phys. 2018;45:5659–65.CrossRef Chen S, Qin A, Zhou D, Yan D. Technical note: U-net-generated synthetic CT images for magnetic resonance imaging-only prostate intensity-modulated radiation therapy treatment planning. Med Phys. 2018;45:5659–65.CrossRef
28.
Zurück zum Zitat Xiang L, Wang Q, Nie D, Zhang L, Jin X, Qiao Y, et al. Deep embedding convolutional neural network for synthesizing CT image from T1-weighted MR image. Med Image Anal. 2018;47:31–44.CrossRef Xiang L, Wang Q, Nie D, Zhang L, Jin X, Qiao Y, et al. Deep embedding convolutional neural network for synthesizing CT image from T1-weighted MR image. Med Image Anal. 2018;47:31–44.CrossRef
29.
Zurück zum Zitat Arabi H, Koutsouvelis N, Rouzaud M, Miralbell R, Zaidi H. Atlas-guided generation of pseudo-CT images for MRI-only and hybrid PET–MRI-guided radiotherapy treatment planning. Phys Med Biol. 2016;61:6531–52.CrossRef Arabi H, Koutsouvelis N, Rouzaud M, Miralbell R, Zaidi H. Atlas-guided generation of pseudo-CT images for MRI-only and hybrid PET–MRI-guided radiotherapy treatment planning. Phys Med Biol. 2016;61:6531–52.CrossRef
30.
Zurück zum Zitat Arabi H, Dowling JA, Burgos N, Han X, Greer PB, Koutsouvelis N, et al. Comparative study of algorithms for synthetic CT generation from MRI: consequences for MRI-guided radiation planning in the pelvic region. Med Phys. 2018;45:5218–33.CrossRef Arabi H, Dowling JA, Burgos N, Han X, Greer PB, Koutsouvelis N, et al. Comparative study of algorithms for synthetic CT generation from MRI: consequences for MRI-guided radiation planning in the pelvic region. Med Phys. 2018;45:5218–33.CrossRef
31.
Zurück zum Zitat Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on machine learning, Lille, France, vol. 37; 2015. pp 448–56. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on machine learning, Lille, France, vol. 37; 2015. pp 448–56.
32.
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60:84–90.CrossRef Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60:84–90.CrossRef
33.
Zurück zum Zitat Zaidi H, Ojha N, Morich M, Griesmer J, Hu Z, Maniawski P, et al. Design and performance evaluation of a whole-body ingenuity TF PET-MRI system. Phys Med Biol. 2011;56:3091–106.CrossRef Zaidi H, Ojha N, Morich M, Griesmer J, Hu Z, Maniawski P, et al. Design and performance evaluation of a whole-body ingenuity TF PET-MRI system. Phys Med Biol. 2011;56:3091–106.CrossRef
34.
Zurück zum Zitat Zaidi H, Montandon M-L, Meikle S. Strategies for attenuation compensation in neurological PET studies. Neuroimage. 2007;34:518–41.CrossRef Zaidi H, Montandon M-L, Meikle S. Strategies for attenuation compensation in neurological PET studies. Neuroimage. 2007;34:518–41.CrossRef
35.
Zurück zum Zitat Mehranian A, Arabi H, Zaidi H. Quantitative analysis of MRI-guided attenuation correction techniques in time-of-flight brain PET/MRI. NeuroImage. 2016;130:123–33.CrossRef Mehranian A, Arabi H, Zaidi H. Quantitative analysis of MRI-guided attenuation correction techniques in time-of-flight brain PET/MRI. NeuroImage. 2016;130:123–33.CrossRef
36.
Zurück zum Zitat Chartsias A, Joyce T, Dharmakumar R, Tsaftaris SA. Adversarial image synthesis for unpaired nulti-modal cardiac data. In: International workshop on simulation and synthesis in medical imaging, SASHIMI; 2017. pp 3–13. Chartsias A, Joyce T, Dharmakumar R, Tsaftaris SA. Adversarial image synthesis for unpaired nulti-modal cardiac data. In: International workshop on simulation and synthesis in medical imaging, SASHIMI; 2017. pp 3–13.
37.
Zurück zum Zitat Huo Y, Xu Z, Bao S, Assad A, Abramson RG, Landman BA. Adversarial synthesis learning enables segmentation without target modality ground truth. In: IEEE 15th International Symposium on Biomedical Imaging (ISBI); 2018. pp 1217–20. Huo Y, Xu Z, Bao S, Assad A, Abramson RG, Landman BA. Adversarial synthesis learning enables segmentation without target modality ground truth. In: IEEE 15th International Symposium on Biomedical Imaging (ISBI); 2018. pp 1217–20.
38.
Zurück zum Zitat Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. IEEE international conference on computer vision. 2017. pp. 2223-2232. Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. IEEE international conference on computer vision. 2017. pp. 2223-2232.
39.
Zurück zum Zitat Su KH, Hu L, Stehning C, Helle M, Qian P, Thompson CL, et al. Generation of brain pseudo-CTs using an undersampled, single-acquisition UTE-mDixon pulse sequence and unsupervised clustering. Med Phys. 2015;42:4974–86.CrossRef Su KH, Hu L, Stehning C, Helle M, Qian P, Thompson CL, et al. Generation of brain pseudo-CTs using an undersampled, single-acquisition UTE-mDixon pulse sequence and unsupervised clustering. Med Phys. 2015;42:4974–86.CrossRef
40.
Zurück zum Zitat Gong K, Yang J, Kim K, El Fakhri G, Seo Y, Li Q. Attenuation correction for brain PET imaging using deep neural network based on Dixon and ZTE MR images. Phys Med Biol. 2018;63:125011.CrossRef Gong K, Yang J, Kim K, El Fakhri G, Seo Y, Li Q. Attenuation correction for brain PET imaging using deep neural network based on Dixon and ZTE MR images. Phys Med Biol. 2018;63:125011.CrossRef
41.
Zurück zum Zitat Liu F, Jang H, Kijowski R, Zhao G, Bradshaw T, McMillan AB. A deep learning approach for 18 F-FDG PET attenuation correction. EJNMMI Phys. 2018;5:24.CrossRef Liu F, Jang H, Kijowski R, Zhao G, Bradshaw T, McMillan AB. A deep learning approach for 18 F-FDG PET attenuation correction. EJNMMI Phys. 2018;5:24.CrossRef
42.
Zurück zum Zitat Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep learning MR imaging–based attenuation correction for PET/MR imaging. Radiology. 2017;286:676–84.CrossRef Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep learning MR imaging–based attenuation correction for PET/MR imaging. Radiology. 2017;286:676–84.CrossRef
43.
Zurück zum Zitat Gong K, Yang J, Kim K, El Fakhri G, Seo Y, Li Q. Attenuation correction for brain PET imaging using deep neural network based on Dixon and ZTE MR images. Phys Med Biol. 2018;63(12):125011.CrossRef Gong K, Yang J, Kim K, El Fakhri G, Seo Y, Li Q. Attenuation correction for brain PET imaging using deep neural network based on Dixon and ZTE MR images. Phys Med Biol. 2018;63(12):125011.CrossRef
44.
Zurück zum Zitat Spuhler KD, Gardus J, Gao Y, DeLorenzo C, Parsey R, Huang C. Synthesis of patient-specific transmission image for PET attenuation correction for PET/MR imaging of the brain using a convolutional neural network. J Nucl Med. 2019;60:555–60.CrossRef Spuhler KD, Gardus J, Gao Y, DeLorenzo C, Parsey R, Huang C. Synthesis of patient-specific transmission image for PET attenuation correction for PET/MR imaging of the brain using a convolutional neural network. J Nucl Med. 2019;60:555–60.CrossRef
Metadaten
Titel
Novel adversarial semantic structure deep learning for MRI-guided attenuation correction in brain PET/MRI
verfasst von
Hossein Arabi
Guodong Zeng
Guoyan Zheng
Habib Zaidi
Publikationsdatum
01.07.2019
Verlag
Springer Berlin Heidelberg
Erschienen in
European Journal of Nuclear Medicine and Molecular Imaging / Ausgabe 13/2019
Print ISSN: 1619-7070
Elektronische ISSN: 1619-7089
DOI
https://doi.org/10.1007/s00259-019-04380-x

Weitere Artikel der Ausgabe 13/2019

European Journal of Nuclear Medicine and Molecular Imaging 13/2019 Zur Ausgabe