Skip to main content
Erschienen in: Molecular Imaging and Biology 1/2022

07.10.2021 | Research Article

Image-to-Images Translation for Multiple Virtual Histological Staining of Unlabeled Human Carotid Atherosclerotic Tissue

verfasst von: Guanghao Zhang, Bin Ning, Hui Hui, Tengfei Yu, Xin Yang, Hongxia Zhang, Jie Tian, Wen He

Erschienen in: Molecular Imaging and Biology | Ausgabe 1/2022

Einloggen, um Zugang zu erhalten

Abstract

Purpose

Histological analysis of human carotid atherosclerotic plaques is critical in understanding atherosclerosis biology and developing effective plaque prevention and treatment for ischemic stroke. However, the histological staining process is laborious, tedious, variable, and destructive to the highly valuable atheroma tissue obtained from patients.

Procedures

We proposed a deep learning-based method to simultaneously transfer bright-field microscopic images of unlabeled tissue sections into equivalent multiple sections of the same samples that are virtually stained. Using a pix2pix model, we trained a generative adversarial neural network to achieve image-to-images translation of multiple stains, including hematoxylin and eosin (H&E), picrosirius red (PSR), and Verhoeff van Gieson (EVG) stains.

Results

The quantification of evaluation metrics indicated that the proposed approach achieved the best performance in comparison with other state-of-the-art methods. Further blind evaluation by board-certified pathologists demonstrated that the multiple virtual stains have high consistency with standard histological stains. The proposed approach also indicated that the generated histopathological features of atherosclerotic plaques, such as the necrotic core, neovascularization, cholesterol crystals, collagen, and elastic fibers, are optimally matched with those of standard histological stains.

Conclusions

The proposed approach allows for the virtual staining of unlabeled human carotid plaque tissue images with multiple types of stains. In addition, it identifies the histopathological features of atherosclerotic plaques in the same tissue sample, which could facilitate the development of personalized prevention and other interventional treatments for carotid atherosclerosis.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
2.
Zurück zum Zitat Pelisek J, Well G, Reeps C et al (2012) Neovascularization and angiogenic factors in advanced human carotid artery stenosis. Circ J 76:1274–1282CrossRef Pelisek J, Well G, Reeps C et al (2012) Neovascularization and angiogenic factors in advanced human carotid artery stenosis. Circ J 76:1274–1282CrossRef
3.
Zurück zum Zitat Pelisek J, Pongratz J, Deutsch L, Reeps C, Stadlbauer T, Eckstein HH (2012) Expression and cellular localization of metalloproteases ADAMs in high graded carotid artery lesions. Scand J Clin Lab Inv 72:648–656CrossRef Pelisek J, Pongratz J, Deutsch L, Reeps C, Stadlbauer T, Eckstein HH (2012) Expression and cellular localization of metalloproteases ADAMs in high graded carotid artery lesions. Scand J Clin Lab Inv 72:648–656CrossRef
4.
Zurück zum Zitat Zhong XY, Ma ZC, Su YS et al (2020) Flavin adenine dinucleotide ameliorates hypertensive vascular remodeling via activating short chain acyl-CoA dehydrogenase. Life Sci 258:118156CrossRef Zhong XY, Ma ZC, Su YS et al (2020) Flavin adenine dinucleotide ameliorates hypertensive vascular remodeling via activating short chain acyl-CoA dehydrogenase. Life Sci 258:118156CrossRef
5.
Zurück zum Zitat Rivenson Y, de Haan K, Wallace WD, Ozcan A (2020) Emerging advances to transform histopathology using virtual staining. BME Frontiers 2020:1–11CrossRef Rivenson Y, de Haan K, Wallace WD, Ozcan A (2020) Emerging advances to transform histopathology using virtual staining. BME Frontiers 2020:1–11CrossRef
6.
Zurück zum Zitat Croce AC, Bottiroli G (2014) Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis. Eur J Histochem 58:2461PubMedPubMedCentral Croce AC, Bottiroli G (2014) Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis. Eur J Histochem 58:2461PubMedPubMedCentral
7.
Zurück zum Zitat Jamme F, Kascakova S, Villette S et al (2013) Deep UV autofluorescence microscopy for cell biology and tissue histology. Biol Cell 105:277–288CrossRef Jamme F, Kascakova S, Villette S et al (2013) Deep UV autofluorescence microscopy for cell biology and tissue histology. Biol Cell 105:277–288CrossRef
8.
Zurück zum Zitat Le TT, Langohr IM, Locker MJ, Sturek M, Cheng JX (2007) Label-free molecular imaging of atherosclerotic lesions using multimodal nonlinear optical microscopy. J Biomed Opt 12:54007CrossRef Le TT, Langohr IM, Locker MJ, Sturek M, Cheng JX (2007) Label-free molecular imaging of atherosclerotic lesions using multimodal nonlinear optical microscopy. J Biomed Opt 12:54007CrossRef
9.
Zurück zum Zitat Zoumi A, Yeh A, Tromberg BJ (2002) Imaging cells and extracellular matrix In vivo by using second-harmonic generation and two-photon excited fluorescence. P Natl Acad Sci USA 99:11014–11019CrossRef Zoumi A, Yeh A, Tromberg BJ (2002) Imaging cells and extracellular matrix In vivo by using second-harmonic generation and two-photon excited fluorescence. P Natl Acad Sci USA 99:11014–11019CrossRef
10.
Zurück zum Zitat Witte S, Negrean A, Lodder JC et al (2011) Label-free live brain imaging and targeted patching with third-harmonic generation microscopy. Proc Natl Acad Sci U S A 108:5970–5975CrossRef Witte S, Negrean A, Lodder JC et al (2011) Label-free live brain imaging and targeted patching with third-harmonic generation microscopy. Proc Natl Acad Sci U S A 108:5970–5975CrossRef
11.
Zurück zum Zitat Ji M, Orringer DA, Freudiger CW et al (2013) Rapid, label-free detection of brain tumors with stimulated Raman scattering microscopy. Sci Transl Med 5:201ra119CrossRef Ji M, Orringer DA, Freudiger CW et al (2013) Rapid, label-free detection of brain tumors with stimulated Raman scattering microscopy. Sci Transl Med 5:201ra119CrossRef
12.
Zurück zum Zitat Orringer DA, Pandian B, Niknafs YS et al (2017) Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat Biomed Eng 1:27CrossRef Orringer DA, Pandian B, Niknafs YS et al (2017) Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat Biomed Eng 1:27CrossRef
13.
Zurück zum Zitat Seeger M, Karlas A, Soliman D, Pelisek J, Ntziachristos V (2016) Multimodal optoacoustic and multiphoton microscopy of human carotid atheroma. Photoacoustics 4:102–111CrossRef Seeger M, Karlas A, Soliman D, Pelisek J, Ntziachristos V (2016) Multimodal optoacoustic and multiphoton microscopy of human carotid atheroma. Photoacoustics 4:102–111CrossRef
14.
Zurück zum Zitat Bayramoglu N, Kaakinen M, Eklund L, Heikkila J (2017) Towards virtual H&E staining of hyperspectral lung histology images using conditional generative adversarial networks. Ieee Int Conf Comp V:64–71. Bayramoglu N, Kaakinen M, Eklund L, Heikkila J (2017) Towards virtual H&E staining of hyperspectral lung histology images using conditional generative adversarial networks. Ieee Int Conf Comp V:64–71.
15.
Zurück zum Zitat Rivenson Y, Wang H, Wei Z et al (2019) Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng 3:466–477CrossRef Rivenson Y, Wang H, Wei Z et al (2019) Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng 3:466–477CrossRef
16.
Zurück zum Zitat Rivenson Y, Liu TR, Wei ZS, Zhang Y, de Haan K, Ozcan A (2019) PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light-Sci Appl 8:23CrossRef Rivenson Y, Liu TR, Wei ZS, Zhang Y, de Haan K, Ozcan A (2019) PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light-Sci Appl 8:23CrossRef
17.
Zurück zum Zitat Christiansen EM, Yang SJ, Ando DM et al (2018) In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173:792CrossRef Christiansen EM, Yang SJ, Ando DM et al (2018) In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173:792CrossRef
18.
Zurück zum Zitat Liu Y, Yuan H, Wang ZY, Ji SW (2020) Global pixel transformers for virtual staining of microscopy images. Ieee T Med Imaging 39:2256–2266CrossRef Liu Y, Yuan H, Wang ZY, Ji SW (2020) Global pixel transformers for virtual staining of microscopy images. Ieee T Med Imaging 39:2256–2266CrossRef
19.
Zurück zum Zitat Li D, Hui H, Zhang YQ et al (2020) Deep learning for virtual histological staining of bright-field microscopic images of unlabeled carotid artery tissue. Mol Imaging Biol 22:1301–1309CrossRef Li D, Hui H, Zhang YQ et al (2020) Deep learning for virtual histological staining of bright-field microscopic images of unlabeled carotid artery tissue. Mol Imaging Biol 22:1301–1309CrossRef
20.
Zurück zum Zitat Zhang Y, de Haan K, Rivenson Y, Li J, Delis A, Ozcan A (2020) Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light Sci Appl 9:78CrossRef Zhang Y, de Haan K, Rivenson Y, Li J, Delis A, Ozcan A (2020) Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light Sci Appl 9:78CrossRef
21.
Zurück zum Zitat Zhou NY, Cai D, Han X, Yao JH (2019) Enhanced cycle-consistent generative adversarial network for color normalization of H&E stained images. Lect Notes Comput Sc 11764:694–702CrossRef Zhou NY, Cai D, Han X, Yao JH (2019) Enhanced cycle-consistent generative adversarial network for color normalization of H&E stained images. Lect Notes Comput Sc 11764:694–702CrossRef
22.
Zurück zum Zitat Gupta L, Klinkhammer BM, Boor P, Merhof D, Gadermayr M (2019) GAN-based image enrichment in digital pathology boosts segmentation accuracy. Lect Notes Comput Sc 11764:631–639CrossRef Gupta L, Klinkhammer BM, Boor P, Merhof D, Gadermayr M (2019) GAN-based image enrichment in digital pathology boosts segmentation accuracy. Lect Notes Comput Sc 11764:631–639CrossRef
23.
Zurück zum Zitat Isola P, Zhu JY, Zhou TH, Efros AA (2017) Image-to-image translation with conditional adversarial networks. Proc Cvpr Ieee:5967–5976. Isola P, Zhu JY, Zhou TH, Efros AA (2017) Image-to-image translation with conditional adversarial networks. Proc Cvpr Ieee:5967–5976.
24.
Zurück zum Zitat Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. Med Imag Comput Comput Assist Interv Pt Iii 9351:234–241 Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. Med Imag Comput Comput Assist Interv Pt Iii 9351:234–241
25.
Zurück zum Zitat Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. Ieee T Image Process 13:600–612CrossRef Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. Ieee T Image Process 13:600–612CrossRef
26.
Zurück zum Zitat Zhang R, Isola P, Efros AA, Shechtman E, Wang O (2018) The unreasonable effectiveness of deep features as a perceptual metric. 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr):586–595. Zhang R, Isola P, Efros AA, Shechtman E, Wang O (2018) The unreasonable effectiveness of deep features as a perceptual metric. 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr):586–595.
27.
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun Acm 60:84–90CrossRef Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun Acm 60:84–90CrossRef
28.
Zurück zum Zitat Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr):8789–8797. Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr):8789–8797.
29.
Zurück zum Zitat He ZL, Zuo WM, Kan MN, Shan SG, Chen XL (2019) AttGAN: facial attribute editing by only changing what you want. Ieee T Image Process 28:5464–5478CrossRef He ZL, Zuo WM, Kan MN, Shan SG, Chen XL (2019) AttGAN: facial attribute editing by only changing what you want. Ieee T Image Process 28:5464–5478CrossRef
30.
Zurück zum Zitat Liu M, Ding YK, Xia M, et al. (2019) STGAN: a unified selective transfer network for arbitrary image attribute editing. 2019 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr 2019):3668–3677. Liu M, Ding YK, Xia M, et al. (2019) STGAN: a unified selective transfer network for arbitrary image attribute editing. 2019 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr 2019):3668–3677.
31.
Zurück zum Zitat Wang W, Zhang Y, Hui H et al (2021) The effect of endothelial progenitor cell transplantation on neointimal hyperplasia and reendothelialisation after balloon catheter injury in rat carotid arteries. Stem Cell Res Ther 12:99CrossRef Wang W, Zhang Y, Hui H et al (2021) The effect of endothelial progenitor cell transplantation on neointimal hyperplasia and reendothelialisation after balloon catheter injury in rat carotid arteries. Stem Cell Res Ther 12:99CrossRef
32.
Zurück zum Zitat Tong W, Hui H, Shang W et al (2021) Highly sensitive magnetic particle imaging of vulnerable atherosclerotic plaque with active myeloperoxidase-targeted nanoparticles. Theranostics 11:506–521CrossRef Tong W, Hui H, Shang W et al (2021) Highly sensitive magnetic particle imaging of vulnerable atherosclerotic plaque with active myeloperoxidase-targeted nanoparticles. Theranostics 11:506–521CrossRef
Metadaten
Titel
Image-to-Images Translation for Multiple Virtual Histological Staining of Unlabeled Human Carotid Atherosclerotic Tissue
verfasst von
Guanghao Zhang
Bin Ning
Hui Hui
Tengfei Yu
Xin Yang
Hongxia Zhang
Jie Tian
Wen He
Publikationsdatum
07.10.2021
Verlag
Springer International Publishing
Erschienen in
Molecular Imaging and Biology / Ausgabe 1/2022
Print ISSN: 1536-1632
Elektronische ISSN: 1860-2002
DOI
https://doi.org/10.1007/s11307-021-01641-w

Weitere Artikel der Ausgabe 1/2022

Molecular Imaging and Biology 1/2022 Zur Ausgabe

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.