Skip to main content
Erschienen in: Molecular Imaging and Biology 5/2020

Open Access 08.06.2020 | Research Article

Deep Learning for Virtual Histological Staining of Bright-Field Microscopic Images of Unlabeled Carotid Artery Tissue

verfasst von: Dan Li, Hui Hui, Yingqian Zhang, Wei Tong, Feng Tian, Xin Yang, Jie Liu, Yundai Chen, Jie Tian

Erschienen in: Molecular Imaging and Biology | Ausgabe 5/2020

Abstract

Purpose

Histological analysis of artery tissue samples is a widely used method for diagnosis and quantification of cardiovascular diseases. However, the variable and labor-intensive tissue staining procedures hinder efficient and informative histological image analysis.

Procedures

In this study, we developed a deep learning-based method to transfer bright-field microscopic images of unlabeled tissue sections into equivalent bright-field images of histologically stained versions of the same samples. We trained a convolutional neural network to build maps between the unstained images and histologically stained images using a conditional generative adversarial network model.

Results

The results of a blind evaluation by board-certified pathologists illustrate that the virtual staining and standard histological staining images of rat carotid artery tissue sections and those involving different types of stains showed no major differences. Quantification of virtual and histological H&E staining in carotid artery tissue sections showed that the relative errors of intima thickness, intima area, and media area were lower than 1.6 %, 5.6 %, and 12.7 %, respectively. The training time of deep learning network was 12.857 h with 1800 training patches and 200 epoches.

Conclusions

This virtual staining method significantly mitigates the typically laborious and time-consuming histological staining procedures and could be augmented with other label-free microscopic imaging modalities.
Begleitmaterial
Hinweise

Electronic supplementary material

The online version of this article (https://​doi.​org/​10.​1007/​s11307-020-01508-6) contains supplementary material, which is available to authorized users.
Dan Li, Hui Hui and Yingqian Zhang contributed equally to this work.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Introduction

Coronary artery disease (CAD) is the leading cause of mortality globally. Fundamental studies regarding CAD pathophysiological mechanisms and potential therapeutic methods have considerable clinical and scientific significance, which highly rely on histology analysis of artery tissue sections [14]. Histological staining can be used to identify arterial features such as intima, media, collagen, and elastic lamina. However, histological staining of artery tissue is an invasive and laborious process that typically includes the artery tissue being fixed and paraffin-embedded, sectioned into 2–10-μm thin slides, chemically stained or fluorescently labeled, mounted on a glass slide, and imaged using a bright-field microscope. Inevitably, the variability of histologically stained tissue sections in these irreversible steps introduces major challenges in histopathological image analysis. These variations are due to human-to-human variability, differences in protocols and microscopes between labs, and color variations in staining procedures [5]. More importantly, to identify various arterial features, multiple stains of the same tissue section are required, whereas the standard histological staining procedure is applied for one type of stain on single artery sections. The time-consuming histological staining procedures also create obstacles for fast pathological diagnosis. Recently, tissue-sectioning microscopies, such as confocal and multiphoton microscopes, have been applied for non-invasive volumetric or quantitative measurements of artery tissue sections to accelerate and improve the microscopic imaging step in this workflow [6, 7]. Nevertheless, tissue-sectioning microscopy requires fluorescence agents as imaging probes in contrast to specific artery tissue compositions.
The use of nonlinear microscopy has been suggested for visualization of unstained tissue samples based on tissue autofluorescence [8, 9], two-photon fluorescence [6, 10], second-harmonic generation [11], third-harmonic generation [12], and Raman scattering [13, 14]. Moreover, optoacoustic imaging has also been investigated for label-free imaging of red blood cells in atheroma [15, 16]. Furthermore, multimodal multiphoton and optoacoustic microscopy was proposed for hybrid imaging of histological features and moieties in excised human carotid atheroma [17]. However, these microscopy methods require ultrafast pulse lasers or spectrometers, which have lower output power, and their implementation on a microscope is a complex process. Therefore, they might not be readily available in most settings and require relatively long scanning times because of the weak optical signals.
Although the aforementioned state-of-the-art microscopy techniques have unique capabilities to visualize different histological moieties in tissue samples using exogenous staining, autofluorescence, or intrinsic information, pathologists are trained to examine histologically stained tissue samples to make diagnostic decisions. Inspired by this, recently, efforts have been focused on virtually creating histological staining images by training microscopic images of tissue samples via deep learning-based methods. Rivenson et al. trained a network that maps the autofluorescence microscopic images of unlabeled tissues to the histological staining images using a convolutional neural network (CNN) [18]. Another study used conditional generative adversarial networks (cGANs) to transfer unstained hyperspectral lung histological images to their corresponding hematoxylin and eosin (H&E) staining images [19]. To address the problem of stain color variations, as approach based on unpaired image-to-image translation using cycle-consistent adversarial networks has also been proposed [20]. Bautista and Yagi [21] proposed a method to transform H&E-stained multispectral images into its Masson’s trichromatic stained equivalent by enhancement spectral transmittance and linear transformation.
In this study, we propose a deep learning-based virtual staining method to generate virtually stained images from bright-field microscopic images of unlabeled rat carotid artery tissue sections imaged with a conventional wide-field microscope (Fig. 1). We trained a deep CNN using the concept of cGAN to match the bright-field microscopic images of unstained tissue sections after obtaining standard histological stains (Fig. 2 and Suppl. Fig. 1). Thus, we could replace the histological staining and bright-field imaging steps with the output of the trained neural network, which is fed with the bright-field microscopic images of the unstained tissue.
We demonstrated the applicability of this deep learning-based virtual histological staining method by imaging fresh rat carotid artery tissue samples. The network output created equivalent images that were well matched with the images of the same samples labeled with three different stains—H&E, picrosirius red (PSR), and orcein stain. Furthermore, the staining efficacy of our approach was blindly evaluated by three board-certificated pathologists. They could recognize histopathological features in images generated with our virtual staining technique, and a high degree of agreement was observed between the histologically stained images and the virtually stained images. Quantification of virtual and histological staining in tissue sections shows that our deep learning-based virtual staining method achieved results comparable to the histological staining technique.

Materials and Methods

Sample Preparation and Image Acquisition

The carotid artery balloon injury rats (male SD, 200–250 g, purchased from experimental Animal center of Chinese PLA general Hospital) model was built using a previously well-established method [22]. The injured artery was excised aseptically from euthanized animals, washed with phosphate-buffered saline (PBS), and fixed with 4 % buffered formalin for 24 h at 4 °C in the dark. The 4-μm cross sections of paraffin-embedded rat carotid artery were collected and immediately imaged with an inverted microscope (Leica DM IL LED) equipped with a × 10/0.22 NA objective. After a bright-field imaging of unlabeled tissue sections, the corresponding slides were histologically stained with hematoxylin and eosin (H&E), picrosirius red (PSR), and orcein. The images from H&E-, PSR-, and orcein-stained sections were acquired with the same configuration of the microscope. These tissue-staining methods are only performed for the training and validation of the proposed approach. The samples were obtained from the Department of Cardiology, Chinese PLA General Hospital, and were prepared by the Histology Laboratory at Chinese PLA General Hospital. All animal experiments were approved in accordance with the guidelines of the Institutional Animal Care and Use Committee (IACUC) of Chinese PLA General Hospital.

Preparing Dataset for Training

Our virtual staining networks transformed unstained tissue images to H&E-, PSR-, and orcein-stained images. Following the bright-field imaging, unstained tissue slides underwent a standard histological staining procedure. To acquire the input and target images with the same resolution and field of view (FOV), microscopic imaging of unstained sections and corresponding stained sections was performed at the same imaging system. However, the unstained tissues may be deformed during histological staining. Therefore, it is crucial to register the FOVs of the input-targeted image pairs. In this work, we performed rigid registration to align unstained and stained images. We acquired 60 images for the unstained, H&E-, PSR-, and orcein-stained groups. In total, 240 whole-slide images (WSIs) were obtained. Each WSI (1079 × 1079 pixels) was randomly cropped into 25 smaller overlapping patches (500 × 500 pixels). After eliminating the patches without intima and media, we obtained training image pairs (1800, 1500, and 1500) and testing image pairs (200, 150, and 150) for virtual H&E, PSR, and orcein staining, respectively.

Conditional Generative Adversarial Network Architecture

To learn the nonlinear mapping from unstained images to standard histological staining of the sample, we utilized a conditional generative adversarial network (cGAN) [23], which is an extension of the generative adversarial network (GAN) [24]. GAN is based on deep convolution learning network, which consists of two networks: generator (G) and discriminator (D). G is responsible for generating new images from prior random data distribution by simulating the real image data distribution. The discriminator D learns a rule to distinguish the images generated by G or the real histological staining images in our case. The two networks have a competitive relationship and are trained at the same time.
In this work, the generator D and the discriminator G of the cGAN comprised a U-net architecture [25] and of PatchGAN [26], respectively, as shown in Fig. 2. The cGAN architecture was updated with the following changes: downsampling path learns image context information and upsampling path target location added to the cGAN model to facilitate input images of 500 × 500 pixels.
To acquire results without generating blurring images, we chose the L1 norm instead of the L2-norm (mean squared error) penalty as a cost function [23, 27]. We defined the loss function of the cGAN as follows:
$$ {L}_{\mathrm{cGAN}}\left(G,D\right)={E}_{x,y\sim {p}_{\mathrm{data}}\left(x,y\right)}\left[\log D\left(x,y\right)\right]+{E}_{x\sim {p}_{\mathrm{data}}(x)}\left[\log \left(1-D\left(x,G(x)\right)\right)\right], $$
where x is the input unstained image to the generator, y is the ground truth image (the corresponding histological stained image in our case), z is a random noise added as dropout in this work, pdata(x, y) is the joint probability distribution of the training data including pairs of input image x and ground truth image y, and \( {E}_{x,y\sim {p}_{\mathrm{data}}\left(x,y\right)} \) is the expectation of log likelihood of (x, y). To reduce blurring and generate shaper images, the L1 regularization term was chosen as follows:
$$ {L}_{L1}(G)={E}_{x,y,z}\Big(\left\Vert {\left.y-G\left(x,z\right)\right\Vert}_1\right). $$
The global cost objective of adversarial learning in cGANs is defined as follows:
$$ \mathrm{G}=\arg \underset{G}{\min}\underset{D}{\max }{L}_{\mathrm{cGAN}}\left(G,D\right)+\lambda {L}_{L1}(G). $$
The regularization parameter λ is empirically chosen to 100 to balance the adversarial loss and global loss. The convolution kernels of cGAN were set to be 4 × 4. These kernels were randomly initialized using a uniform distribution with a minimum value of 0. We set all biases as random uniform distribution. A dropout rate of 0.5 was used in our experiments. We trained our virtual staining network model for 200 epochs with a learning rate of 0.001 for the generator network and 0.0002 for the discriminator network. Adam optimizer with a batch size of 1 and an exponential decay rate of 0.5 was used in our training. For each iteration of the discriminator, there was one iteration of the generator network.

Implementation

Our virtual staining network was implemented using Python version 3.6.9. The cGAN was implemented using TensorFlow version 1.12.0. The other Python libraries used were cv2, os, time, tqdm, the Python imaging library (PIL), SciPy, glob, ops, sys, and numpy. We implemented the software on a desktop computer with a Core i7-8700K CPU at 3.20 GHz (Intel) and 12 GB of RAM, running a Linux 4.15.0 operating system. All the experiments including network training and testing were performed on Nvidia GeForce RTX 2080Ti GPU. The other implementation details, including the number of trained patches, the number of epochs, and the training times, are summarized in Suppl. Table 2.

Evaluation

Calculation of the area of intima, media, and intima-to-media ratio for virtual and standard staining in tissue sections was performed using ImageJ package FIJI (version 1.51) [28]. Data were represented as mean ± SD (standard deviation).

Results

Virtual Staining of Rat Carotid Artery Tissue Sections

We present three virtual staining results for carotid artery tissue sections, which were generated by our trained cGANs from the testing dataset, as shown in Fig. 3. These generated images demonstrated that the cGANs can transform bright-field images of unstained tissue sections (Figs. 3a–g) into the corresponding colorized images that are expected from H&E-, PSR-, and orcein-stained tissue sections. Evaluation of Figs. 3b and c shows that the neointima, media, and elastic lamina are present in both H&E staining techniques. Notably, the presence of neointima formation is clearly displayed in both panels. The replication of neointimal cells can also be identified in both virtual staining and histological staining images.
We further trained our cGAN to virtually stain other tissue types using PSR and Orcein stains. Figures 3e and f show the results for virtual staining of an unstained tissue section that matches very well with a bright-field image of PSR staining. These results illustrate that the cGAN can stain patterns of different histological staining for different tissue sections. The virtual PSR staining technique for carotid artery tissue sections in Fig. 3e correctly displays the distribution of collagen. This result is consistent with the histologic appearance in the bright-field images of the same tissue samples after histological staining (Fig. 3f). Similarly, orcein virtual staining of the tissue section shown in Fig. 3h reveals consistently stained external elastic lamina and histological features that correspond to their appearances shown in the bright-field image after histological staining (Fig. 3i).

Multiple Virtual Stains on the Same Unstained Tissue Section

We trained our deep network to perform multiple virtual stains—H&E stain, PSR stain, and orcein stain—on the same unstained tissue section using StarGAN [29]. This network is capable of learning mappings among multiple stained domains using a single generator. Multiple virtual stains on the same unlabeled tissue section match well with the bright-field images of the three consecutive sections captured after histological staining (Fig. 4). These results illustrate that the StarGAN method can infer multiple types of histologically staining methods from a single bright-field image of an unlabeled sample. In an example of multiple virtual staining, the H&E virtual stain correctly revealed the histological features of the H&E staining and the degree of neointima hyperplasia. Virtual PSR staining captured the histological features of collagen; this is consistent with the histological appearance in the bright-field images captured after histological staining. Similarly, orcein virtual staining revealed consistently stained histological features of elastin in the bright-field images after histological staining.

Blind Evaluation of Virtual and Histological Staining

To demonstrate the efficacy of our approach, H&E, PSR, and orcein virtual staining and the corresponding histological staining of the same tissue sections were blindly evaluated by three board-certified pathologists. These pathologists were blinded to the staining techniques and asked to apply a grade from 1 to 5 for the quality of the different stains: 5, perfect; 4, very good; 3, good; 2, acceptable; 1, unacceptable. The pathologists could recognize histopathological features, including neointima, media, elastic lamina, collagen, and external elastic lamina, presented in both virtual and histological stains. The blind testing results are summarized in Table 1 (H&E staining). Moreover, the pathologists applied the same score scale (1–5) for PSR and orcein staining features, that is, collagen and external elastic lamina, respectively. The results are summarized in Suppl. Table 1.
Table 1.
Blind evaluation of virtual and histological H&E staining in carotid artery tissue sections
Tissue number
Pathologist 1
Pathologist 2
Pathologist 3
Average
NI
M
EL
NI
M
EL
NI
M
EL
NI
M
EL
1 (VS)
3
3
3
3
3
3
4
4
4
3.33
3.33
3.33
1 (HS)
4
4
3
3
4
4
4
4
4
3.67
4.00
3.67
2 (VS)
3
4
3
3
4
3
4
4
4
3.33
4.00
3.33
2 (HS)
4
4
4
3
4
3
4
4
4
3.67
4.00
3.67
3 (VS)
3
3
3
3
4
3
4
5
4
3.33
4.00
3.33
3 (HS)
4
4
4
4
4
3
4
5
4
4.00
4.33
3.67
4 (VS)
3
3
3
3
4
3
4
4
4
3.33
3.67
3.33
4 (HS)
4
3
4
4
4
3
5
4
4
4.33
3.67
3.67
5 (VS)
3
3
3
3
3
3
4
4
4
3.33
3.33
3.33
5 (HS)
4
4
4
3
4
3
5
5
4
4.00
4.33
3.67
Carotid artery tissue sections were stained with H&E and graded for neointima (NI), media (M), and elastic lamina (EL). HS, histologically staining; VS, virtually staining. The winner (and tied) average scores are in italics
Furthermore, the three pathologists were asked to calculate the intima thickness (IT), intima area (IA), media area (MA), and intima-to-media ratio (IMR) to evaluate the neointima hyperplasia in both virtual and histological stains. The IMR was calculated as the mean area of intima divided by the mean area of media. We compared the relative error (RE) between the histologically stained and virtually stained samples to evaluate the quantification accuracy. The quantification results are summarized in Table 2 (H&E staining).
Table 2.
Quantification of virtual and histological H&E staining in carotid artery tissue sections
Tissue number
IT (μm)
RE of IT (%)
IA (μm2)
RE of IA (%)
MA(μm2)
RE of MA (%)
IMR
1 (VS)
228.5 ± 0.9
0.2
261,474.5 ± 2253.0
0.8
175,412.1 ± 7082.3
3.3
1.49
1 (HS)
228.1 ± 1.9
263,530.1 ± 1136.2
169,753.7 ± 3665.8
1.55
2 (VS)
244.6 ± 1.3
1.6
128,655.6 ± 5884.7
4.1
149,911.9 ± 7493.7
0.1
0.86
2 (HS)
248.5 ± 1.9
134,151.7 ± 1535.5
150,026.1 ± 2436.2
0.89
3 (VS)
206.9 ± 2.9
0.1
259,228.6 ± 15,163.9
0.3
183,164.8 ± 14,245.2
4.1
1.42
3 (HS)
206.6 ± 1.1
258,425.0 ± 14,351.3
175,953.6 ± 7619.6
1.47
4 (VS)
140.0 ± 1.4
1.6
254,563.0 ± 12,650.4
3.6
205,179.6 ± 10,924.2
12.7
1.24
4 (HS)
137.9 ± 2.0
264,064.6 ± 8300.7
181,998.4 ± 5369.6
1.45
5 (VS)
223.6 ± 2.5
1.1
128,244.5 ± 1436.1
5.6
162,862.3 ± 4462.2
1.2
0.79
5 (HS)
226.1 ± 0.7
135,838.3 ± 414.6
164,876.6 ± 1722.3
0.82
IT, intima thickness; IA, intima area; MA, media area; IMR, intima-to-media ratio. RE, relative error. HS, histologically staining; VS, virtually staining
Our results indicate that the pathologists could recognize neointima hyperplasia-related histopathological features using both staining techniques and with a high degree of agreement between the techniques.

Discussion

Histological staining analysis is performed as a “golden standard” in diagnostic pathology. It is widely used to identify carotid artery tissue constituents such as intima, media, collagen, and elastic lamina, under light microscope with a high magnification objective. However, the histological staining process is laborious, and image quality of tissue staining images is variable because of the different staining protocols, imaging devices, and human-to-human variations. Such staining processes lead to a barrier in developing standard and fast histological image analysis systems. Notably, the standard histological staining procedure can only be performed on single tissue sections with one type of stain. Therefore, the identification of various components of the same tissue section is highly required.
In this work, we proposed a trained cGAN that virtually stains the label-free fresh tissue sections into the corresponding H&E-, PSR-, and orcein-stained sections of rat carotid artery tissue samples. This deep learning-based method can provide a digital staining of label-free tissue sections, which can bypass the lengthy and laborious tissue preparation process and can mitigate human-to-human variations for standard histological staining of tissue samples. We trained this statistical learning-based network using white light images of unstained sections to learn from histologically stained tissue sections. A blind test was performed to demonstrate the performance of the network by board-certified pathologists. After training, this virtual staining method can be enhanced by combing the unstained images acquired using other advanced label-free imaging techniques, for example, autofluorescence microscopy and Raman microscopy.
For the training process, it is important to match the bright-field images of unstained sections and the corresponding images acquired after histological staining. However, during the whole histological staining process, tissue can be deformed or contaminated by dust, resulting in challenges of the loss function in the training step. To reduce these impacts on the performance of the network, we performed a global co-registration using rigid algorithm (https://​www.​mathworks.​com/​help/​images/​registering-multimodal-mri-images.​html). We further performed a visual inspection of the registered unstained and histological staining image pairs to eliminate the images that included dust or large deformations.
It is important to note that, gradient descent-based GAN training is not always locally convergent partly due to the discontinuous dataset, such as bright-filed microscopic images of unstained tissue sections that used in this study. Here, we transfer bright-field images of unlabeled carotid artery tissue into equivalent images of histologically stained versions of the same samples using conditional generative adversarial network, which are extensions of GANs where both generator and discriminator are conditioned on additional information. To stable the training, we chose L1 regularization term to reduce blurring and generate shaper images. In addition, the regularization parameter λ was used for balancing the adversarial loss and global loss.
Our virtual staining procedure is implemented by training separate cGAN for each staining. To reveal multiple virtual stains, namely H&E, PSR, and orecin stains in our case, we trained the StarGAN network to generate three different stained images from the same unstained image. This is practical for histological applications, that is, the number of staining types on the same unlabeled tissue will be unlimited. This virtual staining method can provide fast pathological diagnosis of various components. In this study, it can generate three different type of stains, including H&E, PSR, and orcein, within 3 s with current hardware configuration. Although this is an improvement compared to the virtual stain cGAN for single tissue stain combinations, the multiple virtual stain image is a little inferior for generating high-quality multiple virtual staining images. Within this network, a generator generates images of multiple domains, solving the problem of image pairing and reducing network parameters. The output results demonstrated that multiple virtual staining images generated from an unstained image simultaneously can express the main staining components according to the stained images.
Our study has some limitations. Firstly, the output image of network needs to be corrected according to the stained images. Substantially, a wide-scale, randomized evaluation of virtual staining images by more pathologists will regulate the network to achieve a high-quality image. This is very useful for fast analysis of bright-field microscopic images of tissue sections in the future. In addition, the combination of high-resolution bright-field images of unlabeled tissue sections with virtual staining in the deep learning-based network results in high-resolution virtual staining images with more detail in histological features. Finally, the quantitative virtual staining evaluation indexes, including the quantitative image-based metrics like SSIM, are required to assist pathologists during routine clinical diagnosis. Notably, this virtual staining method could be transferred into a clinical research on unlabeled human artery tissue samples, which would be necessary to improve the diagnostic efficiency and accuracy of the trained network with the histological stains.

Conclusions

In conclusion, we have developed a deep learning-based virtual staining method that transformed bright-field microscopic images of unlabeled tissue sections into their corresponding images of histological staining of the same samples using a conditional generative adversarial network model. This virtual staining method has been validated by pathologists via blind evaluation. Further improvement of this method will be focused on the combination of advanced other label-free microscopic imaging modalities and the evaluation by large-scale randomized clinical study. We envision that this virtual staining method will provide strong support to the applications of histology analysis in study for CAD.

Acknowledgements

The authors would like to acknowledge the instrumental and technical support of multimodal biomedical imaging experimental platform, Institute of Automation, Chinese Academy of Sciences.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://​creativecommons.​org/​licenses/​by/​4.​0/​.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Unsere Produktempfehlungen

e.Med Interdisziplinär

Kombi-Abonnement

Für Ihren Erfolg in Klinik und Praxis - Die beste Hilfe in Ihrem Arbeitsalltag

Mit e.Med Interdisziplinär erhalten Sie Zugang zu allen CME-Fortbildungen und Fachzeitschriften auf SpringerMedizin.de.

e.Med Orthopädie & Unfallchirurgie

Kombi-Abonnement

Mit e.Med Orthopädie & Unfallchirurgie erhalten Sie Zugang zu CME-Fortbildungen der Fachgebiete, den Premium-Inhalten der dazugehörigen Fachzeitschriften, inklusive einer gedruckten Zeitschrift Ihrer Wahl.

e.Med Radiologie

Kombi-Abonnement

Mit e.Med Radiologie erhalten Sie Zugang zu CME-Fortbildungen des Fachgebietes Radiologie, den Premium-Inhalten der radiologischen Fachzeitschriften, inklusive einer gedruckten Radiologie-Zeitschrift Ihrer Wahl.

Anhänge

Electronic supplementary material

Literatur
1.
Zurück zum Zitat Purushothaman KR, Purushothaman M, Muntner P, Lento PA, O'Connor WN, Sharma SK, Fuster V, Moreno PR (2011) Inflammation, neovascularization and intra-plaque hemorrhage are associated with increased reparative collagen content: implication for plaque progression in diabetic atherosclerosis. Vasc Med 16:103–108CrossRef Purushothaman KR, Purushothaman M, Muntner P, Lento PA, O'Connor WN, Sharma SK, Fuster V, Moreno PR (2011) Inflammation, neovascularization and intra-plaque hemorrhage are associated with increased reparative collagen content: implication for plaque progression in diabetic atherosclerosis. Vasc Med 16:103–108CrossRef
2.
Zurück zum Zitat Bentzon JF, Otsuka F, Virmani R, Falk E (2014) Mechanisms of plaque formation and rupture. Circ Res 114:1852–1866CrossRef Bentzon JF, Otsuka F, Virmani R, Falk E (2014) Mechanisms of plaque formation and rupture. Circ Res 114:1852–1866CrossRef
3.
Zurück zum Zitat Virmani R, Kolodgie FD, Burke AP, Farb A, Schwartz SM (2000) Lessons from sudden coronary death: a comprehensive morphological classification scheme for atherosclerotic lesions. Arterioscler Thromb Vasc Biol 20:1262–1275CrossRef Virmani R, Kolodgie FD, Burke AP, Farb A, Schwartz SM (2000) Lessons from sudden coronary death: a comprehensive morphological classification scheme for atherosclerotic lesions. Arterioscler Thromb Vasc Biol 20:1262–1275CrossRef
4.
Zurück zum Zitat Virmani R, Kolodgie FD, Burke AP, Finn AV, Gold HK, Tulenko TN, Wrenn SP, Narula J (2005) Atherosclerotic plaque progression and vulnerability to rupture: angiogenesis as a source of intraplaque hemorrhage. Arterioscler Thromb Vasc Biol 25:2054–2061CrossRef Virmani R, Kolodgie FD, Burke AP, Finn AV, Gold HK, Tulenko TN, Wrenn SP, Narula J (2005) Atherosclerotic plaque progression and vulnerability to rupture: angiogenesis as a source of intraplaque hemorrhage. Arterioscler Thromb Vasc Biol 25:2054–2061CrossRef
5.
Zurück zum Zitat McCann MT, Ozolek JA, Castro CA, Parvin B, Kovacevic J (2015) Automated histology analysis [opportunities for signal processing]. IEEE Signal Process Mag 32:78–87CrossRef McCann MT, Ozolek JA, Castro CA, Parvin B, Kovacevic J (2015) Automated histology analysis [opportunities for signal processing]. IEEE Signal Process Mag 32:78–87CrossRef
6.
Zurück zum Zitat Yu W, Braz JC, Dutton AM, Prusakov P, Rekhter M (2007) In vivo imaging of atherosclerotic plaques in apolipoprotein E deficient mice using nonlinear microscopy. J Biomed Opt 12:054008CrossRef Yu W, Braz JC, Dutton AM, Prusakov P, Rekhter M (2007) In vivo imaging of atherosclerotic plaques in apolipoprotein E deficient mice using nonlinear microscopy. J Biomed Opt 12:054008CrossRef
7.
Zurück zum Zitat van Zandvoort M, Engels W, Douma K, Beckers L, oude Egbrink M, Daemen M, Slaaf DW (2004) Two-photon microscopy for imaging of the (atherosclerotic) vascular wall: a proof of concept study. J Vasc Res 41:54–63CrossRef van Zandvoort M, Engels W, Douma K, Beckers L, oude Egbrink M, Daemen M, Slaaf DW (2004) Two-photon microscopy for imaging of the (atherosclerotic) vascular wall: a proof of concept study. J Vasc Res 41:54–63CrossRef
8.
Zurück zum Zitat Croce AC, Bottiroli G (2014) Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis. Eur J Histochem 58:2461PubMedPubMedCentral Croce AC, Bottiroli G (2014) Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis. Eur J Histochem 58:2461PubMedPubMedCentral
9.
Zurück zum Zitat Jamme F, Kascakova S, Villette S, Allouche F, Pallu S, Rouam V, Réfrégiers M (2013) Deep UV autofluorescence microscopy for cell biology and tissue histology. Biol Cell 105:277–288CrossRef Jamme F, Kascakova S, Villette S, Allouche F, Pallu S, Rouam V, Réfrégiers M (2013) Deep UV autofluorescence microscopy for cell biology and tissue histology. Biol Cell 105:277–288CrossRef
10.
Zurück zum Zitat Le TT, Langohr IM, Locker MJ, Sturek M, Cheng JX (2007) Label-free molecular imaging of atherosclerotic lesions using multimodal nonlinear optical microscopy. J Biomed Opt 12:054007CrossRef Le TT, Langohr IM, Locker MJ, Sturek M, Cheng JX (2007) Label-free molecular imaging of atherosclerotic lesions using multimodal nonlinear optical microscopy. J Biomed Opt 12:054007CrossRef
11.
Zurück zum Zitat Zoumi A, Yeh A, Tromberg BJ (2002) Imaging cells and extracellular matrix in vivo by using second-harmonic generation and two-photon excited fluorescence. Proc Natl Acad Sci U S A 99:11014–11019CrossRef Zoumi A, Yeh A, Tromberg BJ (2002) Imaging cells and extracellular matrix in vivo by using second-harmonic generation and two-photon excited fluorescence. Proc Natl Acad Sci U S A 99:11014–11019CrossRef
12.
Zurück zum Zitat Witte S, Negrean A, Lodder JC, de Kock CPJ, Testa Silva G, Mansvelder HD, Louise Groot M (2011) Label-free live brain imaging and targeted patching with third-harmonic generation microscopy. Proc Natl Acad Sci U S A 108:5970–5975CrossRef Witte S, Negrean A, Lodder JC, de Kock CPJ, Testa Silva G, Mansvelder HD, Louise Groot M (2011) Label-free live brain imaging and targeted patching with third-harmonic generation microscopy. Proc Natl Acad Sci U S A 108:5970–5975CrossRef
13.
Zurück zum Zitat Ji M, Orringer DA, Freudiger CW et al (2013) Rapid, label-free detection of brain tumors with stimulated Raman scattering microscopy. Sci Transl Med 5:201ra119CrossRef Ji M, Orringer DA, Freudiger CW et al (2013) Rapid, label-free detection of brain tumors with stimulated Raman scattering microscopy. Sci Transl Med 5:201ra119CrossRef
14.
Zurück zum Zitat Orringer DA, Pandian B, Niknafs YS, et al. (2017) Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat Biomed Eng 1 Orringer DA, Pandian B, Niknafs YS, et al. (2017) Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat Biomed Eng 1
15.
Zurück zum Zitat Jansen K, Wu M, van der Steen AF, van Soest G (2014) Photoacoustic imaging of human coronary atherosclerosis in two spectral bands. Photoacoustics 2:12–20CrossRef Jansen K, Wu M, van der Steen AF, van Soest G (2014) Photoacoustic imaging of human coronary atherosclerosis in two spectral bands. Photoacoustics 2:12–20CrossRef
16.
Zurück zum Zitat Wang B, Su JL, Amirian J, Litovsky SH, Smalling R, Emelianov S (2010) Detection of lipid in atherosclerotic vessels using ultrasound-guided spectroscopic intravascular photoacoustic imaging. Opt Express 18:4889–4897CrossRef Wang B, Su JL, Amirian J, Litovsky SH, Smalling R, Emelianov S (2010) Detection of lipid in atherosclerotic vessels using ultrasound-guided spectroscopic intravascular photoacoustic imaging. Opt Express 18:4889–4897CrossRef
17.
Zurück zum Zitat Seeger M, Karlas A, Soliman D, Pelisek J, Ntziachristos V (2016) Multimodal optoacoustic and multiphoton microscopy of human carotid atheroma. Photoacoustics 4:102–111CrossRef Seeger M, Karlas A, Soliman D, Pelisek J, Ntziachristos V (2016) Multimodal optoacoustic and multiphoton microscopy of human carotid atheroma. Photoacoustics 4:102–111CrossRef
18.
Zurück zum Zitat Rivenson Y, Wang H, Wei Z, de Haan K, Zhang Y, Wu Y, Günaydın H, Zuckerman JE, Chong T, Sisk AE, Westbrook LM, Wallace WD, Ozcan A (2019) Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng 3:466–477CrossRef Rivenson Y, Wang H, Wei Z, de Haan K, Zhang Y, Wu Y, Günaydın H, Zuckerman JE, Chong T, Sisk AE, Westbrook LM, Wallace WD, Ozcan A (2019) Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng 3:466–477CrossRef
19.
Zurück zum Zitat Bayramoglu N, Kaakinen M, Eklund L, Heikkila J (2017) Towards virtual H&E staining of hyperspectral lung histology images using conditional generative adversarial networks. 2017 Ieee international conference on computer vision workshops (Iccvw 2017):64-71 Bayramoglu N, Kaakinen M, Eklund L, Heikkila J (2017) Towards virtual H&E staining of hyperspectral lung histology images using conditional generative adversarial networks. 2017 Ieee international conference on computer vision workshops (Iccvw 2017):64-71
20.
Zurück zum Zitat Shaban MT, Baur C, Navab N, Albarqouni S (2019) Staingan: stain style transfer for digital histological images. 2019 Ieee 16th international symposium on biomedical imaging (Isbi 2019):953-956 Shaban MT, Baur C, Navab N, Albarqouni S (2019) Staingan: stain style transfer for digital histological images. 2019 Ieee 16th international symposium on biomedical imaging (Isbi 2019):953-956
21.
Zurück zum Zitat Bautista PA, Yagi Y (2012) Digital simulation of staining in histopathology multispectral images: enhancement and linear transformation of spectral transmittance. J Biomed Opt 17:056013CrossRef Bautista PA, Yagi Y (2012) Digital simulation of staining in histopathology multispectral images: enhancement and linear transformation of spectral transmittance. J Biomed Opt 17:056013CrossRef
22.
Zurück zum Zitat Clowes AW, Reidy MA, Clowes MM (1983) Kinetics of cellular proliferation after arterial injury. I. Smooth muscle growth in the absence of endothelium. Lab Investig 49:327–333PubMed Clowes AW, Reidy MA, Clowes MM (1983) Kinetics of cellular proliferation after arterial injury. I. Smooth muscle growth in the absence of endothelium. Lab Investig 49:327–333PubMed
23.
Zurück zum Zitat Isola P, Zhu JY, Zhou TH, Efros AA (2017) Image-to-image translation with conditional adversarial networks. 30th Ieee conference on computer vision and pattern recognition (Cvpr 2017):5967-5976 Isola P, Zhu JY, Zhou TH, Efros AA (2017) Image-to-image translation with conditional adversarial networks. 30th Ieee conference on computer vision and pattern recognition (Cvpr 2017):5967-5976
24.
Zurück zum Zitat Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. (2014) Generative adversarial nets. Advances in neural information processing systems 27 (nips 2014) 27 Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. (2014) Generative adversarial nets. Advances in neural information processing systems 27 (nips 2014) 27
25.
Zurück zum Zitat Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention, Pt Iii 9351:234–241 Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention, Pt Iii 9351:234–241
26.
Zurück zum Zitat Klages P, Benslimane I, Riyahi S et al (2019) Patch-based generative adversarial neural network models for head and neck MR-only planning. Med Phys Klages P, Benslimane I, Riyahi S et al (2019) Patch-based generative adversarial neural network models for head and neck MR-only planning. Med Phys
27.
Zurück zum Zitat Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA (2016) Context encoders: feature learning by inpainting. 2016 Ieee conference on computer vision and pattern recognition (Cvpr):2536–2544 Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA (2016) Context encoders: feature learning by inpainting. 2016 Ieee conference on computer vision and pattern recognition (Cvpr):2536–2544
28.
Zurück zum Zitat Schneider CA, Rasband WS, Eliceiri KW (2012) NIH image to ImageJ: 25 years of image analysis. Nat Methods 9:671–675CrossRef Schneider CA, Rasband WS, Eliceiri KW (2012) NIH image to ImageJ: 25 years of image analysis. Nat Methods 9:671–675CrossRef
29.
Zurück zum Zitat Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. 2018 Ieee/Cvf conference on computer vision and pattern recognition (Cvpr):8789-8797 Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. 2018 Ieee/Cvf conference on computer vision and pattern recognition (Cvpr):8789-8797
Metadaten
Titel
Deep Learning for Virtual Histological Staining of Bright-Field Microscopic Images of Unlabeled Carotid Artery Tissue
verfasst von
Dan Li
Hui Hui
Yingqian Zhang
Wei Tong
Feng Tian
Xin Yang
Jie Liu
Yundai Chen
Jie Tian
Publikationsdatum
08.06.2020
Verlag
Springer International Publishing
Erschienen in
Molecular Imaging and Biology / Ausgabe 5/2020
Print ISSN: 1536-1632
Elektronische ISSN: 1860-2002
DOI
https://doi.org/10.1007/s11307-020-01508-6

Weitere Artikel der Ausgabe 5/2020

Molecular Imaging and Biology 5/2020 Zur Ausgabe

Akuter Schwindel: Wann lohnt sich eine MRT?

28.04.2024 Schwindel Nachrichten

Akuter Schwindel stellt oft eine diagnostische Herausforderung dar. Wie nützlich dabei eine MRT ist, hat eine Studie aus Finnland untersucht. Immerhin einer von sechs Patienten wurde mit akutem ischämischem Schlaganfall diagnostiziert.

Screening-Mammografie offenbart erhöhtes Herz-Kreislauf-Risiko

26.04.2024 Mammografie Nachrichten

Routinemäßige Mammografien helfen, Brustkrebs frühzeitig zu erkennen. Anhand der Röntgenuntersuchung lassen sich aber auch kardiovaskuläre Risikopatientinnen identifizieren. Als zuverlässiger Anhaltspunkt gilt die Verkalkung der Brustarterien.

S3-Leitlinie zu Pankreaskrebs aktualisiert

23.04.2024 Pankreaskarzinom Nachrichten

Die Empfehlungen zur Therapie des Pankreaskarzinoms wurden um zwei Off-Label-Anwendungen erweitert. Und auch im Bereich der Früherkennung gibt es Aktualisierungen.

Fünf Dinge, die im Kindernotfall besser zu unterlassen sind

18.04.2024 Pädiatrische Notfallmedizin Nachrichten

Im Choosing-Wisely-Programm, das für die deutsche Initiative „Klug entscheiden“ Pate gestanden hat, sind erstmals Empfehlungen zum Umgang mit Notfällen von Kindern erschienen. Fünf Dinge gilt es demnach zu vermeiden.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.