A novel Cerenkov luminescence tomography approach using multilayer fully connected neural network

At first, the orthotopic human glioblastoma U87MG tumor mouse models received ¹¹C-methionine (¹¹C-MET) administration through tail vein (figure 1). The radiopharmaceutical was distributing with blood circulation and accumulated in tumor tissues. The in vivo biological environment provided necessary medium to generate CL. According to the Frank–Tamm theory, the most part of the CL was unable to penetrate through animal tissues. Moreover, the living tissues induced great attenuation and scattering to the CL. As a consequence, only a fraction of the generated CL could be acquired using high-sensitivity electron multiplied charge coupled device (EMCCD). A multilayer fully connected neural network (MFCNN) was presented in this study to investigate the complicated relationship between the acquired CL and the true luminescence sources. Monte Carlo simulation data were employed to train the MFCNN. Then the acquired in vivo CLI results were mapped on the standard numerical mouse, and the surface optical flux was calculated. When the surface optical flux was inputted into the trained MFCNN, the reconstructed photon sources were able to be outputted.

Zoom In Zoom Out Reset image size

The MFCNN in this study consisted of totally seven layers, one for input, one for output, and between them were five hidden layers (figure 2). Input of the MFCNN was the surface photon density, and the output was the reconstructed CL source. Rectified linear unit (ReLU) was applied as the activation function following each connection, and the activation threshold was λ  =  0 as shown in equation (1). A dropout function (probability  =  20%) was used to reduce the overfitting problem of the network. The nonlinear relationship between one layer and the next was described as equation (2). In this study, the reconstruction permissible region was assigned as the mouse brain. To each MFCNN layer, the number of neurons was equal to the number of nodes in the numerical mouse brain. Keras 2.2.4 with Tensorflow 1.12.0 supported by Python 3.6 were employed to implement the network. The MFCNN was trained with 80 epochs, 128 batch size. The optimizer was Adam algorithm with the learning rate: 0.000 01, β₁: 0.9 and β₂: 0.99.

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \begin{matrix} ReLU\left(x \right)=\left\{\begin{matrix} 0, & x<0 \nonumber \\ x, & x\geqslant 0 \nonumber \\ \end{matrix} \right. \nonumber \\ \end{matrix}\nonumber \end{align} \tag{ 1 }$$

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \begin{matrix} x_{j}^{k+1}=ReLU\left(\sum\limits_{i}{{{w}_{i}}x_{i}^{k}} \right)~ \nonumber \\ \end{matrix}.\nonumber \end{align} \tag{ 2 }$$

Zoom In Zoom Out Reset image size

A standard numerical mouse (Dogdas et al 2007) was applied to perform Monte Carlo simulation. The CLT reconstruction mesh was established from the head part of the numerical mouse (figure 3). 1965 nodes of the brain area were included in the reconstruction permissible region. To represent the photon propagation in head, three different related organs including brain, skull and muscle were considered (table 1). The absorption coefficient µ_a and the scattering coefficient µ_s were set based on the previous studies (Liu et al 2015, Guo et al 2017a). All the Monte Carlo simulation was conducted with the Molecular Optical Simulation Environment (MOSE 2.3) (Li et al 2004, Tian et al 2013). The whole calculation procedure ran on a personal computer with Intel Core i7 CPU (3.70 GHz), 16 GB DDR3 RAM and an NVidia GTX1080 Ti GPU.

Zoom In Zoom Out Reset image size

Simulation data of single-source and multiple-sources were collected to train the MFCNN and validate the reconstruction performance. 100 000 photons were set into one single-source sample, and each photon owned the energy of 1.0. Wavelength of the simulated Cerenkov photon was set as 650 nm, which demonstrated suitable penetration and intensity to perform CLI in vivo (Liu et al 2015, Guo et al 2017a). Different sizes and shapes were assigned to the photon sources during simulation (figure 4). Totally 940 different center positions were selected for single-source simulation. 329 center positions were assigned to spheroid sources, 318 to cylindroid sources and 293 to cuboid sources (table 2). The center positions gathered for each source shape covered the brain region as much as possible. In each specific center position, four repetitive simulations were performed, in order to take into consideration the randomness of photon propagation. As a consequence, 3760 single-source samples were collected based on the 940 distinctive center positions.

Zoom In Zoom Out Reset image size

The generated samples of single-source were divided into training-set and test-set based on their center positions. 80% of the 940 center positions were randomly selected for training and the left 20% center positions were for testing. The simulated Cerenkov source samples were gathered into the training-set and the test-set according to their center positions. Moreover, a group of big-sized spheroid samples were produced to evaluate the reconstruction performance. Three different sizes and nine different center positions were applied in the simulation of big-sized samples (table S1 (stacks.iop.org/PMB/64/245010/mmedia)). These sizes and center positions were distinctive from the other single-source samples. Compared to the spheroid training sources, these big-sized test samples were 1.7, 4.1 and 8 times greater in spatial volume. Four repetitive simulations were conducted with the same parameters to reduce the effects from random error. Consequently, 36 big-sized single-source samples were generated, and all of them were used to evaluate the reconstruction performance without training.

The MFCNN was trained by multiple-source data as well, in order to improve the stability and the robustness of CLT reconstruction. Based on equations (3) and (4), the multiple-source samples were produced by assembling different single-source samples from the training-set. Three types including dual, triple, and quadruple-source samples were assembled. Each type owned 1500 samples, and therefore 4500 multiple-source samples were generated to enlarge the training-set.

All the single-source samples in the test-set were applied to evaluate the CLT reconstruction performance. A number of multiple-source test samples were employed for evaluation as well, which were extracted from the single-source test-set. In this way, the proposed MFCNN can be effectively trained, and the reconstruction performance was reliably evaluated with various test samples.

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \begin{matrix} {{b}_{mul}}=\sum\limits_{\forall i\in S}{{{b}_{i}}} \nonumber \\ \end{matrix}\nonumber \end{align} \tag{ 3 }$$

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \begin{matrix} {{x}_{mul}}=\sum\limits_{\forall i\in S}{{{x}_{i}}}. \nonumber \\ \end{matrix}\nonumber \end{align} \tag{ 4 }$$

The reconstruction accuracy was evaluated by center localization error (CLE) and Dice coefficient (DC). CLE indicated the Euclidean distance between the centers of true photon source (x₀, y₀, z₀) and the reconstructed result (x, y , z). The calculation of CLE was described as equation (5). A smaller CLE result indicated higher accuracy of the CLT reconstruction. If the true source center was perfectly coincident to the reconstructed source center, CLE equals to 0. Source shape was also important to evaluate the performance of a reconstruction approach. In this study, DC as equation (6) was employed to reflect the consistency between the true source shape and the reconstructed shape. X in equation (6) indicated the reconstructed region, while Y was the true source region obtained from the CT. The maximum of DC was 1, and the greater value of DC indicated higher similarity between two sources. Moreover, incomplete variables truncated conjugate gradient method (IVTCG) with L1 regularization was introduced as contrast (He et al 2010, Wang et al 2014), which showed ideal performance for CLT reconstruction in the recent publication (Guo et al 2017a).

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \begin{matrix} {\rm CLE}=\sqrt{{{\left(x-{{x}_{0}} \right)}^{2}}+{{\left(y-{{y}_{0}} \right)}^{2}}+{{\left(z-{{z}_{0}} \right)}^{2}}} \nonumber \\ \end{matrix}\nonumber \end{align} \tag{ 5 }$$

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \begin{matrix} {\rm DC}=\frac{2\left| X\cap Y \right|}{\left| X \right|+\left| Y \right|}. \nonumber \\ \end{matrix}\nonumber \end{align} \tag{ 6 }$$

4–6 weeks old BALB/c nude female mice (n  =  3) were used for in vivo CLT experiment. Luciferase (Luc) and green fluorescence protein (GFP) labelled glioma cell line U87MG-Luc-GFP was employed to construct glioma mouse models. Approximately 4  ×  10⁶ U87MG-Luc-GFP cells mixed with phosphate buffered saline (PBS, pH  =  7.0, 10 mM, 10 µl) were administrated into the mouse brain with a microinjector. In vivo bioluminescence imaging (BLI) was performed every three days to monitor the tumor growing status using an IVIS Spectrum system (PerkinElmer Inc., USA). After 9 d for tumor growing, the orthotopic glioma models (n  =  3) were applied for the in vivo CLT experiments.

Gadopentetate dimeglumine (BEILU Pharmaceutical Co., Ltd., China) in the dose of 0.2 ml kg⁻¹ was intraperitoneally inject into the mouse models (n  =  3), in order to perform enhanced T1 MRI (M3^™, Aspect Imaging, Israel) for the orthotropic glioma. Next, in vivo CLI-CT was performed followed by small animal positron emission tomography (PET). ¹¹C-MET (29.6 MBq, 200 µl) was injected into the mouse models (n  =  3) through the tail vein. Twenty minutes after the injection, in vivo CLI-CT was performed with a specially designed small animal pentamodal imaging system (Liu et al 2017). To improve the performance of in vivo CLI, the mice was placed on a black imaging platform in prone position, and black paper pads were used to block the luminescence emitted from mouse abdomen. The in vivo CLI results were acquired with a high-sensitivity EMCCD (iXonEM  +  888, ANDOR, UK). Parameters of 300 s exposure and 4  ×  4 binning were used to obtain the CLI signals from the orthotropic glioma. Then the whole body CT anatomic information was obtained by a micro-CT system. The physical posture of the mice was fixed when performing CLI-CT. Then, the models were immediately transferred for acquisition of PET (GENISYS4, Sofie Biosciences, USA). In the period of in vivo imaging, the living mouse models were anesthetized by 1% isoflurane–oxygen gas mixture.

In the processing of CLT reconstruction, the in vivo imaging data was represented on the standard numerical mouse, and the observed CLI signals were distributed in the numerical mesh. Similar to the reconstruction with simulation data, the surface optical flux was inputted to the well-trained MFCNN and the reconstructed photon sources were generated.

After in vivo imaging, the orthotopic glioma bearing mice were sacrificed and frozen with optimum cutting temperature (O.C.T.) compound (Tissue-Tek, Sakura, USA). 30 µm thick axial frozen sections of the mouse heads were prepared for pathological validation. GFP fluorescence images were taken by a Live Cell Imaging System (AF6000 Modular System, Leica, Germany). These fluorescence signals localized the glioma cells on each frozen section of the mouse brains. Moreover, these frozen sections were also stained by the hematoxylin and eosin (H&E). The MFCNN reconstruction performance was evaluated by contrasting with the GFP fluorescence and the H&E results.

Data analyzed by Prism 7 (GraphPad Software, San Diego, CA, USA) was presented as mean  ±  standard deviation (SD). Student's unpaired t-test was applied to calculate the statistical significance between independent samples, and difference with P  <  0.05 was considered as statistically significant.

The performance of MFCNN was evaluated and shown in both qualitative and quantitative aspects.

From a qualitative aspect, typical single-source reconstruction results using MFCNN were contrasted with the IVTCG method (figure 5).

Zoom In Zoom Out Reset image size

In the evaluation of localization accuracy, the MFCNN results showed great consistency with the true photon sources (figure 5, first and the second row). This consistency of MFCNN results was observed in different source shapes, including spheroid (model 1–3), cylindroid (model-4) and cuboid (model-5). The energy distribution of MFCNN results were also qualitatively proved consistent with the true sources. As contrast, obvious position deviation and unexpected artifacts were observed in the IVTCG results (figure 5, the third row). In reconstruction of untrained different source sizes (figure S1), the MFCNN reconstruction provided desirable performance. Photon sources that were spatially eight times greater than the training samples could be effectively reconstructed by the MFCNN. However, the IVTCG reconstruction suffered through the problem of artifacts and the localization accuracy was obviously interfered. These comparisons demonstrated promising performance of MFCNN in terms of reconstruction accuracy and stability.

In quantitative analysis, the CLE of MFCNN single-source reconstruction was proved to reach the level of 0.0405 mm (table 3). The average CLE from all the single-source test samples was 0.1975 mm by the MFCNN reconstruction, which was a great improvement in accuracy compared with the IVTCG results. The CLE maximum of MFCNN was obtained from the model-3, which owned a greater depth contrast to the other photon sources. This result demonstrated the disturbance of imaging depth to optical tomography. However, under the same depth, the proposed MFCNN demonstrated clearly higher reconstruction accuracy than the IVTCG. The improved performance of MFCNN could also be observed in reconstruction of big-sized samples (table S2). The calculated CLE of MFCNN was obviously less than the IVTCG results in each size level.

The performance of MFCNN was also validated in source shape recovery (tables 3 and S2). By DC calculation, shapes of the MFCNN results showed high consistency with the true sources. Using MFCNN reconstruction, the highest DC was observed greater than 0.90 and all the calculated DC were beyond the level of 0.64. Nevertheless, none of the IVTCG results showed a DC greater than 0.47.

Outperformance of MFCNN was also found in reconstruction of dual, triple and quadruple sources. From the typical dual-source results, the MFCNN results demonstrated high consistency with the true sources. As contrast, deviation of center position was clearly observed between the IVTCG results and the true sources. The accuracy of RTE based IVTCG reconstruction was also significantly disturbed by the undesirable artifacts, exemplified by the results of model-3 in dual-source results (figure 6). With the source number increased, the complexity raised and affected the performance of both MFCNN and IVTCG (figures S2 and S3). Nevertheless, in the results of various sources, MFCNN was observed still outperformed IVTCG. These qualitative contrasts further demonstrated the superiority of the proposed MFCNN reconstruction.

Zoom In Zoom Out Reset image size

The superiority of MFCNN reconstruction was also validated though quantitative CLE comparison (tables 4, S3 and S4). The lower CLE indicated that the MFCNN provided higher accuracy to reconstruct multiple sources. This outperformance was observed in the dual, triple and quadruple-source reconstruction compared with the IVTCG results. In the other hand, the CLE of multiple-source reconstruction was greater than the single-source results (table 3), which was induced by the raise in reconstruction complexity. The analyses with DC further showed that contrasted with IVTCG, the MFCNN results owned better shape consistency to the true multiple sources.

In vivo BLI, CLI, PET and MRI were performed on the orthotropic mouse models (n  =  3) of glioma. In optical imaging (figure 7(a)), BLI showed the distribution of tumor cells. CLI results were obviously consistent to the BLI observation, which indicated high accuracy of in vivo CLI for tumor localization. PET results in various views (figure 7(b)) further verified that the ¹¹C-MET was effectively accumulated in tumor. In the head region, ¹¹C-MET was also observed distributed in submaxillary gland (figure 7(b), arrow 2) and lacrimal gland (figure 7(b), arrow 3). However, CL emitted from these glands were blocked by thick skulls and muscles during imaging, and therefore only the tumor signals were detected. Results of optical imaging and PET demonstrated the in vivo CLI efficiently localized the tumor, which provided solid foundation for accurate CLT reconstruction.

Zoom In Zoom Out Reset image size

Reconstruction results of MFCNN and IVTCG were contrasted with enhanced T1 MRI, GFP and H&E (figure 7(c)). The glioma area was clearly presented in MRI with high signal. The merged CLT-MRI results showed that the gliomas were clearly localized by MFCNN without any artifacts. As a contrast, serious artifacts were observed in the IVTCG results. Meanwhile, compared with MRI and pathological results, MFCNN reconstruction showed higher accuracy than IVTCG in tumor localization. Tumor size in MRI, MFCNN and IVTCG was also compared (table 5). The three maximum dimensions of tumors areas were measured to reflect the size of irregular-shaped tumors. Size of the IVTCG artifacts were calculated as well. In all the mouse models (n  =  3), reconstructed tumor areas of MFCNN were more consistent to the MRI contrasted with IVTCG. In vivo observation verified the feasibility and presented the promising performance of MFCNN CLT reconstruction.