Multimodal image registration for liver radioembolization planning and patient assessment

Selective internal radiation therapy (SIRT) is a type of brachytherapy used in interventional radiology to treat unresectable tumors of the liver, such as hepatocellular carcinoma. The therapy takes advantage of the different supply characteristics of normal liver and liver tumors. The liver parenchyma is perfused mainly by the portal vein and only to a smaller proportion by the hepatic artery. Liver tumors are usually supplied by arterial vessels [6]. Therefore, radioembolization of liver tumors is performed by administering yttrium-90(Y-90)-labeled microspheres to tumor-feeding arterial vessels, targeting the tumor [16]. The microspheres embolize the capillaries and irradiate the surrounding tumor tissue and potentially also surrounding parenchymal regions. Careful treatment planning is required for a successful patient outcome and sparing of normal liver tissue.

Several image modalities are involved in the primary planning process of radioembolization . The multimodal images will influence the patient assessment, which is related to whether the patient-individual liver anatomy allows to successfully perform SIRT. Also the treatment plan, which addresses the issue of an effective and efficient treatment, is strongly influenced by them [6]. Multi-phase contrast-enhanced computer tomography (CT) images, in particular the hepatic venous CT phase (PV CT), and/or dynamic magnetic resonance (MR) images, e.g., contrast-enhanced T1-weighted (CE T1W) images, are performed for volume and tumor burden calculation [14]. They allow segmentations of the normal liver parenchyma and a clear delineation of all liver lesions to be present. Additional T2-weighted MR imaging protocols are helpful in lesion characterization, e.g., in order to differentiate between tumors and cysts [35]. The tumors will be targeted by the therapy, whereas cysts are not of particular interest in radioembolization treatment planning. The hepatic arterial CT (HA CT) gives an overview of the patient-individual arterial vessels, which provide potential locations for Y-90 administration. Diffusion-weighted imaging (DWI) adds valuable information on local liver function. All these imaging modalities make a particular contribution to the patient’s individual organ and function, cf. Fig. 1, but it lacks an overall picture. Besides the use of spatial information fusion for patient assessment, SIRT dosimetry, and advanced treatment planning will benefit from this framework to be able to provide a better patient-specific estimation of the treatment biodistribution. As a future perspective, the introduced image modalities provide the basis for methods modeling the particle distribution, which may replace the controversially discussed MAA-SPECT. In order to enable the combined use of information from the different image modalities, image registration techniques are required [13].

Methods for multimodal CT-T1W liver image registration [11, 28, 31], registration of different MR sequences [3, 33], as well as liver CT registration methods [9, 19] can be found in the literature, whereas a dedicated image registration framework tailored to all introduced image modalities in the context of liver radioembolization has not been demonstrated so far. This paper aims at presenting an image registration framework that enables deformable image registration of HA CT, PV CT, CE T1W, T2 and DWI. To the authors’ knowledge, this is the first attempt of an evaluation for multimodal registration in radioembolization including landmark and deformation field analyses.

The image registration framework should enable deformable registration of five different image modalities (HA CT, PV CT, CE T1W, T2, DWI) relevant in primary liver radioembolization planning. The goal is to introduce one registration framework that is capable of performing all required registrations by adapting only the parameterization. In order to demonstrate the performance of the methods in terms of spatial accuracy and physical plausibility, the image registration methods are evaluated in detail based on anatomic landmarks and based on deformation field characteristics in a total of 103 cases of image pairs of 35 patients.

The results of the landmark-based evaluation of the image registration solutions are summarized in Table 3. The global mean landmark errors are 3.20 mm, 4.49 mm, 5.36 mm, and 4.78 mm for HA CT-PV CT, PV CT-CE T1W, CE T1W-T2, and T2-DWI image registration, respectively. An individual investigation of the x-, y-, and z-component of the landmark errors showed that the z-component has the greatest impact on the global mean landmark error, except for the HA CT-PV CT registration solution. The mean landmark errors of the x- and y-components are quite similar. The diagrams in Fig. 3 visualize the size distribution of landmark errors on a component-wise basis. It underlines the main impact of the z-component on the mean landmark error and a similar distribution of errors for the x- and y-components. The cumulative occurrence indicates that the major amount (> 80%) of landmark errors is smaller than the mean error. Considering the mean template slice thickness, the mean landmark errors of CE T1W-T2 and T2-DWI image registration solutions are within the mean slice thickness, whereas the mean landmark error in case of HA CT-PV CT and PV CT-CT T1W image registration is clearly larger than the mean template slice thickness.

The mean values of the minimum and maximum Jacobian determinants indicate moderate volume changes with maximum volume changes in approximately halving or doubling of cell volume, see Table 4. The deformation fields do not have any foldings. The mean average divergence is rather low and the mean harmonic energy is 0.08, 0.25, 0.12, and 0.58 for HA CT-PV CT, PV CT-CE T1W, CE T1W-T2, and T2-DWI image registration, respectively. We further investigated the local distribution of the Jacobian determinant, the divergence, and the local harmonic energy maps of the deformation fields by mean normalized cross-correlation coefficients calculated from all registration results. A good spatial correlation was demonstrated in case of the Jacobian determinant and the divergence, except for HA CT-PV CT, cf. Table 5. A correlation of the local distribution of the Jacobian determinant and the harmonic energy maps or between divergence and local harmonic energy maps was not found. In case of the PV CT-CE T1W image registration and the CE T1W-T2 image registration, we also visually analyzed the deformation inside the segmented liver and found that a majority of cases show a low harmonic energy in the middle of the liver and larger harmonic energies at the liver periphery, cf. Figs. 4 and 5.

The landmark- and deformation field-based evaluation of the registration methods demonstrate anatomical correspondence and physical plausibility. The results show that the mean landmark errors of the CE T1W-T2 and T2-DWI image registration solutions are within the mean slice thickness and therefore indicate high spatial correspondence. Nevertheless, the definition of three-dimensional landmarks is a very subjective and challenging task causing non-negligible inter-observer errors, which were about 3 mm in the multi-phase CE T1W images [30]. Hence, the mean landmark error in case of the HA CT-PV CT and PV CT-CE T1W image registration might be strongly biased by this phenomenon. The results are comparable to or better than those reported by other groups: In the context of CT-guided liver ablation, a landmark error of \(5.3~\pm ~2.5~\hbox {mm}\) was reported for deformable registration of liver CT images with an intra-observer variation of approximately 2 mm [19]. The mean fiducial errors of follow-up CT registration in two commercial products were stated to be 9.3 mm and 11 mm [9], and the investigation of different algorithms for liver CT-MRI registration reported average errors of 3.9, 4.5, and 6.5 mm [1]. For benchmarking purposes, we also evaluated the liver HA CT-PV CT registration on the DIR-Lab reference dataset [4, 5]. We obtained an average landmark error of 1.93 mm, staying below the values of 2.14 mm and 2.07 mm reported by Heinrich et al. [11] and de Senneville et al. [28]. However, the comparison seems unfair, as our method was not specifically adapted to lung image registration. A performance evaluation on the DIR-Lab dataset with adjusted parameters to the demands of lung image registration (e.g., \(\alpha =1\), \(\rho =\tau =10\)) was presented in [18] and an average landmark error of 0.93 mm was reported there.

By an individual investigation of the x-, y-, and z-component of the landmark errors, we confirmed that the z-component makes the biggest contribution to the mean error in most cases. This leads to the assumption that the definition of landmarks in low resolved spatial directions is challenging. An investigation of the landmark definition in images with non-isotropic image resolution as well as the analysis of inter- and intra-observer errors might provide further insight into this topic. In addition, the accuracy and precision of the registration would benefit from an optimization of the acquisition geometry, allowing for smaller slice thicknesses. Nevertheless, the analysis was performed on data from clinical routine and experienced radiological technicians performed the definition of landmarks. As the achieved registration accuracy is below the slice thickness, below or close to the inter-observer error, the developed methods seem accurate enough for the intended application.

The analysis of the deformation fields demonstrates the absence of any foldings. Therefore, the deformation fields can be considered plausible in the sense of spatially allowed transformations. For the PV CT-CT T1W registration, this behavior was forced by design, adding the volume constraint. There might be a relation to the specific imaging protocols, as the two-phase CT was acquired in one imaging session and the MRI sequences were also acquired in one session, but on different days. This means that also the type of the expected deformation of the PV CT-CE T1W registration might differ from the other registrations.

The Jacobian determinant and the deformation divergence show moderate volume changes, which was expected in this specific intra-patient liver registration scenario. The range of mean minimum and maximum Jacobian determinants of around 0.45 and 1.7 was determined on the voxel dimension and is consistent with findings on the local nonlinear part of the liver deformation being smaller than 5 mm [12]. The correlation between local distributions of the Jacobian determinant and the divergence indicates that the underlying deformation is rather smooth, or more precisely, that the deformation is differentiable and does not fluctuate significantly at any point [22]. Technical details on this are given in Appendix 6. The low correlation in case of HA CT-PV CT and T2-DWI is caused by some outliers exhibiting no correlation as indicated by the high standard deviation. The smoothness of the deformation is also represented by low harmonic energies. Freiman et al. [8] reported median harmonic energies of approximately 0.13 for local-affine diffeomorphic demons, 0.16 for diffeomorphic demons, and 0.2 for demons in controlled experiments on an abdominal CT atlas and artificially generated ground truth deformations. In comparison, our proposed registration solutions reached similar values even on routine image data, acquired for liver radioembolization. Only in case of the T2-DWI registration, the measured harmonic energy is higher. Lower harmonic energy values in the middle of the liver volume and larger values at the liver periphery indicate higher strains at the liver periphery. This should be further analyzed in subsequent studies as well as the relation to local tissue properties. Paulsson et al. [25] investigated respiratory-induced liver deformation. They also observed greater deformation at the periphery than at the center of the liver. To verify these observations and our conclusion, patient-individual measurements of physical parameters would be required for a reliable comparison [15]. In addition, the relation and impact of sliding liver motion could be considered further. The curvature regularization prohibits non-smooth deformations in the current approach and introduces a bias to the displacement vectors at the liver boundary. Considering sliding liver motion and therefore larger displacement vectors will result in higher harmonic energy values at the liver periphery. In order to investigate the quality and stability of the registration parameters, a systematic evaluation and potential optimization should be developed in a next step. Also the selection and stability of the chosen parameters might need further investigation.

The proposed image registration framework focusses on the fusion of multimodal images, namely HA CT, PV CT, CE T1W, T2, and DWI. Therefore, it helps to provide valuable joint information in order to assess the applicability of liver radioembolization. Regarding the registration evaluation criteria, the intended goal of information fusion has been reached with satisfactory precision. Detailed treatment planning requires the integration of additional modalities like SPECT/CT [16] or a multi-slice multi-gradient echo MR-sequence [29] to estimate the intra-hepatic distribution of SIRT particles for dose planning. Therefore, future work includes the integration of these image modalities into the registration framework. In combination with segmentation algorithms and models for activity and dose calculation, it can provide a whole platform for radioembolization planning.

A multimodal image registration framework was presented for patient assessment and primary treatment planning for radioembolization of the liver. In order to register the multimodal CT and MR images, a chain of four pair-wise image registrations was introduced based on a variational registration framework using normalized gradient fields as distance measure and curvature regularization. It was evaluated on a total of 103 cases of 35 patients, yielding good anatomical correspondence and physically plausible deformations. This framework provides a basis for patient assessment and primary treatment planning for liver radioembolization.