Skip to main content
Erschienen in: International Journal of Computer Assisted Radiology and Surgery 3/2018

Open Access 01.03.2018 | Original Article

A hybrid optimization strategy for registering images with large local deformations and intensity variations

verfasst von: Zhang Li, Lucas J. van Vliet, Jaap Stoker, Frans M. Vos

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 3/2018

Abstract

Purpose

To develop a method for intra-patient registration of pre- and post-contrast abdominal MR images with large local deformations and large intensity variations.

Method

A hybrid method is proposed to deal with this problem. It consists of two coupled techniques: (1) descriptor matching (DM) at the original resolution using a discrete optimization strategy to avoid getting trapped in a local minimum; (2) continuous optimization to refine the registration outcome based on autocorrelation of local image structure (ALOST). Our method—called DM-ALOST—has become insensitive to the local uptake of contrast agent by exploiting the mean phase and the phase congruency extracted from the multi-scale monogenic signal. The method was extensively tested on abdominal MR data of 30 patients with Crohn’s disease.

Results

DM-ALOST produced significantly larger mean Dice coefficients than two state-of-the-art methods \(({p}<0.05)\).

Conclusion

Both qualitative and quantitative tests demonstrated improved registration using the proposed method compared to the state-of-the-art. The DM-ALOST method facilitates measurement of corresponding features from different abdominal MR images, which can aid to assess certain diseases, particularly Crohn’s disease.

Introduction

Image registration has been a key topic in the medical imaging community over the past decades. However, there are still many registration problems that remain challenging. We encountered such a challenge upon registering three-dimensional pre- to post-contrast abdominal images acquired by magnetic resonance imaging (MRI) for quantifying Crohn’s disease activity [1]. This registration is far from trivial as peristalsis of the bowel (or organ motion) may cause large local deformations, which makes that the registration algorithm gets trapped in a local minimum as discussed in [2]. Furthermore, the diseased regions of interest are composed of relatively thin bowel structures, which are easily mismatched. Moreover, there can also be large local intensity variations, a.o. due to space-varying contrast uptake and the MRI bias field.
A coarse to fine (or multi-resolution) registration strategy is often used [3] to avoid getting trapped in local minima as much as possible. However, this can be insufficient for large deformations, as we have experienced in the aforementioned case. A coarse to fine registration method can be combined with discrete optimization approaches, such as graph cuts [4] and linear programming [5]. Discrete optimization is typically less sensitive to the initial conditions [5] and suitable to deal with large deformations [2]. However, discrete optimization goes at the expense of the registration precision because of quantization effects.
Alternatively, continuous optimization typically produces more precise solutions than discrete approaches. However, continuous optimization is sensitive to the initial conditions and gets more easily trapped in a local minimum. A hybrid optimization strategy was recently proposed to combine the best of both worlds [6]. As such the discrete optimization strategy initially yields a coarse solution to cope with the largest deformations after which continuous optimization locally improves the result in a refined search space. A good review on hybrid image registration can be found in [7].
It can be foreseen that the aforementioned multi-resolution strategy removes fine structures (e.g. the bowel wall) from the image at coarse resolutions. Furthermore, local deformation due to peristalsis is in essence different from the global motion of the larger structures due to breathing. As large local deformations are not correctly estimated at a coarse scale, an ambiguous registration result will occur locally. Thus, registration based on the highest resolution is preferred without any restriction of the search space for discrete optimization. The large displacement optical flow (LDOF) method [8] integrates rich descriptors such as the scale-invariant feature transform (SIFT [9]) and the histograms of oriented gradients (HOG [10]) into variational optic flow to solve the large deformation problem. To use descriptor matching with continuous optimization for our task, we need an image representation and an energy function that are both insensitive to local intensity variations.
Previously, well-known rich descriptors such as SIFT, SURF [11] and HOG were studied for representing image structure. However, none of these can fully handle intensity and contrast variations. Alternatively, images may be transformed into a representation that is insensitive to these signal variations. An example of such a representation for multi-dimensional signals is the monogenic signal [12]. Thus, a rich descriptor could be calculated based on the monogenic signal representation of these images.
Likewise, there are many energy functions for image registration that tackle the intensity inhomogeneity problem. For example, mutual information (MI) [13, 14] is one of the most widely used energy functions. However, originally being a global objective function, it lacks local, i.e. spatial, information on the local structure and cannot cope with large space-variant signal and contrast differences. Although several methods introduce local information into a global objective function [15, 16], it has been noticed [17] that finding an accurate correspondence remains difficult, especially due to the many local minima that generally accompany most non-rigid deformation models.
Recently, a method employing a local modality independent neighborhood descriptor (MIND) was proposed [17]. MIND was reported to be a “distinctive” descriptor, which was claimed to be important for registering images with many degrees of freedom [17]. However, MIND is not invariant to large intensity inhomogeneities LCC-Demons was introduced to address the local intensity variation issue [18] by using symmetric local correlation coefficient as energy function. A new descriptor called ALOST is proposed to reduce the influence of intensity inhomogeneities [19]. Methods that transform MRI images into a new image representation can be also used to deal with intensity inhomogeneities [20]. Other methods specifically designed for abdominal DCE-MRI registration use the entire DCE-MRI image sequence to reduce the influence [21, 22]. However, all of above-mentioned methods are sensitive to large local deformations of an image.
In this paper, we present a hybrid registration method to facilitate the spatial alignment of pre- to post-contrast abdominal MR images. Our work is inspired by the method described in [8] that initially performs descriptor matching using discrete optimization succeeded by a continuous optimization to refine the registration. Different from [8], we perform the descriptor matching based on a representation derived from the monogenic signal rather than on the original image. The ALOST method [19] follows as a continuous optimization step to refine the registration. Therefore, our work contains two main contributions compared to [8, 19]. 1. It is insensitive to local contrast changes due to the varying uptake of contrast agent as well as to global signal fluctuations from the MR scanner’s bias field. 2. Several elaborate evaluations are performed to assess the performance of our method on bowel images.

Method

Image registration can be treated as an optimization problem [23]. The energy function to be optimized in our experiments consists of three parts, namely a data term, a regularization term and a coupling term:
$$\begin{aligned} E(\mathbf{w})=E_{\mathrm{data}} (\mathbf{w})+\alpha E_{\mathrm{regularize}} (\mathbf{w})+\beta E_{\mathrm{coupling}} (\mathbf{w},\mathbf{w}_1 ),\nonumber \\ \end{aligned}$$
(1)
where \(\mathbf{w}=[u,v,w]\) is the deformation field for each pixel in a 3D moving image, \(\alpha \) and \(\beta \) are weighting parameters.
In our proposed hybrid registration framework, we first find a solution to the descriptor matching term through discrete optimization: \(\mathbf{w}_1 =\arg \min _{{\mathrm{w}}_2 } \{E_{\mathrm{descriptor}} (\mathbf{w}_2 )\}\) (see (5)). Herewith, we aim to estimate the largest deformations in the presence of intensity distortions without sacrificing image resolution. Subsequently, \(\mathbf{w}_{1}\) is used to initialize the minimization (1).
The remainder of the method section is structured as follows. First, we introduce some basic theory of the monogenic signal on which we will build both parts of the hybrid method. Subsequently, we will define the involved energy terms: the data term, and the coupling term. Finally, we will describe our optimization procedure.

Preliminaries

The mean phase and phase congruency [24] are derived from the analytic representation of a signal based on the Riesz transform [25]. They provide measures that are independent of the signal’s amplitude, making them invariant to trends in the signal and variations in contrast. A representative example is shown in Fig. 1. The intensity distortion we added merely concerned the addition of a Gaussian intensity profile centered at a randomly selected point [26]. We plot profiles along a line segment suffering from a large local intensity variation in Fig. 1c, g. In Fig. 1c, the profiles from the original and distorted images clearly deviate because of the intensity distortion. In contrast, this is not the case in Fig. 1g. This example illustrates that the mean phase image is insensitive to local intensity changes.

Definition of the data term

Our data term \(E_{\mathrm{ALOST}} (\mathbf{w})\) exploits the autocorrelation of LOcal STructural information. It was inspired by the MIND concept [17], which is based on the local sum of squared differences (SSD) between two patches P of image I:
$$\begin{aligned}&D_P \left( {I,\mathbf{x}_1, \mathbf{x}_2 } \right) =\mathop \sum \limits _{\mathbf{p}\in P} \left( {I\left( {\mathbf{x}_1 +\mathbf{p}} \right) -I\left( {\mathbf{x}_2 +\mathbf{p}} \right) } \right) ^{2} \nonumber \\&\quad =\mathop \sum \limits _{\mathbf{p}\in P} I^{2}\left( {\mathbf{x}_1 +\mathbf{p}} \right) -2\underbrace{\mathop \sum \limits _{\mathbf{p}\in P} I\left( {\mathbf{x}_1 +\mathbf{p}} \right) I\left( {\mathbf{x}_2 +\mathbf{p}} \right) }_{AC_P \left( {I,\mathbf{x}_1, \mathbf{x}_2 } \right) }\nonumber \\&\quad \quad +\mathop \sum \limits _{\mathbf{p}\in P} I^{2}\left( {\mathbf{x}_2 +\mathbf{p}} \right) \end{aligned}$$
(2)
where \(\mathbf{x}_1 \) and \(\mathbf{x}_2 \) are the centers of the two image patches p of size \(D\times D\times D\). Notice that expanding the square of (2) reveals the inner product between the two patches, which can also be interpreted as a windowed (hence local) autocorrelation as a function of the difference \(\mathbf{x}_1{-}{} \mathbf{x}_2 \). The local SSDs are incorporated in a feature vector defined by:
$$\begin{aligned} \hbox {MIND}\left( {I,\mathbf{x},\mathbf{r}} \right) =\frac{1}{n}\exp \left( {\frac{-D_P \left( {I,\mathbf{x},\mathbf{x}+\mathbf{r}} \right) }{V\left( {I,\mathbf{x}} \right) }} \right) \end{aligned}$$
(3)
with \(\mathbf{r}\) an offset in a predefined neighborhood \(\mathbf{R}\) of size \(R\times R\times R\) around position \(\mathbf{x}\) (spatial coordinator of the image), n a normalization constant (chosen such that the maximum value of the MIND feature at position \(\mathbf{x}\) is one) and \(V\left( {I,\mathbf{x}} \right) \) represents the average SSD in a small neighborhood N around \(\mathbf{x}\) defined by \(V\left( {I,\mathbf{x}} \right) =\frac{1}{\mathrm{size}(N)}\mathop \sum \nolimits _{\mathbf{n}\in N} D_P \left( {I,\mathbf{x},\mathbf{x}+\mathbf{n}} \right) \). In words, the feature vector consists of a 1-D vector of length \(R\times R\times R\), which represents all correlations between a neighborhood P centered at a voxel \((\mathbf{x})\) and a neighborhood P centered on a voxel \((\mathbf{x}+\mathbf{r})\).
We employ the mean phase image \(I_\mathrm{MP} \) and the phase congruency image \(I_\mathrm{PC} \) for image I in (3), since these are both insensitive to local contrast changes. Furthermore, we concatenate the feature vectors for \(I_\mathrm{MP} \) and \(I_\mathrm{PC} \) to form the ALOST representation as
$$\begin{aligned} \mathrm{ALOST}\left( {I,\mathbf{x},\mathbf{r}} \right) =\left[ {\hbox {MIND}\left( {I_\mathrm{MP}, \mathbf{x},\mathbf{r}} \right) ,\hbox {MIND}\left( {I_\mathrm{PC}, \mathbf{x},\mathbf{r}} \right) } \right] . \end{aligned}$$
The ALOST representation has two important properties: (1) it is insensitive to intensity distortions; (2) it emphasizes ‘salient’ features (e.g. edges) in the image. For the sake of simplicity, we use in the remaining text the notation \(\hbox {ALOST}_\mathrm{f} (\mathbf{x})\) and \(\hbox {ALOST}_\mathrm{m} (\mathbf{x})\) to represent ALOST feature vectors calculated from the fixed image and the moving image, respectively. As such, the data term \(E_{\mathrm{data}} (\mathbf{w})\) in (1) can be defined as:
$$\begin{aligned} E_{\mathrm{data}} (\mathbf{w})= \int \limits _\Omega {|\hbox {ALOST}_\mathrm{m} (\mathbf{x}+\mathbf{w}(\mathbf{x}))-\hbox {ALOST}_\mathrm{f} (\mathbf{x})|^{2}} \text {d}{} \mathbf{x}. \end{aligned}$$
(4)
The local diffusion regularization is used to avoid the ill-posedness of the image registration. More details of the ALOST method can be found in [19].

Definition of the descriptor matching and coupling terms

The registration procedure is made robust against getting stuck in a local minimum through descriptor matching in a discrete optimization scheme. To accomplish this, we densely calculate descriptors over both the fixed and moving images at the original resolution (without down-sampling) to preserve the thin bowel structures. Inspired by [8, 27], we use the 3D HOG in our descriptor matching step. Therefore, we first calculate 3D gradients \([\hbox {G}_{{ x}},\hbox {G}_{{y}},\hbox {G}_{{z}}]\) using a coarse 1-D central difference operator in all three dimensions. Then, these 3D gradients are quantized based on a regular polyhedron with \(N_{0}=10\) orientations (i.e. a icosahedron) to limit the computation time. Furthermore, a Gaussian filter with \(\sigma =0.8\,\hbox {mm}\) is applied to smooth along the polyhedron to reduce quantization effects. The final 3D HOG descriptor for each voxel consists of 10 \(\times \) 7 entries (10 orientations times 7 cells, the cell of the current voxel plus cells from its 6 cardinal neighbors in 3D space). For building a cell, we empirically set neighborhoods \(N_{n}=9\).
Notice that the discrete optimization aims to simplify the registration problem after which the continuous optimization follows to refine the registration. Therefore, a highly accurate calculation is not required. At the same time, we found that the HOG was sensitive to large local intensity variations (see see Results section). Accordingly, we propose to calculate HOG on the mean phase image \((\bar{\varphi }(\mathbf{x}))\), which describes the type of local structure and is insensitive to the intensity variations.
The descriptor matching term is defined as:
$$\begin{aligned} E_{\mathrm{descriptor}} (\mathbf{w}_2 )= & {} \sum _i |\hbox {HOG-MP}_\mathrm{m} (\mathbf{x}_i +\mathbf{w}_2 (\mathbf{x}_i ))\nonumber \\&-\hbox {HOG-MP}_\mathrm{f} (\mathbf{x}_i )|^{2}, \end{aligned}$$
(5)
where \(\hbox {HOG-MP}_\mathrm{m} \) and \(\hbox {HOG-MP}_\mathrm{f} \) denote the HOG descriptors of the mean phase images derived from the moving image and the fixed image, respectively; the summation is over all positions \(\mathbf{x}_i\) in the fixed image where a HOG descriptor was calculated; \(\mathbf{x}_j =\mathbf{x}_i +\mathbf{w}_2 (\mathbf{x}_i )\) is a position in the moving image with such a descriptor.
We couple the discrete optimization functional to the continuous one by adding a coupling term to the latter
$$\begin{aligned} E_{\mathrm{coupling}} (\mathbf{w},\mathbf{w}_1 )=\int \limits _\Omega {\rho (\mathbf{x})|\mathbf{w}_1 (\mathbf{x})-\mathbf{w}(\mathbf{x})|^{2}} \text {d}{} \mathbf{x}. \end{aligned}$$
(6)

Optimization procedure

Approximate nearest neighbor (ANN) search method is used for descriptor matching [28]. Compared to exhaustive search method, ANN method allows reduction of the complexity without losing too much accuracy. Similar to [8], we use the ANN method twice (for searching from the moving image to the fixed image and vice versa) to exclude outliers.
A continuous optimization strategy is subsequently used for refining the deformation field, after descriptor matching, by minimizing (1). We apply the Gauss–Newton optimization method in order to do so. We use a multi-resolution registration procedure in order to speed up the convergence. Notice that the multi-resolution strategy is only used in the continuous registration step. At that stage the largest deformations have already been resolved.
The “initialization” through \(\mathbf{w}_1 \) is softly favored through \(E_{\hbox {coupling}} (\mathbf{w},\mathbf{w}_1 )\) (see (1)), but we reduce the “influence” of \(\mathbf{w}_1\) when estimating \(\mathbf{w}\) at finer resolutions. Different from [8], the weighting parameter of \(\beta \) is not fixed but decreasing with increasing iteration number of the registration procedure: \(\beta (k)=\beta _0 /k\), where \(\beta _0 \) is the initial value and k the iteration number.

Results

The proposed hybrid registration method will be compared against two state-of-the-art registration techniques: the deformable registration via attribute matching and mutual-saliency weighting (DRAMMS), [29] and the MIND [17]. Both methods are based on neighborhood structure descriptors. In total, seven different registration methods will be used in the comparisons. These will be referred to as DRAMMS, DM-O (descriptor matching on original image), DM (descriptor matching on mean phase image), MIND, DM-MIND (descriptor matching plus MIND), ALOST [19] and DM-ALOST (descriptor matching plus ALOST, the hybrid method currently proposed).
The computation time for the ALOST and MIND methods on the abdominal dataset (containing the largest images of size 400*400*100) was less than 10 min on a personal computer equipped with an Intel® \(\hbox {Core}^{\mathrm{TM}}2\) Quad Processor Q8400 clocked at 2.66 GHz and 4 GB RAM memory. The DM-ALOST and DM-MIND took around 20 min, which is in the same range as the DRAMMS approach.

Comparison of descriptor matching on intensity and mean phase

Figure 1d visualizes the descriptor matching results using HOG on the intensity images, HOG-I, and Fig. 1h pictures the descriptor matching results using HOG on the mean phase images, HOG-MP, between the same regions (middle left side) in Fig. 1a, b. Red circles indicate the positions around which the descriptors are calculated in Fig. 1a and green dots indicate positions in Fig. 1b. Since there were no spatial deformations, an identity correspondence should be found, i.e. each green dot should directly fit in a red circle, without a displacement. However, there is a clearly observable difference in the matching pairs by using the HOG-I descriptor (Fig. 1d) and the HOG-MP descriptor (Fig. 1h). Where the HOG-MP descriptor mostly finds the identity correspondence, the HOG-I descriptor finds correspondences over very large displacements. This outcome emphasizes that HOG-I is very sensitive to intensity variation, whereas HOG-MP is not.

Registration performance on synthetic abdominal images

We evaluated the performance of our registration framework in the presence of a local geometric deformation and a local intensity distortion. We empirically set \(\alpha =0.1\) for ALOST (same as [19]) and in addition set \(\beta _0 =0.5\) for DM-ALOST. In the ALOST paper [19], we have extensively tested different settings for MIND, but found that the optimal settings were the default settings. We adopted this setting for this paper. For DRAMMS, we also tested several settings in this paper, but without too much difference in performance. Therefore, we used the default settings of DRAMMS and MIND in all our experiments. Furthermore, we selected a typical 2D slice (containing diseased bowel) from a 3D image for this synthetic experiment.
We generated 100 differently deformed images with large local large deformation and local intensity distortion from the selected reference image. First, a global geometric deformation with a thin-plate spline (TPS) model (similar to [26], 7 \(\times \) 7 grid size) was applied to a reference image (Fig. 2a). Second, a local deformation was imposed around one randomly picked point from the image. This deformation was generated by randomly selecting a displacement between 15 to 25 pixels in the x and y direction, respectively. Thereafter, care was taken that the displacement field was continuous by Gaussian filtering, creating a Gaussian distributed displacement field around the picked point. The width of the Gaussian was set to a randomly selected value in the range from 20 to 50 pixels, which effectively controlled the locality of the deformation. Finally, a local intensity distortion was induced by adding a Gaussian-shaped intensity distribution centered at a randomly selected point (again as in [26]). An example is shown in Fig. 2b.
DM-ALOST gave the best overall result on data with only large geometric deformation, although the difference with DM-MIND was not significant (see Fig. 3a). Furthermore, the DM-ALOST outcome performs significantly better than all other approaches as assessed by a two-sided signed rank test \(({p} < 0.01)\) on data with large geometric deformations and local intensity distortions (Fig. 3b). One can also see that the results of DM on original image (DM-O) and DM on mean phase (DM) are similar for images without intensity distortion. On the other hands, DM gave significant better \(({p}< 0.01)\) results comparing to DM-O for images with intensity distortion. It shows that descriptor matching on mean phase images could give better initialization for the following continuous registration step. Therefore, we will only use DM for the rest of experiments.

Abdominal MR image pre- to post-contrast registration

The last series of experiments assess the suitability of the proposed method to register pre- and post-contrast abdominal MR data of patients with Crohn’s disease. These MRI data were taken from a prior study, which has been approved by the medical ethics committee [30]. Thirty out of 33 patients from the prior study gave written consent to use their data for future investigations. The image size of the pre and post-contrast scans was \(400 \times 400\times 100\) voxels with a resolution of \(1 \times 1\times 2 \,\hbox {mm}^{3}\) and was acquired in a breath-hold. All patients underwent ileocolonoscopy within 1 month after the MRI scan was acquired. We used the six methods as described in previous section to register the data (DM-O is excluded). The parameters were the same as in previous section. These settings also proved to yield the best registration result for the abdominal images (data not shown).

Quantitative assessment of the registration performance

Unfortunately, it is not a trivial task to identify landmark points on the bowel wall since there is deformation due to inspiration depth and a continuous deformation over time due to peristalsis between the scans. Therefore, we decided to label relevant ‘landmark’ regions, namely the bowel segments. A research fellow specialized in imaging Crohn’s disease supervised by a radiologist delineated each bowel segment affected by Crohn’s disease in the post-contrast MR images. Subsequently, two research fellows, both specialized in imaging Crohn’s disease, independently annotated the same region in the pre-contrast images. Polygons were drawn in coronal slices. Each lesion was delineated in all slices in which it was observed. In total 30 lesions were identified in 30 bowel segments. A typical example is shown in Fig. 4. The mean Dice coefficient calculated over the annotations in the pre-contrast MR images was 0.71, reflecting the inter-observer agreement. The Dice coefficient was also computed over corresponding pairs of annotations in the pre- and post-contrast images prior to registration and after registration by each of the six methods. The results are summarized in Table 1. One can see that the DM-ALOST outperformed the other methods for the data of both annotators. Particularly, the Wilcoxon signed rank test demonstrated that DM-ALOST was significantly \(({p}<0.05)\) better than the other methods. Notice that the mean Dice coefficients of DM-ALOST, 0.68 and 0.69, closely approximate the inter-observer agreement..

Correlation of manual to semi-automatic measurements of relative contrast enhancement

The manual relative contrast enhancement (RCE) was calculated as defined above using the mean intensities measured over manually delineated regions in the pre- and post-contrast images. Four research fellows each supervised by a different radiologist outlined corresponding regions with the most disease activity in the pre- and post-contrast images of the 30 patients affected by the disease [30]. These regions comprised small focal areas on the order of 100 voxels, as is conventional. These local annotations were made on the most suspicious part of the bowel wall. This is how radiologists conventionally measure RCE. The RCE measurements for these 30 patients were used in previous study [31]. Unfortunately, the annotations for these RCE measurements were not saved. Moreover, one research fellow supervised by a radiologist delineated the entire region affected by the disease only on post-contrast MR images. These annotations could be saved and were used to calculate the semi-automatic RCE measurement. The RCE was semi-automatically measured by first calculating the mean intensity over this full region in the post-contrast image and by averaging over the region in the pre-contrast image that was copied from the post-contrast image. These semi-automatic RCE measures were correlated to the manual RCE measures by means of Pearson’s correlation. The correlation coefficients thus obtained are shown in Table 2, stratified by averaged over four fellows. Notice that DM-ALOST gave a higher correlation than the other registration methods, although none of the differences were significant. It showed that DM-ALOST gave best registration results among all six methods [32].
Table 1
Mean dice metric (MDM) between annotations on pre- and post-contrast abdominal MR images prior to registration (i.e. no registration), and after registration using, respectively, DRAMMS, DM, MIND, DM-MIND, ALOST and DM-ALOST
 
No registration
DRAMMS
DM
MIND
DM-MIND
ALOST
DM-ALOST
Fellow-1
0.50 (0.24)
0.52 (0.22)
0.55 (0.20)
0.63 (0.21)
0.65 (0.20)
0.66 (0.22)
0.68 (0.17)
Fellow-2
0.42 (0.27)
0.52 (0.23)
0.53 (0.21)
0.59 (0.21)
0.62 (0.19)
0.61 (0.21)
0.69 (0.13)
The numbers printed in bold face are the maxima, i.e. the best results, per row. The standard deviation is shown between brackets
Table 2
Average Pearson correlation coefficients of correlations between manual and semi-automatic RCE measurements
 
No registration
DRAMMS
DM
MIND
DM-MIND
ALOST
DM-ALOST
Average
0.55
0.6125
0.6125
0.5975
0.6025
0.605
0.6225
Manual measurements were taken by four research fellows supervised by four radiologists. Semi-automatic RCE measurements were taken prior to and after registration by DRAMMS, MIND, DM-MIND, ALOST and DM-ALOST. The numbers printed in boldface are the maxima, i.e. the best results. The average correlation coefficients by correlating the semi-automatic RCE measurements to all the manual RCE measurements

Discussion and conclusion

We developed a hybrid registration framework called DM-ALOST to register images with large local deformations combined with large local intensity variations. The main target application was the spatial alignment of 3D abdominal pre- and post-contrast MR images of patients with Crohn’s disease. Differences in inspiration depth and peristalsis cause large local geometric deformations, while the administrated contrast agent causes large spatially localized contrast variations. The developed method—DM-ALOST—combines a discrete descriptor matching at the full image resolution with continuous optimization of a powerful representation exploiting the mean phase and the phase congruency. The discrete optimization enabled us to cope with local geometric deformations of small image structures and guides the continuous registration in such a way that it does not get trapped in a local minimum. The use of the mean phase offers the required invariance to large intensity distortions.
It is important for our application that especially the diseased parts of the bowel are correctly registered. Previously, we developed a method to automatically segment the diseased bowel wall from post-contrast MR images [33]. An obvious way to enhance the registration might be to give high priority to such regions [34]. However, we have found that this does not lead to a significant improvement, which we attribute to the small size of the regions.
There are several limitations of our work. The optimal weighting parameters were manually selected for each registration task. Although this might bias the reported figures, we found little difference between the manually tuned settings for the different data sets and the different tasks. Indeed, empirical fine-tuning may be needed for particular task. For example, the scale of Gaussian filter should be increased if the SNR of image is too low. Therefore, we do not expect that it influences the ordering of the six methods in terms of performance. Another limitation is that the HOG-MP descriptor is not invariant to non-rigid deformations. For data hampered by very large local non-rigid deformations, the descriptor matching using HOG-MP might fail. In that case, descriptors could be considered such as described in [35, 36]. However, that would go at the expense of a higher computation time. Furthermore, incorporating segmentation of the entire bowel (i.e. not only the diseased parts) might be useful, although the segmentation on the pre-contrast MR images is far from trivial.

Acknowledgements

The research leading to these results was partly funded by the European Community’s Seventh Framework Programme (FP7/2007-2013): the VIGOR++ Project (Grant Agreement Nr. 270379)

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://​creativecommons.​org/​licenses/​by/​4.​0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Unsere Produktempfehlungen

Die Chirurgie

Print-Titel

Das Abo mit mehr Tiefe

Mit der Zeitschrift Die Chirurgie erhalten Sie zusätzlich Online-Zugriff auf weitere 43 chirurgische Fachzeitschriften, CME-Fortbildungen, Webinare, Vorbereitungskursen zur Facharztprüfung und die digitale Enzyklopädie e.Medpedia.

Bis 30. April 2024 bestellen und im ersten Jahr nur 199 € zahlen!

e.Med Interdisziplinär

Kombi-Abonnement

Für Ihren Erfolg in Klinik und Praxis - Die beste Hilfe in Ihrem Arbeitsalltag

Mit e.Med Interdisziplinär erhalten Sie Zugang zu allen CME-Fortbildungen und Fachzeitschriften auf SpringerMedizin.de.

e.Med Radiologie

Kombi-Abonnement

Mit e.Med Radiologie erhalten Sie Zugang zu CME-Fortbildungen des Fachgebietes Radiologie, den Premium-Inhalten der radiologischen Fachzeitschriften, inklusive einer gedruckten Radiologie-Zeitschrift Ihrer Wahl.

Literatur
1.
Zurück zum Zitat Rimola J, Rodriguez S, García-Bosch O, Ordás I, Ayala E, Aceituno M, Pellisé M, Ayuso E, Ricart C, Donoso L, Panés J (2009) Magnetic resonance for assessment of disease activity and severity in ileocolonic Crohn’s disease. Gut 58(8):1113–1120CrossRefPubMed Rimola J, Rodriguez S, García-Bosch O, Ordás I, Ayala E, Aceituno M, Pellisé M, Ayuso E, Ricart C, Donoso L, Panés J (2009) Magnetic resonance for assessment of disease activity and severity in ileocolonic Crohn’s disease. Gut 58(8):1113–1120CrossRefPubMed
2.
Zurück zum Zitat Heinrich MP, Jenkinson M, Brady SM, Schnabel JA (2013) MRF-based deformable registration and ventilation estimation of lung CT. IEEE Trans Med Imaging 32(7):1239–1248CrossRefPubMed Heinrich MP, Jenkinson M, Brady SM, Schnabel JA (2013) MRF-based deformable registration and ventilation estimation of lung CT. IEEE Trans Med Imaging 32(7):1239–1248CrossRefPubMed
4.
Zurück zum Zitat Boykov Y, Kolmogorov V (2004) An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell 26(9):1124–1137CrossRefPubMed Boykov Y, Kolmogorov V (2004) An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell 26(9):1124–1137CrossRefPubMed
5.
Zurück zum Zitat Glocker B, Komodakis N, Tziritas G, Navab N, Paragios N (2008) Dense image registration through MRFs and efficient linear programming. Med Image Anal 12(6):731–741CrossRefPubMed Glocker B, Komodakis N, Tziritas G, Navab N, Paragios N (2008) Dense image registration through MRFs and efficient linear programming. Med Image Anal 12(6):731–741CrossRefPubMed
6.
Zurück zum Zitat Kwon D, Niethammer M, Akbari H, Bilello M, Davatzikos C, Pohl K (2014) PORTR: pre-operative and post-recurrence brain tumor registration. IEEE Trans Med Imaging 33(3):651–667CrossRefPubMedPubMedCentral Kwon D, Niethammer M, Akbari H, Bilello M, Davatzikos C, Pohl K (2014) PORTR: pre-operative and post-recurrence brain tumor registration. IEEE Trans Med Imaging 33(3):651–667CrossRefPubMedPubMedCentral
7.
Zurück zum Zitat Glocker B, Sotiras A, Komodakis N, Paragios N (2011) Deformable medical image registration: setting the state of the art with discrete methods. Annu Rev Biomed Eng 13:219–244CrossRefPubMed Glocker B, Sotiras A, Komodakis N, Paragios N (2011) Deformable medical image registration: setting the state of the art with discrete methods. Annu Rev Biomed Eng 13:219–244CrossRefPubMed
8.
Zurück zum Zitat Brox T, Malik J (2011) Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans Pattern Anal Mach Intell 33(3):500–513CrossRefPubMed Brox T, Malik J (2011) Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans Pattern Anal Mach Intell 33(3):500–513CrossRefPubMed
9.
Zurück zum Zitat Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110CrossRef Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110CrossRef
10.
Zurück zum Zitat Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Computer vision and pattern recognition (CVPR2005). San Diego, CA Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Computer vision and pattern recognition (CVPR2005). San Diego, CA
11.
Zurück zum Zitat Bay H, Ess A, Tuytelaars T, Gool LV (2008) Speeded-up robust features. Comput Vis Image Underst 110(3):346–359CrossRef Bay H, Ess A, Tuytelaars T, Gool LV (2008) Speeded-up robust features. Comput Vis Image Underst 110(3):346–359CrossRef
12.
Zurück zum Zitat Mellor M, Brady M (2005) Phase mutual information as a similarity measure for registration. Med Image Anal 9(4):330–43CrossRefPubMed Mellor M, Brady M (2005) Phase mutual information as a similarity measure for registration. Med Image Anal 9(4):330–43CrossRefPubMed
13.
Zurück zum Zitat Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P (1997) Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 16(2):187–198CrossRefPubMed Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P (1997) Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 16(2):187–198CrossRefPubMed
14.
Zurück zum Zitat Viola P, III WW (1997) Alignment by maximization of mutual information. Int J Comput Vision 24(2):137–154CrossRef Viola P, III WW (1997) Alignment by maximization of mutual information. Int J Comput Vision 24(2):137–154CrossRef
15.
Zurück zum Zitat Pluim J, Maintz J, Viergever M (2000) Image registration by maximization of combined mutual information and gradient information. IEEE Trans Med Imaging 19(8):809–814CrossRefPubMed Pluim J, Maintz J, Viergever M (2000) Image registration by maximization of combined mutual information and gradient information. IEEE Trans Med Imaging 19(8):809–814CrossRefPubMed
16.
Zurück zum Zitat Studholme C, Drapaca C, Iordanova B, Cardenas V (2006) Deformation-based mapping of volume change from serial brain MRI in the presence of local tissue contrast change. IEEE Trans Med Imaging 25(5):626–639CrossRefPubMed Studholme C, Drapaca C, Iordanova B, Cardenas V (2006) Deformation-based mapping of volume change from serial brain MRI in the presence of local tissue contrast change. IEEE Trans Med Imaging 25(5):626–639CrossRefPubMed
17.
Zurück zum Zitat Heinrich MP, Jenkinson M, Bhushan M, Matin T, Gleeson FV, Brady SM, Schnabel JA (2012) MIND: modality independent neighbourhood descriptor for multi-modal deformable registration. Med Image Anal 16(7):1423–1435CrossRefPubMed Heinrich MP, Jenkinson M, Bhushan M, Matin T, Gleeson FV, Brady SM, Schnabel JA (2012) MIND: modality independent neighbourhood descriptor for multi-modal deformable registration. Med Image Anal 16(7):1423–1435CrossRefPubMed
18.
Zurück zum Zitat Lorenzi M, Ayache N, Frisoni G , Pennec X, ADNI (2013) LCC-demons: a robust and accurate symmetric diffeomorphic registration algorithm. NeuroImage 81:470–483 Lorenzi M, Ayache N, Frisoni G , Pennec X, ADNI (2013) LCC-demons: a robust and accurate symmetric diffeomorphic registration algorithm. NeuroImage 81:470–483
19.
Zurück zum Zitat Li Z, Mahapatra D, Tielbeek JAW, Stoker J, van Vliet LJ, Vos FM (2016) Image registration based on autocorrelation of local structure. IEEE Trans Med Imaging 35(1):63–75CrossRefPubMed Li Z, Mahapatra D, Tielbeek JAW, Stoker J, van Vliet LJ, Vos FM (2016) Image registration based on autocorrelation of local structure. IEEE Trans Med Imaging 35(1):63–75CrossRefPubMed
20.
Zurück zum Zitat Wachinger C, Nassir N (2012) Entropy and Laplacian images: structural representations for multi-modal registration. Med Image Anal 16:1–17CrossRefPubMed Wachinger C, Nassir N (2012) Entropy and Laplacian images: structural representations for multi-modal registration. Med Image Anal 16:1–17CrossRefPubMed
21.
Zurück zum Zitat Hamy V, Dikaios N, Punwani S, Melbourne A, Latifoltojar A, Makanyanga J, Chouhan M, Helbren E, Menys A, Taylor S, Atkinson D (2013) Respiratory motion correction in dynamic MRI using robust data decomposition registration–application to DCE-MRI. Med Image Anal 18(2):301–313CrossRefPubMed Hamy V, Dikaios N, Punwani S, Melbourne A, Latifoltojar A, Makanyanga J, Chouhan M, Helbren E, Menys A, Taylor S, Atkinson D (2013) Respiratory motion correction in dynamic MRI using robust data decomposition registration–application to DCE-MRI. Med Image Anal 18(2):301–313CrossRefPubMed
22.
Zurück zum Zitat Li Z, Tielbeek JAW, Caan MWA, Puylaert C, Ziech MLW, Nio C, Stoker J, van Vliet LJ, Vos FM (2015) Expiration-phase template-based motion correction of free-breathing abdominal dynamic contrast enhanced MRI. IEEE Trans Biomed Eng 62(4):1215–1225CrossRefPubMed Li Z, Tielbeek JAW, Caan MWA, Puylaert C, Ziech MLW, Nio C, Stoker J, van Vliet LJ, Vos FM (2015) Expiration-phase template-based motion correction of free-breathing abdominal dynamic contrast enhanced MRI. IEEE Trans Biomed Eng 62(4):1215–1225CrossRefPubMed
23.
Zurück zum Zitat Klein S, Staring M, Pluim JP (2007) Evaluation of optimization methods for nonrigid medical image registration using mutual information and B-splines. IEEE Trans Image Process 16(12):2879–2890CrossRefPubMed Klein S, Staring M, Pluim JP (2007) Evaluation of optimization methods for nonrigid medical image registration using mutual information and B-splines. IEEE Trans Image Process 16(12):2879–2890CrossRefPubMed
24.
Zurück zum Zitat Kovesi P (2002) Edges are not just steps. In: The 5th Asian conference on computer vision Kovesi P (2002) Edges are not just steps. In: The 5th Asian conference on computer vision
25.
Zurück zum Zitat Felsberg M, Sommer G (2001) The monogenic signal. IEEE Trans Signal Process 49(12):3136–3144CrossRef Felsberg M, Sommer G (2001) The monogenic signal. IEEE Trans Signal Process 49(12):3136–3144CrossRef
26.
Zurück zum Zitat Myronenko A, Song X (2009) Image registration by minimization of residual complexity. In: Computer vision and pattern recognition (CVPR’09). FL, USA, Miami Myronenko A, Song X (2009) Image registration by minimization of residual complexity. In: Computer vision and pattern recognition (CVPR’09). FL, USA, Miami
27.
Zurück zum Zitat Klaeser A, Marszalek M, Schmid C (2008) A Spatio-Temporal descriptor based on 3D-gradients. In: Proceedings of the British machine vision conference (BMVC2008), Leeds, UK Klaeser A, Marszalek M, Schmid C (2008) A Spatio-Temporal descriptor based on 3D-gradients. In: Proceedings of the British machine vision conference (BMVC2008), Leeds, UK
28.
Zurück zum Zitat Shakhnarovich G, Darrell T, Indyk P (2006) Nearest-neighbor methods in learning and vision: theory and practice. MIT Press, Cambridge Shakhnarovich G, Darrell T, Indyk P (2006) Nearest-neighbor methods in learning and vision: theory and practice. MIT Press, Cambridge
29.
Zurück zum Zitat Ou Y, Sotiras A, Paragios N, Davatzikos C (2011) DRAMMS: deformable registration via attribute matching and mutual-saliency weighting. Med Image Anal 15(4):622–639CrossRefPubMed Ou Y, Sotiras A, Paragios N, Davatzikos C (2011) DRAMMS: deformable registration via attribute matching and mutual-saliency weighting. Med Image Anal 15(4):622–639CrossRefPubMed
30.
Zurück zum Zitat Ziech ML, Lavini C, Caan MW, Nio CY, Stokkers PC, Bipat S, Ponsioen CY, Nederveen AJ, Stoker J (2012) Dynamic contrast-enhanced MRI in patients with luminal Crohn’s disease. Eur J Radiol 81(11):3019–3027CrossRefPubMed Ziech ML, Lavini C, Caan MW, Nio CY, Stokkers PC, Bipat S, Ponsioen CY, Nederveen AJ, Stoker J (2012) Dynamic contrast-enhanced MRI in patients with luminal Crohn’s disease. Eur J Radiol 81(11):3019–3027CrossRefPubMed
31.
Zurück zum Zitat Tielbeek J, Makanyanga J, Bipat S, Pendse D, Nio C, Vos F, Taylor S, Stoker J (2013) Grading Crohn disease activity with MRI: interobserver variability of MRI features, MRI scoring of severity, and correlation with Crohn disease endoscopic index of severity. Am J Roentgenol 201(6):1220–1228CrossRef Tielbeek J, Makanyanga J, Bipat S, Pendse D, Nio C, Vos F, Taylor S, Stoker J (2013) Grading Crohn disease activity with MRI: interobserver variability of MRI features, MRI scoring of severity, and correlation with Crohn disease endoscopic index of severity. Am J Roentgenol 201(6):1220–1228CrossRef
32.
Zurück zum Zitat Castillo R, Castillo E, Fuentes D, Ahmad M, Wood AM, Ludwig MS, Guerrero T (2013) A reference dataset for deformable image registration spatial accuracy evaluation using the COPDgene study archive. Phys Med Biol 58(9):2861–2877CrossRefPubMedPubMedCentral Castillo R, Castillo E, Fuentes D, Ahmad M, Wood AM, Ludwig MS, Guerrero T (2013) A reference dataset for deformable image registration spatial accuracy evaluation using the COPDgene study archive. Phys Med Biol 58(9):2861–2877CrossRefPubMedPubMedCentral
33.
Zurück zum Zitat Mahapatra D, Schüffler PJ, Tielbeek JA, Jesica JS, Makanyanga C, Taylor SA, Vos FM, Buhmann JM (2013) Automatic detection and segmentation of Crohn’s disease tissues from abdominal MRI. IEEE Trans Med Imaging 12(32):2332–2348CrossRef Mahapatra D, Schüffler PJ, Tielbeek JA, Jesica JS, Makanyanga C, Taylor SA, Vos FM, Buhmann JM (2013) Automatic detection and segmentation of Crohn’s disease tissues from abdominal MRI. IEEE Trans Med Imaging 12(32):2332–2348CrossRef
34.
Zurück zum Zitat Papież B, Franklin J, Heinrich M, Gleeson F, Schnabel J (2015) Liver motion estimation via locally adaptive over-segmentation regularization. In: MICCAI Papież B, Franklin J, Heinrich M, Gleeson F, Schnabel J (2015) Liver motion estimation via locally adaptive over-segmentation regularization. In: MICCAI
35.
Zurück zum Zitat Lin H, Jacobs DW (2005) Deformation invariant image matching. In: 10th IEEE international conference on computer vision (ICCV 2005), Beijing, China Lin H, Jacobs DW (2005) Deformation invariant image matching. In: 10th IEEE international conference on computer vision (ICCV 2005), Beijing, China
36.
Zurück zum Zitat Moreno-Noguer F (2011) Deformation and illumination invariant feature point descriptor. In: The 24th IEEE conference on computer vision and pattern recognition (CVPR 2011), Colorado Springs, USA Moreno-Noguer F (2011) Deformation and illumination invariant feature point descriptor. In: The 24th IEEE conference on computer vision and pattern recognition (CVPR 2011), Colorado Springs, USA
Metadaten
Titel
A hybrid optimization strategy for registering images with large local deformations and intensity variations
verfasst von
Zhang Li
Lucas J. van Vliet
Jaap Stoker
Frans M. Vos
Publikationsdatum
01.03.2018
Verlag
Springer International Publishing
Erschienen in
International Journal of Computer Assisted Radiology and Surgery / Ausgabe 3/2018
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-017-1697-z

Weitere Artikel der Ausgabe 3/2018

International Journal of Computer Assisted Radiology and Surgery 3/2018 Zur Ausgabe

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.