A review of heart chamber segmentation for structural and functional analysis using cardiac magnetic resonance imaging

Thresholding is often integrated with region-growing. Lee et al. [84] and Codella et al. [85] use region-growing to find the full-blood LV region. They automatically identify a seed point by taking the pixel with the lowest energy in a window during slice propagation. Then in order to prevent the segmented LV region from diffusing to epicardial fat, fluids, and RV, they use an iterative thresholding mechanism that discovers a lower bound of myocardial intensity. Huang et al. [86] employ thresholding to distinguish the blood pool from the myocardium, followed by radial region-growing and extraction of convex hulling to identify the endocardial and epicardial boundaries. Lu et al. [87] apply thresholding to convert a ROI to a binary image for LV localisation and endocardial contour detection (Fig. 8), followed by region-growing to segment the LV epicardium. Ammar et al. [88] take the binary image produced by thresholding as the initial mask for a level-set segmentation method to extract the endocardium. Queiros et al. [89] perform class decomposition following thresholding step to search for the LV centroid. The method sets two thresholds for myocardium and cavity histogram in an EM algorithm to extract the endocardial contour. Kurkure et al. [90] localise LV in the thresholded image by finding a binary component that is closest to the intersection cross-hair generated by LAX vertical and four-chamber view projection in ED phase on a SAX slice. They have also proposed a novel fuzzy connectedness region-growing method taking the spatial adjacency, intensity homogeneity, and multiclass features into consideration. The myocardial boundaries are extracted by dynamic programming, which is an optimal path finding solution of overcoming obstacles such as papillary muscles or trabeculae carneae extrusion, and low liver-to-myocardium tissue contrast. Cousty et al. [91] extract epicardium by developing a spatial–temporal gradient computation for watershed cuts. It is noteworthy that approaches in [84‐87, 89] start their segmentation from mid-ventricular SAX slices, which might involve user interaction, and then propagate their initial results to other slices as prior knowledge. Also, the test image is usually mapped to the polar coordinate since LV roughly has a circular shape [84‐87, 89, 90]. Furthermore, by making use of thresholding on the LV blood pool, the papillary muscles and the trabeculations can be easily outlined [84‐87] due to their intensity diversity with surroundings. Thus, cardiac functional analysis such as LVEDV, LVESV, or LVM estimation be can varied by including or excluding papillary muscles and trabeculations or not, depending on the index definition and requirement of clinicians.

Classification-based methods in cardiac segmentation have also been thoroughly studied. Jolly [92] and Hu et al. [93] propose to classify regions by a 3-GMM with EM, based on the intensity histograms. Jolly [92] separates the muscle, air, blood, and fat, as presented in Fig. 9. Hu et al. [93] separate the muscle, the blood and the background. Pednekar et al. [94] fit a 5-GMM to the intensity histogram of the blood pool, the lung filled with air, myocardium, the region between the blood and myocardium, and the region between the air and myocardium. The EM algorithm is initialised through K-means clustering. Queiros et al. [89] use a 2-GMM. Some classifiers label the features from different tissues without making assumptions on intensity histogram distribution. To label the regions of the lung, the myocardium, and the blood pool, Stalidis et al. [95] make use of a neural network classifier, which is trained via a small number of representative tissue points. The input features of their classifier are the pixel position, pixel intensity, and slice location. Folkesson et al. [96] have presented that a trained KNN classifier is competent for the classification of the LV cavity, myocardium, and background, based on a feature selection scheme. The latter finds the most discriminative features for the pixel classifier or model fitting, aiming at increasing computational efficiency without degrading its accuracy. Bai et al. [97] have shown that the support vector machine (SVM) outperforms KNN in label fusion in a multi-atlas-based cardiac segmentation framework.

Active contour, or deformable model, is one of the most widely applied techniques in heart segmentation. The breakthrough usually comes with the design of the energy function, such as using anatomical assumptions as the constraints on the level-set methods [98, 99]. Paragios [98] propagates coupled endocardial and epicardial contours on SAX slices, where the edge, region, and anatomical constraints are pre-defined. The edge constraint is used to push the curves to the myocardial walls. The region intensity criterion makes the model less sensitive to initial conditions. The GVF snake, a parametric active contour that overcomes the difficulty in evolving the curve to the boundary concavities, is introduced into LV segmentation in these works [100‐102]. Wu et al. [103] claim that the gradient vector convolution (GVC) snake also conquers local minima such as artifacts and papillary muscles. Kaus et al. [104] integrate strong prior knowledge, in the form of PDM, into deformable contours by extending their internal energy, leading to an increase in the robustness of the model. They have considered the inter-spatial relationship of the inner and outer boundaries as well, which compensates for the error produced by incorrect feature detection. Lynch et al. [105] employ the probability density function as the prior information, which is created by a set of manually segmented boundaries on binary images. Furthermore, the evolutions of endocardial and epicardial curves are coupled by an extra level-set constraint. The deformable models proposed in [106, 107] and the level-set approaches presented in [108, 109] incorporate the spatial–temporal LV activation as prior knowledge and track the epicardium/endocardium boundaries on SAX slices in a complete cycle. A typical tracking result is shown in Fig. 10. The constraints in [106, 107] are parameterised by Fourier descriptors. Moreover, an approach for the recognition of intra-ventricular dyssynchrony (IVD) is proposed in [106], where the non-uniform contraction of the ventricular walls brought by the activation delays can be discovered. Jolly [92] makes use of a deformable model to improve further the outcome of EM-based region segmentation. Chen et al. [110] propose to apply deformable models to LVS analysis on SAX slices in DENSE MRI. Their model is driven by minimising an energy function that consists of model intensity, edge attraction, shape prior, contours interaction, and smoothness. The shape prior can eliminate the concavities with negative curvatures in order to remove the papillary muscles from the ventricular walls. Huang et al. [111] have invented a novel deformable model called Metamorphs, whose energy functions are predefined on the distance maps of the object shape and its border. Metamorphs is not particularly designed for MRI cardiac segmentation while it outperforms the GVF snake as a result of better robustness to inferior initial conditions. Based on the motion trajectories in DENSE, representing the movement of myocardial wall between two consecutive phases, the initial manually drawn endocardial and epicardial contours can be propagated slice by slice to other frames [112, 113]. The method proposed in [112] is applicable to both SAX and LAX images. Besides DENSE, Chen et al. [114] have also used tagged MRI to derive LVS. In their work, Gabor filters search the tag intersections. Through matching these intersections, the method is able to track the myocardial motion. The deformable model refines the tracking and displays a dense displacement map. Kermani et al. [115] draw a dense displacement map by fitting a 3D active surface model to an initial sparse displacement map, which is built by establishing point correspondence in cine images. Motion tracking makes LVS easier to be analysed, because the myocardial displacement and temporal scale are known at the meantime. The authors produce visualisations of LV strains in 3D (see Fig. 11). Khalifa et al. [116] measure the wall thickness and thickening with a stochastic speed function based level-set technique extracting inner and outer myocardial walls first. Subsequently, the points on the inner contour and the outer contour are paired. The Euclidean distance between each pair is the wall thickness. Wei et al. [117] use the myocardium contours from cine MRI as prior knowledge to guide the meshing of endocardium and epicardium, which are generated by contour registration to move towards the inner and outer edges in SAX and LAX slices in LGE. Grande et al. [118] model the image likelihood by sampling the intensity and gradient of pixels inside the myocardium or at the boundary of myocardium in different regions. After that, they create a Markov Random Field (MRF) to incorporate the prior and the likelihood models. The prior keeps the curve smooth and excludes the papillary muscles. The deformable model estimates the walls based on the MRF along the SAX radial direction.

Different from image-driven techniques, model-based approaches exploit strong prior knowledge such as by encoding the specific shape variability of the LV, instead of making simple assumptions on the boundaries. By taking advantage of the statistical shape information, segmentation becomes more robust to image noise by restricting the outcome to valid instances statistically. Mitchell et al. [119] introduced early application of 3D-AAM to LV segmentation in 2002. The method showed worthy results in quantifying LVV _epi, LVV _endo , and LVM on SAX volumes. Assen et al. [120] proposed a 3D-ASM segmentation method (SPASM) that can operate on sparse MR images scanned in arbitrary orientations. In most cases, the automated LV segmentation approaches require a stack of parallel SAX images. While SPASM can perform on a datasets of two orthogonal radial LAX slices, four radial LAX slices with a 45-degree angle between two neighbours, 11 equally spaced SAX slices, four SAX slices (one apical, one mid-cavity, two basal), or a combination of two LAX and two SAX slices. The processing pipeline of this method is implemented and shown in Fig. 12. Lekadir et al. [121] improve the 3D-ASM by incorporating an additional shape prior, which is invariant to transforms including translation, rotation, and scaling. This prior is used to detect and correct outliers, thus leading to more robust results. Andreopoulos and Tsotsos [122] use a hierarchical 2D-ASM that incorporates temporal constraints to enhance the fitting outcome of 3D-AAM. Assen et al. [123] replace the absolute intensity 3D-ASM with relative grey scales when ROI is being identified (fuzzy inference). Suinesiaputra et al. [124] propose to employ independent component analysis (ICA) [125] instead of PCA in SSM to extract myocardial contraction from SAX slices. Furthermore, due to the better performance on local description, ICA is used to design a classifier able to detect regional wall motion abnormalities. Lekadir et al. [126] have also assessed myocardial motion through decomposing the global ventricular shape. They calculate the relationships between a series of spatiotemporal inter-landmarks. By tracking the epicardium and endocardium a dysfunction map is drawn to show abnormal contractions. O’Brien et al. [127] model shape, spatial, and temporal variation separately. They use a global contour optimisation instead of conventional ASM fitting. Roohi and Zoroofi [128] propose a kernel PCA (KPCA), in which the modes applied to represent a global ventricle shape are combined non-linearly. The distribution of landmarks is divided into intra- and inter-subspaces. A more recent work proposes to collect all the shapes learned from training data to build a dictionary [129]. The features of segmented frames from the test image are also added to the dictionary to create a patient-specific model dynamically. Each feature is classified into object (myocardial boundaries) or background (blood pool or muscles). A sparse shape model is then used to find the points on the ventricle walls based on their distances to the classified features. Unreliable points are abandoned and the complete LV shape is reconstructed according to the dictionary. Temporal constraints are not considered in this approach as current segmentation relies on the outcomes of the previous frame. Zhu et al. [130] developed a subject specific dynamical model that simultaneously handles inter- and intra-subject variabilities in a recursive Bayesian framework and a combined multi-linear PCA-ICA model. Starting from a manually segmented first frame, subsequent frames are segmented according to the current intensity pattern and a shape prior, predicted from the past frames.

Xenia et al. [131] proposed a framework for LV segmentation that is based on heuristic rules such as the brightness of the blood pool, sphericity of LV, and inter-slice smoothness of segmentation. A graph-cut algorithm was presented to infer the labels of the myocardium for robust optimization; however, the three-dimensional morphology of the heart was not fully exploited. A two-dimensional segmentation framework was proposed in [132] that effectively maps the edge patterns of each slice (centred at LV centroid) from polar into a Cartesian grid. Then a dynamic programming method is used to walk through the grid having the strongest edge values. Ayed et al. [133] propose another 2D segmentation framework that firstly learns intensity and the shape distributions for the blood pool from a manually segmented frame. Then, using a max-flow algorithm, it minimises a lower bound on the Battacharyya distance between the trained and subsequent distributions obtained from other frames. Similarly, Nambakhsh et al. [134] consider learning both intensity and shape constraints from a segmented first frame, and then minimise the distance between the test and trained distributions. However, rather than a graph-cut based method, a series of convex cost functions are solved for exact minimization. Yet, another training based LV segmentation method is proposed by Eslami et al. [135]; the prior information is implemented through kernel based approach, where test image is compared to the training data for the closest neighbour using a random walk paradigm. The method is shown to segment pathological hearts, whose data is usually overlooked using conventional PCA based statistical shape models.

Afshin et al. [14] propose a direct estimation on heart abnormality detection. In their work, each subject comprises three SAX slices, whereas the apical slice is divided into four segments and the mid-ventricular and basal slices are divided into six segments. This user-provided segmentation is only needed in a single frame, which acts as a reference. All other subsequent frames in a complete cycle are automatically divided into 16 segments each, according to their distribution similarity. The local statistical descriptors, whose dimensions are reduced through linear discriminant analysis (LDA), are then constructed based on these segments. Because each segment can reflect the portion of blood filling, their statistics correlate well with the regional LV function. As a result, with a linear SVM trained by the ground-truth given by the radiologists and features from these local LDAs, regional abnormalities can be detected. They have classified 58 subjects (21 normal and 37 abnormal) with an accuracy of 86.09 %. Direct estimation for cardiac functional analysis can be a promising research direction and has achieved competitive performances in comparison with the state-of-the-art segmentation based assessment, with a significantly lower computational complexity.

Maier et al. [136] segment the RV in MRI combining watershed filtering with graph-cut based region merging. They provide two initialisation options: the user either outlines the RV wall in 4–5 slices of the ED phase, or marks two points on the basal slice to register an atlas. Wang et al. [137] use a morphology-based algorithm, which considers the layout, shape, size, and relative locations to locate roughly the LV and RV first. The temporal discrepancy between two consecutive frames is then used to discover RV as the most active part. Ringenberg et al. [138] segment the endocardium by intersecting two ROI constrained binary images as follows. Firstly, an ROI window is selected and converted into binary with an optimal thresholding; next, the same ROI is convolved with a difference of Gaussian filter and thresholded at zero. The RV mask is roughly estimated as the intersection of these images. The window constraints label information from the previous slices and work as prior knowledge. The segmentation begins from the most basal slice in ED and ends at the apex. For ES, the prior is the union of labels from the previous slices at ES and the label of the current slice at ED. Punithakumar et al. [139] base their segmentation on registration and propagation. A 2D mesh delineating the endocardium or epicardium moves across all phases by establishing point-to-point correspondences. The manual segmentation of a single frame is required for initialisation. Mahapatra [140] uses a trained random forest classifier to give voxels two probability values, corresponding to the object and background. Based on this probabilistic map, a final segmentation is achieved by graph-cut. The image features they extracted for the discriminative description consist of intensity statistics, spatial context, textural, and curvature entropy. Nambakhsh et al. [141] propose a method based on the global shape and intensity similarity estimation. Based on the global distribution matching, the shape prior is intrinsically invariant with respect to translation and rotations. Centroid of LV and a small area of RV cavity have to be specified by the user. Compared to the learning-based approach in [140], this algorithm has the advantage of requiring only a single subject for training.

Because of the geometrical complexity of the RV, robustness becomes a major concern in its automated segmentation. Amongst existing techniques, addressing this challenge, multi-atlas-based methods with label fusion [142‐144] have received significant attention. In these frameworks, finding reliable correspondences between the patient and atlas spaces becomes critically important. For instance, Ou et al. [142] present a deformable registration algorithm that uses saliency of the matching for improved robustness versus variation of shape, intensity, and field of view. A “zoom-in” mechanism that uses the first round of RV segmentation to iteratively refine the registration and segmentation outcomes is employed. Alternatively, Grosgeorge et al. [145] employ a PCA-based SSM as the prior model, to guide the segmentation through a graph-cut method. The model is registered to the test case through a rigid transform, with two anatomical landmarks manually placed by the user on the ventricular septum. Oghli et al. [146] apply PCA on signed distance functions extracted from parametrised training contours as the shape prior for a deformable model. In addition, they use region and boundary based energies for improved fitting.

Sermesant et al. [147] have proposed a deformable biomechanical model based on tetrahedral geometric representation. The user specifies a proper mesh size to keep the data amount reasonable and retain a good mesh quality. The mesh is mapped to the test image using a non-rigid registration under influence of the internal and external forces, modelling elastic and imaging constraints, respectively. This method can be used for motion tracking, thus measuring local deformation and LVS is made feasible. Rougon et al. [148] employ a non-rigid registration method to assess myocardial contraction in both SAX and LAX slices. They use tagged MRI to infer the intra-myocardial motion and the cine MRI to extract the myocardial anatomy dynamically. Hautvast et al. [149] suggest a contour propagation scheme from ED to ES images. This method can be applied to SAX, two-chamber or four-chamber LAX slices, but requires manual segmentation on ED for initialisation. Cocosco et al. [150] convert a test image into a binary representation by optimally thresholding the intensity histogram of the ROI. The fat around ventricles is then removed by a thinning operation. All connected components are labelled and region-growing is performed on the SAX slices. Afterwards, they calculate the volume of each component at all frames and the maximum and minimum values are taken out. Two components having the most significant differences between their maximum and minimum volumes are selected as the LV and RV. The final delineation is obtained by merging the voxels classified in the first step along the LAX direction. Grosgeorge et al. [151] use the seminal model of active contours without edges [152] for bi-ventricular segmentation of a large dataset containing 1920 MR images, and obtained satisfactory results comparable to the state-of-the-art. Mahapatra et al. [153] segment bi-ventricles using a graph-cut framework, guided by a shape prior based on the distribution of orientation angles from each pixel to the edge points, as extracted from a single manually annotated image. Wang et al. [154] adaptively use reinforcement learning to assimilate the knowledge provided by the user, such as edge point position correction, in LV/RV segmentation.

Valdes et al. [155] propose to use a probabilistic atlas to guide the EM classification. The atlas provides a spatially and temporally varying probabilistic map for the LV, RV, myocardium, and background including the liver, stomach, lungs, and skin. The results of estimated volume of LV, RV, and myocardium demonstrate that the combination of the EM algorithm and a cardiac atlas improves segmentation accuracy. Bai et al. [156] fuse patched-based labels for a Bayesian formulation within a multi-atlas registration based segmentation framework. Furthermore, they refine the registration using intermediate label information. Figure 13 illustrates the procedure of multi-atlas label fusion and image registration refinement.

Ordas et al. [157] introduce a feature vector, which is invariant under Euclidean transforms in an ASM-based framework. Mitchell et al. [158] propose a hybrid AAM matching mechanism accomplished through three steps. Firstly, AAM alone is fitted to the image. Next, the hybrid AAM/ASM helps avoiding local minima by deploying the shape information. Finally, AAM is reapplied. Zhang et al. [159] also use a combined AAM-ASM model, which is based on novel spatial and temporal features, incorporating the motion. The combination of ASM and AAM yields better segmentation results and overcomes the drawbacks of using ASM or AAM individually. ASM requires good initialisation and can be trapped by incorrect nearby features, though it retains a fine global shape. While AAM performs well in tracking objects, but is easy to be trapped by local minima. Alba et al. [160] segment the LV and RV of highly abnormal hearts by using estimating a mapping between the abnormal image and the space of generic shape model built from a normal population, which can be thus used to segment any types of cardiac abnormality. Increased accuracy is demonstrated for both pulmonary hypertension and hypertrophic hearts.

Wang et al. [15] estimate LV and RV cavity volumes on SAX slices without segmentation. This direct method relies on a likelihood function defined as the area correlation of the LV/RV cavities, and prior function specified by the product of the blobness, edgeness, and homogeneity. The framework consists of a training stage where the prior and likelihood probability functions are inferred. Given a test image, the posterior probability of observing a point in LV/RV is derived using the Bayes rule. The mean cavity area of LV/RV is the expectation of a function of these posterior probabilities and the volumes are estimated using Simpson’s method. However, as indicated by Zhen et al. [16], the limitations of [15] include a simple linear relationship assumption between LV and RV as well as an expensive computational requirement. Zhen et al. [16] make use of a three-layer convolutional deep network, which is learned from unlabelled images, to represent the input test case effectively in feature space. At the meantime, regression forests trained from manually labelled data, as discriminative learning, is responsible for estimating LV and RV volumes (see the flowchart in Fig. 14). They claim that their method significantly outperforms level-set and graph-cut methods. Another advantage of direct estimation is that the inconsistency of boundary and region intensity homogeneity has been excluded from the immediate influences on volumetric quantification.

John and Rahn [161] base their approach on thresholding and region merging. A thresholding roughly separates the blood pool voxels from the image. Afterwards, the Voronoi tessellation of the binarised mask is computed. The tessellated components are finally combined to segment the LA and other structures. Zhu et al. [162] also propose a region-growing framework, where the initial seed is found according to the anatomical knowledge from the middle SAX slice. A shape prior learned from training data is used to attract the growth to the statistically plausible region. This incorporation makes segmentation more robust to spatial variation and image quality.

Karim et al. [163] construct a probabilistic atlas for atrium using 20 manually segmented training images. Given a test image, they apply an optimal thresholding to extract the blood pool and the vessel structures and obtain the Voronoi tessellation for the binarised image [161]. The narrow junctions, which are the connections between the atrium and its neighbouring structures, are then identified (as displayed in Fig. 15). Next, using the probabilistic atlas as prior, they present an MRF based cost function for segmenting cells that belong to the atrium. Additionally, a graph-cut method is applied for global optimisation. In order to deal with LA anatomical variations, Kutra et al. [164] have proposed a multi-component-based LA segmentation. The three most typical variations include the normal, common left trunk (CLT), and right middle pulmonary veins pattern (RMPV). Then a trained SVM is used to automatically select the model that fits the test image best. Eventually the model, which is a mesh of triangles, deforms towards the edge by the external and internal constraints.

Makowski et al. [165] have proposed an active contour-based procedure to segment the heart and vessels in 2D transversal slices. This shape-independent method uses a balloon force to place the segmenting contour roughly and then uses a snake model to refine the segmentation. However, due to the complex geometry of the whole heart, later published works tend to use shape priors to increase robustness.

Lotjonen et al. [166] reconstruct the 3D geometry of atria and ventricles from both SAX and LAX views. The pulmonary artery, pulmonary veins, and vena cava are excluded for volumetric measurement. The shape variability is modelled using PDM, a novel landmark distribution model, and a probabilistic atlas. Then the mean shape model is non-rigidly registered to the test image, and the model deforms towards the boundaries based on the shape priors. Since the performance of SSM-based methods is related to the richness of training samples, Koikkalainen et al. [167] have shown the feasibility to improve the segmentation of four-chamber and major vessels by artificially enlarging the training sets. Wierzbicki et al. [168] build PCA-based models for LV, RV plus RA, LA plus aorta, and the entire heart separately, using high quality training data. Each model is then registered to the mid-diastole frame of a low quality sequence, and propagated to all other frames by animating motion dynamics. Peters et al. [169] developed a deformable model by proposing a novel and robust boundary identifying technique called simulated search, whose mesh matching functions are previously trained. For the prior information-based approaches, model registration is always a critical step. Zhuang et al. [170] and Zuluaga et al. [171] find the breakthrough herein. They present a locally affine registration mechanism, which is further refined by a free-form deformation registration. This atlas-propagation-based method has turned out to be robust against various pathologies. Examples of the segmented whole heart in different views and a visualisation of segmentation errors between the result and the ground-truth are shown in Fig. 16.

Zhen et al. [17] explored the feasibility in applying direct estimation to four-chamber volume measurement as well, by representing the MR images in a compact and discriminative way. The image features are generated using a supervised descriptor learning algorithm. Then the volume estimation becomes a multi-output regression problem solved with random forest.