Data-driven shape parameterization for segmentation of the right ventricle from 3D+t echocardiography

Highlights

• The right ventricle is segmented jointly across multiple echocardiography sequences.

• Segmentation is constrained by a linear basis shape model.

• The linear basis shape model is optimized during segmentation.

• The framework is applied to multiple-view and multiple-subject datasets.

Abstract

Model-based segmentation facilitates the accurate measurement of geometric properties of anatomy from ultrasound images. Regularization of the model surface is typically necessary due to the presence of noisy and incomplete boundaries. When simple regularizers are insufficient, linear basis shape models have been shown to be effective. However, for problems such as right ventricle (RV) segmentation from 3D+t echocardiography, where dense consistent landmarks and complete boundaries are absent, acquiring accurate training surfaces in dense correspondence is difficult.

As a solution, this paper presents a framework which performs joint segmentation of multiple 3D+t sequences while simultaneously optimizing an underlying linear basis shape model. In particular, the RV is represented as an explicit continuous surface, and segmentation of all frames is formulated as a single continuous energy minimization problem. Shape information is automatically shared between frames, missing boundaries are implicitly handled, and only coarse surface initializations are necessary.

The framework is demonstrated to successfully segment both multiple-view and multiple-subject collections of 3D+t echocardiography sequences, and the results confirm that the linear basis shape model is an effective model constraint. Furthermore, the framework is shown to achieve smaller segmentation errors than a state-of-art commercial semi-automatic RV segmentation package.

Introduction

Segmentation of the left ventricle (LV) and right ventricle (RV) from 3D+t echocardiography is an important task for quantifying cardiac function. In practice, this can be challenging because of missing anatomical boundaries and ambiguous edge features. 3D analysis of the LV has received most attention in literature, and model-based methods have been shown to be successful at LV segmentation from 3D echocardiography (Mitchell et al., 2002, Noble and Boukerroui, 2006). The RV literature is less developed, in part as the shape is harder to model, but also because RV ultrasound (US) scans are generally more incomplete due to the relative position of the RV with respect to the transthoracic probe.

Commonly, a surface representing the endocardium is deformed frame-to-frame so that it adheres to the blood-tissue boundary (e.g. Orderud et al., 2007). Specifically, the surface is represented either explicitly or implicitly and an energy minimization problem is formulated which captures the notion of “fitting” the surface to the blood-tissue boundary subject to priors or regularization. An explicit representation directly models the geometry of the segmentation surface (Blake and Isard, 2000). Examples include point distribution models (Cootes et al., 1995), truncated ellipsoids (Orderud, 2006), spline and subdivision surfaces (Piegl and Tiller, 1995, Stam, 1998), and B-spline Explicit Active Surfaces (BEAS) (Barbosa et al., 2012). Implicit representations model the segmentation surface as the level-set of a higher-dimensional function over the image domain (Caselles et al., 1997, Osher and Sethian, 1988).

Implicit representations of the LV and RV have the advantage that they allow for more complex appearance models because they naturally define interior and exterior segmentation regions (Lankton and Tannenbaum, 2008, Huang et al., 2014). The primary drawback of implicit representations, and level sets in particular, is that direct optimization over the entire function is slow. Level sets were made practical by the development of fast marching and narrowband methods, which only update the level set function near the implicit interface boundary or zero level set (Lankton, 2009, Sethian, 1999, Whitaker, 1998). Common to these approaches is that a first-order gradient-based evolution equation is derived from the energy and used to update the level set function. Since the surface fit is specified directly against image intensities—a result of the more complex appearance model—an accurate initialization is necessary so that the first-order optimization does not converge to an unwanted local minimum.

Like level set methods, the surface fit for explicit representations can also be formulated directly against the image (e.g. Kass et al., 1988), but specification of interior and exterior regions is, with the exception of BEAS (Barbosa et al., 2013), typically more difficult. However, if a discrete set of boundary candidates is available—detected independently or based on the current surface state—then the surface fit can instead be defined as the distance between these points and their corresponding points on the explicit surface. In this case, explicit representations are more amenable to non-linear continuous optimization algorithms more powerful than gradient descent.

To handle missing boundaries, regularization is used to constrain the segmentation surface to be smooth and physically plausible. Simple spatio-temporal regularizers are physically motivated and straightforward to implement for explicit or implicit surface representations. However, for large boundary gaps, such as those encountered when acquiring 3D+t US sequences of the RV, the simple interpolating action of these regularizers is insufficient. Instead, shape models can be used to constrain the model surface.

In medical image analysis, point distribution models and triangle meshes have been popular model surface representations. For these definitions, linear basis shape models have been constructed using Principal Component Analysis (PCA) on a set of training surfaces which have been aligned semi-automatically (Bosch et al., 2002, Cootes et al., 1995). The advantage of these Active Shape Models (ASMs) is that the dimensionality of the model surface is reduced. The disadvantage is that the resulting parameterization can be too restrictive and prevent modeling of local deformation unseen in the training examples.

To remedy this, hierarchical ASMs have been proposed which recover scale- and location-specific linear basis shape models. In Davatzikos et al. (2003), this is achieved in 2D by first computing wavelet coefficients of the x- and y-components of the training model contours. The coefficients are then partitioned into bands based on scale and spatial location, and PCA is applied to each. In 3D, spherical wavelets with adaptively selected bands (Nain et al., 2005, Nain et al., 2006), Catmull-Clark subdivision wavelets with fixed bands (Li et al., 2007), and diffusion wavelets (Essafi et al., 2009) with orthomax PCA (Kaiser, 1958, Stegmann et al., 2006), have been proposed. Each of these algorithms produces a linear basis shape model which enables “legal” deformations of the dense model points to be specified with a small number of parameters, where each parameter (by design) controls local shape deformation only.

A key challenge when constructing any of the aforementioned shape models is acquiring accurate training surfaces which are in dense correspondence. Automatically detecting correspondences based on shape features (e.g. positions of high curvature) has been proposed (Brett and Taylor, 2000, Wang et al., 2000), but this is difficult for the RV because of the absence of dense consistent landmarks across the surface, complete boundaries, and increased shape variability (Caudron et al., 2012, Petitjean and Dacher, 2011). Furthermore, while training surfaces are rigidly aligned using Procrustes analysis (Bosch et al., 2002), local incorrect correspondences can remain which introduce artificial shape variation into the model. Therefore, it is necessary to model the parameterization differences between the dense points of the training surfaces. In Davies et al. (2002), this is achieved by mapping each surface to the unit sphere and optimizing predefined parameter transformations for each training surface to construct the ASM of minimum description length (Rissanen, 1983).

In this paper, a framework is described which performs joint 3D segmentation of the RV from multiple 3D+t echocardiography sequences, while simultaneously optimizing all correspondences and an underlying linear basis shape model (Fig. 1). This framework is a modification of Cashman and Fitzgibbon (2013)—where 3D linear basis shape models of animals are learned from 2D exterior silhouettes—to 3D segmentation of multiple-subject and multiple-view collections of 3D+t echocardiography sequences. The key differences are:

• In Cashman and Fitzgibbon (2013), the exterior silhouettes of animals are recovered by semi-automatic segmentation. Therefore, all boundary candidates are valid and ordered, and dynamic programming is used to initialize the boundary candidate correspondences (preimages). Here, boundary candidates are derived using a simple edge detector—only a subset are valid. Furthermore, the boundary candidate positions are noisy and missing boundaries are common. Therefore, a “model-to-data” approach is adopted: the boundary candidate correspondences are initialized by sampling the model surface uniformly, and boundary candidates are subsequently selected based on the current model surface geometry. Robust fitting terms are also used.

• In Cashman and Fitzgibbon (2013), independent rigid transformations and shape parameters are modeled for each frame. Here, scales are introduced for each subject and rigid transformations are introduced for each sequence. Shape similarity is also enforced between frames which are of the same subject at the same point in the cardiac cycle.

Conceptually, the proposed framework is also similar to Zhou et al. (2013), where in 2D, explicit model contours are simultaneously fitted to multiple images in a sequence and constrained to be of similar shape. In Zhou et al. (2013), shape similarity is achieved by minimizing the nuclear norm of the matrix composed of the x- and y-components of all model contours. Here, shape similar is enforced explicitly using a linear basis shape model.

As will be shown, the described framework is suitable for the proposed application for three reasons. First, a Loop subdivision surface is used for the model surface, which by construction, has a small number of parameters but is flexible and can realize local shape deformations. Second, joint optimization of all continuous parameters—including the linear basis shape model control vertices, rigid transformations, and boundary candidate correspondences—mitigates the requirement for any registration or fusion of the input images (e.g. Rajpoot et al., 2011), or accurate training surfaces that are in dense correspondence. Third, it naturally handles missing boundaries.

The structure of the paper is as follows. In Section 2 the complete model energy and optimization algorithm for our framework is presented. In Section 3 and Section 4 we then demonstrate the application to two problems which are hard to solve using prior approaches. First, we show how multiple 3D+t sequences acquired from different viewpoints for a single subject can be segmented jointly while optimizing a subject-specific shape model. Second, we show how multiple 3D+t sequences acquired from multiple subjects—with potentially different viewpoints—can be jointly segmented. Conclusions are given in Section 5.