Nonlocal total variation based on symmetric Kullback-Leibler divergence for the ultrasound image despeckling

Ultrasound imaging is one of the four modern medical imaging techniques that deliver cross-sectional images of a patient’s anatomy and physiology in real time [1]. The use of ultrasound in the diagnosis and assessment of imaging organs and soft tissue structures, as well as human blood, is well established [2]. This technique is progressively achieving an important role in the assessment and characterization of cardiac imaging because it is noninvasive and has a strong ability to identify biological soft tissues [3]. However, ultrasound imaging generates images with low contrast resolution, because of speckle noise.

The speckle pattern, which contains typical light and dark spots in an image, is generated by the interference effect of the ultrasonic echoes and scattering of randomly distributed structure scatters. In fact, speckle is technically not a noise in the typical engineering sense because its texture often carries useful information on the image. Therefore, distinguishing between speckle generated from tissue and that from the received RF signal is essential [3]. The former refers to the image texture and the latter, the speckle noise. Speckle noise is a form of multiplicative noise subjected to Gamma distribution [4‐6]. It is the primary factor that limits the contrast resolution in diagnostic ultrasound imaging, thereby limiting the detectability of small low-contrast lesions and rendering the ultrasound images generally difficult for non-specialist to interpret [3]. Therefore, reducing speckle noise is important for the interpretation and computer-aided analysis of ultrasound images. The aim of despeckling is to eliminate the speckle noise and maintain the image boundaries and image texture.

Decreasing the frequency of the transducer can increase the signal-noise-ratio (SNR) in ultrasound images. However, doing so decreases the resolution of the image, and consequently hinders the capturing of internal details of human organs. Meanwhile, high-resolution ultrasound images usually indicate low SNR, which can be increased by despeckling post processing techniques. One of the classical despeckling methods is the Wiener filter, which is a linear filter based on local statistics [7]. It uses the weighted average calculation of sub-region statistics to estimate the statistical measures over different pixel windows [3]. However, the linear filters have an inherent drawback of over-smoothing anatomical details and transitional boundaries [8]. Thus, in the last two decades, researchers have focused on nonlinear filtering methods to remove speckle noise in ultrasound images.

Similar to ultrasound images, the synthetic aperture radar (SAR) images are usually degraded by multiplicative speckle noise. Some local speckle statistics-based adaptive filters proposed for SAR images are also used in ultrasound images [9‐15]. These adaptive filters use local speckle statistics to improve smoothness in homogeneous regions where the speckle is fully developed and reduce smoothness appreciably in the other regions showing the useful details of an anatomical structure [3]. Adaptive filters include the Lee filter [9], Kuan filter [10], Frost filter [11], adaptive weighted median filter [12], speckle reducing bilateral filter [13], maximum likelihood filer [14] and MRGMAP filter [15]. These adaptive filters use different weighting coefficients calculated from the subregion statistics over different pixel windows to preserve important image structures while removing speckle noise. However, although these adaptive filters can retain the boundaries, they cannot eliminate speckle noise effectively and are sensitive to the size of the pixel window. When the window size is large, the image edges are blurred.

Anisotropic diffusion, proposed by [16], is an efficient nonlinear technique that can enhance contrast and reduce noise simultaneously [17‐19]. It smooths homogeneous image regions and preserves image edges without requiring any information from the image power spectrum [3]. For example, the speckle-reducing anisotropic diffusion filtering has been proposed to enhance image contrast [3]. Wavelet filtering is another method that exploits the decomposition of an image into the wavelet basis and zeros out the wavelet coefficients to despeckle the image. Speckle-reduction filtering in the wavelet domain is based on the concept of the Daubechies Symlet wavelet and soft-thresholding denoising [3]. It was proposed first by Donoho [20] and investigated further by Zhong and Cherkassky [21] and Gupta et al. [22]. However, although diffusion and wavelet filters can suppress speckle noise considerably, they produce excessive smoothing details, particularly in the image boundaries and texture.

Nonlocal denoising methods that use the similarities among small patches have been reckoned to be an effective way to preserve details while decreasing noise [6, 23, 24]. A simple and classic method is the well-known nonlocal means (NLM) filter [23], which measures the similarities among the patches centered at a given pixel in the image and prevents the smoothing of boundaries by assigning only high weights to pixels with similar local properties. However, it cannot be applied to speckle filtering directly since the speckle model of the ultrasound images is not subjected to the Gaussian distribution. The NLM filter has been adapted to Gamma-distributed speckle pattern by introducing a Bayesian estimator to the weighting function and Rayleigh-distributed speckle pattern through the introduction of maximum-likelihood estimation [25, 26]. These adapted methods demonstrate better results than the original NLM filter. The NLM filter generates better structural preservation than the diffusion filters. However, it also generates excessive smoothing textures. Based on the NLM filter, the nonlocal total variation (NLTV) minimization scheme analyzes the patterns around the pixels and performs pattern matching in the global image [27, 28]. The nonlocal total variation norm processes textures and repetitive structures effectively. However, although the NLTV filter performs well in Gaussian noise reduction and sharp boundaries preservation, it cannot be applied to log-compressed ultrasound images directly, because the speckle is not subjected to the Gaussian distribution. Thus, a Bayesian NLTV speckle filter (BNLTV) has been developed for ultrasound images and thus is capable of improving speckle suppression and edge enhancement, outperforming the adaptive filters, diffusion filters, wavelet filters, nonlocal denoising methods, and original NLTV filter [29]. However, the BNLTV filter over-smooth image texture and introduces some artificial traces, while it eliminates the speckle noise. It cannot determine whether or not the speckle received from RF signal.

Another nonlocal denoising method is the blocking matching 3D (BM3D) filter proposed by Dabov et al. [30]. This method combines nonlocal and transform-domain approaches and presents an effective denoising framework by grouping similar patches into a 3D array. It subsequently filters the 3D array by using sparse representation in the transform domain. Finally, it aggregates multiple estimates at each location. Based on the BM3D filter, the SAR-BM3D filter [31] was proposed for despeckling SAR images. The SAR-BM3D filter is generally used to despeckle breast ultrasound images and exhibited the impressive despeckling ability [32].

In this paper, an adapted NLTV speckle filter is proposed to address the problem on speckle noise. This filter can eliminate the speckle noise while maintaining the edges and image texture. For the improvement of the despeckling performance, spatiogram similarity measurement based on symmetric KL divergence is introduced to the similarity calculation between the image patches. The proposed method is validated by considering both real ultrasound images and synthetic noisy images. Before the experiment, a Kolmogorov-Smirnov (KS) test has been conducted on the experimental images to ensure that the images are subjected to Gamma distribution [5]. For the evaluation of the despeckling performance without the ground truth image, the natural image quality evaluator (NIQE) [33] is introduced. NIQE can predict the quality of distorted images (noise, ringing, blur, or blocking), even without any prior knowledge of the reference images or their distortions. The remainder of this paper is organized as follows: Section 2 provides the proposed NLTV filtering procedure. Section 3 reports the experimental results. Finally, Section 4 presents the conclusions.

The synthetic noisy images and clinical ultrasound images from [29] are studied to assess and compare the performances of the despeckling methods quantitatively. In [29], a 2D synthetic image “Modified Shepp-Logan Phantom” available in MATLAB is considered and corrupted with different noise levels. The speckle simulation for the synthetic image is based on the noisy model, the Eq. (1). Three levels of noise are tested by setting standard deviations  sigma = {0.8,1.0,1.3}.The clinical ultrasound images include liver, intravascular, and 3D ultrasound images. Many experiments performed on log-compressed ultrasound images showed that the speckle noise is better described by the Gamma distribution [4‐6, 37]. A simple experiment is then conducted to determine whether or not the speckle noise is subjected to Gamma distribution. In this experiment, the Kolmogorov-Smirnov (KS) test is performed on the experimental images [5]. For each image, the comparison between Gaussian, Gamma, and Rayleigh distribution is done. The formula of the KS test is as follows [5]:where \( \widehat{F_n} \) is the empirical cumulative distribution function (CDF) of the observed images, F _X is the CDF of either Gaussian, Gamma, − Rayleigh, or Fisher-Tippett distribution. The Glivenko-Gantelli theorem states that, if the samples are subjected to distribution F _X , then D _KS converges to 0 [38]. In Table 1, the values in the Gamma column are smaller than the other values, and this means the real ultrasound images and synthetic noisy image are tended to subject to the Gamma distribution.

Three measures are adopted for the quantitative evaluation of the despeckling performance: the signal-to-noise ratio (SNR), structural similarity index (SSIM), and natural image quality evaluator (NIQE). The SNR is established on the variance ratio of the effective signal and noise using the despeckled and noise-free image such thatwhere M is the total number of pixels in the noisy image, \( \widehat{X_i} \) and X _i are the restored value and true value at pixel i, respectively. The SSIM is used to measure the structural similarity between the images [39, 40], and is defined as follows:where μ _X and \( {\mu}_{\widehat{X}} \) are the mean of the ground truth X and denoised image \( \widehat{X} \), respectively, \( {\sigma}_{X\widehat{X}} \) is the covariance of X and \( \widehat{X} \), and c ₁ and c ₂ are constant parameters. In this research, the SSIM is locally measured with an 8 × 8 Gaussian kernel, and the mean SSIM (MSSIM) is estimated as a global SSIM by all the local SSIMs. The NIQE [33] is based on the construction of a “quality aware” collection of statistical features, which are based on a simple and successful space domain natural scene statistic model, represented aswhere v ₁, v ₂ and Σ₁, Σ₂ are the mean vectors and covariance matrices, respectively, of the natural Multivariate Gaussian (MVG) model and MVG model of a distorted image.

The proposed algorithm is compared with two adaptive algorithms, namely, the BNLTV filter [29], which is based on Bayesian framework, and SAR-BM3D filter [31] to provide relevant comparisons. The results of the [29] are provided by the author. The parameters of proposed filter and SAR-BM3D filter are adjusted to obtain the best SNR scores. The detailed parameters for the proposed filter and SAR-BM3D filter are listed in Table 2. All the algorithms are implemented with MATLAB R2012a, and the computer is equipped with 2.80 GHz CPU (10-core, E5) and 64 GB RAM.

The excellent despeckling performance of the proposed filter can be attributed to the spatiogram similarity based on symmetric KL divergence. This similarity measurement is more adaptable to the complex speckle noise, because the KL divergence can measure the difference between two probability distributions and the spatiogram can consider the spatial information of the image patches. Therefore, it demonstrates better performance than the similarity computation based on the non-robust Euclidean distance, because Euclidean distance uses only the intensity information of the pixels in the patch.

Finally, the limitation of the proposed filter is that its filtering parameters (m, p, λ, and β) are currently manually determined by experience. The optimal parameter setting may change with noise levels and image characteristics. Although the experimental results in this study show that the proposed filter can achieve state-of-the-art performance with experiential parameters, the automatic determination of filtering parameters with theoretical foundations is warranted in a future study.

In this paper, an adapted NLTV-based speckle filter has been presented to despeckle the ultrasound images. The filter exploits the spatiogram similarity based on the symmetric KL divergence for the similarity calculation between image patches. As a result, the performance of NLTV is improved. The adapted filter is applied on synthetic and real ultrasound images and then compared with current state-of-the-art methods. The results demonstrate that the proposed filter outperforms current state-of-the-art filters with regard to ultrasound despeckling. We believe that our method can improve the quality of ultrasound images and has benefit effects on visual perception and diagnostic operations.