VOLT: a novel open-source pipeline for automatic segmentation of endolymphatic space in inner ear MRI

D1 and D3 included real-world data sets from consecutive patients from the interdisciplinary German Center for Vertigo and Balance Disorders (DSGZ) of the Munich University Hospital (LMU) between 2015 and 2019. Institutional Review Board approval was obtained before the initiation of the study (no 64115). Included patients had presented with episodic vertigo attacks [19] and undergone iMRI as part of their indicated clinical diagnostic workup to evaluate their ELS. Their data sets were included after they had given oral and written consent following the Declaration of Helsinki. The inclusion criteria were age above 18 years. Exclusion criteria were any MR-related contraindications [20], poor image quality, or missing MR sequences. D1 included 210 inner ears of 105 consecutive patients (50 male; aged 19–84 years, mean age 50.4 ± 17.1 years), and D3 included 20 inner ears of 10 consecutive patients (5 male, aged 31–69 years, mean age: 46.8 ± 14.4 years). D1 and D3 did not differ significantly concerning age, gender, the grade of endolymphatic hydrops (ELH), or data quality (intensity, mean grayscale value). A detailed description of D1and D3 is given in Table 1.

As an artificial data set, D2 provided a known ground truth to test and compare VOLT’s performance to an adapted version of Otsu’s method [21], which is a recognized foreground/background segmentation algorithm based on global thresholding at an optimal histogram-derived cutoff. D2 consisted of an 8-bit cuboid volume using the same voxel-size and grid as real-world data with different sized cylindrical and cuboid-shaped cutouts (signal) whose grayscale values matched the real-world data set D1 (mean 68.7 ± 7.8; range 48.9–92.8). To this structural basis signal, two types of noise were added, which imitate the real-world variability of MRI signals [22]. The noise was added stepwise in the form of increasing blurriness noise (Gaussian blur kernel, SD range 1–6 voxel in x/y/z-direction; SD = standard deviations) or increasing scatter noise (Gaussian, SD range of intensity variation: 0–50 SD). D2 and its varying levels of noise can be viewed in Fig. 1a.

Patients were clinically diagnosed according to the international guidelines, most of the classification committee of the international Bárány Society (www.​jvr-web.​org/​ICVD.​html or www.​baranysociety.​nl) for the diagnosis of vestibular migraine [23], Menière’s disease [24], vestibular paroxysmia [25], bilateral vestibulopathy [26], acute unilateral vestibulopathy/vestibular neuritis [27] and benign paroxysmal positional vertigo [28]. The diagnoses of the patients within D1 and D3 can be viewed in Table 1.

Diagnostic workup included a careful neurological and neuro-otological examination including neuro-orthoptic assessment (e.g., Frenzel goggles; fundus photography and adjustments of the subjective visual vertical (SVV) for graviceptive vestibular function, for methods, see [29]), video-oculography during the head-impulse test (vHIT) for dynamic vestibular function (for methods, see [30, 31]), audiometry, and MR imaging of the whole brain including the cerebellopontine angle and brainstem.

Four hours after intravenous injection of a standard dose (0.1 ml/kg body weight, i.e., 0.1 mmol/kg body weight) of Gadobutrol (Gadovist®, Bayer, Leverkusen, Germany), MR imaging data were acquired in a whole-body 3 T MR scanner (Magnetom Skyra, Siemens Healthcare, Erlangen, Germany) with a 20-channel head coil. Head movements were minimalized in all three axes using a head positioning system for MRI (Crania Adult 01, Pearl Technology AG, Schlieren, Switzerland). A 3D-FLAIR sequence was used to differentiate endolymph from perilymph and bone, and a CISS sequence to delineate the total inner ear fluid space from the surrounding bone. A T2-weighted, three-dimensional, fluid-attenuated inversion recovery sequence (3D-FLAIR) with the following parameters: TR 6000 ms, TE 134 ms, TI 2240 ms, FOV 160 × 160 mm², 36 slices, base resolution 320, averages 1, acceleration factor of 2 using a parallel imaging technique with a generalized auto-calibrating partially parallel acquisition (GRAPPA) algorithm, slice thickness 0.5 mm, acquisition time 15:08 min was carried out. A high-resolution, strongly T2-weighted, 3D constructive interference steady state (CISS) sequence of the temporal bones was performed to evaluate the anatomy of the whole-fluid-filled labyrinthine spaces with the following parameters: TR 1000 ms, TE 133 ms, FA 100°, FOV 192 × 192 mm², 56 slices, base resolution 384, averages 4, acceleration factor of 2 using GRAPPA algorithm, slice thickness of 0.5 mm and acquisition time 8:36 min. The presence of ELH was observed on the 3D-FLAIR images as enlarged negative-signal spaces inside the labyrinth, according to a previously reported method [32, 33]. The decision to apply a single-dose contrast agent was made because of the ongoing discussion about gadolinium deposition within the dentate nucleus and globus pallidus after repeated administration of gadolinium-based contrast agents [34‐37]. It was not considered ethical to apply higher doses of contrast agent if not necessary. Accordingly, only patients with a diagnostic benefit were included in the study.

Evaluation of the iMRI and grading of the ELS was performed independently by two experienced head and neck radiologists and a neurologist who was blinded to the clinical patient data. If discrepancies arose, a consensus was reached by discussion. The characterization of the ELS in the vestibulum and cochlea was based on criteria previously described [12], which constitutes a fusion of two classification systems [38, 39]. D1 and D3 did not differ significantly concerning the grade of ELH. An overview of ELH grade and data quality for data sets D1 and D3 can be viewed in Table 1.

VOLT was developed on the real-world data set D1 using exclusively universal access software, namely 3D Slicer version 4.11 toolbox [40] including the TOMAAT plugin [15], as well as ImageJ Fiji [41] including the “Fuzzy and artificial neural networks image processing toolbox” [42] and the “MorphoLibJ Toolbox” [43](see an overview of the overall pipeline including VOLT-based ELS segmentation in Fig. 2a, b).

VOLT with three different cutoffs (c6, c8, c10) was validated on the artificial data set D2 and the prospective real-world data set D3. Segmentation accuracy was evaluated using the Sørensen-Dice overlap coefficient, which is defined as 2 * |X ∩ Y|/|X| +|Y| for segmentations X and Y [48], as a measure of region overlap between gold standard segmentation and the automatically obtained segmentations from the VOLT pipeline.

Segmentation precision was estimated by comparing the volume of the ELS (V_E) between segmentation methods. Structurally, the human inner ear can be pictured as an external, bony hose system (called the bony labyrinth, containing perilymph) and an inner hose system (called the membranous labyrinth, containing endolymph). The total lymph fluid space includes the inner hose system’s ELS and the surrounding perilymphatic space.

Receiver operating characteristics (ROC) analysis was used to show the (in)dependence of the performance of the methods from the grade of the ELH or the distribution of the fluids within the total fluid space (TFL) and the SNR of the iMRI data set.

The data were analyzed with SPSS 20.0 (SPSS, Chicago, IL, USA). Differences between data sets overall were assessed using a paired t-test, which was Bonferroni-corrected for multiple testing and viewed at p < 0.01 and p < 0.05. Linear agreement between parameter pairs was calculated for each method separately using the two-sided Spearman’s correlation coefficient and reported at a significance level of p < 0.01 and p < 0.05. For Receiver operating characteristics (ROC) analysis, the original Fortran program JLABROC4 (by Charles Metz and colleagues, Department of Radiology, University of Chicago; Java translation by John Eng, Russel H Morgan Department of Radiology and Radiological Science, Johns Hopkins University, Baltimore, Maryland, USA, Version 2.0, March 2017) was used.

After implementation on data set D1, the novel tool for automatic segmentation of the endolymphatic space (ELS) with a novel algorithm based on Volumetric Local Thresholding (VOLT) ran smoothly and showed no operational or stability issues. VOLT does not require especially powerful hardware or closed-source software. The only prerequisite is the installation of universal access software, namely 3D Slicer toolbox [40] including the TOMAAT plugin [15], as well as ImageJ Fiji [41] including the “Fuzzy and artificial neural networks image processing toolbox” [42] and the “MorphoLibJ Toolbox” [43].

The only step requiring user input was the cropping step. While this step required a rough selection of the inner ear and could easily be automatized, it allowed a quick and easy visual assessment of the source images and was therefore considered a suitable quality control mechanism. Cropping was performed in order to reduce computation time as well as allow for easier registration of the inner ears; the registration step was necessary to ensure correct positioning of the CISS-based hull relative to the FLAIR. For both registration and CNN segmentation, the necessary user input was limited to entering parameters and starting the process. After the CNN segmentation the user had to save the segmentations as a new file manually. As an orientation, pre-processing steps of one single-subject data set can be performed in less than ten minutes by an experienced user on a standard consumer laptop (Windows10 (64Bit), Intel^® Core i5-4200U @1,6 GHz, 8 GB RAM).

Data processing steps are fully automated and run without further user input in less than 60 s. Volumetric local thresholds can be adapted to signal-to-noise ratio (SNR) of different data sets. Output files include 3D volumetric quantification of TLS and ELS in mm³ and a 3D visualization of the inner ear. Examples of single-subject VOLT-based inner ear segmentations show different grades of ELH (Fig. 2c).

D2 was created to have a ground truth data set featuring challenges found in inner ear imaging, namely low contrast and high noise. Similar to actual iMRI, the regions of interest were three-dimensional volumes of different sizes. This proofed to be difficult for 2D-algorithms, whereas three-dimensional methods could analyze the data set better.

As an artificial data set, D2 provided a known ground truth to test and compare VOLT’s performance to an adapted version of Otsu’s method (O) [21], which is a recognized foreground/background segmentation algorithm based on global thresholding at an optimal histogram-derived cutoff. Based on empirical observations in the development data set (D1), VOLT was compared to O at three cutoff variations (c6, c8, c10). On average, over all noise conditions, the Dice score (DS) of VOLT cutoff versions (c6: 90%, c8: 92%; c10: 92%) outperformed Otsu’s method (82%). Both VOLT cutoff versions and Otsu’s method fared better with blurriness noise (DS:O: 91%; c6: 92%, c8: 93%; c10: 94%) in comparison with scatter noise (DS O: 82%; c6: 87%, c8: 91%; c10: 90%). More specifically, VOLT cutoff versions showed a high level of agreement in terms of Dice overlap with Otsu’s scores in data sets with low noise levels (Bl 1–4; Sc 10). The higher the noise level, however, the more VOLT cutoff versions outperformed Otsu’s method (Bl 5–6; Sc 20–60), with c8 showing an overall best performance independent of noise levels. All results are presented in Table 2 and Fig. 1b, c.

D3 included previously entirely unseen real-world data sets from 10 consecutive patients (= 20 inner ears) and was used to validate VOLT on entirely unseen data. Ear-specific segmentation accuracy was evaluated using the Sørensen-Dice overlap coefficient (DS), and segmentation precision were estimated by comparing the volume of the ELS (V_E). Performance (DS) of VOLT with the three different cutoffs c6: 97.0% ± 0.7, c8: 96.6% ± 0.8, c10: 95.9% 97% ± 0.9) highly overlapped with the manual segmentation. On average, c8 gave a close representation of the actual volume seen in the manual segmentation, while c6 tended to underestimate and c10 to overestimate the endolymphatic space volume methodically. Note that the grade of ELH correlated significantly with the endolymphatic volume of both the manual segmentation method (two-sided, r(18) = 0.475, p = 0.034) and with VOLT cutoff variations c6 (two-sided, r(18) = 0.553, p = 0.011)–c8 (two-sided, r(18) = 0.566, p = 0.009)–c10 (two-sided, r(18) = 0.569, p = 0.009). Receiver operating characteristics (ROC) analysis showed the grade of the ELH to be a good classifier for the computed volume of the ELS (fitted ROC area: 0.9). Table 2 shows an overview of the performance and accuracy results of each segmentation method. Figure 3 gives an ear-specific overview of each validation parameter.

Objective and volumetric quantification is a necessary step to assess and compare ELH results between studies and hospitals. So far, the clinical gold standard assessment of the ELS is based on a semi-quantitative and subjective grading reliant on a few MR slices in the transversal plane. In addition, different ELH classifications are being used in parallel [38, 39, 49, 50]. While manual volumetric segmentation is the gold standard for volumetric quantification, it is highly subjective and dependent on the rater’s experience and knowledge, not to mention time-consuming. VOLT allows objective, easily reproducible, and reliable stand-alone volumetric ELH quantification and grading, which closely matches manual segmentation, highly correlates with clinical ELH grading, and performs particularly well in data with a low signal-to-noise ratio.

The main advantage of VOLT is its local thresholding algorithm, which enables more flexible and stable results in comparison to global thresholding algorithms (such as Otsu’s method). Inhomogeneous image intensities and local brightness variations are adequately compensated for [48, 51]. The robustness and flexibility of VOLT to image artifacts can be further increased using different radius sizes. Importantly, results in Fig. 1 demonstrate that VOLT does not yield perfect segmentation in the absence of noise (not probable in real-world data), but instead performs more favorably and more stably in the presence of increased noise (very probable in real-world data).

Current ELS MR volumetric assessment approaches remain few and involve a manual or semi-automatic segmentation [6, 18, 50, 52]. Already a considerable improvement, these approaches require lengthy user interaction that is not suitable for use in more extensive group studies or clinical routine. Also, the software used tends to be attached to expensive software, and the uncompiled source is not available for public review.

VOLT runs smoothly and does not require especially powerful hardware or closed-source software. As an orientation, pre-processing steps of one data set can be performed in less than ten minutes by an experienced user on a standard consumer laptop. Data processing steps are fully automated and run without further user input in less than 60 s. Volumetric local thresholds can be adapted to the signal-to-noise ratio (SNR) of different data sets. Output files include 3D volumetric quantification of TLS and ELS in mm³ and a 3D visualization of the inner ear. The endolymphatic volumes conformed to those previously reported [53, 54].

Inner ear segmentation is a prerequisite step for VOLT-based ELH segmentation and is currently performed via a novel CNN-based deep learning approach [43], which is deployed as a module in 3D-Slicer [40]. This CNN was trained in-house at our department, on a separate iMRI data set obtained on the same MRI scanner and with the same imaging sequence parameters as our study. As such, this method was a natural choice for inner ear ROI segmentation in our data set, especially because segmentations were not only highly accurate but also obtainable in comparably fast execution time (< 5 s). A downside is that this network likely has difficulties in generalizing to data from other scanners or imaging sequence settings, e.g., from other clinics. Therefore, we do not assume a TOMAAT/V-Net segmentation as a fixed component of the current ELH segmentation pipeline. The inner ear ROI can also be obtained by other segmentation approaches, most prominently using atlas-based registration. Recently, two in-vivo MRI atlases and templates were proposed, one offering a probabilistic segmentation of the inner ear’s bony labyrinth [46], the other offering a high-resolution multivariate template for T1-, T2- and CISS-weighted MRI imaging [41]. Both atlases can yield accurate segmentation of the inner ear ROI while being much more generalizable to MRI data from previously unseen scanners or acquisition sites, in particular, if multivariate MRI appearances are available as in [41]. The downside of atlas-based segmentation is the high computational complexity of deformable atlas registration algorithms that align the atlas to the target volume. With carefully tuned parametrizations, such algorithms can achieve highly accurate segmentation, but segmentations can take 10 min to 2 h per volume [47], compared to < 5 s computation time for deep neural nets such as [43]. Overall, we, therefore, recommend the usage of deep neural nets for inner ear segmentation predictions; however, the correct way to generalize the network to new sites, e.g., via transfer learning, remains to be established in future work.

There are methodical limitations in the current study that need to be considered in the interpretation of the data. First, the performance of VOLT is highly dependent on the segmentation of the inner ear. An inner ear mask that includes parts of the dark background voxels surrounding the inner ear structures would lead to a false-positive attribution to the ELS. VOLT’s high performance and accuracy values are probably in part attributable to the novel CNN-based deep learning approach. Second, VOLT does not include any anatomical knowledge. In the best case, this means that the algorithm is entirely unbiased, i.e., not influenced by any prior morphological assumptions. The downside is a lack of exclusion of apparent errors that would be noticed by the human examiner.

An example would be segmentation errors that included surrounding structures into the ROI. A human examiner would know not to expect endolymph in the outermost tips of the cochlea or vestibulum. However, an algorithm does not. This is one reason VOLT is designed unusually strict in margin areas. Finally, VOLT (or any ELS segmentation method) is by nature highly dependent upon the resolution and contrast of the MRI raw data to be able to distinguish between endolymphatic and perilymphatic space.

We propose a novel pipeline for the automatic segmentation of endolymphatic hydrops in inner ear MRI. The core component is a novel algorithm based on Volumetric Local Thresholding (VOLT). Tool validation on artificial and real-world data resulted in a high level of performance and accuracy, in particular in low signal-to-noise ratio. ELS volume significantly correlated (p < 0.01) with the clinical grading of the ELS. A generic version of our three-dimensional thresholding algorithm has been made available to the scientific community via GitHub as an ImageJ-Plugin (https://​github.​com/​j-gerb/​3d-thresholding/​tree/​master).