Skip to main content
Erschienen in: International Journal of Computer Assisted Radiology and Surgery 3/2019

16.01.2019 | Original Article

A dataset of laryngeal endoscopic images with comparative study on convolution neural network-based semantic segmentation

verfasst von: Max-Heinrich Laves, Jens Bicker, Lüder A. Kahrs, Tobias Ortmaier

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 3/2019

Einloggen, um Zugang zu erhalten

Abstract

Purpose

Automated segmentation of anatomical structures in medical image analysis is a prerequisite for autonomous diagnosis as well as various computer- and robot-aided interventions. Recent methods based on deep convolutional neural networks (CNN) have outperformed former heuristic methods. However, those methods were primarily evaluated on rigid, real-world environments. In this study, existing segmentation methods were evaluated for their use on a new dataset of transoral endoscopic exploration.

Methods

Four machine learning-based methods SegNet, UNet, ENet and ErfNet were trained with supervision on a novel 7-class dataset of the human larynx. The dataset contains 536 manually segmented images from two patients during laser incisions. The Intersection-over-Union (IoU) evaluation metric was used to measure the accuracy of each method. Data augmentation and network ensembling were employed to increase segmentation accuracy. Stochastic inference was used to show uncertainties of the individual models. Patient-to-patient transfer was investigated using patient-specific fine-tuning.

Results

In this study, a weighted average ensemble network of UNet and ErfNet was best suited for the segmentation of laryngeal soft tissue with a mean IoU of 84.7%. The highest efficiency was achieved by ENet with a mean inference time of 9.22 ms per image. It is shown that 10 additional images from a new patient are sufficient for patient-specific fine-tuning.

Conclusion

CNN-based methods for semantic segmentation are applicable to endoscopic images of laryngeal soft tissue. The segmentation can be used for active constraints or to monitor morphological changes and autonomously detect pathologies. Further improvements could be achieved by using a larger dataset or training the models in a self-supervised manner on additional unlabeled data.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
2.
Zurück zum Zitat Aubreville M, Knipfer C, Oetter N, Jaremenko C, Rodner E, Denzler J, Bohr C, Neumann H, Stelzle F, Maier A (2017) Automatic classification of cancerous tissue in laserendomicroscopy images of the oral cavity using deep learning. Sci Rep. https://doi.org/10.1038/s41598-017-12320-8 Aubreville M, Knipfer C, Oetter N, Jaremenko C, Rodner E, Denzler J, Bohr C, Neumann H, Stelzle F, Maier A (2017) Automatic classification of cancerous tissue in laserendomicroscopy images of the oral cavity using deep learning. Sci Rep. https://​doi.​org/​10.​1038/​s41598-017-12320-8
7.
Zurück zum Zitat Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding. In: IEEE conference on computer vision and pattern recognition, pp 3213–3223. https://doi.org/10.1109/CVPR.2016.350 Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding. In: IEEE conference on computer vision and pattern recognition, pp 3213–3223. https://​doi.​org/​10.​1109/​CVPR.​2016.​350
11.
Zurück zum Zitat Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: Proceedings of the 33rd international conference on machine learning, vol 48, pp 1050–1059 Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: Proceedings of the 33rd international conference on machine learning, vol 48, pp 1050–1059
12.
Zurück zum Zitat García-Peraza-Herrera LC, Li W, Gruijthuijsen C, Devreker A, Attilakos G, Deprest J, Poorten EV, Stoyanov D, Vercauteren T, Ourselin S (2017) Real-time segmentation of non-rigid surgical tools based on deep learning and tracking. In: Lecture Notes on Computer Science LNCS, vol 10170, pp 84–95. https://doi.org/10.1007/978-3-319-54057-3_8 García-Peraza-Herrera LC, Li W, Gruijthuijsen C, Devreker A, Attilakos G, Deprest J, Poorten EV, Stoyanov D, Vercauteren T, Ourselin S (2017) Real-time segmentation of non-rigid surgical tools based on deep learning and tracking. In: Lecture Notes on Computer Science LNCS, vol 10170, pp 84–95. https://​doi.​org/​10.​1007/​978-3-319-54057-3_​8
15.
Zurück zum Zitat Kendall A, Gal Y (2017) What uncertainties do we need in Bayesian deep learning for computer vision? Adv Neural Inf Process Syst 30:5574–5584 Kendall A, Gal Y (2017) What uncertainties do we need in Bayesian deep learning for computer vision? Adv Neural Inf Process Syst 30:5574–5584
17.
21.
Zurück zum Zitat Osma-Ruiz V, Godino-Llorente JI, Sáenz-Lechón N, Fraile R (2008) Segmentation of the glottal space from laryngeal images using the watershed transform. Comput Med Imaging Graph 32(3):193–201CrossRefPubMed Osma-Ruiz V, Godino-Llorente JI, Sáenz-Lechón N, Fraile R (2008) Segmentation of the glottal space from laryngeal images using the watershed transform. Comput Med Imaging Graph 32(3):193–201CrossRefPubMed
23.
25.
Zurück zum Zitat Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in PyTorch. In: 31st Conference on neural information processing systems (NIPS). https://openreview.net/forum?id=BJJsrmfCZ. Accessed 1 Oct 2018 Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in PyTorch. In: 31st Conference on neural information processing systems (NIPS). https://​openreview.​net/​forum?​id=​BJJsrmfCZ. Accessed 1 Oct 2018
29.
Zurück zum Zitat Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention—MICCAI 2015, pp 234–241 Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention—MICCAI 2015, pp 234–241
Metadaten
Titel
A dataset of laryngeal endoscopic images with comparative study on convolution neural network-based semantic segmentation
verfasst von
Max-Heinrich Laves
Jens Bicker
Lüder A. Kahrs
Tobias Ortmaier
Publikationsdatum
16.01.2019
Verlag
Springer International Publishing
Erschienen in
International Journal of Computer Assisted Radiology and Surgery / Ausgabe 3/2019
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-018-01910-0

Weitere Artikel der Ausgabe 3/2019

International Journal of Computer Assisted Radiology and Surgery 3/2019 Zur Ausgabe

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.