Skip to main content

06.05.2024 | Original Article

Non-rigid scene reconstruction of deformable soft tissue with monocular endoscopy in minimally invasive surgery

verfasst von: Enpeng Wang, Yueang Liu, Jiangchang Xu, Xiaojun Chen

Erschienen in: International Journal of Computer Assisted Radiology and Surgery

Einloggen, um Zugang zu erhalten

Abstract

Purpose

The utilization of image-guided surgery has demonstrated its ability to improve the precision and safety of minimally invasive surgery (MIS). Non-rigid scene reconstruction is a challenge in image-guided system duo to uniform texture, smoke, and instrument occlusion, etc.

Methods

In this paper, we introduced an algorithm for 3D reconstruction aimed at non-rigid surgery scenes. The proposed method comprises two main components: firstly, the front-end process involves the initial reconstruction of 3D information for deformable soft tissues using embedded deformation graph (EDG) on the basis of dual quaternions, enabling the reconstruction without the need for prior knowledge of the target. Secondly, the EDG is integrated with isometric nonrigid structure from motion (Iso-NRSFM) to facilitate centralized optimization of the observed map points and camera motion across different time instances in deformable scenes.

Results

For the quantitative evaluation of the proposed method, we conducted comparative experiments with both synthetic datasets and publicly available datasets against the state-of-the-art 3D reconstruction method, DefSLAM. The test results show that our proposed method achieved a maximum reduction of 1.6 mm in average reconstruction error compared to method DefSLAM across all datasets. Additionally, qualitative experiments were performed on video scene datasets involving surgical instrument occlusions.

Conclusion

Our method proved to outperform DefSLAM on both synthetic datasets and public datasets through experiments, demonstrating its robustness and accuracy in the reconstruction of soft tissues in dynamic surgical scenes. This success highlights the potential clinical application of our method in delivering surgeons with critical shape and depth information for MIS.
Literatur
2.
Zurück zum Zitat Dou M, Khamis S, Degtyarev Y, Davidson P, Fanello SR, Kowdle A, Escolano SO, Rhemann C, Kim D, Taylor J, Pushmeet K, Vladimir T, Shahram I (2016) Fusion4d: Real-time performance capture of challenging scenes. ACM Trans Graph (ToG) 35(4):1–13CrossRef Dou M, Khamis S, Degtyarev Y, Davidson P, Fanello SR, Kowdle A, Escolano SO, Rhemann C, Kim D, Taylor J, Pushmeet K, Vladimir T, Shahram I (2016) Fusion4d: Real-time performance capture of challenging scenes. ACM Trans Graph (ToG) 35(4):1–13CrossRef
4.
Zurück zum Zitat Wang Y, Long Y, Fan S, Dou Q (2022) Neural rendering for stereo 3d reconstruction of deformable tissues in robotic surgery. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 431–441 Wang Y, Long Y, Fan S, Dou Q (2022) Neural rendering for stereo 3d reconstruction of deformable tissues in robotic surgery. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 431–441
6.
Zurück zum Zitat Whelan T, Salas-Moreno RF, Glocker B, Davison AJ, Leutenegger S (2016) Elasticfusion: real-time dense slam and light source estimation. Int J Robot Res 35(14):1697–1716CrossRef Whelan T, Salas-Moreno RF, Glocker B, Davison AJ, Leutenegger S (2016) Elasticfusion: real-time dense slam and light source estimation. Int J Robot Res 35(14):1697–1716CrossRef
7.
Zurück zum Zitat Long Y, Li Z, Yee CH, Ng CF, Taylor RH, Unberath M, Dou Q (2021) E-dssr: efficient dynamic surgical scene reconstruction with transformer-based stereoscopic depth perception. In: Medical image computing and computer assisted intervention–MICCAI 2021: 24th international conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24. Springer, pp 415–425 Long Y, Li Z, Yee CH, Ng CF, Taylor RH, Unberath M, Dou Q (2021) E-dssr: efficient dynamic surgical scene reconstruction with transformer-based stereoscopic depth perception. In: Medical image computing and computer assisted intervention–MICCAI 2021: 24th international conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24. Springer, pp 415–425
8.
Zurück zum Zitat Song J, Bai F, Zhao L, Huang S, Xiong R (2020) Efficient two step optimization for large embedded deformation graph based slam. In: 2020 IEEE international conference on robotics and automation (ICRA). IEEE, pp 9419–9425 Song J, Bai F, Zhao L, Huang S, Xiong R (2020) Efficient two step optimization for large embedded deformation graph based slam. In: 2020 IEEE international conference on robotics and automation (ICRA). IEEE, pp 9419–9425
10.
Zurück zum Zitat Lamarca J, Parashar S, Bartoli A, Montiel J (2020) Defslam: tracking and mapping of deforming scenes from monocular sequences. IEEE Trans Rob 37(1):291–303CrossRef Lamarca J, Parashar S, Bartoli A, Montiel J (2020) Defslam: tracking and mapping of deforming scenes from monocular sequences. IEEE Trans Rob 37(1):291–303CrossRef
14.
Zurück zum Zitat Godard C, Mac Aodha O, Firman M, Brostow GJ (2019) Digging into self-supervised monocular depth estimation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3828–3838 Godard C, Mac Aodha O, Firman M, Brostow GJ (2019) Digging into self-supervised monocular depth estimation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3828–3838
15.
Zurück zum Zitat Mur-Artal R, Montiel JMM, Tardós JD (2015) Orb-slam: a versatile and accurate monocular slam system. IEEE Trans Rob 31(5):1147–1163CrossRef Mur-Artal R, Montiel JMM, Tardós JD (2015) Orb-slam: a versatile and accurate monocular slam system. IEEE Trans Rob 31(5):1147–1163CrossRef
16.
Zurück zum Zitat Kavan L, Collins S, Žára J, O’Sullivan C (2007) Skinning with dual quaternions. In: Proceedings of the 2007 symposium on interactive 3D graphics and games. I3D ’07. Association for Computing Machinery, New York, pp 39–46. https://doi.org/10.1145/1230100.1230107 Kavan L, Collins S, Žára J, O’Sullivan C (2007) Skinning with dual quaternions. In: Proceedings of the 2007 symposium on interactive 3D graphics and games. I3D ’07. Association for Computing Machinery, New York, pp 39–46. https://​doi.​org/​10.​1145/​1230100.​1230107
17.
Zurück zum Zitat Jia T, Taylor ZA, Chen X (2021) Long term and robust 6dof motion tracking for highly dynamic stereo endoscopy videos. Comput Med Imaging Graph 94:101995CrossRefPubMed Jia T, Taylor ZA, Chen X (2021) Long term and robust 6dof motion tracking for highly dynamic stereo endoscopy videos. Comput Med Imaging Graph 94:101995CrossRefPubMed
18.
Zurück zum Zitat Parashar S, Pizarro D, Bartoli A (2017) Isometric non-rigid shape-from-motion with Riemannian geometry solved in linear time. IEEE Trans Pattern Anal Mach Intell 40(10):2442–2454CrossRefPubMed Parashar S, Pizarro D, Bartoli A (2017) Isometric non-rigid shape-from-motion with Riemannian geometry solved in linear time. IEEE Trans Pattern Anal Mach Intell 40(10):2442–2454CrossRefPubMed
19.
Zurück zum Zitat Cartucho J, Tukra S, Li Y, Elson SD, Giannarou S (2021) Visionblender: a tool to efficiently generate computer vision datasets for robotic surgery. Comput Methods Biomech Biomed Eng Imaging Vis 9(4):331–338 Cartucho J, Tukra S, Li Y, Elson SD, Giannarou S (2021) Visionblender: a tool to efficiently generate computer vision datasets for robotic surgery. Comput Methods Biomech Biomed Eng Imaging Vis 9(4):331–338
20.
Zurück zum Zitat Chen Z, Marzullo A, Alberti D, Lievore E, Fontana M, De Cobelli O, Musi G, Ferrigno G, De Momi E (2023) Frsr: framework for real-time scene reconstruction in robot-assisted minimally invasive surgery. Comput Biol Med 163:107121CrossRefPubMed Chen Z, Marzullo A, Alberti D, Lievore E, Fontana M, De Cobelli O, Musi G, Ferrigno G, De Momi E (2023) Frsr: framework for real-time scene reconstruction in robot-assisted minimally invasive surgery. Comput Biol Med 163:107121CrossRefPubMed
21.
Zurück zum Zitat Chen Z, Cruciani L, Lievore E, Fontana M, De Cobelli O, Musi G, Ferrigno G, De Momi E (2024) Spatio-temporal layers based intra-operative stereo depth estimation network via hierarchical prediction and progressive training. Comput Methods Programs Biomed 244:107937CrossRefPubMed Chen Z, Cruciani L, Lievore E, Fontana M, De Cobelli O, Musi G, Ferrigno G, De Momi E (2024) Spatio-temporal layers based intra-operative stereo depth estimation network via hierarchical prediction and progressive training. Comput Methods Programs Biomed 244:107937CrossRefPubMed
22.
Zurück zum Zitat Heiselman JS, Jarnagin WR, Miga MI (2020) Intraoperative correction of liver deformation using sparse surface and vascular features via linearized iterative boundary reconstruction. IEEE Trans Med Imaging 39(6):2223–2234 Heiselman JS, Jarnagin WR, Miga MI (2020) Intraoperative correction of liver deformation using sparse surface and vascular features via linearized iterative boundary reconstruction. IEEE Trans Med Imaging 39(6):2223–2234
Metadaten
Titel
Non-rigid scene reconstruction of deformable soft tissue with monocular endoscopy in minimally invasive surgery
verfasst von
Enpeng Wang
Yueang Liu
Jiangchang Xu
Xiaojun Chen
Publikationsdatum
06.05.2024
Verlag
Springer International Publishing
Erschienen in
International Journal of Computer Assisted Radiology and Surgery
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-024-03149-4

Mammakarzinom: Brustdichte beeinflusst rezidivfreies Überleben

26.05.2024 Mammakarzinom Nachrichten

Frauen, die zum Zeitpunkt der Brustkrebsdiagnose eine hohe mammografische Brustdichte aufweisen, haben ein erhöhtes Risiko für ein baldiges Rezidiv, legen neue Daten nahe.

„Übersichtlicher Wegweiser“: Lauterbachs umstrittener Klinik-Atlas ist online

17.05.2024 Klinik aktuell Nachrichten

Sie sei „ethisch geboten“, meint Gesundheitsminister Karl Lauterbach: mehr Transparenz über die Qualität von Klinikbehandlungen. Um sie abzubilden, lässt er gegen den Widerstand vieler Länder einen virtuellen Klinik-Atlas freischalten.

Klinikreform soll zehntausende Menschenleben retten

15.05.2024 Klinik aktuell Nachrichten

Gesundheitsminister Lauterbach hat die vom Bundeskabinett beschlossene Klinikreform verteidigt. Kritik an den Plänen kommt vom Marburger Bund. Und in den Ländern wird über den Gang zum Vermittlungsausschuss spekuliert.

Darf man die Behandlung eines Neonazis ablehnen?

08.05.2024 Gesellschaft Nachrichten

In einer Leseranfrage in der Zeitschrift Journal of the American Academy of Dermatology möchte ein anonymer Dermatologe bzw. eine anonyme Dermatologin wissen, ob er oder sie einen Patienten behandeln muss, der eine rassistische Tätowierung trägt.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.