Skip to main content
Erschienen in: International Journal of Computer Assisted Radiology and Surgery 7/2020

05.06.2020 | Original Article

Evaluation of a marker-less, intra-operative, augmented reality guidance system for robot-assisted laparoscopic radical prostatectomy

verfasst von: Megha Kalia, Prateek Mathur, Keith Tsang, Peter Black, Nassir Navab, Septimiu Salcudean

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 7/2020

Einloggen, um Zugang zu erhalten

Abstract

Purpose

Robot-assisted laparoscopic radical prostatectomy (RALRP) using the da Vinci surgical robot is a common treatment for organ-confined prostate cancer. Augmented reality (AR) can help during RALRP by showing the surgeon the location of anatomical structures and tumors from preoperative imaging. Previously, we proposed hand-eye and camera intrinsic matrix estimation procedures that can be carried out with conventional instruments within the patient during surgery, take < 3 min to perform, and fit seamlessly in the existing surgical workflow. In this paper, we describe and evaluate a complete AR guidance system for RALRP and quantify its accuracy.

Methods

Our AR system requires three transformations: the transrectal ultrasound (TRUS) to da Vinci transformation, the camera intrinsic matrix, and the hand-eye transformation. For evaluation, a 3D-printed cross-wire was visualized in TRUS and stereo endoscope in a water bath. Manually triangulated cross-wire points from stereo images were used as ground truth to evaluate overall TRE between these points and points transformed from TRUS to camera.

Results

After transforming the ground-truth points from the TRUS to the camera coordinate frame, the mean target registration error (TRE) (SD) was \(4.56\pm 1.57\) mm. The mean TREs (SD) in the x-, y-, and z-directions are \(1.93\pm 1.26\) mm, \(2.04\pm 1.37\) mm, and \(2.94\pm 1.84\) mm, respectively.

Conclusions

We describe and evaluate a complete AR guidance system for RALRP which can augment preoperative data to endoscope camera image, after a deformable magnetic resonance image to TRUS registration step. The streamlined procedures with current surgical workflow and low TRE demonstrate the compatibility and readiness of the system for clinical translation. A detailed sensitivity study remains part of future work.
Literatur
1.
Zurück zum Zitat Adebar T, Salcudean S, Mahdavi S, Moradi M, Nguan C, Goldenberg L (2011) A robotic system for intra-operative trans-rectal ultrasound and ultrasound elastography in radical prostatectomy. In: Taylor RH, Yang GZ (eds) Information processing in computer-assisted interventions. Springer, Berlin, pp 79–89CrossRef Adebar T, Salcudean S, Mahdavi S, Moradi M, Nguan C, Goldenberg L (2011) A robotic system for intra-operative trans-rectal ultrasound and ultrasound elastography in radical prostatectomy. In: Taylor RH, Yang GZ (eds) Information processing in computer-assisted interventions. Springer, Berlin, pp 79–89CrossRef
2.
Zurück zum Zitat Chen EC, Morgan I, Jayarathne U, Ma B, Peters TM (2017) Hand-eye calibration using a target registration error model. Healthc Technol Lett 4(5):157–162CrossRef Chen EC, Morgan I, Jayarathne U, Ma B, Peters TM (2017) Hand-eye calibration using a target registration error model. Healthc Technol Lett 4(5):157–162CrossRef
3.
Zurück zum Zitat Cooperberg MR, Lubeck DP, Meng MV, Mehta SS, Carroll PR (2004) The changing face of low-risk prostate cancer: trends in clinical presentation and primary management. J Clin Oncol 22(11):2141CrossRef Cooperberg MR, Lubeck DP, Meng MV, Mehta SS, Carroll PR (2004) The changing face of low-risk prostate cancer: trends in clinical presentation and primary management. J Clin Oncol 22(11):2141CrossRef
5.
Zurück zum Zitat Edgcumbe P, Singla R, Pratt P, Schneider C, Nguan C, Rohling R (2016) Augmented reality imaging for robot-assisted partial nephrectomy surgery. In: International conference on medical imaging and augmented reality. Springer, pp 139–150 Edgcumbe P, Singla R, Pratt P, Schneider C, Nguan C, Rohling R (2016) Augmented reality imaging for robot-assisted partial nephrectomy surgery. In: International conference on medical imaging and augmented reality. Springer, pp 139–150
6.
Zurück zum Zitat Groves L, Vanberlo B, Peters T, Chen E (2019) A deep learning approach for automatic out-of-plane needle localization for semi-automatic ultrasound probe calibration. Healthc Technol Lett 6:204–209CrossRef Groves L, Vanberlo B, Peters T, Chen E (2019) A deep learning approach for automatic out-of-plane needle localization for semi-automatic ultrasound probe calibration. Healthc Technol Lett 6:204–209CrossRef
7.
Zurück zum Zitat Inoue S, Shiina H, Hiraoka T, Mitsui Y, Sumura M, Urakami S, Igawa M (2009) Retrospective analysis of the distance between the neurovascular bundle and prostate cancer foci in radical prostatectomy specimens: its clinical implication in nerve-sparing surgery. BJU Int 104(8):1085–1090CrossRef Inoue S, Shiina H, Hiraoka T, Mitsui Y, Sumura M, Urakami S, Igawa M (2009) Retrospective analysis of the distance between the neurovascular bundle and prostate cancer foci in radical prostatectomy specimens: its clinical implication in nerve-sparing surgery. BJU Int 104(8):1085–1090CrossRef
8.
Zurück zum Zitat Kalia M, Mathur P, Navab N, Salcudean T (2019) A marker-less real time intra-operative camera and hand-eye calibration procedure for surgical augmented reality. Healthc Technol Lett 6:255–260CrossRef Kalia M, Mathur P, Navab N, Salcudean T (2019) A marker-less real time intra-operative camera and hand-eye calibration procedure for surgical augmented reality. Healthc Technol Lett 6:255–260CrossRef
9.
Zurück zum Zitat Lanchon C, Custillon G, Moreau-Gaudry A, Descotes JL, Long JA, Fiard G, Voros S (2016) Augmented reality using transurethral ultrasound for laparoscopic radical prostatectomy: preclinical evaluation. J Urol 196(1):244–250CrossRef Lanchon C, Custillon G, Moreau-Gaudry A, Descotes JL, Long JA, Fiard G, Voros S (2016) Augmented reality using transurethral ultrasound for laparoscopic radical prostatectomy: preclinical evaluation. J Urol 196(1):244–250CrossRef
10.
Zurück zum Zitat Linte CA, Davenport KP, Cleary K, Peters C, Vosburgh KG, Navab N, Jannin P, Peters TM, Holmes DR III, Robb RA (2013) On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic. Comput Med Imaging Graph 37(2):83–97CrossRef Linte CA, Davenport KP, Cleary K, Peters C, Vosburgh KG, Navab N, Jannin P, Peters TM, Holmes DR III, Robb RA (2013) On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic. Comput Med Imaging Graph 37(2):83–97CrossRef
11.
Zurück zum Zitat Mohareri O, Ischia J, Black PC, Schneider C, Lobo J, Goldenberg L, Salcudean SE (2015) Intraoperative registered transrectal ultrasound guidance for robot-assisted laparoscopic radical prostatectomy. J Urol 193(1):302–312CrossRef Mohareri O, Ischia J, Black PC, Schneider C, Lobo J, Goldenberg L, Salcudean SE (2015) Intraoperative registered transrectal ultrasound guidance for robot-assisted laparoscopic radical prostatectomy. J Urol 193(1):302–312CrossRef
12.
Zurück zum Zitat Novara G, Ficarra V, Rosen RC, Artibani W, Costello A, Eastham JA, Graefen M, Guazzoni G, Shariat SF, Stolzenburg JU, Van Poppelk H, Zattonia F, Montorsil F, Mottrieb A, G. Wilson T (2012) Systematic review and meta-analysis of perioperative outcomes and complications after robot-assisted radical prostatectomy. Eur Urol 62(3):431–452CrossRef Novara G, Ficarra V, Rosen RC, Artibani W, Costello A, Eastham JA, Graefen M, Guazzoni G, Shariat SF, Stolzenburg JU, Van Poppelk H, Zattonia F, Montorsil F, Mottrieb A, G. Wilson T (2012) Systematic review and meta-analysis of perioperative outcomes and complications after robot-assisted radical prostatectomy. Eur Urol 62(3):431–452CrossRef
13.
Zurück zum Zitat Pachtrachai K, Allan M, Pawar V, Hailes S, Stoyanov D (2016) Hand-eye calibration for robotic assisted minimally invasive surgery without a calibration object. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 2485–2491 Pachtrachai K, Allan M, Pawar V, Hailes S, Stoyanov D (2016) Hand-eye calibration for robotic assisted minimally invasive surgery without a calibration object. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 2485–2491
14.
Zurück zum Zitat Pratt P, Bergeles C, Darzi A, Yang GZ (2014) Practical intraoperative stereo camera calibration. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 667–675 Pratt P, Bergeles C, Darzi A, Yang GZ (2014) Practical intraoperative stereo camera calibration. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 667–675
15.
Zurück zum Zitat Samei G, Tsang K, Kesch C, Lobo J, Hor S, Mohareri O, Chang S, Goldenberg SL, Black PC, Salcudean S (2019) A partial augmented reality system with live ultrasound and registered preoperative MRI for guiding robot-assisted radical prostatectomy. Med Image Anal 60:101588CrossRef Samei G, Tsang K, Kesch C, Lobo J, Hor S, Mohareri O, Chang S, Goldenberg SL, Black PC, Salcudean S (2019) A partial augmented reality system with live ultrasound and registered preoperative MRI for guiding robot-assisted radical prostatectomy. Med Image Anal 60:101588CrossRef
16.
Zurück zum Zitat Shao J, Luo H, Xiao D, Hu Q, Jia F (2017) Progressive hand-eye calibration for laparoscopic surgery navigation. In: Cardoso J, Arbel T, Luo X, Wesarg S, Reichl T, González Ballester MÁ, McLeod J, Drechsler K, Peters TM, Erdt M, Mori K, Linguraru MG, Uhl A, Oyarzun Laura C, Shekhar R (eds) Computer assisted and robotic endoscopy and clinical image-based procedures. Springer, New York, pp 42–49CrossRef Shao J, Luo H, Xiao D, Hu Q, Jia F (2017) Progressive hand-eye calibration for laparoscopic surgery navigation. In: Cardoso J, Arbel T, Luo X, Wesarg S, Reichl T, González Ballester MÁ, McLeod J, Drechsler K, Peters TM, Erdt M, Mori K, Linguraru MG, Uhl A, Oyarzun Laura C, Shekhar R (eds) Computer assisted and robotic endoscopy and clinical image-based procedures. Springer, New York, pp 42–49CrossRef
17.
Zurück zum Zitat Silberstein JL, Eastham JA (2014) Significance and management of positive surgical margins at the time of radical prostatectomy. Indian J Urol IJU J Urol Soc India 30(4):423CrossRef Silberstein JL, Eastham JA (2014) Significance and management of positive surgical margins at the time of radical prostatectomy. Indian J Urol IJU J Urol Soc India 30(4):423CrossRef
18.
Zurück zum Zitat Thompson S, Stoyanov D, Schneider C, Gurusamy K, Ourselin S, Davidson B, Hawkes D, Clarkson MJ (2016) Hand-eye calibration for rigid laparoscopes using an invariant point. Int J Comput Assist Radiol Surg 11(6):1071–1080CrossRef Thompson S, Stoyanov D, Schneider C, Gurusamy K, Ourselin S, Davidson B, Hawkes D, Clarkson MJ (2016) Hand-eye calibration for rigid laparoscopes using an invariant point. Int J Comput Assist Radiol Surg 11(6):1071–1080CrossRef
19.
Zurück zum Zitat Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22:1330–1334CrossRef Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22:1330–1334CrossRef
Metadaten
Titel
Evaluation of a marker-less, intra-operative, augmented reality guidance system for robot-assisted laparoscopic radical prostatectomy
verfasst von
Megha Kalia
Prateek Mathur
Keith Tsang
Peter Black
Nassir Navab
Septimiu Salcudean
Publikationsdatum
05.06.2020
Verlag
Springer International Publishing
Erschienen in
International Journal of Computer Assisted Radiology and Surgery / Ausgabe 7/2020
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-020-02181-4

Weitere Artikel der Ausgabe 7/2020

International Journal of Computer Assisted Radiology and Surgery 7/2020 Zur Ausgabe

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.