Use of an augmented reality (AR)–based soft tissue navigation system in urologic laparoscopic surgery is an evolving technique.
Objective
To evaluate a novel soft tissue navigation system developed to enhance the surgeon's perception and to provide decision-making guidance directly before initiation of kidney resection for laparoscopic partial nephrectomy (LPN).
Design, setting, and participants
Custom-designed navigation aids, a mobile C-arm capable of cone-beam imaging, and a standard personal computer were used. The feasibility and reproducibility of inside-out tracking principles were evaluated in a porcine model with an artificially created intraparenchymal tumor in vitro. The same algorithm was then incorporated into clinical practice during LPN.
Interventions
Evaluation of a fully automated inside-out tracking system was repeated in exactly the same way for 10 different porcine renal units. Additionally, 10 patients underwent retroperitoneal LPNs under manual AR guidance by one surgeon.
Measurements
The navigation errors and image-acquisition times were determined in vitro. The mean operative time, time to locate the tumor, and positive surgical margin were assessed in vivo.
Results and limitations
The system was able to navigate and superpose the virtually created images and real-time images with an error margin of only 0.5 mm, and fully automated initial image acquisition took 40 ms. The mean operative time was 165 min (range: 135–195 min), and mean time to locate the tumor was 20 min (range: 13–27 min). None of the cases required conversion to open surgery. Definitive histology revealed tumor-free margins in all 10 cases.
Conclusions
This novel AR tracking system proved to be functional with a reasonable margin of error and image-to-image registration time. Mounting the pre- or intraoperative imaging properties on real-time videoendoscopic images in a real-time manner will simplify and increase the precision of laparoscopic procedures.
Introduction
Advances in imaging modalities have led to an increasing knowledge about patients’ individual anatomy. Using this valuable information, computer-assisted surgery promises to facilitate surgical planning and increase surgical precision [1]. It is most widely used in neurosurgery, otolaryngology, and orthopedics, in which the target organs are assumed to be rigid and to have a constant spatial relationship to anatomical landmarks [2]. Especially in soft tissue surgery, organ shift and tissue deformation caused by motion reduce the value of the image data. To overcome the problem of tissue motion, our navigation approach utilizes custom-designed navigation aids that are inserted into the organ and tracked by conventional monocular endoscope. Thus, organ motion is captured in real time during the intervention and compensation can be done inherently.
In this paper, we present our preliminary results with image-guided surgery using a fully automated system on a porcine model as well as the clinical application of the developed AR algorithms during laparoscopic partial nephrectomy (LPN).
Section snippets
Overview
Currently, optical or magnetic tracking systems typically are used for the tracking of surgical instruments and the patient during surgical navigation. In contrast, inside-out tracking only relies on the real-time processing of the endoscopic video; no external tracking systems are involved.
Inside-out tracking follows the endoscope by means of the spatial configuration of custom-developed navigation aids from three-dimensional (3D) imaging data and their two-dimensional (2D) projections in the
In vitro
The system was able to navigate and superpose the virtually created images and real-time images with an error margin of only 0.5 mm (range: 0.2–0.7 mm), and fully automated initial image acquisition took 40 ms (25 pictures per second) with five needles inserted into the kidney. In the case of organ motion during manipulation, the augmented picture with the detected navigation aids could automatically follow the videoendoscopic picture using the navigation aids as landmarks; in this way, shifts
Discussion
In the literature, data about AR-guided minimally invasive urological operations are scarce [9]. Marescaux et al. [10] reported the first real-time AR-assisted laparoscopic adrenalectomy. Mozer et al. [11] then described their AR-guided system for the placement of percutaneous renal access, and Ukimura et al. [12] and Ukimura and Gill [13] reviewed their clinical experience with real-time transrectal US–guided laparoscopic prostatectomy and LPN by means of preoperative CT images (Table 2).
In
Conclusions
We have described a functional and biological model for AR using fully automated tracking with a reasonable error margin and image-to-image registration time. We also managed to create AR in the clinical setting using the same algorithms we described and used in the in vitro stage. Mounting the pre- or intraoperative imaging properties onto the real-time videoendoscopic images in a real-time manner will simplify and increase the precision of laparoscopic procedures.
Research in augmented and virtual reality in congregation with the Internet of Things has opened many avenues in diagnosing and treating neurological disorders. Augmented reality permits inserting virtual content in the real world, while virtual reality is a simulated experience that provides an artificial three-dimensional environment to the user. These are the game-changer technologies as they give a transformational change to existing technologies and methods. Augmented and virtual reality has come out as a significant technology in treating various mental disorders, thereby providing great applications in the field of neuroscience. In this review, we shed light on different components required for developing an augmented and virtual reality-based technology followed by different devices used in the augmented and virtual-reality-based systems. We highlighted the use of this technology in the diagnosis of neurological defects and proposed strategies to use these advancements to provide an immersive experience for both research and educational purposes. Moreover, we show an extensive implementation of augmented and virtual reality in neurological surgery, neuromodulation, and neuroprosthetics. Therefore, the role of the Internet of Things with augmented and virtual reality for diagnosing and treating neurological disorders is the future.
Virtual and augmented reality (VR/AR) technologies hold great promise in various medical fields. The release of a new generation of headsets for medical enhanced VR/AR (MER) opens new possibilities for applications in medicine, particularly in urology, to improve accessibility to everyone. These innovative headsets offer deep immersion without requiring a controller, which represents a novel approach to VR/AR engagement. The potential of these headsets applies to all aspects of urology, including surgical training, virtual meetings, communication between health care providers, patient counseling, telemedicine, delivering patient advice, and pain control. MER has the potential to improve operative planning and enhance intraoperative navigation and spatial awareness. The surgeon’s visualization and overall experience can be significantly enhanced via improved guidance and visualization, ultimately leading to greater precision and safety. This cutting-edge technology has the potential to reshape urology practice, communication methods, and medical procedures, and ultimately to improve patients’ experience of their urological condition.
This mini review explores how a new generation of headsets for medical enhanced virtual reality could revolutionize urology by improving surgical planning, assistance during procedures, and medical education. Patients can benefit from better pain management and a deeper understanding of their conditions. However, challenges such as costs, accuracy, and ethical concerns must be addressed. This technology holds promise for transforming urological practice and patient care.
Augmented Reality (AR) is considered to be a promising technology for the guidance of laparoscopic liver surgery. By overlaying pre-operative 3D information of the liver and internal blood vessels on the laparoscopic view, surgeons can better understand the location of critical structures. In an effort to enable AR, several authors have focused on the development of methods to obtain an accurate alignment between the laparoscopic video image and the pre-operative 3D data of the liver, without assessing the benefit that the resulting overlay can provide during surgery. In this paper, we present a study that aims to assess quantitatively and qualitatively the value of an AR overlay in laparoscopic surgery during a simulated surgical task on a phantom setup. We design a study where participants are asked to physically localise pre-operative tumours in a liver phantom using three image guidance conditions — a baseline condition without any image guidance, a condition where the 3D surfaces of the liver are aligned to the video and displayed on a black background, and a condition where video see-through AR is displayed on the laparoscopic video. Using data collected from a cohort of 24 participants which include 12 surgeons, we observe that compared to the baseline, AR decreases the median localisation error of surgeons on non-peripheral targets from 25.8 mm to 9.2 mm. Using subjective feedback, we also identify that AR introduces usability improvements in the surgical task and increases the perceived confidence of the users. Between the two tested displays, the majority of participants preferred to use the AR overlay instead of navigated view of the 3D surfaces on a separate screen. We conclude that AR has the potential to improve performance and decision making in laparoscopic surgery, and that improvements in overlay alignment accuracy and depth perception should be pursued in the future.
Mixed reality head-mounted displays (MR-HMD) are a novel and emerging tool in healthcare. There is a paucity of data on the safety and efficacy of the use of MR-HMD in the cardiac catheterization laboratory (CCL). We sought to analyze and compare fluoroscopy time, procedure time, and complication rates with right heart catheterizations (RHCs) and coronary angiographies (CAs) performed with MR-HMD versus standard LCD medical displays.
This is a non-randomized trial that included patients who underwent RHC and CA with MR-HMD between August 2019 and January 2020. Their outcomes were compared to a control group during the same time period. The primary endpoints were procedure time, fluoroscopy time, and dose area product (DAP). The secondary endpoints were contrast volume and intra and postprocedural complications rate.
50 patients were enrolled in the trial, 33 had a RHC done, and 29 had a diagnostic CA performed. They were compared to 232 patients in the control group. The use of MR-HMD was associated with a significantly lower procedure time (20 min (IQR 14–30) vs. 25 min (IQR 18–36), p = 0.038). There were no significant differences in median fluoroscopy time (1.5 min (IQR 0.7–4.9) in the study group vs. 1.3 min (IQR 0.8–3.1), p = 0.84) or median DAP (165.4 mGy·cm2 (IQR 13–15,583) in the study group vs. 913 mGy·cm2 (IQR 24–6291), p = 0.17). There was no significant increase in intra- or post-procedure complications.
MR-HMD use is safe and feasible and may decrease procedure time in the CCL.
In laparoscopic liver resection, surgeons conventionally rely on anatomical landmarks detected through a laparoscope, preoperative volumetric images and laparoscopic ultrasound to compensate for the challenges of minimally invasive access. Image guidance using optical tracking and registration procedures is a promising tool, although often undermined by its inaccuracy. This study evaluates a novel surgical navigation solution that can compensate for liver deformations using an accurate and effective registration method. The proposed solution relies on a robotic C-arm to perform registration to preoperative CT/MRI image data and allows for intraoperative updates during resection using fluoroscopic images. Navigation is offered both as a 3D liver model with real-time instrument visualization, as well as an augmented reality overlay on the laparoscope camera view. Testing was conducted through a pre-clinical trial which included four porcine models. Accuracy of the navigation system was measured through two evaluation methods: liver surface fiducials reprojection and a comparison between planned and navigated resection margins. Target Registration Error with the fiducials evaluation shows that the accuracy in the vicinity of the lesion was 3.781.89 mm. Resection margin evaluations resulted in an overall median accuracy of 4.44 mm with a maximum error of 9.75 mm over the four subjects. The presented solution is accurate enough to be potentially clinically beneficial for surgical guidance in laparoscopic liver surgery.
2021, Computer Methods and Programs in Biomedicine
Prostate cancer is a disease with a high incidence of tumors in men. Due to the long incubation time and insidious condition, early diagnosis is difficult; especially imaging diagnosis is more difficult. In actual clinical practice, the method of manual segmentation by medical experts is mainly used, which is time-consuming and labor-intensive and relies heavily on the experience and ability of medical experts. The rapid, accurate and repeatable segmentation of the prostate area is still a challenging problem. It is important to explore the automated segmentation of prostate images based on the 3D AlexNet network.
Taking the medical image of prostate cancer as the entry point, the three-dimensional data is introduced into the deep learning convolutional neural network. This paper proposes a 3D AlexNet method for the automatic segmentation of prostate cancer magnetic resonance images, and the general network ResNet 50, Inception -V4 compares network performance.
Based on the training samples of magnetic resonance images of 500 prostate cancer patients, a set of 3D AlexNet with simple structure and excellent performance was established through adaptive improvement on the basis of classic AlexNet. The accuracy rate was as high as 0.921, the specificity was 0.896, and the sensitivity It is 0.902 and the area under the receiver operating characteristic curve (AUC) is 0.964. The Mean Absolute Distance (MAD) between the segmentation result and the medical expert's gold standard is 0.356 mm, and the Hausdorff distance (HD) is 1.024 mm, the Dice similarity coefficient is 0.9768.
The improved 3D AlexNet can automatically complete the structured segmentation of prostate magnetic resonance images. Compared with traditional segmentation methods and depth segmentation methods, the performance of the 3D AlexNet network is superior in terms of training time and parameter amount, or network performance evaluation. Compared with the algorithm, it proves the effectiveness of this method.
