Comparing a virtual reality head-mounted display to on-screen three-dimensional visualization and two-dimensional computed tomography data for training in decision making in hepatic surgery: a randomized controlled study

Between May and August 2016 90 medical students were recruited and participated in this study. Figure 5 shows the trail recruitment flowchart. Table 2 shows the statistical baseline data of the randomized study groups.

A VR environment with HMD can be used by surgical novices to accurately and quickly determine surgical liver anatomy and tumor involvement in complex liver cases in order to decide on an operative strategy. In the present study, the sum of correct answers in the test was significantly higher and decision time was significantly shorter with VR and 3D compared to 2D while there no differences between 3D and VR. These results were consistent in all three patient cases. In all three study groups, average performance increased per patient case, with case three having the highest number of correct answers and lowest time to answer. In the subjective evaluation, 3D and VR were preferred over 2D in all aspects. VR was evaluated as superior for the identification of anatomic anomalies, risk and target structures and for the transfer of anatomical and pathological information to the intraoperative situation compared to 3D and 2D. While 3D and VR were evaluated as being superior to 2D in overall pleasantness, for planning standard liver resections, and in medical education and training, the overall most favored visualization method was VR.

Other studies have shown 3D visualization to be advantageous in learning surgical liver anatomy with participants in 3D groups consistently answering anatomical questions more correctly and faster than 2D-groups [20, 22‐25, 37]. While the cited studies mainly asked generic questions about liver anatomy with some questions regarding resections, in the present study participants were asked to answer all relevant information needed to determine an operative strategy. Jurgaitis et al. showed that 3D visualizations improved medical students’ ability to localize hepatic tumors and correctly determine the extent of the hepatic resection [25]. The present study has shown that with the help of 3D-models, surgical novices could additionally differentiate between physiological anatomy and the pathology of a patient and make a surgical decision. The 3D visualization system may facilitate the teaching of liver anatomy and pathology and could help medical students to understand the steps in deciding on the type and extent of the hepatic resection. The positive results from previous studies [20, 22‐25, 37] suggest that 3D visualization and virtual reality compare favorably or may be superior to current surgical visualization teaching methods. Continued studies, e.g. focusing on other organ systems or surgical operations, may aid in establishing VR as a valid and modern surgical teaching tool [34]. Furthermore, surgical guidance systems relying on a combination of CT imaging, 3D segmentation and augmented and virtual reality systems are being rapidly developed and becoming increasingly robust [27, 39, 46‐50]. This should give additional weight to the argument that medical students should receive training in, and interact with these systems, particularly in surgical fields [33, 51‐53]. Surgeons may have different opinions on resectability of liver tumors depending on their expertise and experience, but also depending on their understanding of the patients’ imaging data combined with other relevant information [54]. The IMHOTEP tool may help surgical novices better understand the differences in decision-making between different surgeons. It may also help residents more quickly acquire competency in anatomical and pathological assessment of patient data [30, 55]. All three groups in the present study scored better in consecutive cases and significantly reduced the time to answer. The improvement in both speed and accuracy in all three modalities suggests a learning curve for all visualization methods. This is emphasized by the fact that even though patient 3 was deemed the most difficult patient case by the specialists, all three groups had improved their correctness score by patient 3. This adds to the argument that students should receive frequent training in such visualization systems, and demonstrates the need for continuous re-exposure to simulated or real patient cases as one of the most effective methods of learning in the medical/surgical environment.

Another argument for the marked improvement in the 3D and VR-groups may be not only because of the 3D modality, but also due to the fact that interacting with this medium is more intuitive and thus more enjoyable [56, 57]. Participants were most satisfied with VR and the VR-group was least likely to prefer a different visualization method, which again points to VR being an effective medium for keeping the subject engaged with the material, thus facilitating the learning process [40]. This satisfaction with VR training has been noted in a previous publication by Nickel et al. [58]. Pleasantness in both the 3D- and VR-group in the present study was rated very highly. However, for some users, motion sickness can be a problem arising with the use of the VR-glasses [59]. In studies examining this phenomenon, the incidence of motion sickness varies greatly and is dependent on a variety of factors, including individual susceptibility to motion sickness, duration of exposure, postural variation (standing/sitting), actions being performed, and visual motion stimulus, i.e. simulated displacement or simulated motion of the “own” virtual body [60]. The configuration least likely to cause motion sickness appears to be a short exposure time, sitting in place, and no visual motion stimuli, i.e. the virtual avatar remaining in place in concurrence with the user. As the IMHOTEP tool lacks any reason for the user to be exposed to postural variation or visual motion stimulus, the risk of motion sickness can be kept at a minimum. Variability between HMD systems has also been reported [61]. For wearers of glasses the use of VR-glasses may be awkward and make the experience less pleasant. This will likely be corrected by future improved designs of VR-glasses that accommodate for wearers of glasses.

One can argue that with better training of surgical liver anatomy and surgical decision-making, 2D radiological images might be evaluated better after a 3D/VR training period. Surgical topography can be difficult to present and visualize for novice surgeons and medical students, and the addition of 3D visualization of complex anatomy has been shown to improve learning speed and retention when compared to traditional 2D methods [20, 22, 24, 25, 62]. Surgical novices might, for example, improve their reading of CT-images if a correlation between the 3D-model and the 2D images is implemented in the software. Metzler et al. showed that training purely with 3D does not directly transfer to enhance the understanding of 2D CT-images in students [23]. However, the combination of conventional 2D images with simultaneous 3D or VR models may enable a better transfer of understanding. Further integration of conventional 2D-imaging into the IMHOTEP VR software, in order to tie it more closely to the 3D-model, is being planned and its effects will be evaluated in a future study.

The process of image segmentation is a bottleneck for 3D visualizations. In the present study, only open-source segmentation tools with semi-automatic algorithms were used for segmentation. The tools provided accurate results but the process of segmentation was time-consuming, amounting to several hours per patient case, and required board-certified surgeons and radiologists to verify the results. There are commercial segmentation services that can be used to create 3D-models from radiological images [63]. As these services continue to improve and become more commercially viable, VR integration of preoperative planning may find increasing relevance in the clinical setting.

The presented study only used standard clinical imaging modalities, freely downloadable open-source software and commercially available hardware thus enabling a cost effective and easy reproducibility. The imaging data used for this study was computed tomography images, which are generally available for surgical oncologic resection planning. The software used for creating and post-processing the segmentations and 3D-models is open-source software and can be downloaded freely (https://​www.​mitk.​org, www.​slicer.​org, www.​meshmixer.​com). The source code for the virtual reality visualization software IMHOTEP can be downloaded freely (http://​imhotep-medical.​org/​) and further developed as needed. A final consideration to be given to the VR environment is the aspect of telemedicine and telecommunication. As the years since the onset of the COVID-19 pandemic has demonstrated, there is an increasing demand for viable methods of digital communication at a distance, both at work and for social reasons [42, 64]. In light of the shift in many fields to remote work, and the evidence showing that this has not resulted in a loss of productivity, it appears likely that telework will remain attractive for many employers and employees even after cessation of current pandemic restrictions. Aside from the COVID-19 pandemic [44], the need for accurate telecommunication has been increasing in the medical field over the past decade [43, 65], as expert consultations e.g. in multicenter tumor boards become increasingly common, and VR solutions have been proposed for many medical applications, including remote bedside consultation, tumor board discussion and surgical planning and intraoperative guidance systems [66‐69]. Especially in complex cases such as surgical liver planning, the medium should allow for accurate communication and interpretation of information, even in a remote setting. In such cases, an interactive VR platform could allow for precise discussions, for example regarding tumor location and resection possibilities, with reduced risk for error.

The findings in the present study demonstrate that three-dimensional VR visualization is a valid and viable tool for teaching surgical liver anatomy. The VR environment was preferred over the other methods by the participants and it added more enjoyment to the learning process and may thus help create a better learning effect. VR and 3D display of patient anatomy is useful for training of liver surgery for surgical novices, enabling quicker and more accurate assessment of unique patient cases and allowing for improved surgical decision-making compared to 2D display.