Laparoscopic liver resection (LLR) reduces pain and complications resulting in shorter hospital stay with comparable oncological outcomes to open liver resection [
1‐
4]. Uptake of the laparoscopic approach has been slow [
3,
5] but is progressing [
4] with most HPB centres carrying out at least minor liver resections laparoscopically whilst only a few centres perform major hepatectomies or complex liver resections (e.g. superior-posterior segments), laparoscopically [
4,
5]. Expansion of laparoscopic liver surgery is slowed by inherent limitations to depth perception, tactile feedback and field of view which are compounded by the livers varied and complex anatomy [
6,
7]. These limitations have given rise to concern over controlling bleeding and ensuring adequate oncological clearance [
3‐
5,
8‐
10].
Laparoscopic ultrasound, an alternative technique for operative imaging is limited by its two-dimensionality and poor contrast between tumours and normal liver [
7,
18‐
20]. Video-based IGS using augmented reality (AR) can superimposes a 3D liver model directly onto the laparoscopic screen [
21,
22]. Generally application of these systems requires three key steps; the creation of a personalised 3D liver model from a preoperative CT or MRI scan, intraoperative image registration and tracking of the laparoscope to guide the image overlay.
Two commercial IGS designed for open liver surgery [
23,
24], have been adapted for LLR with studies demonstrating comparable accuracy to open surgery [
7,
22]. These systems however are limited by the need for separate screens to demonstrate image guidance [
7] and the use of manual registration [
7,
22] which is a source of errors and delay to the intraoperative workflow [
25]. To address these issues an IGS is being developed with capabilities for AR and semi-automatic registration [
26,
27]. These features may improve the performance and usability of navigated image guidance. The current study tests the feasibility of using the new IGS, Smart Liver, in a clinical setting and is the first study to compare manual with semi-automatic registration [
28].
Discussion
This study has described the development and current performance of the SmartLiver IGS. Focus was on feasibility as opposed to clinical impact because at the outset, navigation accuracy was unknown and therefore no ethical approval was sought to use SmartLiver to adjust surgical strategy. Evaluation was carried out on 18 patients undergoing either LLR or staging laparoscopy. There were no patient safety incidents associated with the use of SmartLiver and perioperative outcomes for patients undergoing LLR were similar to previous reports [
4,
36,
37]. Although IGS are widely used in neurosurgery, orthopaedic surgery and otolaryngology, implementation in LLR has been slow and difficult [
38]. Major challenges are the lack of fixed bony landmarks, paucity of liver surface features, organ motion secondary to diaphragmatic and cardiac movement as well as soft tissue deformation due pneumoperitoneum and surgical manipulation [
7,
38,
39].
The experimental work leading up to this study demonstrated that liver motion and deformation, contribute approximately 7.5 mm to the TRE of SmartLiver [
26,
39,
40]. To achieve a greater level of accuracy requires a deformable 3D model that can adjust its shape and position to reflect intraoperative changes [
26]. The research community has attempted to develop deformable 3D models with varying degrees of success. Because modelling of soft tissue deformation is exceedingly complex and computationally expensive, this technology has not yet reached sufficient maturity for clinical studies [
41,
42].
The primary endpoint of successful registration as assessed by the operating surgeon was achieved in 16 out of 18 patients. Success was indicated by the 3D model maintaining an anatomically appropriate and stable position. It has been previously reported that liver mobilisation results in significant positional shift of landmarks and therefore necessitates repeat registration [
43]. Although not formally quantified, this was also the case in the current study. Hypothetically image guidance should be most beneficial during dissection at the liver hilum and parenchymal transection since the exact position of intrahepatic structures and tumours are crucial during these steps. Liver mobilisation is a standardised process and therefore registration and image guidance may be less important at this stage. Although re-registration issues affect all current IGS, we strongly believe that semi-automatic registration renders this process less cumbersome.
During the first study phase, registration was carried out postoperatively which meant that surgeons could not evaluate the quality AR visualisation. Feedback about equipment handling was positive whereas negative feedback mainly centred on the complexity of the intraoperative setup (e.g. tracker installation) and its impact on surgical workflow. To simplify the setup process, our group developed the crosshair calibration method and a graphic user interface [
29]. In the second study phase, AR visualisation was evaluated intraoperatively. Positive feedback points were that SmartLiver improved intraoperative orientation, aided in the detection of extrahepatic structures and was consistent in the way it displayed anatomy (Table
2). Feedback about the combination of SmartLiver with laparoscopic ultrasound was less favourable, because viewing both, the AR and ultrasound -screen simultaneously was challenging. Our group previously demonstrated how SmartLiver can effectively integrate ultrasound images into AR [
44]. This approach however requires electromagnetically tracked ultrasound, which was not ethically approved for this study. The anatomical precision of the overlay also received negative feedback which probably reflects the fact that there were obvious discrepancies between 3D model position and corresponding liver sections in 4 patients with a higher than average TRE. To improve anatomical precision, efforts were increased to improve manual registration accuracy. In contrast to our groups experience from porcine studies [
26], it was observed that, in patients stereoscopic surface reconstruction may misalign different anatomical regions (e.g. diaphragm with liver), if they have a similar surface structure. Hypothetically the coarser, more lobulated surface of the porcine liver may be more amenable to stereoscopic surface reconstruction because it contains more features to distinguish it from surrounding structures. As demonstrated on this data set, stereoscopic surface reconstruction for the purpose of semi-automatic registration of the human liver is feasible if the liver surface is automatically segmented prior to registration [
35,
45]. To the best of our knowledge this is the first clinical study to compare accuracy of manual and semi-automatic registration in a group of patients. Although the accuracy for manual registration was better than for semi-automatic registration, this did not reach statistical significance. Accuracy of manual registration is comparable to that from other groups previously published in the literature [
22,
46,
47]. Various methodologies for accuracy evaluation have been proposed over time which makes direct comparison between different IGS challenging [
34]. Generally the best published accuracies for video-based IGS are in the range of 10 mm and thus at the current state of the art, any IGS should perhaps be considered as an orientation aid rather than a precise navigation tool [
18,
23,
24]. The utility for visualising intrahepatic structures depended mainly on the quality of the registration. In patients where accuracy ≤ 10 mm was achieved, it was feasible to approximate the position of sectoral branches (e.g. right anterior sector) and major hepatic vein branches. To maximise the potential of IGS it is important to enable smooth integration into the surgical workflow. The main benefit of AR is the intuitive use of image guidance information by obviating the need for two separate screens, therefore reducing the potential for associated errors [
14]. Pending further validation, semi-automatic registration could improve user friendliness and render accuracy less operator dependent compared to manual registration [
48]. Indeed, time efficiency and operator dependence may be crucial advantages of IGS compared to laparoscopic ultrasound. AR visualisation can be switched on and off within seconds whereas ultrasound requires insertion of a laparoscopic probe and manual scanning of the liver surface. Precise use of laparoscopic ultrasound is heavily operator dependent and has a steep learning curve [
49,
50], whereas surgeon feedback indicates that SmartLiver’s AR is easy to mentally integrate.
Although the results from this study have demonstrated the technical feasibility of using SmartLiver intraoperatively, there are some limitations that have to be taken into account. To confirm that the accuracy of semi-automatic registration is non-inferior to manual registration, validation on a larger patient cohort is required. In the second study phase manual registration accuracy was comparable to that of other systems [
22,
46,
47]. Since this is the first clinical report on semi-automatic registration in a clinical series, there are no published data to compare our results to.
It is often criticised that liver surface features may be an inadequate representation of intrahepatic anatomy. Our group and others however recently demonstrated that liver surface landmarks have a good correlation with the anatomical location of intrahepatic structures (e.g. blood vessels) [
21,
51]. Simultaneous localisation and mapping (SLAM) and 3D pose estimation are alternative approaches to image guidance that do not require tracking and therefore may reduce complexity of IGS setup and handling. It remains to be seen however if these technologies can be successfully applied to an intraoperative, clinical setting [
15,
52]. Other groups have demonstrated the feasibility of image-guided laparoscopic liver ablation [
19,
53]. Hypothetically, SmartLiver has the potential to provide image guidance for laparoscopic liver ablation as well but at present further evaluation is required to verify this. In principal IGS can be applied to robotic assisted surgery [
54] without requiring significant alterations. Again our group has not explored this option yet because the main focus is on improving SmartLiver’s performance for LLR first.
In summary, this article has described the clinical development and usability a novel IGS for laparoscopic surgery. For the first time, accuracy metrics for manual and semi-automatic registration have been compared in a clinical series. The next stage of system development will focus on improving SmartLiver’s setup process and to explore alternative methods of semi-automatic registration.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.