A learning robot for cognitive camera control in minimally invasive surgery

The duration of the operation was influenced by an initial learning curve of the operating surgeon during human camera guidance (H1, operations 1–8). The learning curve was determined based on a distribution of durations with a maximum of 4265 s (operation 1) decreasing to 1465 s (operation 8) and an average duration of 2090 ± 919 s. After completion of this learning curve, the average duration of one operation procedure was 1325 s ± 144 s for human guidance (H2, operations 9–20). The duration of the operation supported by cognitive camera control decreased with increasing experience of the robot from L1 (1704 s ± 244 s) over L2 (1406 s ± 112 s) to L3 (1197 s), indicating a learning curve for the robot. Here, Levine’s test shows homogeneity of variances for experiments L1, L2 and L3. One-way ANOVA resulted in a significant difference between the individual experiments (F-value = 4.7, p = 0.045). However, the comparison between operation duration for L1 and L2 did not reach statistical significance (p = 0.1). The resulting durations for human and robotic camera guidance are summarized in Fig. 3A.

Camera guidance quality rated as “good”, “neutral”, “poor” by a surgical expert improved with increasing experience of the robot, such that a higher proportion of “good” and lower proportion of “poor” was observed from L1 over L2 to L3. Levine’s test showed homogeneity of variances for L1, L2 and L3 for both, “good” and “poor” camera guidance. One-way ANOVA showed a significant difference between the experiments (“good”: F-value = 5.52, p = 0.031; “poor”: F-value = 14.58, p = 0.002). However, for the proportion of “good” camera guidance the difference between experiments L1 and L2 did not reach statistical significance. The resulting camera guidance quality for human and robotic camera guidance is summarized in Fig. 3B.

For the proportion of cognitive camera control compared to manual or hands-on mode Levine’s test showed homogeneity of variances for L1, L2 and L3. One-way ANOVA showed a significant difference between the experiments (F-value = 34.2, p < 0.001). The proportion of cognitive camera control increased with increasing experience from L1 (62.4% ± 4.9%) to L2 (85.4% ± 4.2%, p < 0.001) and L3 (85.1%, p = 0.002). Accordingly, the number of times the surgeon had to change to manual or hands-on mode to allow for manual commands to the robot during surgery decreased from L1 (21.8 ± 3.7) to L2 (10.4 ± 2.1, p < 0.001), but not for L3 (14, p = 0.45). Levine’s test showed homogeneity of variances for L1, L2 and L3. One-way ANOVA showed a significant difference between L1, L2 and L3 (F-value = 18.3, p = 0.001).

Our primary achievement is the demonstration of a methodology that represents a paradigm shift from previously programmed robots not suitable to adapt to a surgeon’s needs to a new generation of cognitive robots that will be able to adapt to different surgeons, surgical situations and surgical procedures. Our methodology enables cognitive camera control for a learning robot in minimally invasive surgery. The methodology is applicable to various robot kinematics as demonstrated during experimental validation. Our experiments showed the robot’s ability to learn from human surgeons, from other robots, and on its own.

As early as 2008 the SAGES-MIRA Robotic Surgery Consensus Group envisioned that “Robots could use artificial intelligence to learn from the surgeon operating the device” [29]. In our study the robot positioned itself autonomously after having learned the desired spatial relation of surgical instruments and camera, depending on the phase of a surgical procedure. This autonomy of the robot to position the camera enabled the surgeon to focus on essential surgical tasks, without the need to guide the camera. If the camera position was unsatisfactory to the surgeon, he was able to correct it manually using one of different input methods (touch display or hands-on mode on the LWR). In contrast to previous systems, the robot extended its knowledge base and learned from this experience for future interventions. Thus, the robot changed its behavior over time and improved its performance with every additional procedure resulting in adaptation of the robot to the surgeon and not vice versa. Thereby, different behavioral patterns and individual preferences of various surgeons may be learned by the robot, presenting a matter for future investigation. In addition, due to the provided surgical process model, it was possible to train camera guidance classifiers tailored for each surgical step. The classifiers may also be tailored to the individual surgeon’s preferences over the course of several interventions. Based on the duration of procedures with human camera guidance, we can assume that the learning curve for the surgeon had been completed before initiating the robotic experiments. Thus, a difference in the overall performance of the team “surgeon plus robot” was likely to be caused by the improved robot performance. Here, we focused on the robot learning curve as the difference between the three experiments L1, L2 and L3. The resulting learning curve was characterized by decreasing duration of the surgery with higher proportion of “good” camera guidance quality and lower proportion of “poor” camera guidance quality. Furthermore, less manual corrective feedback was required for L2 and L3 compared to L1. Moreover, apart from the quantitative results the robot proved learning in a qualitative way as perceived by the surgeon. Initially, during several surgical steps, such as opening the lesser pelvic peritoneum, the zooming motion (i.e., moving the camera in or out) was “trained” explicitly, because during experiments Viky and L1 no zooming was conducted by the cognitive camera control. Instead, the robot was manually directed to zoom in when required by the surgeon. Subsequently, in experiments L2 and L3, the robot zoomed in autonomously as it had learned the necessity for this movement from data collected during Viky and L1.

Metrics to measure the quality of the camera guidance and the robot’s learning progress limit the evaluation of learning camera robots. Moreover, only few of these metrics can be utilized as a learning feedback for the robot. Still, feedback to the robot is of utmost importance as better feedback results in better surgical training of the robot [30, 31]. The objective metrics overall procedure time and setup time are a common standard for evaluation of surgical robots. These metrics can be calculated automatically by the robot’s software and are easily comparable (at least for simple surgical procedures). Their main disadvantage is the lack of specificity in the feedback for a learning robot. Thus, in our approach, duration of the surgical procedure was not implemented as a feedback for the robot, but only to measure and evaluate its performance. However, reduced procedure time may also result from the surgeon learning to work with the robot. Subjective metrics, such as questionnaires focusing on the surgeon’s user experience are crucial for user acceptance, and thus, the translation into practice. Unfortunately, most criteria are not well defined and the results of these criteria are not entirely representative for all surgeons as they strongly depend on personal preferences. Additionally, these criteria have to be obtained after the surgical intervention increasing the workload of the surgeon and feedback often is not detailed enough to improve the system’s performance in specific situations. To overcome some of these limitations, we used camera guidance quality as an additional metric. During experimentation, an additional surgical expert assessed the camera guidance quality, rendering it a direct subjective metric. Nevertheless, this metric has proven to be suitable as feedback to a learning robot. The disadvantage of rating the camera quality during experiments is the necessity of a surgical expert in addition to the surgeon, because annotation by the surgeon would interfere with the surgical workflow. Thus, the number of interactions of the surgeon with the system was logged and evaluated, indicating the surgeon’s satisfaction with the (autonomous) camera guidance with less interaction (or correction) representing better outcomes. In future investigations we would like to use these corrective commands directly as a feedback for the learning robot. This would also diminish the time-consuming annotation of camera guidance quality be an additional surgical expert.

Main limitation of our study is its limited number of participants. For this feasibility study a single surgeon performed all operations rendering the procedures standardized and easy to compare, especially for investigating the robot’s learning curve, at the expense of generalization of the results. Additionally, the robot’s capabilities were only evaluated during one surgical procedure. Even though laparoscopic rectal resection is a rather complex procedure, it has yet to be proven, whether the approach is applicable to other procedures (cholecystectomy, hernia repair, pancreatic surgery etc.). Finally, our study is limited to a medical phantom setuo awaiting translation into animal lab studies and subsequent clinical trials. Furthermore, although most surgeons prefer a 30°-optic for laparoscopic anterior rectal resection with total mesorectal excision, in our study we utilized a 0°-optic. Usage of a 30°-optic would result in an additional degree of freedom in camera control posing another major challenge for the machine learning algorithm. Thus, further engineering and computer science research are necessary to fulfill this surgical requirement in future work. In addition, our approach has to proof not only feasibility, but also advantages over previous approaches of visual servoing, i.e. simply following the instrument tip with the camera. Nevertheless, the present study not only demonstrates a completely novel cognitive approach to camera guidance robots and their application in a surgical setting, but also demonstrates the robot’s learning progress and improvement in performance.

Cognitive surgical robots may have a huge clinical impact in the future because of their ability to perceive their environment, interpret the situation based on a knowledge base, and act accordingly. These robots can learn and improve their performance by incorporating new experiences into their knowledge base adapting to different surgeons, different surgical procedures and different patient anatomies. Future research may focus on incorporation of existing or novel technologies, such as surgical phase detection and autonomous manipulation of tissue. In the present study, external cameras continuously track the instruments, however, it has previously been shown that instruments as well as organs can be tracked directly in the laparoscopic video by means of computer vision [32]. Whereas we defined the phase of the operation manually during the operation, research suggests that skilled surgeons can extract it from surgical device data alone [33] leaving room for further automated detection of the surgical phase. For simple procedures, such as cholecystectomy, the phase can even be automatically recovered from the video using deep learning algorithms [34] and procedure duration can be recovered from medical device data in real time [35]. Furthermore, reinforcement learning algorithms are a current trend to solve complex motion tasks in robotics. However, employing reinforcement learning algorithms, that require interaction between learning robot and environment, poses a severe safety issue in medical robotics. Clinical translation of the proposed learning camera guidance system may lay the foundation to gather high quality and quantity of labeled data to employ offline reinforcement learning algorithms to the task of robotic camera guidance to gradually learn and represent more complex behaviors [36]. In a recent review of machine learning techniques in surgical robotics Kassahun et al. describe a number of applications for learning surgical robots, such as tying a knot or steering of vascular catheters [37]. They conclude that machine learning may help to extract human skill and transfer it to surgical robots emphasizing the potential of cognitive surgical robots as demonstrated in our study. Furthermore, the concept of a robot learning from another robot’s experience that we demonstrated in our study, may help to transfer experience easily from one surgical center to another via data transfer instead of having to train the robot all over again.

In the future, cognitive surgical robots and automated systems may become the center of a fully integrated cognitive operating room. They will not only guide the camera or steer catheters, but may also expose tissue with the right force, cut in the correct surgical plane [38], and reconstruct anastomoses after resection [39]. Perception may not only be based on visual information in the laparoscopic image, but also on multimodal data from medical devices and vital parameters from patient monitoring. If the combination of these data with powerful machine learning algorithms continues as outlined in the concept of Surgical Data Science [40], within 15 years cognitive surgical robots will not only enhance human surgeons’ capabilities. They will likely perform autonomous tasks and may even perform simple surgical procedures autonomously, thus changing the face of surgery.