nach oben

Erschienen in:

04.01.2021 | Review Article

Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis

verfasst von: Roi Anteby, Nir Horesh, Shelly Soffer, Yaniv Zager, Yiftach Barash, Imri Amiel, Danny Rosin, Mordechai Gutman, Eyal Klang

Erschienen in: Surgical Endoscopy | Ausgabe 4/2021

Einloggen, um Zugang zu erhalten

Abstract

Background

In the past decade, deep learning has revolutionized medical image processing. This technique may advance laparoscopic surgery. Study objective was to evaluate whether deep learning networks accurately analyze videos of laparoscopic procedures.

Methods

Medline, Embase, IEEE Xplore, and the Web of science databases were searched from January 2012 to May 5, 2020. Selected studies tested a deep learning model, specifically convolutional neural networks, for video analysis of laparoscopic surgery. Study characteristics including the dataset source, type of operation, number of videos, and prediction application were compared. A random effects model was used for estimating pooled sensitivity and specificity of the computer algorithms. Summary receiver operating characteristic curves were calculated by the bivariate model of Reitsma.

Results

Thirty-two out of 508 studies identified met inclusion criteria. Applications included instrument recognition and detection (45%), phase recognition (20%), anatomy recognition and detection (15%), action recognition (13%), surgery time prediction (5%), and gauze recognition (3%). The most common tested procedures were cholecystectomy (51%) and gynecological—mainly hysterectomy and myomectomy (26%). A total of 3004 videos were analyzed. Publications in clinical journals increased in 2020 compared to bio-computational ones. Four studies provided enough data to construct 8 contingency tables, enabling calculation of test accuracy with a pooled sensitivity of 0.93 (95% CI 0.85–0.97) and specificity of 0.96 (95% CI 0.84–0.99). Yet, the majority of papers had a high risk of bias.

Conclusions

Deep learning research holds potential in laparoscopic surgery, but is limited in methodologies. Clinicians may advance AI in surgery, specifically by offering standardized visual databases and reporting.

Nur mit Berechtigung zugänglich

Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25:44–56CrossRef

Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp 1097–1105

Soffer S, Ben-Cohen A, Shimon O, Amitai MM, Greenspan H, Klang E (2019) Convolutional neural networks for radiologic images: a radiologist’s guide. Radiology 290:590–606CrossRef

De Fauw J, Ledsam JR, Romera-Paredes B, Nikolov S, Tomasev N, Blackwell S, Askham H, Glorot X, O’Donoghue B, Visentin D (2018) Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 24:1342–1350CrossRef

Milea D, Najjar RP, Zhubo J, Ting D, Vasseneix C, Xu X, Aghsaei Fard M, Fonseca P, Vanikieti K, Lagrèze WA (2020) Artificial intelligence to detect papilledema from ocular fundus photographs. N Engl J Med 382:1687–1695CrossRef

Brinker TJ, Hekler A, Enk AH, Berking C, Haferkamp S, Hauschild A, Weichenthal M, Klode J, Schadendorf D, Holland-Letz T (2019) Deep neural networks are superior to dermatologists in melanoma image classification. Eur J Cancer 119:11–17CrossRef

Soffer S, Klang E, Shimon O, Nachmias N, Eliakim R, Ben-Horin S, Kopylov U, Barash Y (2020) Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis. Gastrointest Endosc 92:831–839CrossRef

Hashimoto DA, Rosman G, Rus D, Meireles OR (2018) Artificial intelligence in surgery: promises and perils. Ann Surg 268:70–76CrossRef

McInnes MD, Moher D, Thombs BD, McGrath TA, Bossuyt PM, Clifford T, JrmF C, Deeks JJ, Gatsonis C, Hooft L (2018) Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA 319:388–396CrossRef

10.

Moons KG, de Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, Reitsma JB, Collins GS (2014) Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med 11:e1001744CrossRef

11.

Luo W, Phung D, Tran T, Gupta S, Rana S, Karmakar C, Shilton A, Yearwood J, Dimitrova N, Ho TB, Venkatesh S, Berk M (2016) Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res 18:e323CrossRef

12.

Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MM, Sterne JA, Bossuyt PM (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155:529–536CrossRef

13.

Fleuren LM, Klausch TL, Zwager CL, Schoonmade LJ, Guo T, Roggeveen LF, Swart EL, Girbes AR, Thoral P, Ercole A (2019) Machine learning for the prediction of sepsis, a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med 46:383–400CrossRef

14.

Kwong MT, Colopy GW, Weber AM, Ercole A, Bergmann JH (2019) The efficacy and effectiveness of machine learning for weaning in mechanically ventilated patients at the intensive care unit: a systematic review. Bio-Design Manuf 2:31–40CrossRef

15.

Islam MS, Hasan MM, Wang X, Germack HD (2018) A systematic review on healthcare analytics: application and theoretical perspective of data mining. Healthcare 6(2):54CrossRef

16.

de la Fuente LE, Muñoz García Á, Santos Del Blanco L, Fraile Marinero JC, Pérez Turiel J (2020) Automatic gauze tracking in laparoscopic surgery using image texture analysis. Comput Methods Programs Biomed 190:105378CrossRef

17.

Kletz S, Schoeffmann K, Husslein H (2019) Learning the representation of instrument images in laparoscopy videos. Healthc Technol Lett 6:197–203CrossRef

18.

Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH (2005) Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 58:982–990CrossRef

19.

Macaskill P, Gatsonis C, Deeks J, Harbord R, Takwoingi Y (2010) Cochrane handbook for systematic reviews of diagnostic test accuracy. Version 09 0. The Cochrane Collaboration, London

20.

Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C (2019) A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digital Health 1:e271–e297CrossRef

21.

Ioannidis JP, Patsopoulos NA, Evangelou E (2007) Uncertainty in heterogeneity estimates in meta-analyses. BMJ 335:914–916CrossRef

22.

Kletz S, Schoeffmann K, Benois-Pineau J, Husslein H (2019) Identifying surgical instruments in laparoscopy using deep learning instance segmentation. In: 2019 International Conference on Content-Based Multimedia Indexing (CBMI), pp 1–6

23.

Twinanda AP, Shehata S, Mutter D, Marescaux J, Mathelin Md, Padoy N (2017) EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36:86–97CrossRef

24.

Leibetseder A, Petscharnig S, Primus MJ, Kletz S, Münzer B, Schoeffmann K, Keckstein J (2018) Lapgyn4: a dataset for 4 automatic content analysis problems in the domain of laparoscopic gynecology. In: Proceedings of the 9th ACM Multimedia Systems Conference, pp 357–362

25.

Nwoye CI, Mutter D, Marescaux J, Padoy N (2019) Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos. Int J Comput Assist Radiol Surg 14:1059–1067CrossRef

26.

Jin A, Yeung S, Jopling J, Krause J, Azagury D, Milstein A, Li FF (2018) Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision, pp 691–699

27.

Hu XW, Yu LQ, Chen H, Qin J, Heng PA (2017) AGNet: Attention-guided network for surgical tool presence detection. deep learning in medical image analysis and multimodal learning for clinical decision support, pp 186–194

28.

Mishra K, Sathish R, Sheet D (2017) Learning latent temporal connectionism of deep residual visual abstractions for identifying surgical tools in laparoscopy procedures. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 2233–2240

29.

Varytimidis C, Rapantzikos K, Loukas C, Kollias S (2016) Surgical video retrieval using deep neural networks

30.

Wang S, Raju A, Huang J (2017) Deep learning based multi-label classification for surgical tool presence detection in laparoscopic videos. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pp 620–623

31.

Yamazaki Y, Kanaji S, Matsuda T, Oshikiri T, Nakamura T, Suzuki S, Hiasa Y, Otake Y, Sato Y, Kakeji Y (2020) Automated surgical instrument detection from laparoscopic gastrectomy video images using an open source convolutional neural network platform. J Am Coll Surg 230:725-732.e721CrossRef

32.

Zhang B, Wang S, Dong L, Chen P (2020) Surgical tools detection based on modulated anchoring network in laparoscopic videos. IEEE Access 8:23748–23758CrossRef

33.

Choi B, Jo K, Choi S, Choi J (2017) Surgical-tools detection based on Convolutional Neural Network in laparoscopic robot-assisted surgery. Conf Proc IEEE Eng Med Biol Soc 2017:1756–1759

34.

Madad Zadeh S, Francois T, Calvet L, Chauvet P, Canis M, Bartoli A, Bourdel N (2020) SurgAI: deep learning for computerized laparoscopic image understanding in gynaecology. Surg Endosc 34:5377–5383CrossRef

35.

Sahu M, Mukhopadhyay A, Szengel A, Zachow S (2017) Addressing multi-label imbalance problem of surgical tool detection using CNN. Int J Comput Assist Radiol Surg 12:1013–1020CrossRef

36.

Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv preprint arXiv:180402767

37.

Hashimoto DA, Rosman G, Witkowski ER, Stafford C, Navarette-Welton AJ, Rattner DW, Lillemoe KD, Rus DL, Meireles OR (2019) Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann Surg 270:414–421CrossRef

38.

Chittajallu DR, Dong B, Tunison P, Collins R, Wells K, Fleshman J, Sankaranarayanan G, Schwaitzberg S, Cavuoto L, Enquobahrie A (2019) XAI-CBIR: Explainable AI system for content based retrieval of video frames from minimally invasive surgery videos. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp 66–69

39.

Jalal NA, Alshirbaji TA, Möller K (2019) Predicting surgical phases using CNN-NARX neural network. Biomed Tech 64:S188

40.

Kitaguchi D, Takeshita N, Matsuzaki H, Takano H, Owada Y, Enomoto T, Oda T, Miura H, Yamanashi T, Watanabe M, Sato D, Sugomori Y, Hara S, Ito M (2019) Real-time automatic surgical phase recognition in laparoscopic sigmoidectomy using the convolutional neural network-based deep learning approach. Surg Endosc 34:4924–4931CrossRef

41.

Loukas C (2018) Surgical phase recognition of short video shots based on temporal modeling of deep features. arXiv preprint arXiv:180707853

42.

Chen Y, Tang P, Zhong K, Han L, Qi B, Sun Q (2019) Semi-supervised surgical workflow recognition based on convolution neural network. Basic Clin Pharmacol Toxicol 124:52

43.

Jalal NA, Alshirbaji TA, Möller K (2018) Evaluating convolutional neural network and hidden markov model for recognising surgical phases in sigmoid resection. Biomed Tech 63:S251

44.

Petscharnig S, Schöffmann K, Benois-Pineau J, Chaabouni S, Keckstein J (2018) Early and late fusion of temporal information for classification of surgical actions in laparoscopic gynecology. In: 2018 IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS), pp 369–374

45.

Tokuyasu T, Iwashita Y, Matsunobu Y, Kamiyama T, Ishikake M, Sakaguchi S, Ebe K, Tada K, Endo Y, Etoh T, Nakashima M, Inomata M (2020) Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg Endosc. https://doi.org/10.1007/s00464-020-07548-xCrossRefPubMed

46.

Harangi B, Hajdu A, Lampe R, Torok P (2017) Recognizing ureter and uterine artery in endoscopic images using a convolutional neural network. In: 2017 IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS), pp 726–727

47.

Gibson E, Robu MR, Thompson S, Edwards E, Schneider C, Gurusamy K, Davidson B, Hawkesa DJ, Barratta DC, Clarkson MJ (2017) Deep residual networks for automatic segmentation of laparoscopic videos of the liver. Medical Imaging 2017: Image-Guided Procedures, Robotic Interventions, and Modeling

48.

Chittajallu DR, Basharat A, Tunison P, Horvath S, Wells KO, Leeds SG, Fleshman JW, Sankaranarayanan G, Enquobahrie A (2019) content based retrieval of video segments from minimally invasive surgery videos using deep convolutional video descriptors and iterative query refinement. Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling

49.

Müenzer B, Primus MJ, Kletz S, Petscharnig S, Schoeffmann K (2017) Static vs. dynamic content descriptors for video retrieval in laparoscopy. In: 2017 IEEE International Symposium on Multimedia (ISM), pp 216–223

50.

Petscharnig S, Schoffmann K (2018) Learning laparoscopic video shot classification for gynecological surgery. Multimedia Tools Appl 77:8061–8079CrossRef

51.

Twinanda AP, Yengera G, Mutter D, Marescaux J, Padoy N (2019) RSDNet: Learning to predict remaining surgery duration from laparoscopic videos without manual annotations. IEEE Trans Med Imaging 38:1069–1078CrossRef

52.

Bodenstedt S, Wagner M, Mündermann L, Kenngott H, Müller-Stich B, Breucha M, Mees ST, Weitz J, Speidel S (2019) Prediction of laparoscopic procedure duration using unlabeled, multimodal sensor data. Int J Comput Assist Radiol Surg 14:1089–1095CrossRef

53.

Fleuren LM, Klausch TL, Zwager CL, Schoonmade LJ, Guo T, Roggeveen LF, Swart EL, Girbes AR, Thoral P, Ercole A (2020) Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med 46:1–18

54.

Krittanawong C, Virk HUH, Bangalore S, Wang Z, Johnson KW, Pinotti R, Zhang H, Kaplin S, Narasimhan B, Kitai T (2020) Machine learning prediction in cardiovascular diseases: a meta-analysis. Sci Rep 10:1–11CrossRef

55.

Maier-Hein L, Wagner M, Ross T, Reinke A, Bodenstedt S, Full PM, Hempe H, Mindroc-Filimon D, Scholz P, Tran TN (2020) Heidelberg colorectal data set for surgical data science in the sensor operating room. arXiv preprint arXiv:200503501

56.

Stauder R, Ostler D, Kranzfelder M, Koller S, Feußner H, Navab N (2016) The TUM LapChole dataset for the M2CAI 2016 workflow challenge. arXiv preprint arXiv:161009278

57.

Stauder R, Okur A, Peter L, Schneider A, Kranzfelder M, Feussner H, Navab N (2014) Random forests for phase detection in surgical workflow analysis. In: International Conference on Information Processing in Computer-Assisted Interventions, Springer, pp 148–157

Titel: Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis
verfasst von: Roi Anteby
Nir Horesh
Shelly Soffer
Yaniv Zager
Yiftach Barash
Imri Amiel
Danny Rosin
Mordechai Gutman
Eyal Klang
Publikationsdatum: 04.01.2021
Verlag: Springer US
Erschienen in: Surgical Endoscopy / Ausgabe 4/2021
Print ISSN: 0930-2794
Elektronische ISSN: 1432-2218
DOI: https://doi.org/10.1007/s00464-020-08168-1

Neu im Fachgebiet Chirurgie

Mehr Frauen im OP – weniger postoperative Komplikationen

21.05.2024 Allgemeine Chirurgie Nachrichten

Ein Frauenanteil von mindestens einem Drittel im ärztlichen Op.-Team war in einer großen retrospektiven Studie aus Kanada mit einer signifikanten Reduktion der postoperativen Morbidität assoziiert.

„Übersichtlicher Wegweiser“: Lauterbachs umstrittener Klinik-Atlas ist online

17.05.2024 Klinik aktuell Nachrichten

Sie sei „ethisch geboten“, meint Gesundheitsminister Karl Lauterbach: mehr Transparenz über die Qualität von Klinikbehandlungen. Um sie abzubilden, lässt er gegen den Widerstand vieler Länder einen virtuellen Klinik-Atlas freischalten.

Was nützt die Kraniektomie bei schwerer tiefer Hirnblutung?

17.05.2024 Hirnblutung Nachrichten

Eine Studie zum Nutzen der druckentlastenden Kraniektomie nach schwerer tiefer supratentorieller Hirnblutung deutet einen Nutzen der Operation an. Für überlebende Patienten ist das dennoch nur eine bedingt gute Nachricht.

Klinikreform soll zehntausende Menschenleben retten

15.05.2024 Klinik aktuell Nachrichten

Gesundheitsminister Lauterbach hat die vom Bundeskabinett beschlossene Klinikreform verteidigt. Kritik an den Plänen kommt vom Marburger Bund. Und in den Ländern wird über den Gang zum Vermittlungsausschuss spekuliert.

Update Chirurgie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.

Newsletter bestellen

Aktuelle BDC Leitlinien-Webinare

S3-Leitlinie „Diagnostik und Therapie des Karpaltunnelsyndroms“

Karpaltunnelsyndrom BDC Leitlinien Webinare

CME: 2 Punkte

Das Karpaltunnelsyndrom ist die häufigste Kompressionsneuropathie peripherer Nerven. Obwohl die Anamnese mit dem nächtlichen Einschlafen der Hand (Brachialgia parästhetica nocturna) sehr typisch ist, ist eine klinisch-neurologische Untersuchung und Elektroneurografie in manchen Fällen auch eine Neurosonografie erforderlich. Im Anfangsstadium sind konservative Maßnahmen (Handgelenksschiene, Ergotherapie) empfehlenswert. Bei nicht Ansprechen der konservativen Therapie oder Auftreten von neurologischen Ausfällen ist eine Dekompression des N. medianus am Karpaltunnel indiziert.

Prof. Dr. med. Gregor Antoniadis

S2e-Leitlinie „Distale Radiusfraktur“

Radiusfraktur BDC Leitlinien Webinare

CME: 2 Punkte

Das Webinar beschäftigt sich mit Fragen und Antworten zu Diagnostik und Klassifikation sowie Möglichkeiten des Ausschlusses von Zusatzverletzungen. Die Referenten erläutern, welche Frakturen konservativ behandelt werden können und wie. Das Webinar beantwortet die Frage nach aktuellen operativen Therapiekonzepten: Welcher Zugang, welches Osteosynthesematerial? Auf was muss bei der Nachbehandlung der distalen Radiusfraktur geachtet werden?

PD Dr. med. Oliver Pieske

Dr. med. Benjamin Meyknecht

S1-Leitlinie „Empfehlungen zur Therapie der akuten Appendizitis bei Erwachsenen“

Appendizitis BDC Leitlinien Webinare

CME: 2 Punkte

Inhalte des Webinars zur S1-Leitlinie „Empfehlungen zur Therapie der akuten Appendizitis bei Erwachsenen“ sind die Darstellung des Projektes und des Erstellungswegs zur S1-Leitlinie, die Erläuterung der klinischen Relevanz der Klassifikation EAES 2015, die wissenschaftliche Begründung der wichtigsten Empfehlungen und die Darstellung stadiengerechter Therapieoptionen.

Dr. med. Mihailo Andric

Die Highlights vom Kongress des American College of Cardiology 2024

Springer Medizin

Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis