Skip to main content
Erschienen in: Surgical Endoscopy 1/2021

01.01.2021

Development and validation of machine learning models to predict gastrointestinal leak and venous thromboembolism after weight loss surgery: an analysis of the MBSAQIP database

verfasst von: Jacob Nudel, Andrew M. Bishara, Susanna W. L. de Geus, Prasad Patil, Jayakanth Srinivasan, Donald T. Hess, Jonathan Woodson

Erschienen in: Surgical Endoscopy | Ausgabe 1/2021

Einloggen, um Zugang zu erhalten

Abstract

Background

Postoperative gastrointestinal leak and venous thromboembolism (VTE) are devastating complications of bariatric surgery. The performance of currently available predictive models for these complications remains wanting, while machine learning has shown promise to improve on traditional modeling approaches. The purpose of this study was to compare the ability of two machine learning strategies, artificial neural networks (ANNs), and gradient boosting machines (XGBs) to conventional models using logistic regression (LR) in predicting leak and VTE after bariatric surgery.

Methods

ANN, XGB, and LR prediction models for leak and VTE among adults undergoing initial elective weight loss surgery were trained and validated using preoperative data from 2015 to 2017 from Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program database. Data were randomly split into training, validation, and testing populations. Model performance was measured by the area under the receiver operating characteristic curve (AUC) on the testing data for each model.

Results

The study cohort contained 436,807 patients. The incidences of leak and VTE were 0.70% and 0.46%. ANN (AUC 0.75, 95% CI 0.73–0.78) was the best-performing model for predicting leak, followed by XGB (AUC 0.70, 95% CI 0.68–0.72) and then LR (AUC 0.63, 95% CI 0.61–0.65, p < 0.001 for all comparisons). In detecting VTE, ANN, and XGB, LR achieved similar AUCs of 0.65 (95% CI 0.63–0.68), 0.67 (95% CI 0.64–0.70), and 0.64 (95% CI 0.61–0.66), respectively; the performance difference between XGB and LR was statistically significant (p = 0.001).

Conclusions

ANN and XGB outperformed traditional LR in predicting leak. These results suggest that ML has the potential to improve risk stratification for bariatric surgery, especially as techniques to extract more granular data from medical records improve. Further studies investigating the merits of machine learning to improve patient selection and risk management in bariatric surgery are warranted.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Nguyen NT, Varela JE (2017) Bariatric surgery for obesity and metabolic disorders: state of the art. Nat Rev Gastroenterol Hepatol 14:160CrossRef Nguyen NT, Varela JE (2017) Bariatric surgery for obesity and metabolic disorders: state of the art. Nat Rev Gastroenterol Hepatol 14:160CrossRef
3.
Zurück zum Zitat Fry BT, Scally CP, Thumma JR, Dimick JB (2018) Quality improvement in bariatric surgery: the impact of reducing postoperative complications on medicare payments. Ann Surg 268:22–27CrossRef Fry BT, Scally CP, Thumma JR, Dimick JB (2018) Quality improvement in bariatric surgery: the impact of reducing postoperative complications on medicare payments. Ann Surg 268:22–27CrossRef
4.
Zurück zum Zitat Funk LM, Jolles S, Fischer LE, Voils CI (2015) Patient and referring practitioner characteristics associated with the likelihood of undergoing bariatric surgery: a systematic review. JAMA Surg 150:999–1005CrossRef Funk LM, Jolles S, Fischer LE, Voils CI (2015) Patient and referring practitioner characteristics associated with the likelihood of undergoing bariatric surgery: a systematic review. JAMA Surg 150:999–1005CrossRef
5.
Zurück zum Zitat Funk LM, Jolles SA, Greenberg CC, Schwarze ML, Safdar N, McVay MA, Whittle JC, Maciejewski ML, Voils CI (2016) Primary care physician decision making regarding severe obesity treatment and bariatric surgery: a qualitative study. Surg Obes Relat Dis 12:893–901CrossRef Funk LM, Jolles SA, Greenberg CC, Schwarze ML, Safdar N, McVay MA, Whittle JC, Maciejewski ML, Voils CI (2016) Primary care physician decision making regarding severe obesity treatment and bariatric surgery: a qualitative study. Surg Obes Relat Dis 12:893–901CrossRef
6.
Zurück zum Zitat Vidal J, Corcelles R, Jiménez A, Flores L, Lacy AM (2017) Metabolic and bariatric surgery for obesity. Gastroenterology 152:1780–1790CrossRef Vidal J, Corcelles R, Jiménez A, Flores L, Lacy AM (2017) Metabolic and bariatric surgery for obesity. Gastroenterology 152:1780–1790CrossRef
8.
Zurück zum Zitat Mocanu V, Dang J, Ladak F, Switzer N, Birch DW, Karmali S (2019) Predictors and outcomes of leak after Roux-en-Y Gastric Bypass: an analysis of the MBSAQIP data registry. Surg Obes Relat Dis 15:396–403CrossRef Mocanu V, Dang J, Ladak F, Switzer N, Birch DW, Karmali S (2019) Predictors and outcomes of leak after Roux-en-Y Gastric Bypass: an analysis of the MBSAQIP data registry. Surg Obes Relat Dis 15:396–403CrossRef
9.
Zurück zum Zitat Turrentine FE, Denlinger CE, Simpson VB, Garwood RA, Guerlain S, Agrawal A, Friel CM, LaPar DJ, Stukenborg GJ, Jones RS (2015) Morbidity, mortality, cost, and survival estimates of gastrointestinal anastomotic leaks. J Am Coll Surg 220:195–206CrossRef Turrentine FE, Denlinger CE, Simpson VB, Garwood RA, Guerlain S, Agrawal A, Friel CM, LaPar DJ, Stukenborg GJ, Jones RS (2015) Morbidity, mortality, cost, and survival estimates of gastrointestinal anastomotic leaks. J Am Coll Surg 220:195–206CrossRef
10.
Zurück zum Zitat Ward-Smith P (2012) Body mass index, surgery, and risk of venous thromboembolism in middle-aged women. Urol Nurs 32:220–223CrossRef Ward-Smith P (2012) Body mass index, surgery, and risk of venous thromboembolism in middle-aged women. Urol Nurs 32:220–223CrossRef
11.
Zurück zum Zitat Klovaite J, Benn M, Nordestgaard BG (2015) Obesity as a causal risk factor for deep venous thrombosis: a M endelian randomization study. J Intern Med 277:573–584CrossRef Klovaite J, Benn M, Nordestgaard BG (2015) Obesity as a causal risk factor for deep venous thrombosis: a M endelian randomization study. J Intern Med 277:573–584CrossRef
13.
Zurück zum Zitat Aminian A, Andalib A, Khorgami Z, Cetin D, Burguera B, Bartholomew J, Brethauer SA, Schauer PR (2017) Who should get extended thromboprophylaxis after bariatric surgery. Ann Surg 265:143–150CrossRef Aminian A, Andalib A, Khorgami Z, Cetin D, Burguera B, Bartholomew J, Brethauer SA, Schauer PR (2017) Who should get extended thromboprophylaxis after bariatric surgery. Ann Surg 265:143–150CrossRef
15.
Zurück zum Zitat Gaborit B, Aron-Wisnewsky J, Salem J-E, Bege T, Frere C (2018) Pharmacologic venous thromboprophylaxis after bariatric surgery. Ann Surg 268:e51–e52CrossRef Gaborit B, Aron-Wisnewsky J, Salem J-E, Bege T, Frere C (2018) Pharmacologic venous thromboprophylaxis after bariatric surgery. Ann Surg 268:e51–e52CrossRef
16.
Zurück zum Zitat Thereaux J, Lesuffleur T, Czernichow S, Basdevant A, Msika S, Nocca D, Millat B, Fagot-Campagna A (2018) To what extent does posthospital discharge chemoprophylaxis prevent venous thromboembolism after bariatric Surgery? Results from a nationwide cohort of more than 110,000 patients. Ann Surg 267:727–733CrossRef Thereaux J, Lesuffleur T, Czernichow S, Basdevant A, Msika S, Nocca D, Millat B, Fagot-Campagna A (2018) To what extent does posthospital discharge chemoprophylaxis prevent venous thromboembolism after bariatric Surgery? Results from a nationwide cohort of more than 110,000 patients. Ann Surg 267:727–733CrossRef
19.
Zurück zum Zitat Finks JF, English WJ, Carlin AM, Krause KR, Share DA, Banerjee M, Birkmeyer JD, Birkmeyer NJ, Collaborative MBS (2012) Predicting risk for venous thromboembolism with bariatric surgery: results from the Michigan Bariatric Surgery Collaborative. Ann Surg 255:1100–1104CrossRef Finks JF, English WJ, Carlin AM, Krause KR, Share DA, Banerjee M, Birkmeyer JD, Birkmeyer NJ, Collaborative MBS (2012) Predicting risk for venous thromboembolism with bariatric surgery: results from the Michigan Bariatric Surgery Collaborative. Ann Surg 255:1100–1104CrossRef
20.
Zurück zum Zitat Hashimoto DA, Rosman G, Rus D, Meireles OR (2018) Artificial intelligence in surgery: promises and perils. Ann Surg 268:70–76CrossRef Hashimoto DA, Rosman G, Rus D, Meireles OR (2018) Artificial intelligence in surgery: promises and perils. Ann Surg 268:70–76CrossRef
21.
Zurück zum Zitat Kim JS, Merrill RK, Arvind V, Kaji D, Pasik SD, Nwachukwu CC, Vargas L, Osman NS, Oermann EK, Caridi JM (2018) Examining the ability of artificial neural networks machine learning models to accurately predict complications following posterior lumbar spine fusion. Spine 43:853–860CrossRef Kim JS, Merrill RK, Arvind V, Kaji D, Pasik SD, Nwachukwu CC, Vargas L, Osman NS, Oermann EK, Caridi JM (2018) Examining the ability of artificial neural networks machine learning models to accurately predict complications following posterior lumbar spine fusion. Spine 43:853–860CrossRef
26.
Zurück zum Zitat Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. pp 785–794; Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. pp 785–794;
28.
Zurück zum Zitat Pavlou M, Ambler G, Seaman SR, Guttmann O, Elliott P, King M, Omar RZ (2015) How to develop a more accurate risk prediction model when there are few events. BMJ 351:h3868CrossRef Pavlou M, Ambler G, Seaman SR, Guttmann O, Elliott P, King M, Omar RZ (2015) How to develop a more accurate risk prediction model when there are few events. BMJ 351:h3868CrossRef
29.
Zurück zum Zitat Lemaître G, Nogueira F, Aridas CK (2017) Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J Mach Learn Res 18:559–563 Lemaître G, Nogueira F, Aridas CK (2017) Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J Mach Learn Res 18:559–563
30.
Zurück zum Zitat Millman KJ, Aivazis M (2011) Python for scientists and engineers. Comput Sci Eng 13:9–12CrossRef Millman KJ, Aivazis M (2011) Python for scientists and engineers. Comput Sci Eng 13:9–12CrossRef
31.
Zurück zum Zitat Oliphant TE (2007) Python for scientific computing. Comput Sci Eng 9:10–20CrossRef Oliphant TE (2007) Python for scientific computing. Comput Sci Eng 9:10–20CrossRef
33.
Zurück zum Zitat McKinney W (2010) Data structures for statistical computing in python. In: Proceedings of the 9th Python in Science Conference, vol 445. pp 51–56 McKinney W (2010) Data structures for statistical computing in python. In: Proceedings of the 9th Python in Science Conference, vol 445. pp 51–56
34.
Zurück zum Zitat Oliphant TE (2006) A guide to NumPy. Trelgol Publishing USA Oliphant TE (2006) A guide to NumPy. Trelgol Publishing USA
36.
Zurück zum Zitat Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in Pytorch. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in Pytorch.
39.
Zurück zum Zitat Ng A, Katanforoosh K. CS230 Deep learning course notes and code examples Ng A, Katanforoosh K. CS230 Deep learning course notes and code examples
40.
Zurück zum Zitat Guo C, Berkhahn F (2016) Entity embeddings of categorical variables. arXiv preprint. arXiv: 160406737 Guo C, Berkhahn F (2016) Entity embeddings of categorical variables. arXiv preprint. arXiv: 160406737
41.
Zurück zum Zitat Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv
42.
Zurück zum Zitat Caruana R, Lawrence S, Giles CL (2001) Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. Advances in neural information processing systems. MIT Press, Cambridge, pp 402–408 Caruana R, Lawrence S, Giles CL (2001) Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. Advances in neural information processing systems. MIT Press, Cambridge, pp 402–408
43.
Zurück zum Zitat Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958 Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958
44.
Zurück zum Zitat Seabold S, Perktold J (2010) Statsmodels: Econometric and statistical modeling with python. In: Proceedings of the 9th Python in Science Conference, vol 57. p 61 Seabold S, Perktold J (2010) Statsmodels: Econometric and statistical modeling with python. In: Proceedings of the 9th Python in Science Conference, vol 57. p 61
45.
Zurück zum Zitat DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845CrossRef DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845CrossRef
46.
Zurück zum Zitat Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Müller M (2011) pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinf 12:77CrossRef Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Müller M (2011) pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinf 12:77CrossRef
47.
Zurück zum Zitat Team RS (2015) RStudio: integrated development for R. RStudio Inc., Boston, p 42 Team RS (2015) RStudio: integrated development for R. RStudio Inc., Boston, p 42
48.
Zurück zum Zitat Team RC (2013) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna Team RC (2013) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna
49.
Zurück zum Zitat Wickham H (2016) ggplot2: elegant graphics for data analysis. Springer, New YorkCrossRef Wickham H (2016) ggplot2: elegant graphics for data analysis. Springer, New YorkCrossRef
52.
Zurück zum Zitat Johnson KW, Soto JT, Glicksberg BS, Shameer K, Miotto R, Ali M, Ashley E, Dudley JT (2018) Artificial intelligence in cardiology. J Am Coll Cardiol 71:2668–2679CrossRef Johnson KW, Soto JT, Glicksberg BS, Shameer K, Miotto R, Ali M, Ashley E, Dudley JT (2018) Artificial intelligence in cardiology. J Am Coll Cardiol 71:2668–2679CrossRef
53.
Zurück zum Zitat Golas SB, Shibahara T, Agboola S, Otaki H, Sato J, Nakae T, Hisamitsu T, Kojima G, Felsted J, Kakarmath S (2018) A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Med Inform Decis Mak 18:44CrossRef Golas SB, Shibahara T, Agboola S, Otaki H, Sato J, Nakae T, Hisamitsu T, Kojima G, Felsted J, Kakarmath S (2018) A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Med Inform Decis Mak 18:44CrossRef
55.
Zurück zum Zitat Maier-Hein L, Vedula SS, Speidel S, Navab N, Kikinis R, Park A, Eisenmann M, Feussner H, Forestier G, Giannarou S (2017) Surgical data science for next-generation interventions. Nat Biomed Eng 1:691CrossRef Maier-Hein L, Vedula SS, Speidel S, Navab N, Kikinis R, Park A, Eisenmann M, Feussner H, Forestier G, Giannarou S (2017) Surgical data science for next-generation interventions. Nat Biomed Eng 1:691CrossRef
56.
Zurück zum Zitat Lee G, Rubinfeld I, Syed Z (2012) Adapting surgical models to individual hospitals using transfer learning. In: proceedings of the 2012 IEEE 12th International Conference on Data Mining Workshops, pp 57–63 Lee G, Rubinfeld I, Syed Z (2012) Adapting surgical models to individual hospitals using transfer learning. In: proceedings of the 2012 IEEE 12th International Conference on Data Mining Workshops, pp 57–63
57.
Zurück zum Zitat Bihorac A, Ozrazgat-Baslanti T, Ebadi A, Motaei A, Madkour M, Pardalos PM, Lipori G, Hogan WR, Efron PA, Moore F (2019) MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery. Ann Surg 269:652–662CrossRef Bihorac A, Ozrazgat-Baslanti T, Ebadi A, Motaei A, Madkour M, Pardalos PM, Lipori G, Hogan WR, Efron PA, Moore F (2019) MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery. Ann Surg 269:652–662CrossRef
58.
Zurück zum Zitat Natekin A, Knoll A (2013) Gradient boosting machines, a tutorial. Front Neurorobot 7:21CrossRef Natekin A, Knoll A (2013) Gradient boosting machines, a tutorial. Front Neurorobot 7:21CrossRef
60.
Zurück zum Zitat Qu Y, Cai H, Ren K, Zhang W, Yu Y, Wen Y, Wang J (2016) Product-based neural networks for user response prediction. In: Proceedings of the 2016 IEEE 16th International Conference on Data Mining. pp 1149–1154 Qu Y, Cai H, Ren K, Zhang W, Yu Y, Wen Y, Wang J (2016) Product-based neural networks for user response prediction. In: Proceedings of the 2016 IEEE 16th International Conference on Data Mining. pp 1149–1154
61.
Zurück zum Zitat Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems. MIT Press, Cambridge, pp 3111–3119 Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems. MIT Press, Cambridge, pp 3111–3119
62.
Zurück zum Zitat Covington P, Adams J, Sargin E (2016) Deep neural networks for youtube recommendations. In: Proceedings of the 10th ACM conference on recommender systems. pp 191–198 Covington P, Adams J, Sargin E (2016) Deep neural networks for youtube recommendations. In: Proceedings of the 10th ACM conference on recommender systems. pp 191–198
Metadaten
Titel
Development and validation of machine learning models to predict gastrointestinal leak and venous thromboembolism after weight loss surgery: an analysis of the MBSAQIP database
verfasst von
Jacob Nudel
Andrew M. Bishara
Susanna W. L. de Geus
Prasad Patil
Jayakanth Srinivasan
Donald T. Hess
Jonathan Woodson
Publikationsdatum
01.01.2021
Verlag
Springer US
Erschienen in
Surgical Endoscopy / Ausgabe 1/2021
Print ISSN: 0930-2794
Elektronische ISSN: 1432-2218
DOI
https://doi.org/10.1007/s00464-020-07378-x

Weitere Artikel der Ausgabe 1/2021

Surgical Endoscopy 1/2021 Zur Ausgabe

Update Chirurgie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.

S3-Leitlinie „Diagnostik und Therapie des Karpaltunnelsyndroms“

Karpaltunnelsyndrom BDC Leitlinien Webinare
CME: 2 Punkte

Das Karpaltunnelsyndrom ist die häufigste Kompressionsneuropathie peripherer Nerven. Obwohl die Anamnese mit dem nächtlichen Einschlafen der Hand (Brachialgia parästhetica nocturna) sehr typisch ist, ist eine klinisch-neurologische Untersuchung und Elektroneurografie in manchen Fällen auch eine Neurosonografie erforderlich. Im Anfangsstadium sind konservative Maßnahmen (Handgelenksschiene, Ergotherapie) empfehlenswert. Bei nicht Ansprechen der konservativen Therapie oder Auftreten von neurologischen Ausfällen ist eine Dekompression des N. medianus am Karpaltunnel indiziert.

Prof. Dr. med. Gregor Antoniadis
Berufsverband der Deutschen Chirurgie e.V.

S2e-Leitlinie „Distale Radiusfraktur“

Radiusfraktur BDC Leitlinien Webinare
CME: 2 Punkte

Das Webinar beschäftigt sich mit Fragen und Antworten zu Diagnostik und Klassifikation sowie Möglichkeiten des Ausschlusses von Zusatzverletzungen. Die Referenten erläutern, welche Frakturen konservativ behandelt werden können und wie. Das Webinar beantwortet die Frage nach aktuellen operativen Therapiekonzepten: Welcher Zugang, welches Osteosynthesematerial? Auf was muss bei der Nachbehandlung der distalen Radiusfraktur geachtet werden?

PD Dr. med. Oliver Pieske
Dr. med. Benjamin Meyknecht
Berufsverband der Deutschen Chirurgie e.V.

S1-Leitlinie „Empfehlungen zur Therapie der akuten Appendizitis bei Erwachsenen“

Appendizitis BDC Leitlinien Webinare
CME: 2 Punkte

Inhalte des Webinars zur S1-Leitlinie „Empfehlungen zur Therapie der akuten Appendizitis bei Erwachsenen“ sind die Darstellung des Projektes und des Erstellungswegs zur S1-Leitlinie, die Erläuterung der klinischen Relevanz der Klassifikation EAES 2015, die wissenschaftliche Begründung der wichtigsten Empfehlungen und die Darstellung stadiengerechter Therapieoptionen.

Dr. med. Mihailo Andric
Berufsverband der Deutschen Chirurgie e.V.