Skip to main content
Erschienen in: Journal of Medical Systems 1/2023

01.12.2023 | Original Paper

Improving Hospital Outpatient Clinics Appointment Schedules by Prediction Models

verfasst von: Orel Babayoff, Onn Shehory, Shamir Geller, Chen Shitrit-Niselbaum, Ahuva Weiss-Meilik, Eli Sprecher

Erschienen in: Journal of Medical Systems | Ausgabe 1/2023

Einloggen, um Zugang zu erhalten

Abstract

Patient no-shows and suboptimal patient appointment length scheduling reduce clinical efficiency and impair the clinic’s quality of service. The main objective of this study is to improve appointment scheduling in hospital outpatient clinics. We developed generic supervised machine learning models to predict patient no-shows and patient’s length of appointment (LOA). We performed a retrospective study using more than 100,000 records of patient appointments in a hospital outpatient clinic. Several machine learning algorithms were used for the development of our prediction models. We trained our models on a dataset that contained patients’, physicians’, and appointments’ characteristics. Our feature set combines both unstudied features and features adopted from previous studies. In addition, we identified the influential features for predicting LOA and no-show. Our LOA model’s performance was 6.92 in terms of MAE, and our no-show model’s performance was 92.1% in terms of F-score. We compared our models’ performance to the performance of previous research models by applying their methods to our dataset; our models demonstrated better performance. We show that the major effector of such differences is the use of our novel features. To evaluate the effect of our prediction results on the quality of schedules produced by appointment systems (AS), we developed an interface layer between our prediction models and the AS, where prediction results comprise the AS input. Using our prediction models, there was an 80% improvement in the daily cumulative patient waiting time and a 33% reduction in the daily cumulative physician idle time.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Merrit Hawkins, "survey of physician appointment wait times and medicare and medicaid acceptance rates," merritthawkin, 2017. Merrit Hawkins, "survey of physician appointment wait times and medicare and medicaid acceptance rates," merritthawkin, 2017.
2.
Zurück zum Zitat A. Baker, "Crossing the Quality Chasm: A New Health System for the 21st Century," National Academies Press, 2001. A. Baker, "Crossing the Quality Chasm: A New Health System for the 21st Century," National Academies Press, 2001.
3.
Zurück zum Zitat D. Gupta and B. Denton, "Appointment scheduling in health care: Challenges and opportunities," IIE Transactions, vol. 40, pp. 800-819, 2008.CrossRef D. Gupta and B. Denton, "Appointment scheduling in health care: Challenges and opportunities," IIE Transactions, vol. 40, pp. 800-819, 2008.CrossRef
4.
Zurück zum Zitat R. Lucas, H. Farley, J. Twanmoh, A. Urumov, N. Olsen, B. Evans and H. Kabiri, "Emergency department patient flow: the influence of hospital census variables on emergency department length of stay," the Society for Academic Emergency Medicine, pp. 597–602, 2009. R. Lucas, H. Farley, J. Twanmoh, A. Urumov, N. Olsen, B. Evans and H. Kabiri, "Emergency department patient flow: the influence of hospital census variables on emergency department length of stay," the Society for Academic Emergency Medicine, pp. 597–602, 2009.
5.
Zurück zum Zitat J. L. Wiler, C. Gentle, J. M. Halfpenny, A. Heins, A. Mehrotra, M. G. Mikhail and D. Fite, "Optimizing Emergency Department Front-End Operations," Annals of Emergency Medicine, pp. 142–160, 2010. J. L. Wiler, C. Gentle, J. M. Halfpenny, A. Heins, A. Mehrotra, M. G. Mikhail and D. Fite, "Optimizing Emergency Department Front-End Operations," Annals of Emergency Medicine, pp. 142–160, 2010.
6.
Zurück zum Zitat T. Cayirli and E. A. Veral, "Outpatient scheduling in health care: A review of literature," Production and Operations Management, vol. 12, pp. 519-549, 2009.CrossRef T. Cayirli and E. A. Veral, "Outpatient scheduling in health care: A review of literature," Production and Operations Management, vol. 12, pp. 519-549, 2009.CrossRef
7.
Zurück zum Zitat S. Barnes, E. Hamrock, M. Toerper, S. Siddiqui and S. Levin, "Real-time prediction of inpatient length of stay for discharge prioritization," Journal of the American Medical Informatics Association, 2015. S. Barnes, E. Hamrock, M. Toerper, S. Siddiqui and S. Levin, "Real-time prediction of inpatient length of stay for discharge prioritization," Journal of the American Medical Informatics Association, 2015.
8.
Zurück zum Zitat M. Rowan, T. Ryan, F. Hegarty and N. O’Hare, "The use of artificial neural networks to stratify the length of stay of cardiac patients based on preoperative and initial postoperative factors," Artificial Intelligence in Medicine, 2007. M. Rowan, T. Ryan, F. Hegarty and N. O’Hare, "The use of artificial neural networks to stratify the length of stay of cardiac patients based on preoperative and initial postoperative factors," Artificial Intelligence in Medicine, 2007.
9.
Zurück zum Zitat D. Ben Tayeb, N. Lahrichi and L.-M. Rousseau, "Patient scheduling based on a service-time prediction model: A data-driven study for a radiotherapy center," Springer Health Care Management Science, 2018. D. Ben Tayeb, N. Lahrichi and L.-M. Rousseau, "Patient scheduling based on a service-time prediction model: A data-driven study for a radiotherapy center," Springer Health Care Management Science, 2018.
10.
Zurück zum Zitat H. Lenzi, Â. J. Ben and A. T. Stein, "Development and validation of a patient no-show predictive model at a primary care setting in Southern Brazil," PLOS ONE, 2019. H. Lenzi, Â. J. Ben and A. T. Stein, "Development and validation of a patient no-show predictive model at a primary care setting in Southern Brazil," PLOS ONE, 2019.
11.
Zurück zum Zitat D. Carreras-García, D. Delgado-Gómez, F. Llorente-Fernández and A. Arribas-Gil, "Patient no-show prediction: A systematic literature review," Entropy, 2020. D. Carreras-García, D. Delgado-Gómez, F. Llorente-Fernández and A. Arribas-Gil, "Patient no-show prediction: A systematic literature review," Entropy, 2020.
12.
Zurück zum Zitat B. Zeng, A. Turkcan, J. Lin and M. Lawley, "Clinic scheduling models with overbooking for patients with heterogeneous no-show probabilities," Annals of Operations Research, vol. 178, pp. 121-144, 2009.CrossRef B. Zeng, A. Turkcan, J. Lin and M. Lawley, "Clinic scheduling models with overbooking for patients with heterogeneous no-show probabilities," Annals of Operations Research, vol. 178, pp. 121-144, 2009.CrossRef
13.
Zurück zum Zitat M. Heshmat, K. Nakata and A. Eltawil, "Solving the patient appointment scheduling problem in outpatient chemotherapy clinics using clustering and mathematical programming," Computers & Industrial Engineering, pp. 347–358, 2018. M. Heshmat, K. Nakata and A. Eltawil, "Solving the patient appointment scheduling problem in outpatient chemotherapy clinics using clustering and mathematical programming," Computers & Industrial Engineering, pp. 347–358, 2018.
14.
Zurück zum Zitat K. Muthuraman and M. Lawley, "A stochastic overbooking model for outpatient clinical scheduling with no-shows," IIE Transactions, 2008. K. Muthuraman and M. Lawley, "A stochastic overbooking model for outpatient clinical scheduling with no-shows," IIE Transactions, 2008.
15.
Zurück zum Zitat R. M. Goffman, S. L. Harris, J. H. May, A. S. Milicevic, R. J. Monte, L. Myaskovsky, K. L. Rodriguez, Y. C. Tjader and D. V. L., "Modeling patient No-Show history and predicting future outpatient appointment behavior in the Veterans Health Administration," Military Medicine, 2017. R. M. Goffman, S. L. Harris, J. H. May, A. S. Milicevic, R. J. Monte, L. Myaskovsky, K. L. Rodriguez, Y. C. Tjader and D. V. L., "Modeling patient No-Show history and predicting future outpatient appointment behavior in the Veterans Health Administration," Military Medicine, 2017.
16.
Zurück zum Zitat D. Hanauer and Y. Huang, "Patient no-show predictive model development using multiple data sources for an Effective Overbooking Approach," Applied Clinical Informatics, 2014. D. Hanauer and Y. Huang, "Patient no-show predictive model development using multiple data sources for an Effective Overbooking Approach," Applied Clinical Informatics, 2014.
17.
Zurück zum Zitat L. F. Dantas, S. Hamacher, F. L. Cyrino Oliveira, S. D. Barbosa and F. Viegas, "Predicting patient no-show behavior: A study in a bariatric clinic," Obesity Surgery, 2018. L. F. Dantas, S. Hamacher, F. L. Cyrino Oliveira, S. D. Barbosa and F. Viegas, "Predicting patient no-show behavior: A study in a bariatric clinic," Obesity Surgery, 2018.
18.
Zurück zum Zitat T. E. Raghunathan, J. M. Lepkowski, J. V. Hoewyk and P. Solenberger, "A Multivariate Technique for Multiply Imputing Missing Values Using a Sequence of Regression Models," Survey methodology, pp. 58–96, 2001. T. E. Raghunathan, J. M. Lepkowski, J. V. Hoewyk and P. Solenberger, "A Multivariate Technique for Multiply Imputing Missing Values Using a Sequence of Regression Models," Survey methodology, pp. 58–96, 2001.
19.
Zurück zum Zitat S. Snowden, P. Weech, R. McClure, S. Smye and P. Dear, "A neural network to predict attendance of paediatric patients at outpatient clinics," Neural Computing & Applications, pp. 234–241, 1995. S. Snowden, P. Weech, R. McClure, S. Smye and P. Dear, "A neural network to predict attendance of paediatric patients at outpatient clinics," Neural Computing & Applications, pp. 234–241, 1995.
20.
Zurück zum Zitat H. B. Harvey, C. Liu, J. Ai, C. Jaworsky, E. C. Guerrier, E. Flores and O. Pianykh, "Predicting no-shows in radiology Using Regression modeling of data available in the electronic medical record," Journal of the American College of Radiology, vol. 14, pp. 1303-1309, 2017.CrossRefPubMed H. B. Harvey, C. Liu, J. Ai, C. Jaworsky, E. C. Guerrier, E. Flores and O. Pianykh, "Predicting no-shows in radiology Using Regression modeling of data available in the electronic medical record," Journal of the American College of Radiology, vol. 14, pp. 1303-1309, 2017.CrossRefPubMed
22.
Zurück zum Zitat T. Chen and C. Guestrin, "XGBoost: A Scalable Tree Boosting System," in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, 2016. T. Chen and C. Guestrin, "XGBoost: A Scalable Tree Boosting System," in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, 2016.
23.
Zurück zum Zitat G. Ke, Q. Meng, T. Finley, T. Wan, W. Chen, W. Ma, Q. Ye and T.-Y. Liu, "LightGBM: A Highly Efficient Gradient Boosting Decision Tree," in 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, 2017. G. Ke, Q. Meng, T. Finley, T. Wan, W. Chen, W. Ma, Q. Ye and T.-Y. Liu, "LightGBM: A Highly Efficient Gradient Boosting Decision Tree," in 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, 2017.
24.
Zurück zum Zitat H. Ramchoun, M. Amine, J. Idrissi, Y. Ghanou and M. Ettaouil, "Multilayer Perceptron: Architecture Optimization and Training," International Journal of Interactive Multimedia and Artificial Intelligence, pp. 26–30, 2016. H. Ramchoun, M. Amine, J. Idrissi, Y. Ghanou and M. Ettaouil, "Multilayer Perceptron: Architecture Optimization and Training," International Journal of Interactive Multimedia and Artificial Intelligence, pp. 26–30, 2016.
25.
Zurück zum Zitat G. Luo, "A review of automatic selection methods for machine learning algorithms and hyper-parameter values," Network Modeling Analysis in Health Informatics and Bioinformatics, 2016. G. Luo, "A review of automatic selection methods for machine learning algorithms and hyper-parameter values," Network Modeling Analysis in Health Informatics and Bioinformatics, 2016.
26.
Zurück zum Zitat W. Li, X. Xing, F. Liu and Y. Zhang, "Application of Improved grid search algorithm on SVM for classification of Tumor Gene," International Journal of Multimedia and Ubiquitous Engineering, pp. 181–188, 2014. W. Li, X. Xing, F. Liu and Y. Zhang, "Application of Improved grid search algorithm on SVM for classification of Tumor Gene," International Journal of Multimedia and Ubiquitous Engineering, pp. 181–188, 2014.
27.
Zurück zum Zitat H. A. A. Mohamad, "Agarwood oil QUALITY classification using support Vector classifier and grid Search cross Validation hyperparameter tuning," International Journal of Emerging Trends in Engineering Research, vol. 8, pp. 2551-2556, 2020.CrossRef H. A. A. Mohamad, "Agarwood oil QUALITY classification using support Vector classifier and grid Search cross Validation hyperparameter tuning," International Journal of Emerging Trends in Engineering Research, vol. 8, pp. 2551-2556, 2020.CrossRef
28.
Zurück zum Zitat M. Aladeemy, L. Adwan, A. B. Booth, M. T. Khasawneh and S. Poranki, "New feature selection methods based on opposition-based learning and self-adaptive cohort intelligence for predicting patient no-shows," Applied Soft Computing, vol. 86, p. 105866, 2020.CrossRef M. Aladeemy, L. Adwan, A. B. Booth, M. T. Khasawneh and S. Poranki, "New feature selection methods based on opposition-based learning and self-adaptive cohort intelligence for predicting patient no-shows," Applied Soft Computing, vol. 86, p. 105866, 2020.CrossRef
29.
Zurück zum Zitat E. Štrumbelj and I. Kononenko, "Explaining prediction models and individual predictions with feature contributions," Knowledge and Information Systems, vol. 41, pp. 647-665, 2013.CrossRef E. Štrumbelj and I. Kononenko, "Explaining prediction models and individual predictions with feature contributions," Knowledge and Information Systems, vol. 41, pp. 647-665, 2013.CrossRef
30.
Zurück zum Zitat S. Lipovetsky and M. Conklin, "Analysis of regression in game theory approach," Applied Stochastic Models in Business and Industry, vol. 17, pp. 319-330, 2001.CrossRef S. Lipovetsky and M. Conklin, "Analysis of regression in game theory approach," Applied Stochastic Models in Business and Industry, vol. 17, pp. 319-330, 2001.CrossRef
31.
Zurück zum Zitat Batunacun, R. Wieland, T. Lakes and C. Nendel, "Using Shapley additive explanations to interpret extreme gradient boosting predictions of grassland degradation in Xilingol, China," Geoscientific Model Development, vol. 14, pp. 1493–1510, 2021. Batunacun, R. Wieland, T. Lakes and C. Nendel, "Using Shapley additive explanations to interpret extreme gradient boosting predictions of grassland degradation in Xilingol, China," Geoscientific Model Development, vol. 14, pp. 1493–1510, 2021.
32.
Zurück zum Zitat A. Ahmadi-Javid, Z. Jalali and K. J. Klassen, "Outpatient appointment systems in healthcare: A review of optimization studies," European Journal of Operational Research, vol. 258, pp. 3-34, 2017.CrossRef A. Ahmadi-Javid, Z. Jalali and K. J. Klassen, "Outpatient appointment systems in healthcare: A review of optimization studies," European Journal of Operational Research, vol. 258, pp. 3-34, 2017.CrossRef
33.
Zurück zum Zitat W. Liang, S. Luo, G. Zhao and H. Wu, "Predicting hard ROCK Pillar Stability Using GBDT, XGBoost, and Lightgbm algorithms," Mathematics, vol. 8, p. 765, 2020.CrossRef W. Liang, S. Luo, G. Zhao and H. Wu, "Predicting hard ROCK Pillar Stability Using GBDT, XGBoost, and Lightgbm algorithms," Mathematics, vol. 8, p. 765, 2020.CrossRef
34.
Zurück zum Zitat C. Bentéjac, A. Csörgő and G. Martínez-Muñoz, "A comparative analysis of gradient boosting algorithms," Artificial Intelligence Review, vol. 54, pp. 1937-1967, 2020.CrossRef C. Bentéjac, A. Csörgő and G. Martínez-Muñoz, "A comparative analysis of gradient boosting algorithms," Artificial Intelligence Review, vol. 54, pp. 1937-1967, 2020.CrossRef
35.
Zurück zum Zitat N. Schilling, M. Wistuba, L. Drumond and L. Schmidt-Thieme, "Machine Learning and Knowledge Discovery in Databases," Hyperparameter optimization with Factorized multilayer perceptrons, pp. 87–103, 2015. N. Schilling, M. Wistuba, L. Drumond and L. Schmidt-Thieme, "Machine Learning and Knowledge Discovery in Databases," Hyperparameter optimization with Factorized multilayer perceptrons, pp. 87–103, 2015.
36.
Zurück zum Zitat H. Ramchoun, M. Amine, J. Idrissi, Y. Ghanou and M. Ettaouil, "Multilayer Perceptron: Architecture," International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, p. 26, 2016.CrossRef H. Ramchoun, M. Amine, J. Idrissi, Y. Ghanou and M. Ettaouil, "Multilayer Perceptron: Architecture," International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, p. 26, 2016.CrossRef
38.
Zurück zum Zitat S. Srinivas and H. Salah, "Consultation length and no-show prediction for improving appointment scheduling efficiency at a cardiology clinic: A data analytics approach," International Journal of Medical Informatics, vol. 145, p. 104290, 2021.CrossRefPubMed S. Srinivas and H. Salah, "Consultation length and no-show prediction for improving appointment scheduling efficiency at a cardiology clinic: A data analytics approach," International Journal of Medical Informatics, vol. 145, p. 104290, 2021.CrossRefPubMed
39.
Zurück zum Zitat H. Salah and S. Srinivas, "Predict, then schedule: Prescriptive analytics approach for machine learning-enabled Sequential Clinical Scheduling," Computers & Industrial Engineering, vol. 169, p. 108270, 2022.CrossRef H. Salah and S. Srinivas, "Predict, then schedule: Prescriptive analytics approach for machine learning-enabled Sequential Clinical Scheduling," Computers & Industrial Engineering, vol. 169, p. 108270, 2022.CrossRef
40.
Zurück zum Zitat A. Kuiper and R. H. Lee, "Appointment scheduling for multiple servers," Management Science, vol. 68, pp. 7422-7440, 2022.CrossRef A. Kuiper and R. H. Lee, "Appointment scheduling for multiple servers," Management Science, vol. 68, pp. 7422-7440, 2022.CrossRef
Metadaten
Titel
Improving Hospital Outpatient Clinics Appointment Schedules by Prediction Models
verfasst von
Orel Babayoff
Onn Shehory
Shamir Geller
Chen Shitrit-Niselbaum
Ahuva Weiss-Meilik
Eli Sprecher
Publikationsdatum
01.12.2023
Verlag
Springer US
Erschienen in
Journal of Medical Systems / Ausgabe 1/2023
Print ISSN: 0148-5598
Elektronische ISSN: 1573-689X
DOI
https://doi.org/10.1007/s10916-022-01902-3

Weitere Artikel der Ausgabe 1/2023

Journal of Medical Systems 1/2023 Zur Ausgabe