Skip to main content
main-content
Erschienen in: Journal of Clinical Monitoring and Computing 4/2020

20.07.2019 | Original Research

Towards development of alert thresholds for clinical deterioration using continuous predictive analytics monitoring

verfasst von: Jessica Keim-Malpass, Matthew T. Clark, Douglas E. Lake, J. Randall Moorman

Erschienen in: Journal of Clinical Monitoring and Computing | Ausgabe 4/2020

Einloggen, um Zugang zu erhalten

Abstract

Patients who deteriorate while on the acute care ward and are emergently transferred to the Intensive Care Unit (ICU) experience high rates of mortality. To date, risk scores for clinical deterioration applied to the acute care wards rely on static or intermittent inputs of vital sign and assessment parameters. We propose the use of continuous predictive analytics monitoring, or data that relies on real-time physiologic monitoring data captured from ECG, documented vital signs, laboratory results, and other clinical assessments to predict clinical deterioration. A necessary step in translation to practice is understanding how an alert threshold would perform if applied to a continuous predictive analytic that was trained to detect clinical deterioration. The purpose of this study was to evaluate the positive predictive value of ‘risk spikes’, or large abrupt increases in the output of a statistical model of risk predicting clinical deterioration. We studied 8111 consecutive patient admissions to a cardiovascular medicine and surgery ward with continuous ECG data. We first trained a multivariable logistic regression model for emergent ICU transfer in a test set and tested the characteristics of the model in a validation set of 4059 patient admissions. Then, in a nested analysis we identified large, abrupt spikes in risk (increase by three units over the prior 6 h; a unit is the fold-increase in risk of ICU transfer in the next 24 h) and reviewed hospital records of 91 patients for clinical events such as emergent ICU transfer. We compared results to 59 control patients at times when they were matched for baseline risk including the National Warning Score (NEWS). There was a 3.4-fold higher event rate for patients with risk spikes (positive predictive value 24% compared to 7%, p = 0.006). If we were to use risk spikes as an alert, they would fire about once per day on a 73-bed acute care ward. Risk spikes that were primarily driven by respiratory changes (ECG-derived respiration (EDR) or charted respiratory rate) had highest PPV (30–35%) while risk spikes driven by heart rate had the lowest (7%). Alert thresholds derived from continuous predictive analytics monitoring are able to be operationalized as a degree of change from the person’s own baseline rather than arbitrary threshold cut-points, which can likely better account for the individual’s own inherent acuity levels. Point of care clinicians in the acute care ward settings need tailored alert strategies that promote a balance in recognition of clinical deterioration and assessment of the utility of the alert approach.
Zugang erhalten Sie mit:
e.Med Interdisziplinär

Für Ihren Erfolg in Klinik und Praxis - Die beste Hilfe in Ihrem Arbeitsalltag

Mit e.Med Interdisziplinär erhalten Sie Zugang zu allen CME-Fortbildungen und Fachzeitschriften auf SpringerMedizin.de.

Literatur
1.
Zurück zum Zitat Gold CA, Mayer SA, Lennihan L, et al. Unplanned transfers from hospital wards to the neurological intensive care unit. Neurocrit Care. 2015;23:159–65. PubMed Gold CA, Mayer SA, Lennihan L, et al. Unplanned transfers from hospital wards to the neurological intensive care unit. Neurocrit Care. 2015;23:159–65. PubMed
2.
Zurück zum Zitat Escobar GJ, Greene JD, Gardner MN, et al. Intra-hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6:74–80. PubMed Escobar GJ, Greene JD, Gardner MN, et al. Intra-hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6:74–80. PubMed
3.
Zurück zum Zitat Escobar GJ, Greene JD, Scheirer P, et al. Risk-adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46:232–9. PubMed Escobar GJ, Greene JD, Scheirer P, et al. Risk-adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46:232–9. PubMed
4.
Zurück zum Zitat O’Callaghan DJ, Jayia P, Vaughan-Huxley E, et al. An observational study to determine the effect of delayed admission to the intensive care unit on patient outcome. Crit Care. 2012;16:R173. PubMedPubMedCentral O’Callaghan DJ, Jayia P, Vaughan-Huxley E, et al. An observational study to determine the effect of delayed admission to the intensive care unit on patient outcome. Crit Care. 2012;16:R173. PubMedPubMedCentral
5.
Zurück zum Zitat Delgado MK, Liu V, Pines JM, et al. Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system. J Hosp Med. 2013;8:13–9. PubMed Delgado MK, Liu V, Pines JM, et al. Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system. J Hosp Med. 2013;8:13–9. PubMed
6.
Zurück zum Zitat Reese J, Deakyne SJ, Blanchard A, et al. Rate of preventable early unplanned intensive care unit transfer for direct admissions and emergency department admissions. Hosp Pediatr. 2015;5:27–34. PubMed Reese J, Deakyne SJ, Blanchard A, et al. Rate of preventable early unplanned intensive care unit transfer for direct admissions and emergency department admissions. Hosp Pediatr. 2015;5:27–34. PubMed
7.
Zurück zum Zitat Rosenberg AL, Hofer TP, Hayward RA, et al. Who bounces back? Physiologic and other predictors of intensive care unit readmission. Crit Care Med. 2001;29:511–8. PubMed Rosenberg AL, Hofer TP, Hayward RA, et al. Who bounces back? Physiologic and other predictors of intensive care unit readmission. Crit Care Med. 2001;29:511–8. PubMed
8.
Zurück zum Zitat Wellner B, Grand J, Canzone E, et al. Predicting unplanned transfers to the intensive care unit: a machine learning approach leveraging diverse clinical elements. JMIR Med Informatics. 2017;5:e45. Wellner B, Grand J, Canzone E, et al. Predicting unplanned transfers to the intensive care unit: a machine learning approach leveraging diverse clinical elements. JMIR Med Informatics. 2017;5:e45.
9.
Zurück zum Zitat Bittman J, Nijjar AP, Tam P, et al. Early warning scores to predict noncritical events overnight in hospitalized medical patients. J Patient Saf. 2017;00:1. Bittman J, Nijjar AP, Tam P, et al. Early warning scores to predict noncritical events overnight in hospitalized medical patients. J Patient Saf. 2017;00:1.
11.
Zurück zum Zitat Alam N, Hobbelink EL, van Tienhoven AJ, et al. The impact of the use of the Early Warning Score (EWS) on patient outcomes: a systematic review. Resuscitation. 2014;85:587–94. PubMed Alam N, Hobbelink EL, van Tienhoven AJ, et al. The impact of the use of the Early Warning Score (EWS) on patient outcomes: a systematic review. Resuscitation. 2014;85:587–94. PubMed
12.
Zurück zum Zitat Smith MEB, Chiovaro JC, O’Neil M, et al. Early warning system scores for clinical deterioration in hospitalized patients: a systematic review. Ann Am Thorac Soc. 2014;11:1454–65. PubMed Smith MEB, Chiovaro JC, O’Neil M, et al. Early warning system scores for clinical deterioration in hospitalized patients: a systematic review. Ann Am Thorac Soc. 2014;11:1454–65. PubMed
13.
Zurück zum Zitat Bartkowiak B, Snyder AM, Benjamin A, et al. Validating the electronic cardiac arrest risk triage (eCART) score for risk stratification of surgical inpatients in the postoperative setting. Ann Surg. 2018;269:1059. Bartkowiak B, Snyder AM, Benjamin A, et al. Validating the electronic cardiac arrest risk triage (eCART) score for risk stratification of surgical inpatients in the postoperative setting. Ann Surg. 2018;269:1059.
14.
Zurück zum Zitat Green M, Lander H, Snyder A, et al. Comparison of the between the flags calling criteria to the MEWS, NEWS and the electronic cardiac arrest risk triage (eCART) score for the identification of deteriorating ward patients. Resuscitation. 2018;123:86–91. PubMed Green M, Lander H, Snyder A, et al. Comparison of the between the flags calling criteria to the MEWS, NEWS and the electronic cardiac arrest risk triage (eCART) score for the identification of deteriorating ward patients. Resuscitation. 2018;123:86–91. PubMed
15.
Zurück zum Zitat Churpek MM, Edelson DP, Lee JY, et al. Association between survival and time of day for rapid response team calls in a national registry. Crit Care Med. 2017;45:1677–82. PubMedPubMedCentral Churpek MM, Edelson DP, Lee JY, et al. Association between survival and time of day for rapid response team calls in a national registry. Crit Care Med. 2017;45:1677–82. PubMedPubMedCentral
16.
Zurück zum Zitat Churpek MM, Snyder A, Twu NM, et al. Accuracy comparisons between manual and automated respiratory rate for detecting clinical deterioration in ward patients. J Hosp Med. 2018;13:2017–8. Churpek MM, Snyder A, Twu NM, et al. Accuracy comparisons between manual and automated respiratory rate for detecting clinical deterioration in ward patients. J Hosp Med. 2018;13:2017–8.
17.
Zurück zum Zitat Rojas J, Carey K, Edeslon D, et al. Predicting intensive care unit readmission with machine learning using electronic health record data. Ann Am Thorac Soc. 2018;15:846–53. PubMedPubMedCentral Rojas J, Carey K, Edeslon D, et al. Predicting intensive care unit readmission with machine learning using electronic health record data. Ann Am Thorac Soc. 2018;15:846–53. PubMedPubMedCentral
18.
Zurück zum Zitat Rothman MJ, Rothman SI, Beals J. Development and validation of a continuous measure of patient condition using the electronic medical record. J Biomed Inform. 2013;46:837–48. PubMed Rothman MJ, Rothman SI, Beals J. Development and validation of a continuous measure of patient condition using the electronic medical record. J Biomed Inform. 2013;46:837–48. PubMed
19.
Zurück zum Zitat Ruminski CM, Clark MT, Lake DE, et al. Impact of predictive analytics based on continuous cardiorespiratory monitoring in a surgical and trauma intensive care unit. J Clin Monit Comput. 2018;33:703. PubMed Ruminski CM, Clark MT, Lake DE, et al. Impact of predictive analytics based on continuous cardiorespiratory monitoring in a surgical and trauma intensive care unit. J Clin Monit Comput. 2018;33:703. PubMed
20.
Zurück zum Zitat Moss TJ, Lake DE, Calland JF, et al. Signatures of subacute potentially catastrophic illness in the ICU. Crit Care Med. 2016;44:1639. PubMedPubMedCentral Moss TJ, Lake DE, Calland JF, et al. Signatures of subacute potentially catastrophic illness in the ICU. Crit Care Med. 2016;44:1639. PubMedPubMedCentral
21.
Zurück zum Zitat Moss TJ, Lake DE, Moorman JR. Local dynamics of heart rate: detection and prognostic implications. Physiol Meas. 2014;35:1929–42. PubMed Moss TJ, Lake DE, Moorman JR. Local dynamics of heart rate: detection and prognostic implications. Physiol Meas. 2014;35:1929–42. PubMed
22.
Zurück zum Zitat Fairchild K, Schelonka R, Kaufman D, et al. Septicemia mortality reduction in neonates in a heart rate characteristics monitoring trial. Pediatr Res. 2013;74:570–5. PubMedPubMedCentral Fairchild K, Schelonka R, Kaufman D, et al. Septicemia mortality reduction in neonates in a heart rate characteristics monitoring trial. Pediatr Res. 2013;74:570–5. PubMedPubMedCentral
23.
Zurück zum Zitat Lake DE, Richman JS, Griffin MP, et al. Sample entropy analysis of neonatal heart rate variability. Am J Physiol Regul Integr Comp Physiol. 2010;22908:789–97. Lake DE, Richman JS, Griffin MP, et al. Sample entropy analysis of neonatal heart rate variability. Am J Physiol Regul Integr Comp Physiol. 2010;22908:789–97.
24.
Zurück zum Zitat Lake DE, Fairchild KD, Moorman JR. Complex signals bioinformatics: evaluation of heart rate characteristics monitoring as a novel risk marker for neonatal sepsis. J Clin Monit Comput. 2014;28:329–39. PubMed Lake DE, Fairchild KD, Moorman JR. Complex signals bioinformatics: evaluation of heart rate characteristics monitoring as a novel risk marker for neonatal sepsis. J Clin Monit Comput. 2014;28:329–39. PubMed
25.
Zurück zum Zitat Nemati S, Holder A, Razmi F, et al. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med. 2017;46:547. Nemati S, Holder A, Razmi F, et al. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med. 2017;46:547.
26.
Zurück zum Zitat Calvert JS, Price DA, Chettipally UK, et al. A computational approach to early sepsis detection. Comput Biol Med. 2016;74:69–73. PubMed Calvert JS, Price DA, Chettipally UK, et al. A computational approach to early sepsis detection. Comput Biol Med. 2016;74:69–73. PubMed
27.
Zurück zum Zitat Desautels T, Calvert J, Hoffman J, et al. Prediction of sepsis in the intensive care unit with minimal electronic health record data: a machine learning approach. JMIR Med Inform. 2016;4:e28. PubMedPubMedCentral Desautels T, Calvert J, Hoffman J, et al. Prediction of sepsis in the intensive care unit with minimal electronic health record data: a machine learning approach. JMIR Med Inform. 2016;4:e28. PubMedPubMedCentral
28.
Zurück zum Zitat Clark M, Vergales B, Paget-Brown A, et al. Predictive monitoring for respiratory decompensation leading to urgent unplanned intubation in the neonatal intensive care unit. Pediatr Res. 2013;73:104–10. PubMed Clark M, Vergales B, Paget-Brown A, et al. Predictive monitoring for respiratory decompensation leading to urgent unplanned intubation in the neonatal intensive care unit. Pediatr Res. 2013;73:104–10. PubMed
29.
Zurück zum Zitat Keim-Malpass J, Kitzmiller R, Skeeles-Worley A, et al. Advancing continuous predictive analytics monitoring: moving from implementation to clinical action in a learning health system. Crit Care Nurs Clin North Am. 2018;30:273. PubMed Keim-Malpass J, Kitzmiller R, Skeeles-Worley A, et al. Advancing continuous predictive analytics monitoring: moving from implementation to clinical action in a learning health system. Crit Care Nurs Clin North Am. 2018;30:273. PubMed
30.
Zurück zum Zitat Moss TJ, Clark MT, Calland JF, et al. Cardiorespiratory dynamics measured from continuous ECG monitoring improves detection of deterioration in acute care patients: a retrospective cohort study. PLoS ONE. 2017;12:1–16. Moss TJ, Clark MT, Calland JF, et al. Cardiorespiratory dynamics measured from continuous ECG monitoring improves detection of deterioration in acute care patients: a retrospective cohort study. PLoS ONE. 2017;12:1–16.
31.
Zurück zum Zitat Lake DE, Moorman JR. Accurate estimation of entropy in very short physiological time series: the problem of atrial fibrillation detection in implanted ventricular devices. AJP Hear Circ Physiol. 2011;300:H319–25. Lake DE, Moorman JR. Accurate estimation of entropy in very short physiological time series: the problem of atrial fibrillation detection in implanted ventricular devices. AJP Hear Circ Physiol. 2011;300:H319–25.
32.
Zurück zum Zitat Carrara M, Carozzi L, Moss TJ, et al. Classification of cardiac rhythm using heart rate dynamical measures: validation in MIT-BIH databases. J Electrocardiol. 2015;48:943–6. PubMed Carrara M, Carozzi L, Moss TJ, et al. Classification of cardiac rhythm using heart rate dynamical measures: validation in MIT-BIH databases. J Electrocardiol. 2015;48:943–6. PubMed
33.
Zurück zum Zitat Carrara M, Carozzi L, Moss TJ, et al. Heart rate dynamics distinguish among atrial fibrillation, normal sinus rhythm and sinus rhythm with frequent ectopy. Physiol Meas. 2015;36:1873–88. PubMed Carrara M, Carozzi L, Moss TJ, et al. Heart rate dynamics distinguish among atrial fibrillation, normal sinus rhythm and sinus rhythm with frequent ectopy. Physiol Meas. 2015;36:1873–88. PubMed
34.
Zurück zum Zitat Huber P. The behavior of maximum likelihood estimation under nonstandard conditions. In: Proceedings Fifth Berkeley symposium math and statistics. 1967, pp. 221–233. Huber P. The behavior of maximum likelihood estimation under nonstandard conditions. In: Proceedings Fifth Berkeley symposium math and statistics. 1967, pp. 221–233.
35.
Zurück zum Zitat White H. Maximum likelihood estimation of misspecified models. Econometrica. 1982;50:1–25. White H. Maximum likelihood estimation of misspecified models. Econometrica. 1982;50:1–25.
36.
Zurück zum Zitat Harrell F. Regression modeling strategies. New York: Springer; 2015. Harrell F. Regression modeling strategies. New York: Springer; 2015.
37.
Zurück zum Zitat Keim-Malpass J, Enfield KB, Calland JF, et al. Dynamic data monitoring improves predictive analytics for failed extubation in the ICU. Physiol Meas. 2018;39:075005. PubMed Keim-Malpass J, Enfield KB, Calland JF, et al. Dynamic data monitoring improves predictive analytics for failed extubation in the ICU. Physiol Meas. 2018;39:075005. PubMed
38.
Zurück zum Zitat Lawless J, Singhal K. Efficient screening of nonnormal regression methods. Biometrics. 1978;34:318–27. Lawless J, Singhal K. Efficient screening of nonnormal regression methods. Biometrics. 1978;34:318–27.
39.
Zurück zum Zitat Ho D, Imai K, King G, et al. MatchIt: nonparametric preprocessing for parametric causal inference. J Stat Softw. 2011;42:1–28. Ho D, Imai K, King G, et al. MatchIt: nonparametric preprocessing for parametric causal inference. J Stat Softw. 2011;42:1–28.
40.
Zurück zum Zitat Sullivan BA, Grice SM, Lake DE, et al. Infection and other clinical correlates of abnormal heart rate characteristics in preterm infants. J Pediatr. 2014;164:775–80. PubMedPubMedCentral Sullivan BA, Grice SM, Lake DE, et al. Infection and other clinical correlates of abnormal heart rate characteristics in preterm infants. J Pediatr. 2014;164:775–80. PubMedPubMedCentral
41.
Zurück zum Zitat Swanson JR, King WE, Sinkin RA, et al. Neonatal intensive care unit length of stay reduction by heart rate characteristics monitoring. J Pediatr. 2018;198:1–6. Swanson JR, King WE, Sinkin RA, et al. Neonatal intensive care unit length of stay reduction by heart rate characteristics monitoring. J Pediatr. 2018;198:1–6.
42.
Zurück zum Zitat Fidler RL, Pelter MM, Drew BJ, et al. Understanding heart rate alarm adjustment in the intensive care units through an analytical approach. PLoS ONE. 2017;12:1–10. Fidler RL, Pelter MM, Drew BJ, et al. Understanding heart rate alarm adjustment in the intensive care units through an analytical approach. PLoS ONE. 2017;12:1–10.
45.
Zurück zum Zitat Winters BD, Cvach MM, Bonafide CP, et al. Technological distractions (Part 2): a summary of approaches to manage clinical alarms with intent to reduce alarm fatigue. Crit Care Med. 2018;46:130–7. PubMed Winters BD, Cvach MM, Bonafide CP, et al. Technological distractions (Part 2): a summary of approaches to manage clinical alarms with intent to reduce alarm fatigue. Crit Care Med. 2018;46:130–7. PubMed
46.
Zurück zum Zitat Srinivasa E, Mankoo J, Kerr C. An evidence-based approach to reducing cardiac telemetry alarm fatigue. Worldviews Evid.-Based Nurs. 2017;14:265–73. PubMed Srinivasa E, Mankoo J, Kerr C. An evidence-based approach to reducing cardiac telemetry alarm fatigue. Worldviews Evid.-Based Nurs. 2017;14:265–73. PubMed
47.
Zurück zum Zitat Curry JP, Jungquist CR. A critical assessment of monitoring practices, patient deterioration, and alarm fatigue on inpatient wards: a review. Patient Saf Surg. 2014;8:1–20. Curry JP, Jungquist CR. A critical assessment of monitoring practices, patient deterioration, and alarm fatigue on inpatient wards: a review. Patient Saf Surg. 2014;8:1–20.
48.
Zurück zum Zitat Min A, Scott LD. Evaluating nursing hours per patient day as a nurse staffing measure. J Nurs Manage. 2016;24:439–48. Min A, Scott LD. Evaluating nursing hours per patient day as a nurse staffing measure. J Nurs Manage. 2016;24:439–48.
50.
Zurück zum Zitat Shah ND, Robert D, Clinic M, et al. Big data and predictive analytics recalibrating expectations. JAMA. 2018;320:5–6. Shah ND, Robert D, Clinic M, et al. Big data and predictive analytics recalibrating expectations. JAMA. 2018;320:5–6.
51.
Zurück zum Zitat Moskowitz A, McSparron J, Stone DJ, et al. Preparing a new generation of clinicians for the Era of big data. Harvard Med student Rev. 2015;2:24–7. Moskowitz A, McSparron J, Stone DJ, et al. Preparing a new generation of clinicians for the Era of big data. Harvard Med student Rev. 2015;2:24–7.
52.
Zurück zum Zitat Celi L. Big data in the intensive care unit. Am J Respir Crit Care. 2013;187:1157–60. Celi L. Big data in the intensive care unit. Am J Respir Crit Care. 2013;187:1157–60.
53.
Zurück zum Zitat Mehta N, Pandit A. Concurrence of big data analytics and healthcare: a systematic review. Int J Med Inform. 2018;114:57–65. PubMed Mehta N, Pandit A. Concurrence of big data analytics and healthcare: a systematic review. Int J Med Inform. 2018;114:57–65. PubMed
Metadaten
Titel
Towards development of alert thresholds for clinical deterioration using continuous predictive analytics monitoring
verfasst von
Jessica Keim-Malpass
Matthew T. Clark
Douglas E. Lake
J. Randall Moorman
Publikationsdatum
20.07.2019
Verlag
Springer Netherlands
Erschienen in
Journal of Clinical Monitoring and Computing / Ausgabe 4/2020
Print ISSN: 1387-1307
Elektronische ISSN: 1573-2614
DOI
https://doi.org/10.1007/s10877-019-00361-5

Weitere Artikel der Ausgabe 4/2020

Journal of Clinical Monitoring and Computing 4/2020 Zur Ausgabe

Neu im Fachgebiet AINS

Newsletter

Bestellen Sie unseren kostenlosen Newsletter Update AINS und bleiben Sie gut informiert – ganz bequem per eMail.