- •
Continuous predictive analytics monitoring synthesizes data from a variety of inputs into a risk estimate that clinicians can observe in a streaming environment.
- •
For continuous predictive analytics monitoring to be useful, clinicians must engage with the data in a way that makes sense for their clinical workflow in the context of a learning health system (LHS).
- •
Clinicians described the processes needed to move to clinical action through the following themes: (1) understand the science behind the
Advancing Continuous Predictive Analytics Monitoring: Moving from Implementation to Clinical Action in a Learning Health System
Section snippets
Key points
Study Design and Sample
This study used a longitudinal qualitative descriptive design using both focus group and individual interviews collected from an academic surgical trauma ICU in central Virginia.22, 23 Participants were recruited through a convenience sampling strategy that included any point-of-care clinician (registered nurse [RN], respiratory therapist, nurse practitioner, attending physician) who worked in the unit and were exposed to CoMET® display monitoring for any period of time. There were no exclusion
Results
Demographic characteristics of the sample were not maintained to protect anonymity because all clinicians were recruited from the same ICU. Several themes and subthemes emerged from the data that described the necessary process to move predictive analytics monitoring from implementation to clinical action. These included
- 1.
Understand the science behind the algorithm (subthemes: first-to-test or lacks standard evidence, and moving to alarm)
- 2.
Trust the data inputs (subtheme: noise).
- 3.
Integrate with the
Discussion
There is a gap in the literature of research focused on implementation of continuous predictive analytics monitoring. Even less known about how to optimize the implementation among a clinician user group that is the first to test the modality in practice. Point-of-care clinicians in this study articulated that understanding these implementation processes in an LHS are critical steps that are needed before any resulting clinical action. The LHS offers a system of care that allows for exploration
References (30)
- et al.
Predicting the need for urgent intubation in a surgical/trauma intensive care unit
Surgery
(2013) - et al.
Heart rate dynamics preceding hemorrhage in the intensive care unit
J Electrocardiol
(2015) - et al.
External validation in an intermediate unit of a respiratory decompensation model trained in an intensive care unit
Surgery
(2017) - et al.
Early heart rate characteristics predict death and morbidities in preterm infants
J Pediatr
(2016) - et al.
A roadmap for national action on clinical decision support
J Am Med Inform Assoc
(2007) - et al.
The impact of the use of the Early Warning Score (EWS) on patient outcomes: a systematic review
Resuscitation
(2014) - et al.
Developing a patient safety surveillance system to identify adverse events in the intensive care unit
Crit Care Med
(2010) - et al.
Worldwide demand for critical care
Curr Opin Crit Care
(2011) Big data in the intensive care unit. Closing the data loop
Am J Respir Crit Care Med
(2013)- et al.
Complex signals bioinformatics: evaluation of heart rate characteristics monitoring as a novel risk marker for neonatal sepsis
J Clin Monit Comput
(2014)
A review of dashboards for data analytics in nursing
Comput Inform Nurs
New-onset atrial fibrillation in the critically Ill
Crit Care Med
Septicemia mortality reduction in neonates in a heart rate characteristics monitoring trial
Pediatr Res
Stochastic modeling of central apnea events in preterm infants
Physiol Meas
Signatures of subacute potentially catastrophic illness in the ICU: model development and validation
Crit Care Med
Cited by (27)
Engaging clinicians early during the development of a graphical user display of an intelligent alerting system at the bedside
2022, International Journal of Medical InformaticsCitation Excerpt :Analysis identified five emerging themes: trend evolution, context, evaluation/interpretation/explanation (sub theme- continuity of evaluation), clinical intuition support, and clinical operations utility. Early-stage, qualitative work that engages research teams with clinical experts is a critical component of health care-focused AI research that is often overlooked [14–18]. Studies demonstrate that AI models identify predictive patterns in heterogenous patient data [8,19–22].
Nursing and precision predictive analytics monitoring in the acute and intensive care setting: An emerging role for responding to COVID-19 and beyond
2021, International Journal of Nursing Studies AdvancesCitation Excerpt :These elements have been demonstrated in a variety of settings including aviation/air traffic control, military command, and among clinicians in complex healthcare environments (Endsley, 1995). Through our preliminary qualitative research among over 40 adult surgical and neonatal clinicians, we have found that elements of situational awareness are a critical antecedent necessary prior to initiating clinical action in response to a change in risk score presented by precision continuous predictive analytics monitoring (Keim-Malpass et al., 2018; Kitzmiller et al., 2019). A balanced approach between the development of meaningful analytics and engaged situational awareness among nurses on the acute care floor or ICU is critical, so attention and working memory can be properly devoted to dynamic decision making and translation to relevant clinical actions (Stubbings et al., 2012).
Physiological machine learning models for prediction of sepsis in hospitalized adults: An integrative review
2021, Intensive and Critical Care NursingCitation Excerpt :The excitement around the potential for significant improvement in patient outcomes must be approached with rigorous methodological underpinnings for model development and validation. Beyond rigor in model development, research investigating the use of machine learning models in patient care should focus on ensuring the models are integrated into the clinical environment successfully through frameworks of implementation for predictive analytics in the healthcare system (Keim-Malpass et al., 2018). Machine learning models for sepsis prediction demonstrate promise towards the continued goal of reducing events of clinical deterioration and improving outcomes for patients at risk for sepsis.
Interpreting a recurrent neural network's predictions of ICU mortality risk
2021, Journal of Biomedical InformaticsPredictive analytics and early intervention in healthcare social work: a scoping review
2024, Social Work in Health Care
Disclosure Statement: J. Keim-Malpass was supported through a grant from the University of Virginia Translational Health Research Institute of Virginia (THRIV) Scholars award. R. Kitzmiller was supported by the National Center for Translational Sciences and National Institutes of Health (NIH) through grant KL2TR001109. The content is solely the responsibility of the authors and does not necessarily represent official views of the NIH. C. Lindberg, R. Anderson, and R. Kitzmiller are supported by MITRE funding agreements 19140 and 11348 (accelerating staff engagement in predictive monitoring development, implementation, and use). The MITRE Corporation operates the Centers for Medicare & Medicaid Services (CMS) Alliance to Modernize Healthcare (CAMH), a federally funded research and development center dedicated to strengthening the nation’s health care system. The MITRE Corporation operates CAMH in partnership with the CMS and the Department of Health and Human Services.
Conflicts of Interest: Drs J.R. Moorman and M.T. Clark have equity in, and are officers of, the Advanced Medical Predictive Devices, Diagnostics, and Displays in Charlottesville, VA, USA (AMP3D). Dr M.T Clark is employed by AMP3D.
- 1
1215 Lee Street, Charlottesville, VA 22908.