BiomarkersImproving risk classification of critical illness with biomarkers: A simulation study☆
Introduction
More than 4 million patients receive intensive care each year, and broad variation exists in critical care delivery [1]. As such, the Institute of Medicine and multiple critical care professional societies called for a coordinated system of emergency and critical care [2], [3]. Similar to trauma care, one approach to improve coordination is to identify and systematically triage highest-risk patients to a higher level of care. This system of tiered regionalization could improve survival of high-risk patients while reducing costs [4].
The first challenge to effective regionalization is the accurate identification of patients who are most likely to benefit from triage to referral centers. Evidence suggests that emergency medical services (EMS) personnel could play a key role as well as clinicians evaluating new patients in the emergency department [5]. However, critical illness risk prediction tools during emergency care of patients without trauma demonstrate imperfect discrimination because they would redistribute many patients without critical illness to regional centers while assigning high-risk patients with critical illness to hospitals without critical care resources [5]. Without better tools, emergency care providers may inappropriately allocate thousands of low-risk patients to referral centers while still overlooking patients at greatest risk.
The complex diagnoses and overlapping mechanisms of disease leading to critical illness among the noninjured are unlikely captured by clinical data alone. This fact is well recognized in the hospital, where clinicians often combine biomarkers with clinical data to improve assessments of risk and guide treatment [6], [7], [8]. Now, because many biomarkers are measured using point-of-care platforms [9], the potential to move biomarker measurement to the forefront of emergency care—the prehospital phase—is close to reality, and yet, few empiric data help guide which biomarkers could be most helpful or how they may improve classification beyond easily measured clinical data. Nor has existing work used state-of-the-art methods to measure incremental benefit for candidate markers [10].
In this study, we determined how strongly associated with outcome a biomarker must be to meaningfully improve classification of critical illness risk compared with clinical data alone. We hypothesized that in silico biomarkers that are strongly associated with critical illness would provide incremental benefit over clinical data alone and that large studies would be needed to definitively document their value.
Section snippets
Conceptual approach
We sought to determine the strength of a biomarker necessary to meaningfully impact classification of emergency patients as high or low risk for critical illness. Emergency care personnel routinely combine physiological measurements (eg, heart rate and blood pressure) with diagnostic aids (such as electrocardiograms) to make these critical triage decisions [11]. In fact, physiological measurements, diagnostic aids, or traditional blood tests could all be considered “biomarkers” of critical
Results
Of 57 647 encounters, 3121 (5.4%) were hospitalized with critical illness (cases) and 54 526 (94.6%) without critical illness (controls). The clinical risk model alone had moderate discrimination between cases and controls (Table 1). When even weakest biomarkers (OR, 1.5) were added to the clinical model, we observed statistically significant increases in the AUC (Fig. 1) and IDI (Table 1). The model discrimination steadily increased when we added stronger biomarkers, whereas model calibration
Discussion
We demonstrate that the addition of biomarkers could significantly improve clinical risk stratification for critical illness during emergency care. Even weak biomarkers reclassified cases at higher risk for critical illness, whereas only strong biomarkers could reclassify controls as lower risk. We found that our results required large cohort sizes to attain significance but were robust to correlation between biomarkers and clinical predictors. These estimates inform future design of risk
Conclusions
In summary, clinical models for triage of critical illness could be significantly improved, especially by incorporating biomarker measurements. When designing such empirical cohort studies, substantial sample sizes may be required to uncover incremental benefit from candidate biomarkers or biomarker panels.
The following are the supplementary data related to this article.
Acknowledgment
This study is supported, in part, by funding from the National Institutes of Health (KL2 RR025015, K23GM104022 to C.W.S.; GM054438 and CA129934 to M.S.P.).
References (31)
- et al.
The diagnostic impact of prehospital 12-lead electrocardiography
Ann Emerg Med
(1990) - et al.
Out-of-hospital characteristics and care of patients with severe sepsis: a cohort study
J Crit Care
(2010) - et al.
Critical care delivery in the United States: distribution of services and compliance with Leapfrog recommendations
Crit Care Med
(2006) Hospital based emergency care: at the breaking point
- et al.
Prioritizing the organization and management of intensive care services in the United States: the PrOMIS Conference
Crit Care Med
(2007) - et al.
Potential value of regionalized intensive care for mechanically ventilated medical patients
Am J Respir Crit Care Med
(2008) - et al.
Prediction of critical illness during out-of-hospital emergency care
JAMA
(2010) - et al.
Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and d-dimer
Ann Intern Med
(2001) - et al.
Implementation of a sensitive troponin I assay and risk of recurrent myocardial infarction and death in patients with suspected acute coronary syndrome
JAMA
(2011) - Jones AE, Shapiro NI, Trzeciak S, et al. Lactate clearance vs central venous oxygen saturation as goals of early sepsis...
The prognostic value of blood lactate levels relative to that of vital signs in the pre-hospital setting: a pilot study
Crit Care
Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures
Ann Intern Med
Serum lactate is associated with mortality in severe sepsis independent of organ failure and shock
Crit Care Med
Early lactate clearance is associated with improved outcome in severe sepsis and septic shock
Crit Care Med
Emergency department lactate is associated with mortality in older adults admitted with and without infections
Acad Emerg Med
Cited by (17)
ML models for severity classification and length-of-stay forecasting in emergency units
2023, Expert Systems with ApplicationsSoluble urokinase receptor as a predictor of non-cardiac mortality in patients with percutaneous coronary intervention treated ST-segment elevation myocardial infarction
2020, Clinical BiochemistryCitation Excerpt :Identification of STEMI patients with a high risk of non-cardiac mortality is challenging but may guide clinicians to identify patients where focus after the acute phase of STEMI should include non-cardiac disease. Several biomarkers have shown great promise in risk stratification of patients [7–12], but the use of a biomarker to predict non-cardiac mortality in patients with STEMI has to the best of our knowledge never been examined. One candidate biomarker with this potential is the soluble urokinase plasminogen activator receptor (suPAR).
Abnormal routine blood tests as predictors of mortality in acutely admitted patients
2020, Clinical BiochemistryCitation Excerpt :Additionally, the comparison to other algorithms strengthens the blood test score by showing that the decrease in accuracy, while significant, is acceptable to achieve a simpler model. The use of biomarkers in early risk stratification has drawn a lot of interest in recent years [28]. Using a different approach, routine biomarkers have been shown as predictors of 30-day mortality in early patient care [16], and they have been linked to mortality in older patients [20].
Charting a roadmap for heart failure biomarker studies
2014, JACC: Heart FailureCitation Excerpt :We prefer that the components of the NRI are presented separately for events and nonevents. Furthermore, following the newest research developments in this area, we recommend that no statistical testing be conducted and no p values reported for measures of incremental value in model performance (30). Instead, once the new biomarker has been shown to be significantly associated with the outcome in a multivariable model, CIs should be given to quantify the precision of the estimate of incremental value.
Associations between molecular characteristics of colorectal serrated polyps and subsequent advanced colorectal neoplasia
2020, Cancer Causes and Control
- ☆
Conflict of interest: The authors declare no conflicts of interest.