The simple observational critical care studies: estimations by students, nurses, and physicians of in-hospital and 6-month mortality

Patients admitted to intensive care units (ICUs) may suffer from various illnesses and comorbidities. ICU patients are known to have high mortality rates, and survivors may suffer from long-term impairments in both cognitive and overall function, leading to a reduced quality of life [1]. Although all patients admitted to the ICUs are in an acute state of critical illness, they are often remarkably different in terms of characteristics and prognosis. Every day, healthcare providers judge clinical expectations, including possible health outcomes, and provide such information to patients and their relatives. Accurate estimates of survival are essential, as these may influence clinical decisions, especially in resource-limited situations. In an ICU setting, physicians and nurses typically apply a combination of deductive and inductive reasoning, along with intuitive (non-methodical reasoning) analysis. Discussing mortality estimations in an inter-disciplinary setting can therefore offer new insights and improvements in the reasoning process and improved prediction [2, 3].

During the training of physicians and nurses, various theories are used to teach them how to actively combine acquired clinical knowledge, acquired diagnostic experience, and personal intuition. This approach is needed to make as accurate prognostic statements as possible, including statements about the likelihood that the applied treatment will lead to survival [4].

One intriguing study evaluated the discriminative accuracy of physicians and nurses’ estimations of 303 critically ill patients' clinical outcomes and found that the estimations of physicians and nurses added significantly to the discriminative accuracy of existing prediction models [5]. However, the results of this study have not been validated in a sufficiently large population [6], and research in other settings has shown contrasting results [7]. Nevertheless, the estimation would be valuable in the early stages of critical diseases and should be made shortly after ICU admission. Furthermore, the accuracy of estimations can evolve and differ for different healthcare providers [8‐10].

The ability of healthcare providers to predict which patients will survive is relevant for clinical decision-making. However, intuition alone may not always lead to correct diagnostic judgments, and both managing this intuition and clinical experience are prerequisites for reliable estimates of the outcome. The estimations of experienced physicians and nurses may be substantially more accurate than those of students. Whether this assumption is correct is unknown, however. This study aimed to explore the accuracy of students’, nurses’, and physicians’ estimations of mortality and to evaluate the association of combined estimations by students, nurses, and physicians with the outcome.

Between May 14, 2018, and July 10, 2019, a total of 3357 ICU admissions were assessed for eligibility. In total, 1,104 patients fulfilled the inclusion criteria for clinical examination. Data were not obtained for 94 patients: 45 patients died before inclusion, continuous resuscitation efforts were made for 26 patients, and there was no access to 23 patients due to logistic reasons. In total, 1,010 patients were included in the SICS-II cohort. For 183 patients (18%), estimations were not available, resulting in 827 patients (82%) being included in the SOCCS cohort (Fig. 1; Table 1). Median time from ICU admission to clinical examination was 1.8 h (IQR 1.1–2.7). In total, 178 patients (22%) died during the hospital stay. By the 6-month follow-up, 238 patients (29%) had died. Ten patients were lost to follow-up.

In this study, the estimations of medical and nursing students, nurses, and physicians of both in-hospital and 6-month mortality within 3 h after admission to the ICU were found to have moderate discriminative accuracy. In addition, the variation in accuracy between students, nurses, and physicians was not statistically different. The most valuable observation was that, in case where more students, nurses, and physicians estimated non-survival, both the risk of death and the discriminative accuracy were significantly higher, suggesting the benefits of multidisciplinary clinical care and prognostication.

Several studies have evaluated the accuracy of physicians and nurses’ estimations of clinical outcomes in critically ill patients [5, 15‐19]. Overall, these studies show an additional value of physicians and nurses’ estimations compared or added to existing clinical prediction models. Below, we present the previous studies most relevant to the current study. Detsky et al. investigated the discriminative accuracy of both nurses' and physicians’ estimations and showed that, of multiple outcomes estimated, nurses could best estimate in-hospital mortality in critically ill patients [5]. In our study, physicians were most accurate in estimating in-hospital mortality, followed by students and nurses. No previous study has evaluated the accuracy of medical students’ estimations of patient mortality. Buehler et al. showed that the overall accuracy of predictions made by ICU physicians was moderate [20]. However, their study was a literature review comparing several observational studies that were limited to physicians.

We are unaware of any studies investigating the cumulative estimation of multiple healthcare providers considering mortality. However, one study showed that predictions considering ineffective treatment were more valuable when based on collaborative decision-making [21]. Another study focusing on detecting sepsis showed that the chance of severe sepsis was highest when estimated by two healthcare providers [22].

The most significant difference between our study and previous research efforts is that we intended to assess patients at the earliest ICU admission stage. In previous studies, the estimations were made within 24 h after ICU admission, leaving time for interventions and treatment responses that which might influence outcome predictions, and were mainly made by fewer physicians [23]. Treatment decisions and interventions influence prognostication only minimally in the first few hours after admission. Thereafter, many other factors, such as deterioration or improvement of organ function, logically influence the (prediction of the) outcome. To eventually guide clinical decision-making, it is essential to estimate mortality as soon as possible after, or even at, ICU admission to trigger reconsideration of ICU admittance and treatment restrictions.

The moderate accuracy of clinical predictions by individuals has been well established [24]. It is assumed that many supportive technical innovations in clinical decision-making will exponentially become available [25]. The challenge is to find and especially use the best possible algorithm to predict patient outcomes [26]. Nevertheless, healthcare providers can take into account improbable events and take variables that are currently unavailable for algorithms. It remains essential to keep an eye on the patient at the bedside, as previous studies have shown that often simple variables can predict adverse outcomes [27, 28]. Furthermore, the healthcare providers' “gut feeling,” or “clinical gestalt,” will probably never be fully understood but may be of crucial additive value. For example, Ferreyro et al. investigated physician- and patient-level factors influencing physicians' predictions of 6-month mortality in critically ill patients based on variables available at admission [29]. The authors found that older age, malignancy, and higher APACHE III scores were associated with the physicians’ predictions [29]. Although we did not investigate students, nurses, and physicians factors, we report similar observations for patient-level factors such as the SAPS II and APACHE IV score, the use of vasopressors, prolonged capillary refill time, and lower level of consciousness in our study. Using data from studies like from Ferreyro et al. or our study using Bayesian network analysis may further unravel how, why, and based on what variables healthcare providers make their predictions [30, 31]. Eventually, insight into the drivers of clinical predictions could be valuable for developing and evaluating decision algorithms and may inform future healthcare providers' education [32].

Several limitations of this study must be acknowledged. First, students’, nurses', and physicians’ estimations are subjective and may be influenced by personal experience, beliefs, or specialty training. Second, unlike other studies, we did not report any characteristics of the students, nurses, and physicians, who provided the estimations of mortality [5, 23]. Furthermore, we reported the estimations based on a binary scale, using “Yes” and “No.” Physicians may be more likely to make mortality predictions on a continuous scale. Using a rating scale based on likelihood and providing more characteristics of the healthcare providers could have helped further characterize our results. Third, the results of the estimations of the nurses and physicians need to be interpreted carefully since the a priori chance of a good outcome is, in general, relatively high and might therefore influence the overall accuracy. Moreover, the students were instructed to complete their assessments first, and the physicians and nurses were also instructed to let the student make their estimates first. However, the students were not completely blinded to the estimations of the physicians and nurses, which could have influenced their estimations. Furthermore, there might have been a selection bias due to the relatively high number of missing estimations by physicians. These estimations, and those of others, may be missing because the physician or nurse was absent or occupied at the time students made their estimations. However, there was no significant clinical difference between patients with and without an estimation of mortality by the physician, and as we included a large cohort of unselected patients, we deem it unlikely that this could have influenced our results.

Based on the initial examination, students, nurses, and physicians can only moderately predict in-hospital and 6-month mortality in critically ill patients. Combined estimations led to more accurate predictions and may serve as an example of the benefit of multidisciplinary clinical care and future research efforts.