Including oxygen supplement in the early warning score: a prediction study comparing TOKS, modified TOKS and NEWS in a cohort of emergency patients

Early warning scores (EWS) are widely used in emergency departments and on general wards to detect critical illness and deterioration among hospitalized patients [1]. These systems are implemented to prevent serious adverse events e.g. cardiac arrest, admission to the intensive care unit (ICU) or death [1, 2].

Patient deterioration is often preceded by deviations in vital signs [3], and logically early warning scores are primarily based on these. For each vital parameter, cut-off points are set to determine the “normal range” and grade the extent of deviation. Points allocated for each vital sign add up to an aggregate EWS.

EWS is part of a rapid response system. A rapid response system consists of an afferent limb with staff measuring vital parameters and calculating a score, and an efferent limb consisting of an algorithm to trigger a call for a physician or medical emergency team [4].

Several EWS have been developed, implemented and adjusted worldwide [5]. In Denmark, the five regions use different locally established scores [6‐8]. The score used in Central Region Denmark is named TOKS (Table 1) and was developed locally by a group of doctors and nurses. A guideline was released in 2010 [7]. TOKS was established by consensus decisions and has not been validated in an emergency setting. TOKS differs from other EWS by the selected cut-off points, and by not including oxygen supplement, despite high discriminatory performance of other scores including oxygen [2].

The aim of this study was to compare TOKS with a modified TOKS (mTOKS) including 2 points for oxygen treatment. Moreover, we aim to compare the discriminatory ability between TOKS, mTOKS and the National Early Warning Score (NEWS) [9].

We hypothesize, that the ability of TOKS to discriminate patients at risk of dying could be improved, if two additional points are allocated to patients in supplemental oxygen therapy.

In this study of emergency department patients, we found that including 2 points for oxygen supplement in TOKS improved the score’s ability to predict death within 7 days. This finding suggest that it could be beneficial to include oxygen therapy in TOKS. Furthermore, our study shows that NEWS had a statistically better discriminatory ability, when it comes to predicting 7-day mortality, than both TOKS and mTOKS respectively. However, all models showed moderate to good discriminatory ability.

Discussion and continuous improvement of tools to detect critical illness in hospitals is essential to ensure patient safety. In this study we chose to look at oxygen therapy, because this treatment influences other vital signs, altering the aggregate EWS. Oxygen therapy is used for patients with respiratory distress and helps decrease the respiratory rate, increase saturation thus reducing hypoxemia [15]. In the acute setting, oxygen therapy is merely a symptomatic treatment, which does not necessarily improve the underlying cause of respiratory distress. Because oxygen therapy might to some degree mask an underlying threatening condition, it could justify compensatory points. Furthermore, oxygen supplement could simply be prescribed based on a gut feeling before vital signs are measured.

Studies from United Kingdom (UK) have shown that the national early warning score (NEWS) has the best discriminatory ability of in hospital mortality, cardiac arrest and transfer to ICU when compared to 33 other early warning scores [2]. NEWS was developed empirically by a group consisting of doctors and nurses appointed by the Royal College of physicians in 2012 [9]. They first developed a forerunner called Vitalpack EWS (ViEWS) [1]. Pryterch et al. stated that ViEWS emerged by a “trial and error” approach, to gain the best AUROC discrimination of 24-h in-hospital mortality. This was obtained by adjusting the weightings and vary the ranges for each EWS component. Two points for oxygen supplement was included in the final version of NEWS, because it improved the overall discriminatory ability, as evident by an increase in AUROC. The clinical guideline on TOKS was published in 2010 [7]. NEWS was not yet developed at the time and other studies on EWS generally showed low sensitivity [16, 17]. Since no specific scoring system seemed optimal, TOKS was developed based on available international literature combined with consensus decisions.

We excluded 37% of the patients because they did not have a full set of vital parameters to calculate TOKS, mTOKS and NEWS. Many patients attended the EDs in Central Region Denmark due to minor injuries e.g. a sprained ankle or a broken finger. These patients will, according to the triage algorithm, automatically not have vital signs measured (supplementary Table 1). Patients may also be excluded due to inadequate documentation of vital signs. A Danish study from Petersen et al. found that inadequate documentation is often due to lack of time [18]. Further, since oxygen supplement is not a part of TOKS, it is questionable how well this parameter is documented. However, we found that a minority of patients were excluded solely because they lacked documentation on oxygen supplement, indicating documentation of oxygen supplement to be a part of the daily routine.

The outcome was death within 7 days after the first set of vital signs were measured. We could have chosen 24- or 48-h mortality. These short-term endpoints might have a stronger correlation to the EWS score, thus presumably improving the discriminatory ability of all three early warning scores. Thus, choosing 7-day mortality could underestimate the discriminatory ability.

AUROC may be optimistic, since no data splitting was conducted. However, using univariable models we do not expect the true AUROC to differ significantly.

Many different endpoints have been used in the search of the best EWS. e.g. Cardiac arrest, transfer to ICU, death or in-hospital mortality. Concerns about the value of these endpoints have been discussed. Critics say that one should be careful not to optimize EWS only to find patients who cannot be saved, even with timely and correct care [19]. Thus, it is important that we develop an EWS that can predict critical illness within patients whose condition can be turned around.

We were not able to exclude patients with “do not resuscitate” orders. Unfortunately, these patients may fall into the group of patients who cannot be saved and are therefore more likely to die. This could underestimate the discriminatory ability of all three early warning scores. In future studies this group should be eliminated.

We chose to convert the GCS into AVPU in order to use the vital parameters from triage. The applied conversion algorithm has not been validated. TOKS allocates one point to the agitated person, but GCS does not account for agitation. This means that some agitated patients will get 0 points because level of consciousness was assessed using GCS. This could underestimate the discriminatory ability of TOKS and mTOKS if agitation has a significant correlation to death.

This study has multiple strengths. We used a large database including patients from five different hospitals in Central Denmark Region. Data on vital signs were collected as a part of the daily routine. This makes the study population representative to the Danish population in general. In future studies it could be interesting to compare all the EWS systems in Denmark, test other cut-off values for each vital parameter and incorporate other endpoints than death.

Inclusion of oxygen supplement in the early warning score TOKS improves the ability to discriminate patients at risk of death within 7 days. The discriminatory ability of NEWS is statistically superior to both TOKS and mTOKS when it comes to discriminating 7-day mortality, but all models show moderate to good discriminatory ability.