The performance of the EMS triage (RETTS-p) and the agreement between the field assessment and final hospital diagnosis: a prospective observational study among children < 16 years

This study is an observational prospective study with a retrospective analysis of paediatric patients assessed by an EMS nurse and transported to the PED. Prospectively, we informed the EMS staff about the study and educated the staff at several workplace meetings at all nine of the ambulance stations in the EMS organisation. The education was made up of a repetition of the triage process with the RETTS-p triage protocol and also repetition of patient assessment according to local guidelines. This was further emphasised in official weekly newsletters sent out to all the EMS staff before and during the study. The purpose was to minimise bias and to achieve a uniform patient assessment and also to increase the data quality of the prehospital patient records.

The study was conducted in a single-site urban setting in the western part of Sweden. The EMS organisation covers 900 km², with a population of 660,000 inhabitants. The EMS carries out more than 80,000 assignments every year, of which 58,575 assignments are identified as primary assessments and approximately 3100 are children < 16 years of age.

All health care in Sweden is tax funded and free to residents of Sweden, regardless of the type of disease. In Sweden, the twenty-one county councils are responsible for providing health care for the residents within the county, including the EMS. The EMS can be organised within the body of a university hospital, county or contracted to a private company. In the study organisation, the EMS is organised under the university hospital. The EMS has 22 units, of which 18 are ambulances all with advanced life-saving (ALS) capacity and four special units; two single responders manned by one specialised trained registered nurse, one emergency physician-manned unit and a scene-command unit. Within the urban area, there is a designated children’s hospital with full capacity to handle all kinds of paediatric ailment, resulting in short transportation times from the scene to the PED. In total, the EMS carries out approximately 5.4% runs annually with children < 16 years of age. The retrospective patient data were collected from the EMS organisation patient records and from the hospital medical records for the specific in-hospital care event. Each record was reviewed manually and unidentified data such as vital signs, triage level, EMS nurse assessment and ICD code were then entered into a registry of which statistical analyses were conducted in a aggregated form.

The RETTS-p triage system is similar to the MTS triage system in its organisational structure, with flowcharts for each presentation. In order to identify patients at risk of deteriorating, the RETTS-p has included vital signs (VS) in each flowchart. The RETTS-p was developed and is maintained and licensed by a Swedish company (Predicare AB). The RETTS-p consists of two components, VS and emergency symptoms and signs (ESS). VS (SaO2, respiratory rate, pulse rate, temperature and level of consciousness) are recorded for each patient, together with an ESS stating the level of severity. The highest triage level of the VS or ESS constitutes the final triage level. The patient is triaged to a triage level colour ranging from red, defined as life threatening, orange potentially life threatening, yellow, green and blue. Red and orange are considered to be in need of a direct medical evaluation by a physician to what is defined as an acute process. The triage levels of yellow and green can wait without jeopardising the individual medical risk, but yellow is regarded as a higher level due to deviating VS or signs and symptoms (ESS) that should be evaluated by a physician before green-triaged patients. The blue level is the lowest triage colour and alternatives other than the PED may be more appropriate. At the time of this study, only triage levels green to red were used in the EMS nurse triage.

When a patient dials the Swedish emergency number (112), a dispatcher assesses the patient according to a medical index and assigns the patient a priority of 1 to 4. Priority 1 is life threatening and one or two ALS units respond to the patient with blue lights and sirens. Priority 2 is considered urgent and a unit should respond within 30 min. Priority 3 can wait without medical risk. Priority 4 is assigned when there is a need for transport only and it is carried out by a non-emergency patient transport manned by one emergency medical technician. For priority 1–3, it is mandatory according to legislation in Sweden that a registered nurse conducts the assessment at the scene and is responsible for the administration of drugs. All ambulances in Sweden are therefore manned by a registered nurse, often with a postgraduate education in prehospital emergency care. Ambulance crew set-ups can be two registered nurses or one nurse and one EMT. The EMS nurse has been given the responsibility independently to decide upon the level of care which includes 1. Transport to hospital, 2. Arrange an appointment at a primary care centre and 3. Treat and release and/or give advice on self-care. To aid in the assessment, the EMS nurse uses the triage system (RETTS-p) to help discriminate between degrees of severity and a telephone consultation with a paediatrician at the PED, especially in cases where the patient is non-conveyed, is encouraged. The EMS nurse’s on-the-scene triage level becomes the triage level at the PED. To increase patient safety, the handover to the PED nurse always takes place together with the patient/ significant other.

Defining a true emergency patient is problematic and, as a result, no gold standard exists [10]. Predominantly, two options are available when aiming to test the internal validity of triage systems: 1. by proxy variables, for example, interventions performed and/or the need for hospitalisation and 2. physiological factors such as vital signs or blood samples. We have used a mixture of these in this study and a similar approach has been used when evaluating the MTS [10‐12]. An “emergent” patient is defined as having abnormal VS according to the definition or a potentially life-threatening condition according to the final diagnosis. Reference VS that are compared against the RETTS-p VS are based on the paediatric increased risk of mortality acute physiology score III (PRISM III-APS) [13]. An increased risk of mortality in the PRISM-APS has been regarded as a qualifier for an “emergent” patient. Furthermore, SaO2 is part of the RETTS-p VS, but, in the PRISM III-APS, PaO2 is reported. SaO2 has been converted to PaO2 using a formula presented by Severinghaus, defining SaO2 of < 91% as the outside reference with an increased risk of mortality [14] Additional file 1. Potentially life-threatening conditions are defined according to conditions presented in previous studies [10, 11]. Examples of these conditions are sepsis, meningitis, high-energy trauma, substantial blood loss and intoxication Additional file 2. We have used a sequential step comparison of 1. qualification against VS and 2. comparison against defined potentially life-threatening conditions. This is an approximation of criterion validation (true emergency) and the RETTS-p triage levels of red or orange fulfilling the criteria are regarded as a true positive.

The EMS assessment, in addition to the triage level, has been collected from EMS records, where the EMS nurse registers the assessed condition and/or symptoms. In order to compare the EMS field assessment with the final physician diagnosis in hospital, a diagnosis classification instrument has been used. This instrument categorises life-threatening diagnoses and non-life-threatening diagnoses and has previously been used in another study [15]. It has also been further explained in a study protocol [16]. The instrument comprises five categories (A-E) and is described in Table 4. In cases where it was difficult to determine the category, the case was discussed in a group to reach consensus about the classification.

The results in this study are presented as numbers and percentages or as the median with 25th and 75th percentiles. Missing data have been denoted in each table and, in Table 3, only patients recorded with a final triage colour in the ambulance were included. In Table 1, group comparisons were performed using Fisher’s exact test for categorical variables and the Mann-Whitney U test for continuous variables. To calculate sensitivity, specificity, positive and negative predicted values and likelihood ratios, data from Table 3 were used, where RETTS-p levels red and orange were regarded as a positive test and an emergent patient was defined as having either life-threatening VS or a potentially life-threatening condition. Under-triage and over-triage was defined as 1-Sensitivity (proportion of yellow/green triaged among all emergent patients) and 1-Specificity (proportion of red/orange triaged among all non-emergent patients) respectively. All tests are two-sided and p-values below 0.01 are considered significant due to the number of tests in Table 1. SPSS version 22 was used to perform data processing and statistical analysis.

Of the 454 assessed patients, 390 (85.9%) patients received a triage colour according to RETTS-p from the EMS nurse. The group associated with a triage colour was significantly older. The majority of the patients in both groups were males, had no previous medical history, arrived at the PED between 4 pm and 12 pm, visited the PED for a medical ailment and were discharged from the PED with an intervention by a physician such as X-ray, ultrasound, suturing, blood sample, medical treatment or prescription. Of all the children, 11% were identified with VS associated with an increased risk of death and 7% were diagnosed with a potentially life-threatening condition. A total of 31.7% of all children were discharged from the PED without any intervention (Table 1).

A total of 390 patients transported to the PED were assessed with a triage colour according to RETTS-p by the EMS nurse. Of them, 149 patients (38.2%) were triaged to a level red (life threatening) or orange (potentially life threatening) condition, according to RETTS-p guidelines. Of them, 55% of the red- and 31% of the orange- triaged patients were hospitalised (Table 2). According to the definition of an emergent patient, based on the classification of VS and a potentially life-threatening condition, only 40 patients (26.8, 95% CI [22.5,31.7]) at the red and orange triage level qualified as emergent patients. We found that the majority of the patients admitted to in-patient care were not classified as emergent patients according to VS or a potentially life-threatening condition (Table 3). Eight patients with life-threatening VS according to the definition had more than one VS outside the reference interval. An abnormal heart rate (n = 24) and an abnormal temperature (n = 12) were the most frequent presentations among the patients identified with VS according to the emergent patient category. If triaged to yellow or green, the predictive value of not being life threatening or potentially life threatening was 91.7, 95% CI [88.5,94.1]. The RETTS-p triage levels of red and orange had a sensitivity of 66.7% for detecting an emergent patient, 95% CI [53.3,78.3], i.e. an under-triage of 33.3%. The corresponding specificity was 67.0, 95% CI [61.6,72.0], i.e. an over-triage of 33.0%. The positive likelihood ratio was 2.02, 95% CI [1.59, 2.56], and the negative likelihood ratio was 0.47, 95% CI [0.33, 0.67].

A total of 412 of 454 patients transported to the PED received a hospital diagnosis (ICD-10-SE). The remainder were treated by a nurse practitioner in the PED or the patient left the PED before being examined by a physician. Thirteen patients (3.2%) were classified with a life-threatening specified hospital diagnosis and 320 patients (77.6%) were classified with a specified hospital diagnosis but not life threatening. Another 79 patients had a final diagnosis described as a symptom (for example, dyspnoea) or a diffuse assessment such as observation or evaluation of a non-defined incident. The EMS nurse’s field assessment was in agreement with the hospital diagnosis in 329 (80%) of the cases. Among the remaining 83 cases, the field assessment was not in agreement with the hospital diagnosis or the symptoms reported by the EMS nurse were atypical or uncommon for the hospital diagnosis. In 33 patients (8%), the EMS field assessment diverged from the hospital diagnosis. One of these patients had a life-threatening hospital diagnosis with a fatal outcome in the PED (Table 4).

Of the total number of children that were transported to the PED in this study, 13 (3.2%) were given a specified hospital diagnosis identified as life threatening, such as toxic shock syndrome, intracerebral haemorrhage or anaphylaxis. The prevalence of critical conditions in the PED has been reported in studies, with rates of 0.28–4.2%, depending on the region, patient cohort and the definition of a critically ill child [31‐34]. The fact that this study only involved the patients initially in contact with the EMS may have increased the prevalence, albeit low overall. In the majority of patients, the field assessment was in agreement with the final hospital diagnosis, with only one patient classified as having a life-threatening final diagnosis which was not in agreement with the field assessment. However, the diagnosis for another 32 patients of those classified with a non-life-threatening final diagnosis was not in agreement with the EMS field assessment. Paramedics’ ability to predict hospital admission and the aetiology of the condition in adults has been shown to be moderate in several studies [35‐37]. In one study, 19% of the paediatric transports with lights and sirens were later identified as unnecessary [38]. In a study of adults from Ireland, advanced paramedics showed agreement in 70% of the cases when the field diagnosis and the ED physicians’ working diagnosis were compared. Areas of less agreement included patients with no abnormalities [39]. These results suggest that it is particularly difficult to exclude a life-threatening condition, as further tests and examinations are required after arrival at hospital. Overall limited exposure and lack of training may be one reason for a less appropriate field assessment, which has been previously reported [40, 41]. Furthermore, environmental characteristics also play a role in the assessment, where the EMS is confronted by various challenges, such as worried parents, language barriers with no support in interpretation on the scene or no observable abnormality.

In this study, 14.1% of the patients were not triaged to a colour, which can be explained in several ways. Firstly, short transportation times may have influenced the triage before arrival at the PED. Secondly, patients were significantly younger in the non-triage group, indicating difficulties either in the EMS nurse’s competence assessing infants according to the triage system or in obtaining VS in these patients. A lack of VS in paediatric patients in the pre-hospital setting has also been previously reported, with an increasing rate of fewer recorded VS in younger children [42, 43]. This is worrying, as the youngest children run a higher risk of serious bacterial infections [10, 44]. Thirdly, some of these patients were transported directly to the emergency room and the EMS nurse may have been occupied with patient interventions before arrival at the PED. As a result, the patient was assessed to the highest triage level, even though this was not recorded. However, the number of patients not triaged to a colour in this study appears to be similar to studies of other triage systems, such as the MTS in the PED, which reported a missing triage level in 5–16% of the patients [10‐12, 45]. On average, the EMS nurses in the study organisation assess approximately 50 paediatric patients every year in an unselected patient population. This may indicate that the lack of exposure has an impact on triage knowledge and assessment. On the other hand, the use of a triage system by the EMS clinician at the scene in every case may help to identify patients at risk in a systematic approach when assessing patient severity. A triage system with both VS and ESS may contribute to increased patient safety when utilised in a setting and used by nurses with limited exposure to paediatric conditions, especially since the ESS part of the system acts as a checklist based on severity for each condition within the triage system. However, adherence to guidelines and protocols has been reported with large variability and suboptimal adherence for certain patient groups in the EMS (7.8–95%) [46]. One survey of EMS nurses reported adherence to guidelines of 84% and one factor explaining the variation in adherence was the characteristics of the guidelines [47]. Using manual triage systems, where the EMS nurse has to recall from memory when documenting the triage process, can be problematic. A study of the adult CTAS revealed improved performance when implementing a computerised triage system compared with traditional manual triage [48]. Introducing an electronic version of the RETTS-p adapted for pre-hospital use and incorporated in a decision support system, in combination with regular training on paediatric assessment, may increase adherence to and confidence in the system. Furthermore, the introduction of dedicated units for paediatric cases with staff rotation into the PED would increase exposure and competence in the at-the-scene assessments.