Effects of the application of a checklist during trauma resuscitations on ATLS adherence, team performance, and patient-related outcomes: a systematic review

A systematic search of studies listed in the electronic databases of Pubmed, Embase, CINAHL, and Cochrane was performed from their inception till January 2019. We combined the search terms derived from our research aim in separate search strings for each database (Table 1). Our search terms covered the intended population (Trauma resuscitation, including accessory search terms) and intervention (the application of a checklist, including accessory search terms). By purpose, we did not use search terms for outcomes, in order to avoid the introduction selection bias, as we intended to research all possible outcomes. In addition, all reference lists of included articles were screened for relevant additional citations.

Studies were included if the effect of the application of a checklist during resuscitation of trauma patients by a trauma team in an in-hospital setting was evaluated. We included studies with a randomized controlled—or controlled before-and-after study design. All other forms of observational study designs, reviews, case series or case reports, animal studies, and simulation studies were excluded. Solely English and Dutch publications from peer-review journals were reviewed. There were no restrictions on the year of publication, on the age of patients or on composition or method of application of a checklist during trauma resuscitation.

First, all eligible studies were selected by screening the title and abstract. Second, all selected papers were screened full-text. During these two phases, two investigators independently selected articles according to the predefined eligibility criteria and discussed their results. In case of any unresolved disagreements, a third reviewer made the final decision using the same eligibility criteria. The resultant articles after the two phases of screening were included in our systematic review.

Two reviewers independently extracted and inserted the following data into tables: Authors, year of publication, study design, research population, checklist’s items, composition, form, and application during trauma resuscitation and all identified effects reported by the included articles. To assess the methodological quality of the included studies, the Effective Public Health Practice Project Quality Assessment Tool (EPHPP tool) was applied [23]. This tool has been judged suitable to be used in systematic reviews of effectiveness [24] and has been reported to have content and construct validity [25, 26]. Furthermore, this tool is a reliable instrument to assess the quality of randomized controlled trials and before-and-after studies, whereas the overall inter-observer coefficient was found to be 0.77 (95% CI 0.51–0.90) [27]. This inter-observer coefficient value is considered as an excellent agreement [28] (Table 2).

The search strategy yielded 625 potentially relevant titles and abstract. After screening title and abstract, 20 articles remained for full-text screening (Fig. 1). After screening the full texts of remaining articles, three studies were included in our systematic review. No additional studies were found by assessing the reference lists of included studies.

All three included studies had a before-and-after study design to investigate the effect of the application of a checklist during trauma resuscitation. Two included studies of Kelleher et al. [29, 30] originated from the same pediatric trauma center and had the same sample. Two separate analyses and corresponding results were published in two different articles [30]. In their studies all trauma resuscitations had been captured on video during two 4-month periods and were analyzed retrospectively. They included 435 and 437 cases in their analyses, respectively. The third included study, of Lashoher et al. [31], was a multicenter study, including 11 hospitals in eight different countries. A random sample of 20% of all trauma resuscitations was assessed by live observants. Data collection duration varied among the different hospitals and ranged from 3 to 8 months pre- and 3–11 months post-implementation. In total, 3422 resuscitations had been assessed and the results were analyzed.

Using the EPHPP quality assessment tool, the quality of the two studies of Kelleher et al. [29, 30] and Lashoher et al. [31] were found to be moderate and weak, respectively (Table 3). Most concerning point that affected the quality of all three studies was the awareness of the participants (trauma team members) of being studied. This effect, also known as the Hawthorne effect [32], could have influenced behavior during trauma resuscitations. Two other points of concern were found in the study of Lashoher et al. [31], whereas no description of the randomization, withdrawal, and dropout of trauma resuscitations was provided.

Two different checklists were applied in the three included studies. A paper checklist was used in the two studies of Kelleher et al. [28, 29]. The checklist was applied by the surgical team leader (surgical senior resident, fellow, or attending) or a member of the emergency medicine leadership team (emergency medicine fellow or attending) during trauma resuscitation. The checklist was composed of 30 items extracted from the ATLS^™ guidelines, of which 14 items belong to the primary survey and 16 items belong to the secondary survey. The checklist was designed to track task completion by the bedside clinician (surgical junior resident or trauma nurse practitioner) and other team members.

In the study of Lashoher et al. [31], the application of the World Health Organization Trauma Care Checklist was investigated. The WHO trauma care Checklist is based on a literature review of medical errors during initial resuscitation of severely injured patients [33], and contains 19 core items, of which 11 items are tasks also described in the ATLS^™ guidelines. Hospital staff of all 11 hospitals participating in the study received checklist- and patient safety training.  Furthermore, hospital staff of each participating hopsital were encouraged to modify the checklist to be relevant in their unique setting. The study team of the core checklist approved site-specific final versions. The responsible member of the trauma team and form (e.g. paper or electronic) of the checklist during resuscitation was not described in the included article.

Six different outcomes-related to the applications of a checklist during trauma resuscitation were identified: Mortality, complications, missed injuries, task adherence, mean time to task completion, and workflow.

Lashoher et al. [31] found no difference in odds of mortality in the overall study sample after adjusting for patient demographics and injury severity. After an injury-severity-stratified analysis, checklist implementation was associated with a 50% reduction (adjusted odds ratio (aOR) 0.51, 95% Confidence Interval (CI 0.30–0.89) in mortality among patients with the most severe injuries (Injury severity score > 25). No such association was found among patients who were less severely injured (injury severity score < 25) [31].

Only the incidence of one of the ten complications, pneumonia, was slightly higher after the introduction of the checklist after adjusting for patient characteristics (aOR 1.69, 95% CI 1.03–2.80) in the study of Lashoher et al. [31]. No differences were found for the other nine assessed complications [31].

The incidence of missed injuries was zero in both groups and did not differ before and after implementation of a checklist after adjusting for patient characteristics and injury severity (aOR 0.62; 95% CI 0.19–2.03; p = 0.437) in the study of Lashoher et al. [31].

Fourteen of the thirty ATLS-related tasks that had been assessed in the study of Kelleher et al. [29] were completed significantly more frequently after checklist introduction (for all p ≤ 0.05). None of the 30 ATLS-related were performed less frequently. After adjustment for type of resuscitation factors and type of tasks (tasks were not further specified), the odds of completing primary survey tasks were 2.66 (95% CI 2.07–3.42) times higher after implementation, and the adjusted odds of secondary survey task completion were 2.46 (95% CI 2.04–2.98) times higher. Lashohor et al. [31] found 18 of the 19 clinical tasks to be significantly (p < 0.05) more frequently performed after implementation of the WHO trauma care checklist.

Kelleher et al. [30] found that the average adjusted time to task completion was 9 s faster with the application of a checklist (95% CI − 13.8 to − 4.8 s) Furthermore, six of the 12 primary survey tasks (full exposure, temperature measurement, blood pressure, heart rate measurement, respiratory rate measurement, and oxygen saturation) were performed significantly (all p < 0.05) faster (− 20 till − 43 s) [29].

A predefined ideal process model of six ATLS primary survey tasks was created in the study of Kelleher et al. [30] The six ATLS tasks were categorized into Airway (A), Breathing (B), Circulation (C), and Disability (D) which is well known from the ABCDE mnemonic as described in the ATLS Guidelines. The model required that “A” tasks are done before “B” tasks, “B” tasks are done before “C” tasks, and “C” tasks are done before “D” task. Model fitness (degree of agreement with the ideal process model, ranging from 0 to 1), conformance (“yes” if fitness 1; “no” if fitness < 1) was measured and the frequency of unique workflow traces were analyzed. Besides the comparison of the application of a checklist, analysis of a brief pre-arrival notification was performed. The authors found that after implementation of a checklist workflow improved. Model fitness (0.80 vs 0.91; p = 0.007), conformance frequency (26.1% vs 59.4%; p = 0.01), and number of unique workflow traces (19 vs 16; p = 0.01) improved for resuscitations without notification. Comparable improvement was shown after the implementation of the checklist when a brief notification was given, in which case Model fitness (0.90 vs 0.96; p < 0.001), conformance frequency (50.8% vs 77.6%; p < 0.001), and frequency of unique workflow traces (63 vs 35; p = 0.005) also improved. Finally, the effect of the absence of pre-arrival notification on model fitness was also smaller after checklist implementation compared to before the use of the checklist (Cohen’s d = 0.75 (95% CI 0.73–0.82) vs. Cohen’s d = 0.52 (95% CI 0.51–0.56).