The implication of a translational triage tool in mass casualty incidents: part three: a multinational study, using validated patient cards

For this study, authentic patient cases representing victims of real MCIs have been collected and made into patient cards, utilized in disaster management courses [15‐18]. The Medical Response to Major Incident (MRMI) is a module-based simulation course that uses authentic and scientifically validated patient cards (MACSIM®), derived from victims of MCIs [19]. The patients have a necessary medical history, critical input, and output data for prehospital and hospital assessments [15‐18].

Five patient cards were selected by the representative for the MACSIM® system The study administrators had no involvement in this selection. Five patient cards were chosen because of the results of the previous study, in which the number of patients per healthcare provider defining an MCI was set to three [5]. It was assumed that five patients were enough to create an MCI, although no time pressure could be applied to the participants due to the study design. Figure 2 shows all MACSIM® patients, which were given to each participant. Participants received a normal MACSIM card initially to familiarize themselves with the characteristics of the card, and relevant signs before triaging the study´s five patients (Appendix A, Appendix B, Table 1).

The researchers actively engaged in a multinational research group, Research Alliance in Disaster and Emergency Medicine (RADEM) were contacted regarding possible participation. Additionally, researchers from countries previously engaged in the evaluation of TTT were invited. Purposefully, these commonly accepted subject-matter experts recruited new participants from their research networks. Although the current study aimed to compare the results of the traditionally used primary triage model in participating centers with TTT, the type of triage system used in each country was not an inclusion criterion.

Physicians, nurses, and paramedics actively engaged in primary triage in the fields of prehospital, emergency care, and or disaster medicine. Staff with no experience in triage procedures were excluded.

Representatives from eight countries (India, Italy, New Zealand, Poland, Saudi Arabia, Sri Lanka, Sweden, and Thailand), called subject-matter specialists, joined the study, and together recruited 164 participants from their countries. One of the participants did not conduct the second round, thus, was excluded. Another 5 participants were excluded due to incomplete data (Table 2).

Each subject-matter specialist recruiting national participants for the study was informed regarding the study and received information about MACSIM® cards, and how to interpret the cards [15‐19]. Moreover, they were informed about the study design and the period of study (November 2022-March 2023). Participants were not familiar with TTT and received lectures, face-to-face information, and/or written information. They had opportunities to test the tool and ask questions from each country leader. Each participant had to triage five (5) patients (numbered 1 to 5) by their national or institutional primary triage system in the first round and the TTT tool in the second round (the same patient cards). The time between the two assessments varied between 2 and 21 days (Table 2). This deliberate spacing aimed to serve the purpose of a washout period, ensuring that the effects of prior triage experiences did not influence subsequent assessments. Furthermore, neither the participants nor the country leaders were informed about the triage outcomes/keys. The results of each round were compiled and transferred to an Excel file and sent to the lead author. Each participant received full information regarding the study and how to participate and act in each round of the study. The results obtained after two triage rounds were to be statistically analyzed by one of the authors, with experience in statistical analyses, using validated statistical methods.

Each country leader was asked to comment on the performance of the participants and the feasibility and ease of understanding and performing the triage, using TTT. Participants could comment on the tool and its feasibility, and any concerns experienced during the trial. These qualitative data were collected by each country leader and were sent to the lead author for analysis and further assessment.

The statistical analysis was pursued to observe similarities or differences between the two systems (i.e., own system vs. TTT). Thus, testing whether one mutual triage system (TTT) can be used as a universal system. Professional statisticians validated the process and results. The diagnostic accuracy of the triage systems in the first round and TTT in the second round were assessed using a 2 × 2 contingency table. Cases were dichotomized into two groups: Immediate/Red/P 1 and non-immediate. The sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), over-triage, under-triage, positive likelihood ratio (LR+), and negative likelihood ratio (LR-) were calculated as compared with the reference standard, i.e., the outcome and the corresponding pre-determined color of the MACSIM® cards unknown to the respondents. The over-triage was calculated as 1- PPV based on the Cribari matrix method whereas under-triage rates were calculated as 1-sensitivity [20]. Wilson's method was used to calculate the 95% confidence intervals (CI) for all estimates of diagnostic accuracy (Tables 3, 4).

The study follows Swedish law's ethical guidelines and principles and is exempted from ethical approval requirements. In Sweden, where the study is conducted, ethical approval is mandatory if the research includes sensitive data on the participants such as race, ethnical heritage, political views, religion, sexual habits, and health or physical interventions or employs a method that aims to affect the person physically or psychologically [21]. Written and verbal information, including the study’s design, purpose, and data use, were provided for each participant. However, some countries required their ethical approval: The University of Otago, Wellington, ref D22/350 issued New Zealand ethics approval. Thailand ethical approval was issued by Mahidol University, Bangkok, ref MURA 2022/753.

There were several institutional, and national triage systems. The most used systems were Sieve and START triage, followed by other systems (the latter were grouped as one, called Mixed). Sieve and START are both recognized processes for prioritizing victims of MCIs.

Five patient cards were triaged by 164 participants (one missed the second round) on two rounds, resulting in 1575 triaged cases, of which, 790 cases were in the first round and n = 785 in the second round with TTT (n = 5 missing). START triage was used in n = 330 cases, Sieve in n = 290 cases, and in 170 cases mixed triage systems were used. The TTT algorithm had similar sensitivity as the Sieve primary triage, 100% (95% CI 96.8–100), and 98% (95% CI 93.9–99.5), respectively but a higher sensitivity than the START primary triage, 99.2% (95% CI 95.8–99.9) vs. 91.7% (95% CI 85.7–95.3). The TTT algorithm had a lower specificity, compared to Sieve, 36.8% (95% CI 30.0–44.2) vs. 73.6% (95% CI 66.6–79.6), whereas START triage systems used in the first round had a specificity of 77.2% (95% CI 65.6–78.0) compared to 65.7% (95% CI 58.8–71.9) for TTT. The positive predictive value (PPV) was lower for TTT at 51.3% (95% CI 44.8–57.8) compared with Sieve’s 71.3% (95% CI 63.8–77.7). PPV for the START triage system was 68.8% (95% CI 44.8–57.8), compared to 65.8% (95% CI 59.0–72.1) for TTT. Consequently, a higher rate of false positives for the TTT algorithm yielded an over-triage for the TTT of 48.7% (95% CI 42.2–55.2), compared to Sieve at 28.8% (95% CI 22.3–36.2). For countries using START in the first round TTT yielded an over-triage of 34.2% (95% CI 27.9–41.0) vs. 31.2% (95% CI 24.9–38.4) for START. Regarding under-triage, the TTT had almost zero under-triage (0–0.8%) in the largest Sieve and START group (Table 3). In the mixed system group, TTT had an under-triage of 4.5% but out of the three countries with mixed systems, the finding originated from one country, which had a relatively high misclassification rate overall of 10–26.1% under-triage (Table 4).

A variation in diagnostic accuracy was found between countries for the TTT algorithm. However, out of eight countries the TTT yielded a 100% sensitivity in all but one country, where TTT showed a sensitivity of 90.9%. The range of specificity was between 25.5 and 66.7% (Table 4). Most of the triage systems, and the TTT algorithm had good accuracy for triaging patients 1, 2, and 4. However, patients 3, and 5 caused some issues, leading to over-triage in almost all systems. When adjusting for the variance of country and patients regarding over-triage, under-triage, and accuracy we found that TTT in round two had a significantly higher over-triage compared to all other systems in the first round (estimate = 1.565, SE = 0.198, z = 7.904, p < 0.001). Under-triage was significantly lower for TTT compared to all triage systems (estimate = −1.828, SE = 0.407, z = −4.493, p < 0.001). Moreover, the accuracy for TTT was found to be significantly lower (estimate = -0.812, SE = 0.161, z = −5.055, p < 0.001). In- between system comparisons, the TTT algorithm had a higher over-triage as compared to Sieve (estimate = 3.2758, SE = 0.4391, z = 7.461, p < 0.001). There was no significant difference regarding under-triage (estimate = -17.523, SE = 1716, z = −0.01, p = 0.992). TTT had a significantly lower accuracy than Sieve triage (estimate = −2.774, SE = 0.388, z = −7.152, p < 0.001). For those countries (n = 5) using START in the first round, the TTT algorithm had a significantly higher over-triage (estimate = 1.036, SE = 0.350, z = 2.964, p < 0.01) and a significantly lower under-triage (estimate = −2.250, SE = 0.768, z = −2.298, p < 0.01). There was no significant difference in accuracy between TTT and START (estimate = −0.189, SE = 0.270, z = −0.699, p = 0.484). TTT appears to be superior in terms of identifying critically ill patient cases with lower under-triage compared to triage systems used in this study but at the expense of a higher over-triage rate and thus a lower accuracy.

The statistical values in the tables, such as sensitivity, specificity, accuracy, PPV, negative predictive value, over-triage, and under-triage, are all presented with their corresponding 95% confidence intervals. Additionally, the positive and negative likelihood ratios (LR + and LR-) are included for each system and country, providing a comprehensive overview of the performance of each triage system in the study.

Almost all participants expressed that the TTT algorithm was easy to understand and follow, and it took a shorter time to assess each patient and perform the triage. According to each country leader´s report participants could easily use and understand the algorithm. Some participants expressed difficulties in using the blue tag for some of the patients. The Blue color corner tag in SMART triage is referred to as expectance or the fourth priority. These are immediate and severe cases that cannot be handled immediately due to the lack of resources and or the fact that they cannot survive despite the best available resources.

Several limitations to this study should be acknowledged. First and foremost, the number of participants in the study was relatively small. In addition, the selection of the participants, people with knowledge of triage, may result in a selection bias of choosing those who may be more familiar/in favor of new tools such as TTT. A larger sample size and heterogeny population would provide a more robust statistical analysis and increase the generalizability of the findings. Second, the study did not measure the time taken to perform the triage procedures using the TTT algorithm compared to other triage systems. Time efficiency is an essential factor to consider when evaluating the effectiveness of a triage tool, particularly in mass casualty incidents where rapid decision-making is crucial. Third, the study utilized validated patient cards rather than real-life mass casualty incidents for triage evaluation. While this approach allows for a controlled comparison of the TTT algorithm and occurring triage systems, it may not fully capture the complexities and challenges that emerge in real-world scenarios. Fourth, the same cards were used on two different occasions, which might be considered as a bias. However, the deliberate spacing between the two tests was supposed to serve the purpose of a washout period, ensuring that the effects of prior triage experiences did not influence the next assessments. Furthermore, neither the participants nor country leaders knew the outcome of each card triage. Lastly, the study focused primarily on the feasibility of the TTT algorithm, with less emphasis on other factors such as resource allocation, patient flow, and inter-agency coordination. A comprehensive evaluation of the TTT algorithm should also consider the aforementioned factors to provide a more holistic understanding of its implications in MCIs. Future studies should explore these aspects in greater detail, preferably in a simulation exercise, to fully assess the potential benefits and drawbacks of implementing the TTT algorithm in disaster response efforts before its implementation in real-life events.