https://orcid.org/0000-0002-4496-0893
Figures
Abstract
Background
The Global Trigger Tool (GTT) has become a worldwide used method for estimating adverse events through a retrospective patient record review. However, little is known about the facilitators and the challenges in the GTT-implementation process. Thus, this study followed two aims: First, to apply a comprehensive set of feasibility criteria to qualitatively and systematically assess the GTT-implementation process in three departments of German university hospitals. Second, to identify the facilitators and the obstacles met in the GTT-implementation process and to derive recommendations for supporting other hospitals in implementing the GTT in clinical practice.
Methods
The study used a qualitative documentary method based on process documentation, with written and verbal feedback from the reviewer, as well as evaluating the study sites during the implementation process. The study was conducted in three departments, each in a different German university hospital. The authors applied a comprehensive set of 22 feasibility criteria assessing the level of challenge in GTT implementation. The results were synthesized and they focused on the facilitators and the challenges.
Results
Of these 22 feasibility criteria, nine were assessed as a low-level challenge, eleven regarded as a moderate-level challenge, and two with a problematic level of challenge. In particular, the lack of time and staff resources, the quality of the information in the patient records, organizational procedures, and local issues, posed major challenges in the implementation process. By contrast, the use of local coordinators and an external expert made important contributions to the GTT implementation.
Citation: Brösterhaus M, Hammer A, Gruber R, Kalina S, Grau S, Roeth AA, et al. (2022) Using the Global Trigger Tool in surgical and neurosurgical patients: A feasibility study. PLoS ONE 17(8): e0272853. https://doi.org/10.1371/journal.pone.0272853
Editor: George Vousden, Public Library of Science, UNITED KINGDOM
Received: April 14, 2020; Accepted: July 28, 2022; Published: August 16, 2022
Copyright: © 2022 Brösterhaus et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: In accordance with the German data security law (DSG-VO) and the requirement of the ethical committee, data from the SafeCulture study may not be shared in the public domain, as the number of cases and departments participating in this study will not guarantee anonymity of participants. However, data will be used by students of the Institute of Patient Safety and a participating department for their theses. Data will be stored in accordance with national and regional data security standards and may be obtained on request by contacting the first authors. The Institute of patient safety in Bonn (ifps@ukbonn.de) is the institutional point of contact that can field data access queries.
Funding: The SafeCulture study was funded by the BONFOR-research funding program of the Faculty of Medicine of the University of Bonn [grant number: 2015-1-25]. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. URL of the funder website: https://www.medfak.uni-bonn.de/de/forschung/foerderung/interne-foerderung/bonfor.
Competing interests: The authors have declared that no competing interests exist.
Background
Routinely collected data (including the data in the medical records) was identified as one of the most important components of an effective system for safety measurements that hospitals worldwide should implement [1]. To learn from and act on adverse events (AEs), this was critical to provide suitable instruments for quantifying and characterizing the AEs [2]. The patient record review (PRR) (also known as the medical record review, or the retrospective chart review) is currently the most common method for quantifying patient safety outcomes, such as the AEs, for research purposes [3].
One PRR tool that is used is the Institute for Healthcare Improvement’s (IHI) Global Trigger Tool (GTT). Since its development, the GTT has been used worldwide as a standardized two-stage method for the retrospective PRR of randomly selected patient records [4–10]. International studies have shown that the GTT is an effective tool for an AE assessment in hospitals [11–13]. It has several strengths (for instance, an active involvement of the clinical staff in the quality assurance processes; a standardized and rapid method for recording and monitoring the AEs; and an identification of the AEs, independently of the employees’ willingness to report incidents) [3, 14–16]. The GTT has become a standard measure for patient safety in several countries (for example, Sweden and Norway [17–19]). However, in other countries, such as Germany, the GTT is not yet routinely used for an AE assessment. Furthermore, very limited information is available concerning the feasibility of implementing the GTT into routine practice. A study by Schildmeijer et al. [20] evaluated the process of applying the GTT using focus groups. Adler et al. [21] and Von Plessen et al. [22] both described the lessons learned in setting up the GTT in a hospital setting and they presented recommendations for the implementation thereof, while Adler et al. [21] considered solely the entire implementation process, including leadership commitment, reviewer training, and the review process. Even so, no systematic assessment of feasibility when using predefined criteria to identify the facilitators and the challenges in the implementation process is available. Such information could be used to further promote the GTT implementation internationally.
The authors, therefore, designed this study followed two aims: First, to apply a comprehensive set of feasibility criteria to qualitatively and systematically assess the GTT-implementation process in three departments of German university hospitals. Second, to identify the facilitators and the obstacles in the GTT-implementation process, including local management, reviewer recruitment, reviewer training, patient record review, and the reporting of results to derive recommendations. This would support other hospitals in implementing the GTT, as a patient safety assessment tool in daily clinical practice.
Methods
Study object and setting
This feasibility assessment investigated the application of the GTT in a retrospective cross-sectional SafeCulture study. More details of the study are published elsewhere [5]. The study was conducted from 07/2016 to 09/2017 in three departments each in a different university hospital in Germany. The GTT monitors the AEs, by using a series of warning signals, so-called triggers, indicating that an AE might have occurred. In this study, the researchers used two adapted versions of the German GTT, one the PRR in general surgery, and the other in the neurosurgery departments [5]. The general surgery tool included 48 triggers, and the neurosurgery tool had 59 triggers. The GTT was carried out in two departments for general surgery (GS1 and GS2), with 72 and 91 beds, respectively, as well as in one neurosurgery department with 44 beds (NS). The entire implementation process of the GTT is summarized in Table 1. The process involved (1) local management (for example, information and approval from the hospital management and staff council; information on the local staff; and the appointment of a local coordinator); (2) reviewer recruitment; (3) a standardized one-day reviewer training (two-phases), with each reviewer team (consisting of two reviewers and one secondary reviewer) in the three departments; (4) a two-stage PRR process, with primary reviewers (medical students (n = 2), registered nurse (n = 1), physicians (n = 3)), and secondary reviewers (physician (n = 3)), per each hospital. These individuals all possessed the clinical background knowledge about patient record content and the layout, as well as the care provisions [23]; and (5) reporting and discussion of the identified AEs. Before the two-stage PRR, each team was trained on the GTT. The reviewer training was conducted with the support of an external expert. In line with the IHI recommendations, the reviewer training consisted of two phases, without any time limit. In the first phase, two to three local inpatient records per department were reviewed by the primary and secondary reviewers, together with the assistance of an external expert. Afterward, the reviewers discussed their findings and the key findings of each patient record. After the discussion, and with the support of the external expert, the reviewer team agreed on a common understanding of the process and the definitions for PRR. In the second phase, the reviewer team completed a pilot review by using a selected set of three further local inpatient records per department. As well as in the real PRR, and in contrast to the first phase of the training, the primary reviewers reviewed the patient records without the secondary reviewer. The secondary reviewer only assessed the identified adverse events. After the assessment, the found triggers and adverse events were discussed together. The local management and reviewer recruitment began in August 2016, and the reviewer training took place in November 2016 and January 2017. The patient records were drawn from two consecutive months for the patients who were discharged during the given period, between 12/2016 and 01/2017. The two-stage review process was conducted between 01/2017 and 06/2017, depending on the individual time resources of the reviewer within the three departments. The reporting and the discussion of the results took place between 07/2017 and 09/2017.
Data collection
The study used a qualitative documentary method [25] to systematically assess the feasibility of the GTT-implementation process. The data for the feasibility assessment was gathered at the Institute of Patient Safety, by two of the research associates, MB and AH, and was supervised by TM, between 07/2016 and 09/2017. For the assessment basis, two different sources were used: 1) Documented process logs on the implementation by the two research associates, MB and AH; and 2) Verbal and written feedback from the reviewers and the local coordinators, by MB maintaining regular telephone and written contact at the three departments throughout the implementation processes (including local management, reviewer recruitment, reviewer training, patient record review, and the reporting of the results).
Ethical approval
This SafeCulture study was approved by the Ethics Committee of the Faculty of Medicine, the University of Bonn, Germany (Ldf. Nr. 310/14). In addition, this research was sanctioned by the local Ethics Committees of the participating hospitals. Following the data security policies and the German professional code of conduct for physicians (§15 of the professional code), informed consent of the participants was not required, as the data was exclusively collected in-house by the medical staff of the respective hospitals; and the data did not include any personal or identifiable information of the patients, which was respectively analyzed anonymously.
Data analysis
For the analysis of the process logs, as well as the verbal and written feedback, a set of qualitative feasibility criteria was adapted from Orsmond and Cohn [26]. The adaptation of the set was conducted by the four authors, MB, RG, AH, and TM. In the first step, the three authors, MB, RG, and AH independently rated all of the feasibility criteria from Orsmond and Cohn, concerning their suitability for the study purpose and whether any criteria were missing. The feasibility criteria that were chosen were then compared, discussed, and agreed upon on a consensus involving TM. In a second step, the set of feasibility criteria was adapted for the context of the study and linguistically. For example, Orsmond and Cohn worded the individual feasibility criteria as questions that were allotted into five topics. For better comprehensibility, the authors linguistically adapted the feasibility criteria from Orsmond and Cohn and transformed the questions into statements. For example, criterion 13 was changed from ‘What are the recruitment rates?’ to ‘Sufficient recruitment of appropriate reviewers’. Furthermore, the structure of Orsmond and Cohn’s list of criteria was generally retained but five feasibility criteria were combined from two topics into a new topic. Hence the creation of ‘rating the reviewer recruitment and capabilities’. The feasibility criterion of ‘What are the retention and follow-up rates as the participants move through the study and intervention?’ was deleted. Instead, one criterion to assess the quality of the information in the patient records was added, as this was an important aspect of the implementation process. Moreover, the order of some criteria regarding the implementation process was changed. For the third step, this set of criteria was reviewed by TM, resulting in minor changes, for example, spelling, and the use of terms, prior to finalization.
The final assessment tool consisted of 22 feasibility criteria that concerned five topics: (1) Evaluation of the resources and the ability to manage and implement the GTT (five feasibility criteria); (2) Evaluation of the availability of the patient records and the resulting sample characteristics (six feasibility criteria); (3) Evaluation of the reviewer recruitment and capability (five feasibility criteria); (4) Evaluation and the refinement of the data collection procedures and measures (three feasibility criteria); and (5) Preliminary evaluation of the implementation of the GTT (three feasibility criteria). The adapted and the original feasibility criteria from Orsmond and Cohn are presented in Table 2.
Based on the documentation and guided by the feasibility criteria, MB, RG, and AH synthesized the process logs and the verbal and written feedback, focusing on the facilitators and the challenges in the implementation process, by allocating them to the individual feasibility criteria. MB and AH then independently assessed the level of challenge during the implementation process for each criterion. Again, by involving TM in the next step, the results of the independent assessment were combined, compared, discussed, and agreed on a consensus between the three authors. For a better understanding, the single feasibility criterion was visualized as follows: red = problematic (several challenges), yellow = moderate (a few challenges), and green = low (almost no challenges). Based on this synthesis, the authors critically discussed and derived general recommendations regarding the five topics to support the hospitals in their approach when implementing the GTT into local practice.
Results
Out of the 22 feasibility criteria, nine were assessed with a low level of challenge, eleven with a moderate challenge, and two with a problematic level of challenge. Table 3 provides an overview of the levels of challenge during the GTT implementation for all of the criteria. The following syntheses of the results are structured according to the five topics and the 22 feasibility criteria from Orsmond and Cohn [26].
Resources and the ability to manage and implement the GTT
Out of the five feasibility criteria in this topic area, two were assessed as presenting minor challenges, and three were designated as moderate challenges.
(1) Availability of administrative capability, expertise, skills, time to conduct the study, and the implementation of the GTT (moderate level of challenge).
The implementation of the GTT was supported by a local coordinator that was experienced with the local requirements, and this person organized the implementation process at each site (for example, reviewer recruitment, selection of the patient records, organization of the equipment for the data collection). Prior to the PRR, the reviewer training was conducted with an external expert, who was familiar with the GTT application but who was not directly involved in the local practices. The external expert was important for ensuring that the local reviewers understood the purpose of the measurements. Additionally, the external expert maintained a neutral point of view in the initial reviews, while providing advice, as appropriate. However, while the external expert was planned and budgeted in the study, the involvement of an external expert would require additional financial resources and efforts in engagement by the hospitals. Concerning the general management of the implementation of the GTT, the process was properly scheduled, by ensuring that there was enough time for training the reviewers in each department, and for preparing and conducting the data collection when using the GTT.
(2) Fulfillment of ethical and data security requirements (moderate level of challenge).
For the entire study, approval was obtained from the Ethics Committee of the Medical Faculty of the University of Bonn, Germany. As the implementation of the GTT was part of a research project, all of the physicians involved in the data collection required ethical and professional counseling, in accordance with German law (Art. 15 of the Professional Code for Physicians in Germany). Moreover, clarification on data security and ethical issues for all members of the reviewer teams was ensured. A minimal challenge in direct communication between the physicians and the responsible ethical committees was experienced. The entire process of ethical and professional counseling took, on average, three to four weeks. In Germany, only professionals who work in the departments treating the patients have access to the patient records. Thus, the main challenge in this area was the lack of access rights for the reviewer teams to the patient records in GS1 and NS at the beginning of the data collection, as the reviewers did not work in these departments. Furthermore, the physicians had more extensive access rights to the patient records than the other professionals, such as the medical students. Therefore, the first step was to set up full access rights to the patient records. In GS2, no problems occurred, as all of the reviewers were physicians and they had full access to the patient data.
(3) Availability of the technology and the equipment to adequately conduct the data collection using the GTT, including the collection, management, and the analyses of the data (low level of challenge).
The organization of the training equipment (GTT manual, computer with access to the patient records, GTT-template in Excel, and a total of five to six patient records per department for the training) was a minor challenge. For the data collection, the reviewer required a workspace equipped with a computer, with access to the patient records, and the GTT template in Excel. Concerning the workspace, GS1 conducted the PRR in a meeting room that was often occupied, despite being reserved, so another room had to be found at short notice.
(4) Implementation of the GTT within the designated budget (low level of challenge).
The implementation of the GTT did not require extensive financial resources for the data collection equipment. An external expert for the reviewer training, as well as the reviewer’s reimbursement to cover expenses, was previously budgeted. Thus, the implementation of the GTT was within budget. However, while all the three hospital managements supported the implementation of the study, none of them could provide any additional resources to exempt the reviewers from their daily work during the PRR.
(5) Efficient and effective management of the data entry and analyses when using the GTT (moderate level of challenge).
The data was collected by primary reviewers in the departments at the scheduled times. However, the need for scheduling appointments with a secondary reviewer for confirming and categorizing the severity of the AEs posed a moderate challenge for the reviewers.
Availability of the patient records and the resulting sample characteristics
The availability of the patient records and the resulting sample characteristics were categorized as a low level of challenge in three out of the six feasibility criteria. Two of the feasibility criteria resulted in moderate challenges and one was a major challenge.
(6) Assistance with and the obstacles to the patient record selection (moderate level of challenge).
The patient record selection was carried out by the local coordinators. In GS1, the coordinator was supported by a primary reviewer, and in GS2 and NS, by the respective department office. For the randomized patient record selection, the patients were filtered by the discharge date. The initial difficulties with the hospital information system in GS1 prevented the filtering regarding the discharge date but these problems were solved with the support of the departmental staff.
(7) Availability of potential patients in the respective departments (low level of challenge).
Each department had a sufficient patient volume for a random selection of 40 patient records (20 per month over a period of two consecutive months, by those patients that were admitted between 01/2017 and 03/2017).
(8) Availability of the patient records in the departments that meet the inclusion and exclusion criteria (low level of challenge).
In each department, all of the randomly selected patients met the IHI recommendations for the inclusion criteria (patient stay > 24h, ≥18 years, discharged for at least 30 days, with a complete patient record). This allowed for an oversampling of 12 additional patient records (six per month and the department named) for any replacement required. However, only one patient record had to be substituted in GS1 since the original patient record was assigned to another department during the period of the PRR, thus it was inaccessible.
(9) Feasibility and suitability of the inclusion and exclusion criteria for the patient record selection (moderate level of challenge).
In all of the departments, the inclusion criterion of a "complete patient record" only became apparent during the PRR, as some documents, such as extra medication plans, only became available in-patient records, weeks after the discharge date. In these cases, the PRR was interrupted and the patient records were marked and reviewed after the documents were added.
(10) Dealing with an insufficient quality of the information in the patient records (problematic level of challenge).
All of the departments were in transition from a solely paper-based documentation to electronic patient records. As part of this process, the patient records were scanned after the patient’s discharge and then accessed as digital patient records in the local clinical information systems. While the quality of the scanned documents was generally good, it was detrimental when red ink, pencil, and colored templates were sometimes not scanned appropriately and thus not visible. Likewise, the handwriting on the scanned documents was often illegible. Accordingly, some triggers could not be evaluated appropriately. In accordance with the IHI, the GTT trigger was only rated if the reviewer identified relevant information in the PR. The reviewers were instructed to assign triggers wherever the information was not applicable, or where it had not been documented, with an ‘NA’. While the risk of overlooking the information was covered by the independent review process in the first stage, there was no information on the completeness and the quality of the patient’s record contents.
(11) Relevance of the implementation of the GTT to the respective departments (low level of challenge).
All of the departments identified between ten and twenty-four AEs [5], with most of the AEs identified as temporary harms requiring an intervention (category E), or initial or prolonged hospitalization (category F). The local coordinator gave feedback that to their knowledge, the harms categorized in E and F had not been revealed by other methods (administrative routine data, critical incident reporting). In GS2 and NS, the AEs of severity categories E and F (for example, intraoperative blood loss, pneumonia) occurred repeatedly in the reviewed patient records.
Reviewer recruitment and capability
The reviewer recruitment and capabilities were associated with five feasibility criteria, of which four were rated with a low level of challenge, and one with a problematic level of challenge.
(12) Assistance with the recruitment and the organizational obstacles concerning the recruitment (low level of challenge).
The participating hospitals’ management and staff council was informed about the procedures and approval was received, with full support prior to implementing the GTT. Moreover, the reviewer recruitment was conducted and organized by a responsible local coordinator, who provided all of the required information to the potential reviewers and served as a contact in case of queries.
(13) Sufficient recruitment of appropriate reviewers (low level of challenge).
The effort and time required for the recruitment varied across the departments, due to differing staff resources. However, each department was able to recruit a sufficient number of appropriate reviewers within the scheduled time.
(14) Feasibility and suitability of the inclusion and exclusion criteria for the reviewer selection (low level of challenge).
As recommended by the IHI, all of the departments designated at least two primary reviewers and one secondary reviewer, with all of the secondary reviewers being physicians. However, due to staff resources, the composition of the reviewer teams varied across the departments. In all of the departments, the IHI recommendations were followed in the recruitment of the reviewers (reviewer team with a clinical background, knowledge of the patient’s record content and layout, as well as care provision at the hospital) [23]. In NS, only the secondary reviewer worked in the department, while the two primary reviewers were a physician and a registered nurse working in the hospital’s quality management department. In GS2, the reviewer team consisted of three physicians working in the department. In GS1, the reviewer team was solely supported by one local physician, who served as a secondary reviewer. Since the GTT was used for the first time within a SafeCulture study in Germany, the primary and secondary reviewers of the three departments had no expertise in PRR with the GTT. Due to a lack of personnel, two medical students served as primary reviewers, thus they required additional support and training to become familiar with the patient records. Overall, and despite the different compositions of the reviewer teams, the IHI recommendations were met, and all of the teams successfully completed the PRR.
(15) Feasibility and suitability of the GTT-based data collection for the reviewers (problematic level of challenge).
In all of the three departments, the reviewers reported that the PRR was not exclusively carried out during the reviewers’ regular working hours but it required overtime. Due to workload and the composition of the reviewer teams, especially in NS and GS1 (such as physicians from the departments versus the medical students and personnel from the other departments), the primary and secondary reviewers had difficulties scheduling the discussion of the results in the second stage of the PRR process. Consequently, the categorization of the severity of the identified AEs was delayed in these two departments.
(16) Level of safety of the procedures for the implementation of the GTT (low level of challenge).
While the participating hospitals did not exempt the reviewers from their daily work, for PRR, the data collection was conducted on top of their regular clinical duties. This additional workload, with working overtime, may pose a risk to patient care. However, within this study, no additional safety issues other than those that were measured with the GTT were identified.
Data collection procedures and measures
In this section, all of the three feasibility criteria were classified as a moderate level of challenge.
(17) Feasibility and suitability of the GTT-based data collection (moderate level of challenge).
Before the data collection, the authors used the German version of the GTT [27], and for the neurosurgery department, selected triggers from the National Institute for Health and Welfare of Finland were used [28]. Both of them were slightly adapted in terms of linguistics and content. New triggers were only added when the essential required content was not covered by existing triggers (for instance, ‘relevant increase of leukocytes or other serological infection values during the hospital stay’). To ensure consistent data content, no triggers were removed from the GTT. However, the GTT required some local tailoring to the different conditions in the individual departments (for example, the different use of drug names). In addition, the order of the triggers was changed in some departments, to reflect the composition of the local patient records, and to facilitate continuous reviewing, without jumping between the triggers or the patient record documents. To facilitate the trigger search and the evaluation, and to ensure the reviewers understood the individual GTT triggers, the Excel template for the data entry contained additional information on each trigger (for instance, specific examples of the adverse events) and guidelines for completing the form. Hence, the missing data resulted only from the missing or unidentifiable information in the patient records.
(18) Feasibility and suitability of the amount of data collected when using the GTT in general (moderate level of challenge).
In GS2, the secondary reviewer verified the AEs found and determined the severity immediately after the primary reviewers screened the patient records. In contrast, the PRR in GS1 and NS delayed the data collection, due to additional appointments that were required for discussions between the primary and the secondary reviewers. For the GTT, the average PRR time was limited to 20 minutes per patient record. To ensure compliance with the recommended 20 minutes per patient record, stopwatches were used in all departments. The reviewers reported that this was particularly helpful for their own control, as there was often a risk that they would start reading the patient records in their entirety, instead of screening them. Still, the local coordinator informed that the average time for the PRR varied, and in some cases, the reviewers required more time. This especially became prevalent in the early stages of the PRR, or in the extensive patient records. Nonetheless, all of the reviewers improved rapidly, as they practiced their reviewing skills and became more familiar with the local documentation standards and patient record structures. This became more apparent when the PRRs were performed by students or quality managers, who were less familiar with the local documentation standards in the patient records. Based on their limited experience, the students had more difficulty evaluating the medical triggers than did the physicians or nurses.
(19) Consistency of the data collection when using the GTT (moderate level of challenge).
In order to ensure consistent data collection, a one-day reviewer training was conducted, providing all of the reviewers with uniform definitions and an explanation of the triggers. The AE assessment and the categorization of AE severity were in accordance with the National Coordination Council for Medication Error Reporting and Prevention Index [24]. During the training and the data collection, the professionals often struggled with the categorization of the AEs. This typically happened with the harms of severity, namely, the categories E and F, with temporary harm to patients requiring an additional intervention, or an initial or prolonged stay, respectively. During the training, the professionals tended to downplay the identified AEs. This was not because they considered them unimportant. This was because the GTT does not distinguish between avoidable and unavoidable AEs, and minor interventions were considered an important treatment to the patient, but not harmful. For example, before the training, the professionals understood the use of restraints (one trigger of the general module of the GTT), as a measure to strap patients, as a means to protect them from injuries. However, this trigger may either indicate that the AEs were caused by the restraint (bruises or anxieties), or the restraint was a consequence of an AE (delirium). By using the GTT, this trigger would identify an AE in both cases, even if greater harm (a fall) could have been prevented through this intervention. If required, the research team was consulted during the data collection to answer questions on the categorization of patient harms, as this was complex and highly dependent on professional experience.
Implementation of the GTT
Overall, the implementation of the GTT resulted in three feasibility criteria, with a moderate level of challenge.
(20) Success of the GTT implementation, as shown in the collected data (moderate level of challenge).
The identification of 53 AEs proved that the GTT was successfully implemented, supporting the importance of a GTT implementation. This was supported by the statement of the reviewers, that most of these AEs, especially those categorized in E and F, would have not been identified through other measures (administrative routine data, critical incident reporting). This was also reflected by the reviewer feedback that was in favor of the GTT and its further use. Some efforts were made at GS1 and NS to continue the use of GTT. For example, the implementation was continued in GS1 as part of a doctorate thesis. The hospital of the NS department adapted the tool for perinatology.
(21) Success of the GTT implementation, as shown by the feedback of the people involved in the GTT implementation (moderate level of challenge).
The reviewers found the results very valuable. However, it was also noted that staff resources and performing the PRR outside of working hours present an obstacle. During the PRR, no changes were made to the instrument. However, the reviewers made some suggestions for improving the GTT template and some triggers (for example, adding a row in the template for special information per patient record and the adjustment of triggers based on specific departmental conditions).
(22) Indications for the failure of the GTT implementation in general (moderate level of challenge).
Despite some deviations from the IHI recommendations (for example, the composition of the reviewers), the GTT was generally implemented in the intended manner. Further adaptations to the tool were not required during the data collection. The identification of the AEs revealed four recurring AEs in two departments. However, prolonged use of the GTT independently from this study was recommended and both the local coordinators and the reviewers emphasized the importance of the GTT. An integration ideally into routine patient safety activities within the departments was encouraged; however, none of the participating departments was able to invest further resources for continuing the GTT reviews.
Facilitators and the obstacles in the GTT implementation process
As a result of the systematic assessment regarding the feasibility of the GTT implementation, a list of recommendations regarding the facilitators and the obstacles in the GTT-implementation process was summarized (Table 4). This list was structured along with the five feasibility topics, as it was designed to support the hospitals in the implementation process of the GTT when rendered in daily clinical practice.
Discussion
This is the first study, which has systematically assessed the feasibility of the GTT implementation process (including local management, reviewer recruitment, reviewer training, patient record review, and the reporting of the results) when using 22 feasibility criteria and derived recommendations in the three hospitals. The study has revealed various levels of challenge. The criteria with a problematic level of challenge (two out of 22) mainly involved either a poor quality of the information in the patient records or a lack of time, especially due to the time-consuming activities (for example, the meetings with the secondary reviewer). The challenges regarding the quality of the patient records have also been described in other studies. In particular, insufficient documentation was noted on several occasions. For example, the AEs were not documented, and accordingly, they could not be identified in the patient records, due to different standards, clinical ignorance, and worries about the liability risks [17, 20, 29]. This was because scanning the original patient records to create an electronic patient record affected the quality of readability [30], with the use of the original patient records, instead of illegible scans, is recommended. Concerning the time resources, it is also recommended to consider alternate reviewer compositions (for example, involving the medical students) and ensure that the reviewers are relieved of some of their daily tasks. Even this is in discrepancy with Adler et al. [21], who solely recommended reviewers with at least five years of experience in the clinical field, and even students with enough experienced medical knowledge, with at least some clinical experience, who could then be eligible for the PRR.
The criteria that were classified as a moderate level of challenge (eleven out of 22), mostly involved the organizational procedures and the local issues (for example, planning required for the local tailoring of the GTT and the data collection procedure), or a lack of staff resources (for instance, the inability to exempt the reviewers from their daily tasks). Moreover, this moderate challenge also presented the need for an external expert. This expert should be familiar with the GTT and, even more importantly, not be directly involved in the local practices. The external expert would be an important facilitator, ensuring a neutral point of view in the reviewer training and the initial PRR, making objective remarks and recommendations regarding the data collection and evaluation. This became most obvious during the training when the professionals tended to downplay the identified AEs. Through the external expert, the study was able to inform reviewers of the importance of their decisions. This was especially important during the sampling procedure and the data collection process, as an expert can oversee the PRR and counteract the sampling biases [22]. Consequently, repetitive trainings and discussion of the identified AEs, within the reviewer teams, would ensure a shared understanding and a consistent application of the GTT [31]. Thus, the hospitals should, therefore, train several reviewers simultaneously as a team [22].
Prior to the data collection, it was important to adapt the GTT to the local conditions in the hospitals. Since the tool does not cover all specialties and department-specific situations, there is a conflict between when using the tool and adapting it locally, which can result in significant deviations from the GTT. With the development of triggers and new modules, clinical relevance and benefit must be considered and continuously evaluated [32]. Another facilitator was the supplementary information provider on each trigger and guidelines for completing the form, which simplified the trigger search and the evaluation. The moderate level of challenge additionally resulted from the time limit of 20 minutes per patient record, and this was difficult to be administered. This challenge was also found by Schildmeijer et al. [20]. Therefore, stopwatches are recommended. Further to identify the recurring AEs and allow more time for the detailed analyses, the hospitals must use the GTT for a longer period of time than was possible in this study. Only then would it be possible to investigate if the AEs were unique events or a consequence of problematic structures and practices in the department (routine use of restraints instead of choosing alternatives). Moreover, using additional data (administrative routine data, or critical incident reporting) and more detailed analyses, such as root-cause-analyses, can support the identification of problematic structures and processes, and thus learn from the errors and improve the patient safety work [33, 34]. Furthermore, when using and analyzing the GTT, different additional data should be noted and critically reviewed to learn for improvement. For example, Hibbert et al. 2016 [35] recommended that the AEs that were associated with the omissions and the preventability scores for a priority setting were important data, which should be considered. The focus of using the GTT should not be exclusively on counting the AEs. Rather, the aim of use should also be able to understand and characterize the AEs.
The lowest level of challenge (nine out of 22 feasibility criteria) was identified in terms of support by the hospital management and the professionals in the departments through the GTT-implementation process.
Moreover, the designation of at least one responsible person (for instance, the local coordinator), who is acquainted with the local processes and coordinates the reviewer recruitment, together with the organization of the required equipment (for example, for the training and the data collection), was proved to be an important facilitator in the implementation process.
Limitations of the study
For assessing the feasibility of the GTT implementation, the feasibility criteria that were originally developed were adapted to assess the feasibility of the interventional studies. The evaluation of the GTT implementation was limited to a two-month data collection, within the three departments. Thus, the challenges that were identified were not necessarily generalizable but they have provided a realistic picture of the challenges in the GTT-implementation process. Besides, these results have related to the implementation of the GTT in Germany. Some of the obstacles arose exclusively from specific circumstances (health care system/data protection) in Germany. As a result, some of the recommendations regarding the facilitators might be specific to Germany. Nonetheless, it is believed that most of the recommendations apply to other countries as well, especially those that have not yet implemented the GTT as a routine assessment. Furthermore, this GTT implementation was conducted in the framework of a study, which comes with the support of the researchers. Facilitators and challenges may arise in any setting, without such support. However, the results of this study can provide initial insights into the feasibility of a GTT implementation. In the context of the SafeCulture study, the review process was restricted to two months. Be that as it may, the implementation of the GTT requires local management, reviewer recruitment, reviewer training, and the reporting of the results, together with the effort required regarding these parts, in a setting without a study context. Lastly, but not least, the results of this study might have been biased by the researcher’s personal experiences and assumptions prior to the study. These have been reduced to a minimum by the independent ratings in the selection of feasibility criteria for the assessment (rating of criteria) itself, by MB, RG, and AH. Moreover, all of the results and decisions have been discussed and agreed on a consensus when involving TM.
Conclusion
The GTT implementation in practice comes along with various levels of challenge. Thus, the success of a GTT implementation process might be supported by considering potential facilitators and obstacles before the implementation process begins. Through a systematic assessment of the feasibility, essential aspects to be considered during the GTT-implementation process were derived. In particular, critical challenges relating to time, staff resources, and organizational aspects can be reduced in advance through effective planning. The identification of the AEs, especially the severity categories E and F, illustrates the GTT relevance for the departments. While the harms of categories G and I might be identified through other methods (for example, administrative routine data, and critical incident reporting), the harms of categories E and F have not been exposed in this study. By identifying these AEs with the GTT, problematic processes and structures can be identified early, and more serious harms may be avoided. The GTT can thus be a valuable supplement to other instruments for measuring the AEs.
Acknowledgments
The authors would like to thank the reviewer teams in the participating departments. They would also like to thank Marie Debrou were and Nikoloz Gambashidze from the Institute for Patient Safety for their valuable advice and support at various stages of the project.
References
- 1.
Organisation for Economic Co-operation and Development (OECD) Measuring Patient Safety. Opening the Black Box 2018. Available: https://www.oecd.org/health/health-systems/Measuring-Patient-Safety-April-2018.pdf. Accessed 27 May 2022.
- 2. Pannick S, Sevdalis N, Athanasiou T (2016). Beyond clinical engagement: a pragmatic model for quality improvement interventions, aligning clinical and managerial priorities. BMJ Quality & Safety 25 (9): 716–725. Available: Accessed 27 May 2022. pmid:26647411
- 3. Vassar M, Holzmann M (2013). The retrospective chart review: important methodological considerations. Journal of Educational Evaluation for Health Professions 10: 12. Available: Accessed 27 May 2022. pmid:24324853
- 4. Suarez C, Menendez MD, Alonso J, Castano N, Alonso M et al. (2014). Detection of adverse events in an acute geriatric hospital over 6 years using the Global Trigger Tool. Journal of the American Geriatrics Society 62 (5): 896–900. Available: Accessed 27 May 2022. pmid:24697662
- 5. Brösterhaus M, Hammer A, Kalina S, Grau S, Roeth AA et al. (2019). Applying the Global Trigger Tool in German Hospitals. J Patient Saf: 1. Available: https://doi.org/10.1097/PTS.0000000000000576. Accessed 27 May 2022.
- 6. Hoffmann-Völkl G, Kästenbauer T, Mück U, Zottl M, Huf W et al. Detektion unerwünschter Ereignisse mittels IHI Global Trigger Tool im Kontext der Einführung eines Risikomanagement-Systems. Eine retrospektive Studie über einen Zeitraum von 3 Jahren an der Herz- und Gefäßchirurgischen Abteilung eines Wiener Schwerpunktkrankenhauses. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen. Available: http://doi.org/10.1016/j.zefq.2017.09.013. Accessed 27 May 2022.
- 7. Haukland EC, Mevik K, Plessen C von, Nieder C, Vonen B (2019). Contribution of adverse events to death of hospitalized patients. BMJ Open Qual 8 (1): e000377. Available: Accessed 27 May 2022. pmid:30997413
- 8. Hwang J-I, Kim J, Park J-W (2018). Adverse Events in Korean Traditional Medicine Hospitals: A Retrospective Medical Record Review. J Patient Saf 14 (3): 157–163. Available: Accessed 27 May 2022. pmid:26001551
- 9. Carnevali L, Krug B, Amant F, van Pee D, Gerard V et al. (2013). Performance of the adverse drug event trigger tool and the global trigger tool for identifying adverse drug events. Experience in a Belgian hospital. The Annals of Pharmacotherapy 47 (11): 1414–1419. Available: Accessed 27 May 2022. pmid:24285758
- 10. Good VS, Saldana M, Gilder R, Nicewander D, Kennerly DA (2011). Large-scale deployment of the Global Trigger Tool across a large hospital system. Refinements for the characterization of adverse events to support patient safety learning opportunities. BMJ Quality & Safety 20 (1): 25–30. Available: http://doi.org/10.1136/bmjqs.2008.029181. Accessed 27 May 2022.
- 11. Classen DC, Resar R, Griffin F, Federico F, Frankel T et al. (2011). ‘Global trigger tool’ shows that adverse events in hospitals may be ten times greater than previously measured. Health Affairs (Project Hope) 30 (4): 581–589. Available: Accessed 27 May 2022. pmid:21471476
- 12. Naessens JM, Campbell CR, Huddleston JM, Berg BP, Lefante JJ et al. (2009). A comparison of hospital adverse events was identified by three widely used detection methods. International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care 21 (4): 301–307. Available: Accessed 27 May 2022. pmid:19617381
- 13. Rutberg H, Borgstedt Risberg M, Sjodahl R, Nordqvist P, Valter L et al. (2014). Characterizations of adverse events detected in a university hospital: a 4-year study using the Global Trigger Tool method. BMJ Open 4 (5): e004879. Available: Accessed 27 May 2022. pmid:24871538
- 14. Thomas EJ, Petersen LA (2003). Measuring errors and adverse events in health care. Journal of General Internal Medicine 18 (1): 61–67. Available: Accessed 27 May 2022. pmid:12534766
- 15. Rozich JD, Haraden CR, Resar RK (2003). Adverse drug event trigger tool: a practical methodology for measuring medication-related harm. Quality & Safety in Health Care 12 (3): 194–200. Available: Accessed 27 May 2022. pmid:12792009
- 16. Preen DB, Holman CDJ, Lawrence DM, Baynham NJ, Semmens JB (2004). A hospital chart review provided more accurate comorbidity information than data from a general practitioner survey or an administrative database. Journal of Clinical Epidemiology 57 (12): 1295–1304. Available: Accessed 27 May 2022. pmid:15617956
- 17. Nilsson L, Borgstedt-Risberg M, Soop M, Nylén U, Ålenius C et al. (2018). Incidence of adverse events in Sweden during 2013–2016. A cohort study describing the implementation of a national trigger tool. BMJ Open 8 (3): e020833. Available: Accessed 27 May 2022. pmid:29602858
- 18. Deilkås ET, Risberg MB, Haugen M, Lindstrøm JC, Nylén U et al. (2017). Exploring the similarities and the differences in the hospital adverse event rates between Norway and Sweden using Global Trigger Tool. BMJ Open 7 (3): e012492. Available: Accessed 27 May 2022. pmid:28320786
- 19. Deilkås ET, Bukholm G, Lindstrom JC, Haugen M (2015). Monitoring adverse events in Norwegian hospitals from 2010 to 2013. BMJ Open 5 (12): e008576. Available: Accessed 27 May 2022. pmid:26719311
- 20. Schildmeijer K, Nilsson L, Perk J, Arestedt K, Nilsson G (2013). Strengths and weaknesses of working with the Global Trigger Tool method for retrospective record review: focus group interviews with team members. BMJ Open 3 (9): e003131. Available: Accessed 27 May 2022. pmid:24068761
- 21. Adler L, Denham CR, McKeever M, Purinton R, Guilloteau F et al. (2008). Global Trigger Tool: Implementation basics. J Patient Saf 4 (4): 245–249. Available: https://doi.org/10.1097/PTS.0b013e31818e8a87. Accessed 27 May 2022.
- 22. Plessen C von, Kodal AM, Anhoj J (2012). Experiences with global trigger tool reviews in five Danish hospitals. An implementation study. BMJ Open 2 (5). Available: https://doi.org/10.1136/bmjopen-2012-001324. Accessed 27 May 2022.
- 23.
Griffin FA, Resar RK (2009). IHI Global Trigger Tool for Measuring Adverse Events. Second Edition. Available: http://app.ihi.org/webex/gtt/ihiglobaltriggertoolwhitepaper2009.pdf. Accessed 27 May 2022.
- 24. Hartwig SC, Denger SD, Schneider PJ (1991). Severity-Indexed, Incident Report-Based Medication Error-Reporting Program. American Journal of Health-System Pharmacy 48 (12): 2611–2616. Available: https://doi.org/10.1093/ajhp/48.12.2611. Accessed 27 May 2022. pmid:1814201
- 25.
Bohnsack R, Nentwig-Gesemann I, Nohl A-M, editors (2013). Die dokumentarische Methode und ihre Forschungspraxis. Grundlagen qualitativer Sozialforschung. Wiesbaden: Springer VS. 402 p.
- 26. Orsmond GI, Cohn ES (2015). The Distinctive Features of a Feasibility Study: Objectives and Guiding Questions. OTJR: Occupation, Participation and Health 35 (3): 169–177. Available: Accessed 27 May 2022. pmid:26594739
- 27.
Institute for Healthcare Improvement (IHI) (2009). Aktives Messinstrument der Patientensicherheit–das IHI Global Trigger Tool. Projekt-Version. Interreg 4A Projekt: Stärkung der Patientensicherheit zwischen Dänemark und Deutschland. Available: https://docplayer.org/24711831-Aktives-messinstrument-der-patientensicherheit-das-ihi-global-trigger-tool.html. Accessed 27 May 2022.
- 28.
Öhman J, Keski-Kuha T, Henttonen H, Kaartinen M, Kujansuu E (2013). Use of the Global Trigger Tool in neurosurgery: A comparison of manual and text mining detection of triggers. In: Doupi P, Peltomaa K, Kaartinen M, Öhman J, editors. IHI Global Trigger Tool and patient safety monitoring in Finnish hospitals. Current experiences and future trends. [Helsinki]: National Institute for Health and Welfare. pp. 56–63.
- 29. Weingart SN, Pagovich O, Sands DZ, Li JM, Aronson MD et al. (2005). What can hospitalized patients tell us about adverse events? Learning from patient-reported incidents. Journal of General Internal Medicine 20 (9): 830–836. Available: Accessed 27 May 2022. pmid:16117751
- 30. Clarke A, Adamson J, Watt I, Sheard L, Cairns P et al. (2016). The impact of electronic records on patient safety: a qualitative study. BMC medical informatics and decision making 16: 62. Available: Accessed 27 May 2022. pmid:27260193
- 31. Parand A, Dopson S, Renz A, Vincent C (2014). The role of hospital managers in quality and patient safety: a systematic review. BMJ Open 4 (9): e005055. Available: Accessed 27 May 2022. pmid:25192876
- 32. Kaafarani HMA, Rosen AK, Nebeker JR, Shimada S, Mull HJ et al. (2010). Development of trigger tools for surveillance of adverse events in ambulatory surgery. Quality & Safety in Health Care 19 (5): 425–429. Available: Accessed 27 May 2022. pmid:20513790
- 33. Maass C, Kuske S, Lessing C, Schrappe M (2015). Are administrative data valid when measuring patient safety in hospitals? A comparison of data collection methods using a chart review and administrative data. International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care 27 (4): 305–313. Available: https://doi.org/10.1093/intqhc/mzv045. Accessed 27 May 2022.
- 34. Zhan C, Miller MR (2003). Administrative data based patient safety research: a critical review. Quality & Safety in Health Care 12 Suppl 2: ii58–63. Available: Accessed 27 May 2022. pmid:14645897
- 35. Hibbert PD, Molloy CJ, Hooper TD, Wiles LK, Runciman WB et al. (2016). The application of the Global Trigger Tool: a systematic review. International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care: 1–10. Available: Accessed 27 May 2022. pmid:27664822