Implementation of checklists in health care; learning from high-reliability organisations

To obtain a range of views, eight informants were recruited from six different HROs based on the literature and the authors' personal knowledge (table 2) [20, 21]. The organisations were formally asked to appoint the informants. The candidates then underwent a pilot interview to ensure that they had comprehensive experience in checklist development and "sharp-end" use of checklists in addition to a comprehensive cultural understanding of their own organisation. All the informants were males, with 10-30 years of experience in high-risk operations. As the organisations all had restrictions on sharing experiences or standing operating procedures with the public, the informants had to have an organisational standing giving them permission to disclose and discuss potentially sensitive information.

Six of the experts underwent a semi structured interview at their work place and two by telephone. The interview guide had broad, open-ended questions regarding the informants' personal experiences with checklists. The interviews (lasting 45 to 90 minutes) were tape recorded and transcribed verbatim. Furthermore, one of the authors (OT) made field visits to all the HROs to observe their checklist use and organisation. Field notes were taken during parts of interviews when audio recording was inconvenient or not permitted (e.g. during a tour of the nuclear plant or inside the parachute packing area). All transcripts and field notes were reviewed by two of the authors (OT & GB) who identified and agreed on 8-12 assertions from each informant regarding important issues and elements in checklist development and implementation. These assertions were returned to the informants for validation (member check), resulting in minor revisions. The authors (except AE), who all have previous experience with quality improvement projects, further analysed the assertions in a Delphi approach by e-mail and during a final consensus meeting [22, 23]. In the first step of the process, each author proposed groups and subgroups of assertions. In the second step, the most important assertions were pinpointed. Finally, there was a consensus meeting where the groups and subgroups where further discussed. The lead authors (OT & GB) then performed the final analysis, and all authors agreed on the result. Figure 1 illustrates the entire process.

Several of the informants underlined that a checklist is a tool, not a goal in itself. In order to build a reliable safety culture and achieve checklist acceptance, two basic assumptions must be met; firstly, "There must be a predefined problem that a checklist is the right tool for solving" (C) and secondly, "The end user must not get the feeling that he or she is deprived of the opportunity to apply common sense" (A).

All of the informants expressed that proximity to the checklist development process is very important. One informant cautioned about merely copying checklists; "There must be a built-in scepticism as to whether the experiences of others are relevant for my organisation" (B). Two informants said; "Proximity between the checklist owner and the end user is important" (D) and "Checklists should be made within the organisation that intends to use it" (G).

Within an organisation, there are many relevant parties. Table 3 lists some stakeholders involved in checklist development and implementation.

"The checklist must foster resilient communication" (G). The informants underlined the importance of understanding and communication to minimize the risk of misunderstandings and to optimize flow during checklist use (Table 4).

All the informants described a thorough system for checklist revisions. One informant (E) said that revisions are important for two reasons; firstly, "To maintain a dynamic and up-to-date checklist" and secondly, "To build a culture where the sharp-end feels that their feedback is valuable to the organization". All informants used the terms "revision" and "validation" imprecisely and interchangeably. The key "validation" measure described by the informants was the absence of serious accidents and adverse events after introducing the checklists. Table 5 lists the different methods and approaches for validation and revision.

HROs undoubtedly have thorough and trustworthy experience with checklists, but some have questioned whether health care has gone too far in translating ideas from HROs in general [24]. HROs and medicine have many similarities concerning human factors and complex operations. They both fall prey to the limitations of human pathophysiology and the many challenges of man-machine interface. This study is limited to experiences with checklists and will not discuss the many additional issues concerned with building a safety culture.

Implementing checklists involves many conflicting interests, including the organisational culture and workflow, which are often affected [25]. Oversimplification of potential challenges could easily lead to conflicts between fractions of adopters and opponents. There are several reports of low compliance when checklists are introduced in health care [10, 26]. Our data highlights some important assumptions about achieving checklist acceptance. Firstly, there must be an identified need or problem for which a checklist is considered the right solution. A process is doomed to fail if the safety manager or department head try to force the "sharp-end" personnel to use a checklist to solve an issue that is not recognised as a problem. Secondly, be aware of challenging physicians' self-esteem and historically strong cultural autonomy. Human beings (including physicians) have cognitive limitations such as difficulty in carefully attending to several things simultaneously [21]. An effective checklist will enhance performance during high workload and stressful conditions by freeing mental capacity to perform important tasks in the correct manner and order [27]. However, checklists should not deprive the operator of the opportunity to use common sense or independently make more reasonable decisions.

The participants in our study emphasized that all stakeholders must have ownership, not only for the final checklist but for the entire development process. These views are consistent with the encouragement to make local modifications to the WHO Surgical Safety Checklist [28]. Recent critical reports from the UK call for an urgent nationwide checklist revision [29, 30]. These challenges regarding checklist acceptance may be the result of failed checklist implementation locally, and lack of proximity between the checklist-owner and end-users.

The visual elements of a checklist are important and will directly influence its efficacy and feasibility [14]. There are several guidelines for checklist lay-out and design from both medicine and aviation [31‐33]. All informants emphasized the importance of short checklists. Long lists should be divided into shorter sections or separate lists. One of the informants from aviation (E) described how they had divided the long "normal procedures checklist" into three chronological sections; "before take off", "in-flight" and "before landing". The WHO safe surgery checklist follows the same logic ("check-in", "time-out", and "sign-out"). Checklists must be easy to find and use, easy to understand, and short.

All but one of the informants had regular and mandatory checklist-training in full scale simulators. Such simulation is an expensive and time consuming training method, but after decades of experience this is still the preferred method (gold standard) for team and checklist training in these HROs. We also believe that team training is the key for successful checklist implementation in health care. Local team training with low cost and simple technical fidelity is a feasible method for rehearsing complex operations [34]. Success in checklist implementation is probably more a matter of will, enthusiasm, and organisational competence than of available time and financial resources [35].

Health care has traditionally had a strong hierarchical structure. Such a culture can discourage information exchange, resulting in anxiety or unwillingness to provide feedback on a checklist [36]. HROs cultivate a diversity of expertise; hence, authority migrates to the personnel with process-specific knowledge, independent of their hierarchical organisational position [17]. All the informants described a system where the sharp-ends' experiences are regarded as a valuable and essential part of the revision process. Resilience is a key issue in this context, and has been defined as "the intrinsic ability of a system to adjust its functioning prior to, during, or following changes and disturbances."[37] One cornerstone of resilience is to understand and obtain experiences from both the sharp and blunt end [38]. A self-reinforcing and destructive process may develop if a hierarchical structure prevents feedback from important stakeholders during checklist implementation.

Prospective memory (PM) describes the ability to remember to carry out actions (in the near future) that are planned after a delay or interruption [39]. Unpredictability, delays, and interruptions are frequent in health care [40]. During a stressful situation, memory is also likely to be more error-prone [41]. Cognitive aids like checklists are shown to increase performance in health care when solving complex and time-critical tasks [42, 43]. All the informants in our study described a culture where cognitive aids had an essential role. Medicine has traditionally relied on memory as the basis of diagnosis and treatment [42]. It has been argued that "PM failures in medicine must not be forgotten any more" [39].

The key strength of this study is the diversity of the informants, who provided a broad information source. Despite the nature of their often classified operations, none of those invited to take part in the study declined to participate. This enabled insight from an often inaccessible safety culture. Additionally, each informant had 10-30 years of cultural experience and an understanding of their organisations. Still, the informants' experiences may not have reflected all the relevant issues regarding checklist development and implementation from their own HRO, and the lack of female informants may also have limited the data to some degree.

In this study we have obtained experiences from informants representing six different HROs. It is likely to believe that these informants, together in an expert consensus process, could have developed and provided further understanding in checklist implementation.

Checklists in HROs are highly valued and given resources to develop, use and validate. Critical comments in health care are concerned if checklists may be cost-ineffective [44]. This line of criticism is debated claiming that checklists are a cost saving quality improvement strategy [45]. Studies concerning cost - benefit should be encouraged.