Maximising harm reduction in early specialty training for general practice: validation of a safety checklist

Patients worldwide are unintentionally but avoidably harmed as a result of their interactions with health care [1]. A global campaign to highlight the patient safety problem and recommend potential solutions is being led by the World Health Organisation (WHO) [2]. In the United Kingdom (UK), making patient care safer has been an explicit policy priority in the National Health Service (NHS) for over a decade [3]. Until recently the predominant safety improvement focus was on acute hospital settings where there is well-established evidence of the scale and consequences of adverse health care events [4]. In UK general medical practice comparable research and knowledge of the nature and impact of avoidable harm is growing, but is methodologically limited compared with secondary care [5, 6]. However, the emerging evidence from a range of international sources suggests that the safety of general practice may be compromised in a significant minority of cases [7‐9]. Specialty trainees are known to be involved in a proportion of these incidents largely because of a range of systems, knowledge, cognitive, training and behaviour based reasons.

In the NHS, a raft of interventions designed to make patient care safer are being implemented or are under development. For example: multi-centre collaborative programmes of safety improvement that aim to reduce harm in specific areas of acute hospital care and, more recently, primary care are currently underway [10]; a high-profile and long-running campaign to minimize healthcare acquired infections is ongoing [11]; while a concerted attempt to improve the reporting and learning from patient safety incidents has also taken place [12].

From an educational perspective, it is now recognised that there is a need for patient safety education to be more explicitly integrated and prominently positioned within existing undergraduate and postgraduate training curricula for all health care professions [13]. The UK Royal College of General Practitioners (RCGP) - which has responsibility for the content of the specialty training curriculum - has responded by developing a curriculum statement on ‘patient safety’ [14] and defining specific learning objectives (Table 1). However, it is left to individual GP educational supervisors to determine how the RCGP curriculum is best delivered and the related learning needs of trainees are identified and acted upon during the training period. The role of the postgraduate deaneries in this regard is to verify that the evidence provided by training practices to support the delivery of the patient safety element of the curriculum is of an adequate standard.

Ensuring that all essential educational issues are identified, prioritised and satisfactorily covered during training is not straightforward given the complexity of the tasks to be undertaken and the high volume of topics to be addressed. Within the framework of the RCGP curriculum, GP educational supervisors currently guide the activities undertaken by the trainee using a combination of locally developed induction packs, nationally promoted learning interventions and assessments, and professional experience in the workplace to match across to RCGP curriculum competencies (Table 2). However, it is currently unclear how the patient safety-related learning needs of trainees are specifically addressed when in the training environment. Given the lack of standardized guidance on how and what specific learning issues are to be covered, it is possible there is variation in GP educational supervisors’ interpretation and delivery of this safety-critical element of the curriculum at the ‘sharp end’ of frontline educational practice.

Overall there appears to be a paucity of evidence on how postgraduate training in general practice (or lack of) may have a visible and negative impact on the safety of patients – arguably it may be impossible to ever demonstrate clear causation. However, we know that a range of issues connected with the postgraduate training environment may act as proxy indicators of the safety of patient care being compromised unnecessarily and avoidably. For instance, medical educators may be involved in failures of, or inadequate, clinical supervision [15]; or fail to respond appropriately to trainees’ seeking professional guidance [16]; or conduct insufficient joint reviews of the management of complex clinical cases [17]; or provide limited feedback on drug prescribing performance [18]; and may let poorly developed attitudes and behaviour (such as lack of insight) continue unchecked [19]. Other salient issues that are potentially safety-critical include trainees’ possessing different levels of clinical knowledge [20] and an inability to prioritise their clinical workloads and manage time [17]. The quality of the learning environment in which trainees are based may also affect the safety of patient care [21], while doctors-in-training are known to be susceptible to medical errors [6, 22].

Given what is known about human error theory in the healthcare workplace [23], and the marked differences in local safety cultures [24] and the reliability of practice systems [5‐9], it is inevitable that variation in the quality of training provision exists and that some issues will be inadequately covered or even overlooked completely. If this happens with fundamental training topics which are considered to be safety-critical then there is a likelihood that the risk of patients being harmed and other quality of care issues arising could potentially increase.

Evidence is accumulating from healthcare safety and improvement initiatives that routine adherence to the adoption and use of checklist reminders can improve the overall reliability with which care processes and tasks are undertaken and so potentially mitigate future risks [25‐28]. Against this background, we aimed to identify and prioritise the most safety-critical issues to be addressed by educational supervisors and trainees in the first 12-weeks of specialty training in the general practice environment – the training period judged to be specifically high risk and beyond which trainees are given greater clinical freedom. In doing this we may help maximize early opportunities to address safety-critical issues proactively via a checklist reminder, which may lead to a reduced risk of patients being unintentionally harmed during and after the training period.

We used a six-stage mixed methods approach to develop and content validate the checklist for educational supervisors (Table 3).

The study identified 14 safety-critical domains and 47 related items that were judged to be essential components of specialty training in general practice to be addressed by educational supervisors and trainees in the first 12-week period of training in the general practice setting. Each item was also aligned with one or more of the 12 RCGP curriculum competencies to assist and guide supervisors in delivering the curriculum and collecting supporting evidence (Table 4). All checklist items were endorsed by a minimum of 20 of 24 GP educational supervisors who rated each item ≥3 on the 4-point scale. The overall content validity index ratio for the tool was 0.98 (Table 5).

We identified and prioritised a whole range of safety-critical issues that were judged to be ‘essential’ as part of the high risk 12-week phase of specialty training in the general practice environment. We demonstrated that the content validity of the checklist tool is adequate, providing the first steps towards validating this approach as a method for educational supervisors to guide and support safety-critical education interventions at this phase of training and potentially in future training periods (Additional file 1). At the suggestion of supervisors, the checklist was also adapted to enable trainees to self-assess safety-related learning needs early in their specialty training and monitor future progress (Additional file 2).

All of these factors are of prime importance because of the potential medico-legal implications for the GP educational supervisor if a trainee is inadequately tutored and supervised and patient safety is subsequently compromised. GP educators need to have mechanisms in place to make an early assessment of a trainee’s competencies and to undertake regular performance reviews and feedback sessions as part of an overall strategy to minimise risks. It should be noted that as trainees’ progress through training they are exposed to a more ‘positive’ and less ‘reductionist’ view of care quality in general practice via engagement with a range of improvement methods and exposure to the Quality & Outcomes Framework as part of routine clinical practice.

Theory suggests that because human memory and attention are imperfect [24], one way to mitigate errors or lapses in the execution of tasks is through the use of procedural checklists to help us perform more reliably. Compliance checklists to improve the safety or reliability of processes are standard practice in high-reliability industries [35, 36], most notably in commercial aviation and petro-chemical organisations. In health care checklist use is well documented, particularly in surgical [37], obstetric [38] and intensive care [39] settings where their implementation is associated with improved clinical outcomes, team working and productivity efficiencies.

However, despite these positive reports there is some scepticism over whether the introduction of a checklist on its own is the single greatest factor attributable to improvement success. Bosk et al. [40] suggest that the widespread deployment of a checklist may not be successful without a clear understanding that it is a ‘technical’ solution being introduced to a complex dynamic working environment where pre-existing ‘socio-cultural’ issues may need to be contained or resolved. Clinicians may resist or feel threatened by checklist implementation because they perceive it to impinge on their expertise, interfere with their decision-making or over-simplify the working environment. We also know that there is variation in perceptions of local safety climate within and between general practice teams and that some may unintentionally inflate this measure because of inexperience with, or lack of knowledge of, local safety concerns [41]. Similar variation exists in the strength of team working [42], the maturity of the learning environment [21] and commitment to quality improvement [43]. Attending to these types of socio-cultural issues is, therefore, as equally important as supporting efforts into checklist development and implementation if momentum is to be gained and a successful impact made.

Adaptation of the checklist as a self-assessment tool for trainees was an unexpected study development. Self-assessment is a method of measuring and interpreting one’s own performance and is a well-established educational intervention amongst all health care professions and as part of specialty training [44]. The use of self-assessment tools by specialty trainees to identify learning needs is routine in medicine [45, 46], but does not appear to be well developed in terms of highlighting specific educational interventions related to patient safety. The activity takes on a greater significance when the performance focus is related to the identification of learning need associated with safety-critical education and training. Although the limitations of self-assessmentis well established [44], it is still promoted as a valuable educational activity because it can be used as a baseline measure which prompts joint discussion and monitoring of performance over time by the trainee and supervisor.

The dissemination and implementation of the checklist and rating scale are proceeding on a national, voluntary basis in Scottish GP specialty training practices with plans in place to evaluate their feasibility and educational impact in the short-term. There is clear potential for patients to be unintentionally and avoidably harmed because of what happens (or does not happen) educationally in the GP training environment. The checklist concept is but one small (albeit untested) intervention to help minimise the related risks to patients and the potential medico-legal consequences for supervisor and trainee alike [47]. The use of a checklist measure to ensure the reliable delivery of ‘essential’ patient safety education has potential relevance for all medical specialties and other clinical professions with similar training arrangements in the UK and internationally.