Embracing Errors in Simulation-Based Training: The Effect of Error Training on Retention and Transfer of Central Venous Catheter Skills

doi:10.1016/j.jsurg.2015.08.002

Journal of Surgical Education

Volume 72, Issue 6, November–December 2015, Pages e158-e162

https://doi.org/10.1016/j.jsurg.2015.08.002 Get rights and content

Introduction

Error management training is an approach that encourages exposure to errors during initial skill acquisition so that learners can be equipped with important error identification, management, and metacognitive skills. The purpose of this study was to determine how an error-focused training program affected performance, retention, and transfer of central venous catheter (CVC) placement skills when compared with traditional training methodologies.

Methods

Surgical interns (N = 30) participated in a 1-hour session featuring an instructional video and practice performing internal jugular (IJ) and subclavian (SC) CVC placement with guided instruction. All interns underwent baseline knowledge and skill assessment for IJ and SC (pretest) CVC placement; watched a “correct-only” (CO) or “correct + error” (CE) instructional video; practiced for 30 minutes; and were posttested on knowledge and IJ and SC CVC placement. Skill retention and transfer (femoral CVC placement) were assessed 30 days later. All skills tests (pretest, posttest, and transfer) were videorecorded and deidentified for evaluation by a single blinded instructor using a validated 17-item checklist.

Results

Both the groups exhibited significant improvements (p < 0.001) in knowledge and skills after the 1-hour training program, but the increase of items achieved on the performance checklist did not differ between conditions (CO: IJ Δ = 35%, SC Δ = 29%; CE: IJ Δ = 36%, subclavian Δ = 33%). However, 1 month later, the CO group exhibited significant declines in skill retention on IJ CVC placement (from 68% at posttraining to 44% at day 30; p < 0.05) and SC CVC placement (from 63% at posttraining to 49% at day 30; p < 0.05), whereas the CE group did not have significant decreases in performance. The CE group performed significantly better on femoral CVC placement (i.e., transfer task; 62% vs 38%; p < 0.01) and on 2 of the 3 complication scenarios (p < 0.05) when compared with the CO group.

Conclusions

These data indicate that incorporating error-based activities and discussions into training programs can be beneficial for skill retention and transfer.

Introduction

As noted by DaRosa and Pugh,¹ surgical education curricula often overlook the beneficial role of errors. Frequently, faculty spend so much time teaching residents the correct way to perform a procedure that they neglect to explicitly discuss how to recognize and manage errors.² Increasingly, however, educators are concluding that errors play a valuable role in medical training.1, 3, 4, 5, 6 When trainees are explicitly encouraged to make, discuss, and reflect on errors during the initial acquisition of skills, it is believed that they would have a deeper understanding of the procedure, subsequently affecting retention and transfer of skills.⁴ In fact, work from other domains has demonstrated that trainees who are encouraged to actively explore and make errors exhibit better skill acquisition,⁷ decision making,⁸ transfer of training, and performance.⁹ Simulation-based settings offer an ideal opportunity to examine the value of this educational paradigm for surgical procedures, as trainees can make errors in a safe setting without negatively affecting patient care.

One procedure that is frequently trained in simulated settings¹⁰ and that can be fraught with errors is central venous catheter (CVC) placement. If done incorrectly, the patient can contract infection, pneumothorax, arterial puncture, or bleeding.¹¹ Thus, trainees need to have a well-developed understanding of the rationale and consequences for the various steps in the procedure, as well as an appropriate awareness of how to detect and manage any errors that may occur. However, the ideal way to design curricula to facilitate this understanding within simulation-based training programs remains unclear. The aim of this study was to investigate the effect of an error-training curriculum on the acquisition, retention, and transfer of CVC skills.

Section snippets

Materials and Methods

This study was approved by the institutional review board at the University of Texas Southwestern. A total of 30 general surgery postgraduate year–1 residents (13 categorical, 9 preliminary, 4 oral and maxillofacial surgery, and 4 urology) at University of Texas Southwestern were required to complete the training program.

All interns participated in a 1-hour educational session featuring an instructional video and faculty-guided instruction with a CVC simulator and ultrasound. Interns were

Results

Overall, 73% of the participants indicated that they had never participated in any formal CVC training. Groups performed similarly on all pretests before training. Specifically, at baseline, participants in the CO group correctly performed an average of 32% of items required for IJ CVC placement and 30% for SC CVC placement and scored 66% correct on the knowledge test. Similarly, participants in the CE group correctly performed an average of 35% of items required for IJ CVC placement and 33%

Discussion

The current study sought to examine if purposeful inclusion and discussion of errors into simulation-based training programs can be beneficial for the short- and long-term retention and transfer of newly acquired surgical skills, compared with curricula solely focusing on correct methods. Our findings revealed that the individuals in the error-training group did indeed demonstrate better skill retention 1 month later, despite no differences immediately after the training session. In fact, the

Conclusion

The results from this study indicate that incorporating error-based activities and discussions into CVC training programs can be beneficial for skill retention and transfer. Individuals in the error-trained group exhibited similar posttraining scores, better retention, and better adaptive transfer when compared with individuals trained solely under correct conditions. This project contributes to the literature seeking to identify characteristics of training programs that can reduce

References (19)

D.A. DaRosa et al.
Error training: missing link in surgical education
Surgery
(2012)
R.M. Satava
The future of surgical simulation and surgical robotics
Bull Am Coll Surg
(2007)
M.A. Fischer et al.
Learning from mistakes. Factors that influence how students and residents learn from medical errors
J Gen Intern Med
(2006)
A. King et al.
Errors as allies: error management training in health professions education
BMJ Qual Saf
(2013)
A.K. Gardner et al.
Error management training and simulation education
Clin Teach
(2014)
F.A. Rogers et al.
A role for error training in surgical technical skill instruction and evaluation
Am J Surg
(2002)
K. Chillarege et al.
Learning from our mistakes: error exposure training for mature learners
J Bus Psycol
(2003)
S. Gully et al.
The impact of error training and individual differences on training outcomes: an attribute-treatment interaction perspective
J Appl Psycol
(2002)
N. Keith et al.
Effectiveness of error management training: a meta-analysis
J Appl Psycol
(2008)

There are more references available in the full text version of this article.

Cited by (41)

Using process mining algorithms for process improvement in healthcare
2024, Healthcare Analytics
Healthcare professionals must provide their patients with the best possible service and be well-informed and expert at carrying out complex surgical procedures to fulfill this responsibility. The aim of the medical treatments is fewer complications, shorter hospital stays, and a better patient experience. Through continuous learning and training, medical practitioners trained in up-to-date and state-of-the-art surgical techniques and technologies make productive and effective healthcare systems possible. Healthcare systems often report on problems with surgical processes, skipped procedures, unusual activities during operations, and lengthy transition times. This event log data allows implementing process mining methods to deliver medical professionals with simple and understandable findings using Petri nets for process analysis and enhancement. This study identifies the parallels and discrepancies between the pre-and post-stages and their respective frequency on each typical Central Venous Catheter (CVC) installation activity. The Process Mining for Python (PM4Py) frameworks used four major mining algorithms to view the event log (i.e., alpha miner, directly-follows graph (DFG), heuristic miner, and inductive miner). This study's findings indicate that medical residents are more susceptible to error during pre-operative procedures.
Creating an Optimal Learning Environment in the Operating Room: Lessons From the Soccer Field
2023, Journal of Surgical Education
For trainees, the operating room (OR) often represents an educational sanctuary, where for a few hours they can focus on their craft, rather than on phone calls, pages, and the never-ending task of electronic health record documentation. The OR provides a unique opportunity for unfettered one-on-one time with an attending surgeon at their side, where they can learn the art of surgery without interruption. It is vitally important to maximize learning in the OR, yet evidence suggests that it is not always an ideal educational environment. Considering the recent excitement over the World Cup soccer tournament (full disclosure: the senior author is an Argentine immigrant and soccer fan), in this article, we provide evidence-based ideas and suggestions on how to optimize learning in the OR using some analogies from the soccer field.
Formalizing the Hidden Curriculum of Performance Enhancing Errors
2023, Journal of Surgical Education
Despite its inevitability, error remains an uncomfortable topic for discussion amongst surgeons. There are a range of reasons cited for this; significantly, there is an inextricable link between a surgeon's actions and their patient's outcomes. Attempts to reflect on error are often unstructured and without a defined end point, and modern surgical curricula lack content to guide residents’ learning on recognizing and reflecting on sentinel events. There is a need to develop a tool to guide a standardized, safe, and constructive response to error.
The current educational paradigm revolves around error avoidance. However, there is an evolving evidence base surrounding the inclusion of error management theory (EMT) into surgical training. This method explores and incorporates positive discussions surrounding errors, and has been demonstrated to improve long-term skill acquisition and training outcomes.
We must harness the performance enhancing effects of our errors in the same way we do our successes. Implicated in all surgical performance is human factors science/ergonomics (HFE) - the interface between psychology, engineering, and performance. Developing a national HFE curriculum in the context of EMT would provide a common language to facilitate objective reflections regarding surgeons’ operative performance and manage the stigma associated with fallibility.
A conceptual model to inform the design of healthcare simulations that promote errors as a catalyst for learning: A discussion paper
2022, Nurse Education in Practice
Previous studies have demonstrated that students who are engaged in learning tasks and make errors before receiving instruction on how to complete them, achieve better learning outcomes than students who first receive instruction and then complete the learning activities with the aim of avoiding errors. Although simulation literature often refers to errors as learning opportunities, to date, there is limited understanding of how pedagogical approaches that promote learning from errors can guide the design of simulation-based learning in healthcare education.
To (a) present the Learning from Errors conceptual model; and (b) provide an example of how educators can use this model.
The Learning from Errors model is drawn from critical elements of two pedagogical approaches, productive failure and error management training and pedagogical features of high-quality healthcare simulations.
We describe the Learning from Errors model, which emphasises the need for adopting pedagogical methods that explicitly use errors as learning opportunities and ultimately inform simulation design. We then illustrate the application of this model to a simulation example.
The model includes the following elements: i) normalisation of errors, ii) challenging simulation scenarios, iii) self-directed learning, iv) collaborative teamwork and v) comparison with best practice.
This discussion paper presents the Learning from Errors conceptual model, an evidence-based approach that can assist educators in the design of simulations that embrace errors as a catalyst for learning.
Effects of patient deterioration simulation using inattentional blindness for final year nursing students: A randomized controlled trial
2021, Nurse Education Today
Citation Excerpt :
This may allow students to experience how easily patient safety can be violated. Nursing students learn the importance of patient safety by experiencing their mistakes (DaRosa and Pugh, 2012; Gardner and Rich, 2014; Gardner et al., 2015; Kim and Lee, 2020). Experiencing errors can reinforce PSC (DaRosa and Pugh, 2012), which is supported by our findings.
Patient deterioration should be detected early and responded appropriately for patient safety. It is necessary to strengthen situational awareness regarding patient deterioration. Inattentional blindness is a major factor that hinders situational awareness about patient deterioration in the clinical setting.
To analyze the impact of patient deterioration simulation using inattentional blindness (PDS-IB) on situational awareness and patient safety competency-attitude among final year nursing students.
A randomized controlled trial.
Final year nursing students at a university in South Korea.
Students were randomly assigned to an experimental or control group. The experimental group (n = 47) was given a PDS-IB. The control group (n = 44) received a simple patient deterioration simulation. Situational awareness and patient safety competency-attitude were measured at baseline, post intervention, and at 2 weeks follow-up. Data were analyzed using a two-way repeated measures ANOVA.
There were statistically significant group effects, time effects, and group and time interaction effects in situational awareness and patient safety competency-attitude.
PDS-IB is an effective educational strategy that increases situational awareness and patient safety competency-attitude in final year nursing students.
Piloting the FIRE: A Novel Error Management Training Simulation Curriculum for Fasciotomy Instruction
2021, Journal of Surgical Education
Citation Excerpt :
Initially incorporated in software skills training, EMT has demonstrated encouraging results in medical simulation training. Specifically, it has been shown to be noninferior to traditional teaching methods for immediate recall and has shown superior retention and adaptive task transfer at 30 days after instruction.27,28 FIRE merged SBML and EMT as the backbone of an educational curriculum for LEF training, and leveraged deliberate practice strengths from both instructional approaches.
Multiple studies have demonstrated poor performance of lower extremity fasciotomy (LEF), highlighted by missed and/or inadequately released compartments. Incorporating error management training (EMT) into surgical simulation has been promoted as a way to gain deeper understanding of procedural errors and overall performance. The purpose of this study was to evaluate LEF performance using a Fasciotomy Improvement through Recognition of Errors (FIRE) simulation training curriculum to train novice surgical trainees.
A mastery learning-based EMT curriculum was developed, and surgical residents were enrolled and pretested with a multiple-choice question (MCQ) written test, and a simulated fasciotomy using a lower leg model. Each trainee then watched a 15-minute narrated presentation followed by 2 rounds of fasciotomy error recognition and management training exercises to a mastery standard. During each round, trainees performed hands-on assessment of unique premade fasciotomy leg models containing a variable number of procedural errors. They were required to identify and propose corrective action for all errors. Serial rounds of remediation were implemented until the mastery standard was attained on both error identification rounds. All trainees were post-tested with the same MCQ and another simulated fasciotomy.
All 14 residents had minimal experience with only 0.3 ± 0.6 fasciotomies performed prior to instruction. There were 3 ± 1.6 missed or inadequately released compartments on the pretest. Residents examined 14 ± 2.5 legs, including 2 ± 2.5 legs during remediation to attain mastery. All residents demonstrated significant improvement following the FIRE of Error curriculum for the MCQ (57% ± 16% vs 78% ± 13%; p = 0.01; Cohen's d = 1.4), fasciotomy score (10 ± 7.1 vs 28 ± 1.9; p < 0.001; Cohen's d = 3.6), and achieving a complete fasciotomy (14% ± 36% vs 93% ± 27%; p < 0.001; Cohen's d = 2.5). Only a single cumulative compartment was missed on post-testing.
Implementation of a mastery learning-based EMT curriculum for fasciotomy simulation training results in significant improvement in fasciotomy technique without reliance on repeated procedure performance nor clinical fasciotomy exposure. This curriculum is a highly effective option for surgical trainees lacking fasciotomy training during residency.

View all citing articles on Scopus

: Presented at the 15th Annual Meeting of the Association of Program Directors in Surgery. Seattle, WA. April 21-22, 2015.

View full text

2015 APDS Spring MeetingEmbracing Errors in Simulation-Based Training: The Effect of Error Training on Retention and Transfer of Central Venous Catheter Skills

Introduction

Methods

Results

Conclusions

Introduction

Section snippets

Materials and Methods

Results

Discussion

Conclusion

Surgery

The future of surgical simulation and surgical robotics

Bull Am Coll Surg

Learning from mistakes. Factors that influence how students and residents learn from medical errors

J Gen Intern Med

Errors as allies: error management training in health professions education

BMJ Qual Saf

Error management training and simulation education

Clin Teach

A role for error training in surgical technical skill instruction and evaluation

Am J Surg

Learning from our mistakes: error exposure training for mature learners

J Bus Psycol

The impact of error training and individual differences on training outcomes: an attribute-treatment interaction perspective

J Appl Psycol

Effectiveness of error management training: a meta-analysis

J Appl Psycol

2015 APDS Spring Meeting
Embracing Errors in Simulation-Based Training: The Effect of Error Training on Retention and Transfer of Central Venous Catheter Skills