Developing a coaching mechanism for practicing surgeons

doi:10.1016/j.surg.2016.03.036

Surgery

Volume 160, Issue 3, September 2016, Pages 536-545

Presented from the podium at the annual consortium meeting of the American College of Surgeons–Accredited Education Institutes in Chicago, IL, on March 14th, 2015.

https://doi.org/10.1016/j.surg.2016.03.036 Get rights and content

Background

While performance feedback and assessment are hallmarks of surgical training, they abruptly cease after training is completed. In their absence, performance may stagnate and poor habits persist. Our aim was to develop a coaching mechanism for practicing surgeons with feedback provision based on objective performance assessment.

Methods

Technical and nontechnical intraoperative video recordings from laparoscopic or robotic cholecystectomies, colectomies, and hysterectomies were assessed by a blinded surgeon and a human factors expert, respectively. Aspects of performance in need of improvement were noted, and a coaching session was developed for feedback provision to participating surgeons. This 4-hour coaching session consisted of a didactic lecture with video review and hands-on practice using procedural and mannequin-based simulation.

Results

Thirty-two practicing surgeons (18 general; 14 gynecologists) from 6 different hospitals were assessed, and 9 of them participated in coaching. Technical aspects identified for performance improvement included suboptimal trocar placement, inadequate critical view achievement during laparoscopic cholecystectomies, poor visualization of the operating field, bimanual dexterity, and dissection techniques, while nontechnical aspects included inappropriate handling of distractions and interruptions, poor ergonomic positioning and situational awareness, and inadequate mitigation of delays. Most surgeons appropriately accomplished some of the objectives of the distraction scenario, but none was able to achieve expert levels on Fundamentals of Laparoscopy tasks. Participants perceived the coaching sessions as highly valuable.

Conclusion

Our study identified several technical and nontechnical skill sets of practicing surgeons in need of improvement and provided support for the implementation of coaching programs for surgeons on an ongoing basis.

Section snippets

Methods

This project was an Institutional Review Board–approved, prospective quality improvement study with voluntary enrollment of practicing general and gynecologic surgeons. The study was conducted in 6 hospitals of Carolinas HealthCare System (CHS), a large not-for-profit health care system comprising 43 hospitals and more than 60,000 employees. The Center for Medicare and Medicaid Services Partnership for Patients funded the project as part of 5 Leading Edge Advanced Practice Topics projects. The

Results

As shown in Fig 2, 90 surgeons from 6 different CHS hospitals were invited to participate. Of these 90, 32 surgeons (18 general; 14 gynecologists) agreed to be videotaped and evaluated. Forty-two technical videos and 45 nontechnical videos were assessed by the surgeon experts and the human factors expert, respectively (Table II²²). In a few cases, there was a technical problem capturing a recording, resulting in the discrepant number of total recordings. Seven surgeons volunteered to

Discussion

In this study, we aimed to identify technical and nontechnical skill deficiencies of practicing surgeons and to develop a coaching mechanism that would address the lack of performance feedback available to surgeons today. To accomplish this, we embarked on a video-based assessment of technical and nontechnical skill of attending surgeons and created a coaching program that was based on an interactive video review and simulation-based practice.

Our approach allowed us to demonstrate the

References (35)

L. Hull et al.
The impact of nontechnical skills on technical performance in surgery: A systematic review
J Am Coll Surg
(2012)
S. Yule et al.
Surgeons’ non-technical skills
Surg Clin N Am
(2012)
D. Stefanidis
Optimal acquisition and assessment of proficiency on simulators in surgery
Surg Clin North Am
(2010)
M.C. Vassiliou et al.
A global assessment tool for evaluation of intraoperative laparoscopic skills
Am J Surg
(2005)
A.C. Goh et al.
Global evaluative assessment of robotic skills: Validation of a clinical assessment tool to measure robotic surgical skills
J Urology
(2012)
N. Sevdalis et al.
Reliability of a revised NOTECHS scale for use in surgical teams
Am J Surg
(2008)
C.O. Longenecker et al.
The practices of effective managerial coaches
Bus Horizons
(2005)
E.N. de Vries et al.
The incidence and nature of in-hospital adverse events: A systematic review
Qual Saf Health Care
(2008)
Agenda for research and development in patient safety
(2016)
J.D. Bikrmeyer et al.
Surgical skill and complication rates after bariatric surgery
N Engl J Med
(2013)

K.E. Hauer et al.

Remediation of the deficiencies of physicians across the continuum from medical school to practice: A thematic review of the literature

Acad Med

(2009)

D.A. Davis et al.

Accuracy of physician self-assessment compared with observed measures of competence: A systematic review

JAMA

(2006)

R.S. Wigton et al.

The effect of feedback in learning clinical diagnosis

J Med Educ

(1986)

C.C. Greenberg et al.

Surgical coaching for individual performance improvement

Ann Surg

(2015)

P. Singh et al.

A randomized controlled study to evaluate the role of video-based coaching in training laparoscopic skills

Ann Surg

(2015)

A.J. Becker

It’s not what they do, it’s how they do it: Athlete experiences of great coaching

Int J Sports Sci Coach

(2009)

S.J. Cole et al.

Randomized controlled trial on the effect of coaching in simulated laparoscopic training

Surg Endosc

(2014)

Cited by (33)

Experience of practicing obstetrician-gynecologists in a surgical training program in total laparoscopic hysterectomy
2023, AJOG Global Reports
Advanced surgical techniques, such as total laparoscopic hysterectomy, are often challenging to acquire beyond fellowship training programs for practicing obstetrician-gynecologists. A lack of formative data currently exist for continuing medical education programs, limiting our understanding of how improvement in surgical skills and training programs occur.
This study aimed to investigate how practicing obstetrician-gynecologists acting as trainees experience a program that aims to teach them total laparoscopic hysterectomy, and to assess whether their surgical skills improve according to data from formative assessment tools and qualitative data from open-ended survey questions and in-depth interviews.
We report a process analysis of formative data collected during a pilot implementation trial of a surgical training program targeting practicing obstetrician-gynecologists. Eleven consultant obstetrician-gynecologists and 4 experienced surgical mentors participated in 4 hospitals in Queensland, Australia. Total laparoscopic hysterectomy was performed in 700 patients over the course of the study. A total laparoscopic hysterectomy surgical mentorship training program of 10 training days with up to 3 total laparoscopic hysterectomy procedures per day was performed. Both the obstetrician-gynecologists and the surgical mentor completed a formative assessment questionnaire analyzing the trainee's performance after each surgical procedure. Mentors were formatively assessed by the Structured Training Trainer Assessment Report (STTAR) and at the completion of the study by the mini-STTAR, a summative assessment of quality of mentorship. Obstetrician-gynecologists, mentors, hospital leaders, and surgical administrative staff participated in qualitative interviews about the training program.
Over time, there was a demonstrated improvement in trainee performance reported by both mentors and trainees in all competency assessment tool domains as the case number increased, with mentors consistently rating trainees’ performance higher than the trainees themselves. Most trainees were satisfied with their mentor in all 31 areas during formative assessment, and at the end of the training, structure, attributes, and role modeling were all rated high (average score >4.5; range, 3.79–5.00), whereas training behavior was rated slightly lower at 4.1 (range, 3.79–4.45). Qualitative interviews demonstrated that the trainees found the training to be a beneficial, hands-on experience.
Formative assessment clearly documented improvement in surgical skills during a total laparoscopic hysterectomy training program for consultant obstetrician-gynecologists.
Peer Coaching as a Form of Performance Improvement: What Surgeons Really Think
2021, Journal of Surgical Education
Coaching has been proposed as a potential form of continuing professional development (CPD) for surgeons. Our study aimed to elicit qualified surgeons’ perceptions of peer coaching as a form of CPD and to determine the effect of demographic factors on surgeons’ views.
A cross-sectional paper survey was conducted across 2 South Australian metropolitan hospitals from November 2018 to January 2019. This comprised 5 demographic questions and 6 Likert items eliciting views on peer-based coaching and was distributed at departmental unit meetings to surgical consultants and fellows. Participation was voluntary and a definition of “peer-based coaching” was provided.
Hundred and eighteen surgeons of a possible 125 (94.4% response rate) from 8 surgical specialties completed the survey with 45.4% (n = 54) having received coaching since obtaining their fellowship. The majority of participants (72.9%, n = 86) reported consultant surgeons would benefit from peer coaching and that one-on-one coaching in an individual setting would be a useful CPD activity (73.7%, n = 87). Just over half the participants (53.4%, n = 63) felt that coaching by a nonsurgeon such as a psychologist would benefit their nonoperative skills. Many participants (61.8%, n = 73) felt more inclined to participate if CPD points were awarded. Despite the support in favor of coaching, a significant percentage of participants (45.8%, n = 54) wanted further evidence of its efficacy.
There is support amongst surgeons for peer coaching and its inclusion as a form of CPD, however, many require more evidence of its benefits, thus highlighting the need for ongoing research studies, consultation and pilot coaching programs.
Surgical coaching to achieve the ABMS vision for the future of continuing board certification
2021, American Journal of Surgery
In February 2019, the American Board of Medical Specialties (ABMS) released the final report of the Continuing Board Certification: Vision for the Future initiative, issuing strong recommendations to replace ineffective, traditional mechanisms for physicians’ maintenance of certification with meaningful strategies that strengthen professional self-regulation and simultaneously engender public trust. The Vision report charges ABMS Member Boards, including the American Board of Surgery (ABS), to develop and implement a more formative, less summative approach to continuing certification. To realize the ABMS’s Vision in surgery, new programs must support the assessment of surgeons’ performance in practice, identification of individualized performance gaps, tailored goals to address those gaps, and execution of personalized action plans with accountability and longitudinal support.
Peer surgical coaching, especially when paired with video-based assessment, provides a structured approach that can meet this need. Surgical coaching was one of the approaches to continuing professional development that was discussed at an ABS-sponsored retreat in January 2020; this commentary review provides an overview of that discussion. The professional surgical societies, in partnership with the ABS, are uniquely positioned to implement surgical coaching programs to support the continuing certification of their membership. In this article, we provide historical context for board certification in surgery, interpret how the ABMS’s Vision applies to surgical performance, and highlight recent developments in video-based assessment and peer surgical coaching. We propose surgical coaching as a foundational strategy for accomplishing the ABMS’s Vision for continuing board certification in surgery.
Acquiring and maintaining lifelong expertise in surgery
2020, Surgery (United States)
Expertise in surgery is essential to ensure the best outcomes across various phases of surgical care. Expertise is multidimensional and is both domain- and context-specific. It encompasses a broad range of knowledge, skills, and professional attributes. General expertise is characterized by pattern recognition, rapid and effective decision-making, fluency of actions, and efficiency in performance. Adaptive expertise involves complex decision-making and exceptional performance within the context of unexpected findings, unusual circumstances, complex systems, and the rapidly changing surgical environment. Traditionally, surgeons have acquired expertise through extensive experiences in a variety of difficult and demanding situations. Over the past few decades, considerable emphasis has been placed on a more planned and efficient approach to the development of surgical expertise through extended periods of deliberate practice coupled with specific feedback, and through establishment of progressively higher goals and benchmarks for the surgeon. Reflection and metacognition have been recognized as playing key roles in achieving expertise. More recently, an expert performance approach to training has been described that involves defining expert performance, focusing efforts on achieving expertise, and monitoring the outcomes of such efforts. The cycle of practice-based learning and improvement should serve as the essential framework for implementation of the expert performance approach. Simulation-based education and training, coaching, and communities of practice can be very helpful in achieving and maintaining expertise. Professional organizations, such as the American College of Surgeons, need to continue playing pivotal roles in supporting surgeons in their quest to achieve and maintain expertise.
The System for Telementoring with Augmented Reality (STAR): A head-mounted display to improve surgical coaching and confidence in remote areas
2020, Surgery (United States)
The surgical workforce particularly in rural regions needs novel approaches to reinforce the skills and confidence of health practitioners. Although conventional telementoring systems have proven beneficial to address this gap, the benefits of platforms of augmented reality-based telementoring in the coaching and confidence of medical personnel are yet to be evaluated.
A total of 20 participants were guided by remote expert surgeons to perform leg fasciotomies on cadavers under one of two conditions: (1) telementoring (with our System for Telementoring with Augmented Reality) or (2) independently reviewing the procedure beforehand. Using the Individual Performance Score and the Weighted Individual Performance Score, two on-site, expert surgeons evaluated the participants. Postexperiment metrics included number of errors, procedure completion time, and self-reported confidence scores. A total of six objective measurements were obtained to describe the self-reported confidence scores and the overall quality of the coaching. Additional analyses were performed based on the participants’ expertise level.
Participants using the System for Telementoring with Augmented Reality received 10% greater Weighted Individual Performance Score (P = .03) and performed 67% fewer errors (P = .04). Moreover, participants with lower surgical expertise that used the System for Telementoring with Augmented Reality received 17% greater Individual Performance Score (P = .04), 32% greater Weighted Individual Performance Score (P < .01) and performed 92% fewer errors (P < .001). In addition, participants using the System for Telementoring with Augmented Reality reported 25% more confidence in all evaluated aspects (P < .03). On average, participants using the System for Telementoring with Augmented Reality received augmented reality guidance 19 times on average and received guidance for 47% of their total task completion time.
Participants using the System for Telementoring with Augmented Reality performed leg fasciotomies with fewer errors and received better performance scores. In addition, participants using the System for Telementoring with Augmented Reality reported being more confident when performing fasciotomies under telementoring. Augmented Reality Head-Mounted Display–based telementoring successfully provided confidence and coaching to medical personnel.
Do Not Blame the Resident: the Impact of Surgeon and Surgical Trainee Experience on the Occurrence of Intraoperative Adverse Events (iAEs) in Abdominal Surgery
2018, Journal of Surgical Education
Citation Excerpt :
As such, the patient safety efforts should focus less on curbing resident autonomy and more on establishing formal operative support programs that facilitate culturally and logistically “calling the senior partner into the OR” before an iAE occurs in challenging surgical procedures. Formalized coaching programs in which an experienced senior surgeon provides real-time guidance and intraoperative feedback to junior partners may also hold promise as a preventative strategy for iAE occurrence for less experienced surgeons.39-43 The Society of Gastrointestinal and Endoscopic Surgeons conducted a multi-institutional study in which they used tele-monitoring as a form of coaching for experts in laparoscopic sleeve gastrectomy to have a “remote presence” and provide feedback to mentees.44
Intraoperative adverse events (iAEs) are defined as inadvertent injuries that occur during an operation and are associated with increased mortality, morbidity, and health care costs. We sought to study the impact of attending surgeon experience as well as resident training level on the occurrence of iAEs.
The institutional American College of Surgeons-National Surgical Quality Improvement Program and administrative databases for abdominal surgeries were linked and screened for iAEs using the International Classification of Diseases, Ninth Revision, Clinical Modification-based Patient Safety Indicator “accidental puncture/laceration.” Each flagged record was systematically reviewed to confirm iAE occurrence and determine the number of years of independent practice of the attending surgeon and the postgraduate year (PGY) of the assisting resident at the time of the operation. The attending surgeon experience was divided into quartiles (<6 years, 6-13 years, 13-20 years, >20 years). The resident experience level was defined as Junior (PGY-1 to PGY-3) or Senior (PGY-4 or PGY-5). Univariate/bivariate then multivariable logistic regression analyses adjusting for patient demographics, comorbidities, and operation type and/or complexity (using RVUs as a proxy) were performed to assess the independent impact of resident and attending surgeon experience on the occurrence of iAEs.
A large tertiary care teaching hospital.
Patients included in the 2007-2012 ACS-NSQIP that had an abdominal surgery performed by both an attending surgeon and a resident.
A total of 7685 operations were included and iAEs were detected in 159 of them (2.1%). Junior residents participated in 1680 cases (21.9%), while senior residents were involved in 6005 (78.1%). The iAE rates for attending surgeons with <6, 6-13, 13-20, and >20 years of experience were 2.7%, 1.7%, 2.4%, and 1.4%, respectively. In multivariable analyses, the risk of occurrence of an iAE was significantly decreased for surgeons with >20 years of experience compared to those with <6 years of experience (odds ratio=0.52, 95% confidence interval 0.32-0.86, p = 0.011). On bivariate analyses, iAEs occurred in 1.2% of junior resident cases, while senior residents had an iAE rate of 2.3%. However, after risk adjustment on multivariable analyses, the resident experience level did not significantly impact the rate of iAEs.
The surgeon's level of experience, but not the resident's, is associated with the occurrence of iAEs in abdominal surgery. Efforts to improve patient safety in surgery should explore the value of pairing junior surgeons with the more experienced ones thru formalized coaching programs, rather than focus on curbing resident operative autonomy.

View all citing articles on Scopus

: Supported through a Leading Edge Advanced Practice Topics grant by the Center for Medicare and Medicaid Services contract number HHSM-500-2012-00028C/000005.

View full text

8th Annual American College of Surgeons Accredited Educational Institutes (ACS-AEI) ConsortiumDeveloping a coaching mechanism for practicing surgeons

Background

Methods

Results

Conclusion

Section snippets

Methods

Results

Discussion

J Am Coll Surg

Surg Clin N Am

Surg Clin North Am

Am J Surg

J Urology

Am J Surg

Bus Horizons

The incidence and nature of in-hospital adverse events: A systematic review

Qual Saf Health Care

Agenda for research and development in patient safety

Surgical skill and complication rates after bariatric surgery

N Engl J Med

Remediation of the deficiencies of physicians across the continuum from medical school to practice: A thematic review of the literature

Acad Med

Accuracy of physician self-assessment compared with observed measures of competence: A systematic review

JAMA

The effect of feedback in learning clinical diagnosis

J Med Educ

Surgical coaching for individual performance improvement

Ann Surg

A randomized controlled study to evaluate the role of video-based coaching in training laparoscopic skills

Ann Surg

It’s not what they do, it’s how they do it: Athlete experiences of great coaching

Int J Sports Sci Coach

Randomized controlled trial on the effect of coaching in simulated laparoscopic training

Surg Endosc

8th Annual American College of Surgeons Accredited Educational Institutes (ACS-AEI) Consortium
Developing a coaching mechanism for practicing surgeons