Association for Surgical EducationQuantitative and qualitative analysis of performance during advanced laparoscopic fellowship: a curriculum based on structured assessment and feedback
Section snippets
Materials and Methods
During a minimally invasive surgery fellowship at the University of North Carolina at Chapel Hill, a fellow completed daily self-assessments and reviewed video recordings of the operations in which he participated. The attending also completed assessments of the fellow's performance. Three commonly performed advanced laparoscopic procedures in the fellowship were selected: Roux-en-Y gastric bypass (RYGB), paraesophageal hernia repair with Nissen fundoplication (PEH/NF), and Heller myotomy (HM).
Quantitative data
A total of 76 procedures (HM: n = 16; PEH/NF: n = 19; RYGB: n = 41) were included (152 total assessments). The attending's response rate was 92%. The reliability scores of the assessment tools (items, not raters) were high, with estimates above .85 (range: .85 to .91). There were significant differences between ratings by the trainee and attending. Only the GOALS had moderately good inter-rater agreement, .63 and .64 for RYGB and HM, respectively. The other tools had moderately low agreement,
Comment
In this study, we prospectively collected yearlong data on assessment and instruction during a fellowship with a structured curriculum. We used a single-subject design and measured the performance of fellow over an extended period of time after introduction of an intervention (ie, fellowship training).6, 11 In this design, the subject serves as his or her own control. Our assessments were based on existing, extensively studied measures and used early, frequently, and consistently to capture
Conclusions
Our study demonstrates the feasibility of using operative assessment tools for the measurement of surgical performance in a structured curriculum based on a reflective-practice instructional model. These tools can be used as a quantitative index to monitor a fellow's learning curve. In combination with immediate and specific feedback, these tools can provide a valuable source of instruction and guide performance improvement through self-directed assessments. The successful delivery of such a
References (18)
- et al.
Verbal feedback from an expert is more effective than self-accessed feedback about motion efficiency in learning new surgical skills
Am J Surg
(2007) - et al.
A global assessment tool for evaluation of intraoperative laparoscopic skills
Am J Surg
(2005) - Fellowship Council website. Retrieved May 6, 2014 from:...
- ACGME website. Retrieved May 6, 2014 from...
- et al.
Technical skills assessment toolbox: a review using the unitary framework of validity
Ann Surg
(2014) - et al.
Teaching surgical skills—changed in the wind
N Engl J Med
(2006) - et al.
The use of single-subject research to identify evidence-based practice in special education
Exceptional Child
(2005) - et al.
Objective structured assessment of technical skill (OSATS) for surgical residents
BMJ
(1997) - et al.
Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills
Surg Endosc
(2004)
Cited by (10)
Toward effective feedback: From concept to practice
2020, Surgery (United States)Citation Excerpt :Educators should be flexible and adopt new trends into their curricular design,24 as long as they do not compromise the quality of training. Video-based training can be effective and allows trainees to learn in a low-stress environment.25 This, however, does not replace the reality that surgical trainees need to progress through a comprehensive training program and to achieve competency in performing surgical operations.
Targeted surgical coaching can improve operative self-assessment ability: A single-blinded nonrandomized trial
2020, Surgery (United States)Citation Excerpt :The identified improvements in accuracy and correlation for self-assessment may be identified as “training to the scale” rather than augmented real-world ability. At worst, familiarity with the OSATS scale allows for deliberate structured reflection, which can be a valuable process.3,35,36 Self-assessment can be compared to the gold standard expert ratings using group-level analysis or direct comparisons of scores.27
Faculty feedback versus residents’ self-assessment of operative performance: Different but complementary
2018, American Journal of SurgeryCitation Excerpt :While we are aware of the limitations of self-assessment,10 we aimed to examine the utility of self-assessment when used in combination with expert feedback. This study builds on our previous investigation of qualitative and quantitative analysis of an advanced laparoscopic fellow's performance in the operating theater11 by comparing surgical residents' self-assessment of performance to the evaluation of surgical skills by experienced faculty. The goal of the study is to identify the cognitive focus of residents at various levels of training (junior and senior residents) and evaluate the differences between their thought process and that of the faculty by analyzing the narrative feedback portion of standard operative performance rating tools.
Development of a certification examination for orthopedic sports medicine fellows
2020, Canadian Journal of SurgeryValidity and reliability evidence support task-specific metrics for laparoscopic fundoplication
2024, Surgical EndoscopyAssessment of surgical performance of laparoscopic benign hiatal surgery: a systematic review
2019, Surgical Endoscopy
Timothy M. Farrell receives consultancy fees from NuWave ($500 to $1000 per year for 2 years) and grant support from the Fellowship Council ($35,000 to $50,000 per year). The other authors declare no conflicts of interest.