Validation of a new box trainer-related tracking device: the TrEndo
verfasst von:
Pieter J. van Empel, Lennart B. van Rijssen, Joris P. Commandeur, Mathilde G. E. Verdam, Judith A. Huirne, Fedde Scheele, H. Jaap Bonjer, W. Jeroen Meijerink
There is an increasing demand for structured objective ex vivo training and assessment of laparoscopic psychomotor skills prior to implementation of these skills in practice. The aim of this study was to establish the internal validity of the TrEndo, a motion-tracking device, for implementation on a laparoscopic box trainer.
Methods
Face validity and content validity were addressed through a structured questionnaire. To assess construct validity, participants were divided into an expert group and a novice group and performed two basic laparoscopic tasks. The TrEndo recorded five motion analysis parameters (MAPs) and time.
Results
Participants demonstrated a high regard for face and content validity. All recorded MAPs differed significantly between experts and novices after performing a square knot. Overall, the TrEndo correctly assigned group membership in 84.7 and 95.7% of cases based on two laparoscopic tasks.
Conclusion
Face, content, and construct validities of the TrEndo were established. The TrEndo holds real potential as a (home) training device.
Minimally invasive surgery (MIS) has been recognized as a contribution to the field of surgery by the majority of general surgeons, gynecologists, and urologists due to a distinct set of advantages, including lower short-term morbidity and mortality rates for laparoscopic resections compared to open surgery, cosmetic advantages, and an associated improved postoperative recovery [1, 2]. Technical–surgical demands in MIS differ from those in open surgery, including reduced depth perception [3‐6], longer instruments, counterintuitive instrument movement, and loss of joint dexterity [7, 8]. The operating room (OR) as a primary, complex, and expensive teaching environment is no longer desirable and it also carries legal and ethical concerns, amplified by increasing pressure on OR efficiency [9‐12]. There is a demand for structured objective ex vivo training and assessment of laparoscopic skills prior to implementation in practice.
Simulation-based practice does not put patient safety at risk and avoids interference with the efficiency of health-care resources [13, 14]. Other advantages include practice of (exclusively difficult aspects of) procedures at one’s own pace and with constructive feedback. Therefore, skill acquisition is more efficient. Assessment and therefore a minimal competency level of skills prior to implementation in practice are possible. Non-patient-bound simulation-based practice has already been demonstrated to improve MIS skills [15, 16], which are subsequently transferable to the OR [15, 17‐20]. Consequently, many surgical programs are incorporating simulation-based practice into their curriculum. However, evaluation of laparoscopic skills is currently still performed mainly by subjective expert observation [21‐24].
Anzeige
The laparoscopic box trainer, a traditional MIS simulator, has been shown to improve MIS skills and appears to be effective as a high-fidelity training device [25‐29]. The objective of this study was to investigate face, content, and construct validities of a new motion-tracking device, the TrEndo (Training in Endoscopy, Delft University of Technology, Delft, The Netherlands), implemented on a traditional laparoscopic box trainer. The TrEndo is an augmented-reality (AR) simulator that records various task-efficiency parameters (motion analysis parameters, MAPs) during simulated laparoscopic tasks. Inclusion of a motion-tracking device on the laparoscopic box trainer is a relatively new training option.
Before a surgical simulator can be used as a training and assessment device, its validity should be proven by vigorous and objective evaluation [30, 31]. Face validity assesses simulator realism [31‐33]. Content validity describes the simulator’s usefulness as a training tool [31, 32]. Construct validity reflects a simulator’s abilities to discriminate between different levels of competence, e.g., experienced surgeons and novices [33‐35].
Materials and methods
A prospective observational cohort study was conducted in The Netherlands and Belgium between February 1 and November 31, 2010.
Participants
Participants were divided into two groups based on prior laparoscopic experience. Experts were defined as having performed over 100 basic laparoscopic procedures and were recruited at two Dutch conferences. Medical students with no laparoscopic experience were defined as novices and were recruited at the VU University Medical Center. Trainees in urology, gynecology, and surgery participating in a laparoscopic suturing course organized by the VU University Medical Center at hospitals in The Netherlands and Belgium were additionally recruited for face and content validity evaluation purposes [36]. All participants voluntarily participated in this study. Participants with prior TrEndo experience were excluded. A brief introduction to the TrEndo was given to all participants.
Anzeige
Systems and hardware
Laparoscopic training boxes (Camtronics Nederland B.V., Son, The Netherlands) simulate an abdominal cavity using an aluminum frame and allow regular insertion of traditional trocars with conventional laparoscopic instruments (B. Braun Medical B.V., Melsungen, Germany) and a camera connected to a video monitor on which the simulated environment is viewed.
The TrEndo is constructed as a trocar on a laparoscopic training box through which laparoscopic instruments may still be regularly inserted (Fig. 1). Instrument movement is measured in four degrees of freedom: X, Y, and Z axes and axis rotation [37]. Five motion analysis parameters (MAPs) are recorded with a sample frequency of 100 Hz individually for the right and left hands, including: path length (mm, length of curve described by the tip of the instrument), insertion distance (mm, total distance traveled by the instrument along its axis), angular area (deg2, related to the distances between the farthest positions of the instrument during a task), volume (mm3, thee-dimensional space used), and time (s) [38].
Fig. 1
A Schematic and photo of TrEndo motion-tracking device on a laparoscopic box trainer. B Knot-tying task on a laparoscopic box trainer equipped with the TrEndo tracking device
×
Face and content validities
Twenty questions adapted from a previously described study on the face validation of another laparoscopic training device were used to inquire experts’ and trainees’ first impression of the TrEndo with a laparoscopic box trainer [33]. Participants additionally rated six possible advantages of the TrEndo with a laparoscopic box trainer compared to a virtual reality (VR) system on a 5-point Likert scale (1 = unimportant, 5 = very important) which we represent as percentages rated low–moderately important (score = 1–3) to highly important (score = 4–5). To assess content validity, participants reflected on the training capacities of the TrEndo within a standardized surgical curriculum using an open questionnaire.
Construct validity
Participants were asked to correctly position a curved tapered needle into a laparoscopic needle holder (task 1) and complete a standard laparoscopic square knot (task 2) in the laparoscopic box trainer on an artificial skin patch using a 15-cm single 3-0 silk suture. A 5-min time limit was set for task 2. Equipment and instruments used were kept identical. Construct validity was determined by comparison of expert and novice MAPs for both tasks. To explore whether MAPs predict individual laparoscopic skill, a logistic regression analysis was conducted to predict group membership (novice or expert) based on TrEndo performance. Finally, we used a Wald analysis to determine the most contributive MAPs in predicting group membership by calculating individual contributions per MAP to the predictions made in the logistic regression analysis.
Statistical analysis
Statistical analysis was performed using SPSS 15.0 for Windows (SPSS Inc., Chicago, IL, USA). A nominal significance level of 0.05 was used. All tests were performed two-sided. Values for continuous variables are given as mean (SD), with regard to the multiple TrEndo parameters this was performed using MANOVA analysis, followed by separated ANOVAs. Comparisons with regard to the questionnaire data were performed with two-sided Mann–Whitney U analysis. These results are presented as median values. Values for categorical data are specified as frequency (%) and were analyzed using χ2 or Fisher’s exact test. A logistic regression analysis was used to test the predictive value of the TrEndo parameters for classification of participants as either novice or expert according to their performance.
Results
Demographics, experience, and face validity
The majority of respondents were active in general surgery (Table 1). Thirty-eight experts and 24 trainees returned the questionnaire. Not all respondents completed the entire form, but to calculate mean scores there were never more than four value points missing.
Table 1
Participant characteristics
Demographics
Total
Experts
Trainees
Total (male; female)
57 (39; 18)
38 (29; 9)
24 (12; 12)
Mean age (range)
46 (33–60)
32 (28–37)
Respondent specialties
N
%
%
General surgery
43
71.1
66.7
Gynecology
9
13.2
16.7
Urology
10
15.8
16.7
Mean no. laparoscopic procedures/year
114.86
40.87
Mean no. complex laparoscopic procedures/year
62.97
7.05
Table 2 depicts mean first impression scores. All questions on first impression were rated above a score of 3 on the 5-point Likert scale. Trainees were slightly more positive than experts, with a significant difference for design and overall functionality.
Table 2
Face validity
Total mean
Experts
Trainees
p*
Mean
SD
Mean
SD
Design
3.57
3.36
0.79
3.88
0.74
0.01
Realism
3.30
3.30
0.81
3.29
0.67
0.82
User-friendliness
3.69
3.68
0.66
3.71
0.69
0.81
Overall functionality
3.40
3.08
1.02
3.88
0.74
0.02
Trocar positions
3.79
3.78
0.71
3.79
0.51
0.95
Instrument movement
3.67
3.70
0.85
3.63
0.71
0.65
* Mann–Whitney test, two-sided, expert versus resident
Content validity
All experts and trainees rated procedural functioning, hand–eye coordination, and depth perception above 3 on the 5-point Likert scale (Table 3). Seventy-five percent of the participants scored 4–5 on the 5-point Likert scale regarding didactic quality compared to a low–moderate score (1–3) by 12.5%. This question was not answered by 12.5% of participants. No significant differences were observed between experts and trainees.
Table 3
Content validity
Training capacities
Total mean
Experts
Trainees
p*
Mean
SD
Mean
SD
Procedural functioning
4.26
4.16
0.68
4.42
0.56
0.13
Hand–eye coordination
4.31
4.29
0.64
4.35
0.58
0.72
Depth perception
3.48
3.34
0.95
3.69
1.02
0.26
* Mann–Whitney test, two-sided, expert versus trainee
Anzeige
Over 60% of the participants rated each selected advantage of the TrEndo with laparoscopic box trainer as highly advantageous compared to a VR system. No significant difference was observed between experts and trainees (Table 4).
Table 4
Selected possible advantages of the box trainer with TrEndo, compared to VR systems
Totala
Expertsa
Traineesa
p*
1–3
4–5
1–3
4–5
1–3
4–5
Intro and examples DVD
35.1
64.9
32.4
67.6
39.1
60.9
0.59
Home practice
14.0
86.0
17.6
82.4
8.7
91.3
0.91
Real instruments
5.3
94.7
5.9
94.1
4.3
95.7
0.79
Real needle and thread
0.0
100.0
0.0
100.0
0.0
100.0
1.00
Realistic haptic feedback
13.2
86.8
15.6
84.4
9.5
90.5
0.52
Objective assessment
28.1
71.9
26.5
73.5
30.4
69.6
0.74
Categories are divided into low-to-moderate (1–3) and highly (4–5) advantageous compared to a VR system. All values are expressed as percentages on a 5-point Likert scale
* χ2 test, two-sided, expert versus trainees
aValid percent
Construct validity
Task 1 was completed significantly faster by experts (n = 46) than by novices (n = 65) (p < 0.001). Experts used significantly less path length than novices with both right and left hand and utilized a significantly smaller right-hand area and right-hand volume compared to novices. There were no significant differences between experts and novices with the remaining MAPs, although left-hand area, left-hand volume, and right-hand depth showed a trend in favor of experts.
No novices were able to complete task 2 within 5 min. Table 5 gives the MAPs for task 2, showing a significant difference between novices and experts on all MAPs for both hands.
Table 5
Motion analysis parameters for square knot-tying task (task 2)
MAP
Novices
Experts
p
Mean
SD
Mean
SD
Left path (mm)
6,288.47
1,994.74
2,792.92
1,733.70
<0.01
Left depth (mm)
19.01
5.77
16.19
4.05
<0.01
Left area (deg2)
142.53
86.24
89.93
55.13
<0.01
Left volume (mm3)
2,746.13
1,790.54
1,318.09
870.48
<0.01
Right path (mm)
5780.33
2096.37
2862.95
1587.96
<0.01
Right depth (mm)
17.46
4.08
14.98
3.62
<0.01
Right area (deg2)
104.40
51.52
85.34
40.48
0.03
Right volume (mm3)
2,170.82
1,128.60
1,451.15
863.43
<0.01
Time (s)
277.53
39.97
117.59
77.80
<0.01
Based on recorded MAPs it was possible to correctly classify 84.8% of novices (56 of 66) and 84.4% of experts (38 of 45) for task 1. The TrEndo correctly classified 98.3% of novices (58 of 59) and 92.9% of experts (52 of 56) for task 2. Overall, the TrEndo correctly classified 84.7% of participants (94 of 111) at the first task and 95.7% of participants (110 of 115) at the second task.
Anzeige
Left path (Wald score 10.97), left depth (Wald score 4.63), and time (Wald score 15.67) were the most relevant parameters in determining participant level at task 1. At task 2, left volume (Wald score 4.80) and time (Wald score 15.37) were the most contributory parameters to correctly classifying participants as either expert or novice (Table 6).
Table 6
Calculated Wald values for needle positioning (task 1) and square knot-tying task (task 2)
Dependent variable
Needle positioning
Square knot
Wald
p
Wald
p
Left path (mm)
10.97
<0.01
0.05
0.82
Left depth (mm)
4.63
0.03
0.09
0.76
Left area (deg2)
0.43
0.51
2.38
0.12
Left volume (mm3)
1.20
0.27
4.80
0.03
Right path (mm)
0.01
0.92
1.52
0.22
Right depth (mm)
0.06
0.81
0.01
0.92
Right area (deg2)
0.27
0.61
0.38
0.54
Right volume (mm3)
0.49
0.49
0.60
0.44
Time (s)
15.67
<0.01
15.37
<0.01
Discussion
Structured objective training and assessment of surgical skills prior to implementation in the OR is a hot topic in current surgical, gynecologic, and urologic education [39]. This is one of the first studies to evaluate laparoscopic suturing tasks using a motion-tracking device on a traditional laparoscopic box trainer. Besides the laparoscopic box trainer, MIS simulators include augmented-reality (AR) and virtual reality (VR) systems [21, 32, 40, 41]. AR simulators combine the physical reality (such as in a box trainer) with virtual reality into one system. VR simulators are completely computer-based; software replicates entire MIS procedures.
AR simulators similar to the TrEndo include the Red Dragon (EDGE) (SimuLab, Seattle, WA, USA) and the ProMis (Haptica Inc., Boston, MA, USA). Both track instrument movements using a passive vision-tracking system, with cameras additionally capturing video images of internal laparoscopic instrument movement. The Red Dragon also provides force measurement on instruments and tissue. These latter two instruments are, however, sensitive to interference and usable only in a laboratory setting [42‐44]. Compared to the TrEndo they are also relatively expensive (training option) and are unable to deploy a faculty’s own instruments.
Most AR and VR simulators provide objective feedback after an exercise using incorporated metrics calculated by motion tracking. Such feedback allows supervisors and trainees to monitor performance and progress objectively without supervisor’s presence. Tracking systems are inherently present in VR simulators; however, their use for assessment is often not tested or validated [45]. All MIS simulators aim for a maximum realistic setting but basic laparoscopic skills training in particular remains unrealistic as AR and VR simulators are computer-based. Furthermore, most AR and VR simulators do not provide realistic tactile (haptic) feedback [46]. Previous studies illustrated the importance of haptic feedback during laparoscopic training [47, 48], demonstrating significant improvement of surgical skills in the presence of haptic feedback compared to training in the absence of haptic feedback [32, 48]. VR simulators tend to be expensive and immobile and the advanced technology compromises user-friendliness. Realism and procedural training are limited as VR simulation tends to focus on hand–eye coordination training [49].
Anzeige
Based on recorded MAPs, the TrEndo correctly classified 95.7% of a large and diverse study group into appropriate expert or novice groups. MAPs time, left depth, and left path were most contributory to correctly predicting group membership when positioning a needle into a needle holder, most likely because the left hand is often not used by novices during this task. After the performance of the square knot task, all MAPs differed significantly between experts and novices. Time and left-hand volume were the most contributory MAPs for this task. We believe a difference in novice versus expert dexterity is determined mainly by left-hand skill.
This study follows on the work described by Chmarra et al. [38], in which the TrEndo was able to correctly classify 23 of 31 gynecologists based on laparoscopic nonsuturing tasks, and on the work of Yamaguchi et al. [50], who demonstrated the efficacy of an objective evaluation of psychomotor skills for laparoscopic suturing using an electromagnetic motion-tracking system for MAPs time, left path length, and right speed between experienced surgeons and novice surgeons. In accordance with prior results, this study showed that time and path length help distinguish between expert and novices [22, 51]. We recruited a large and diverse study group, employed an extensive set of MAPs, and focused on laparoscopic tasks, including intracorporeal suturing and knot-tying.
Many novices had difficulty concentrating a full 5 min, and not all recruited experts had equal experience or skill in laparoscopy, and we did not acknowledge a subject’s dominant hand. This may have influenced recorded MAPs.
Metrics currently must still be interpreted by a faculty member. We are investigating facilitation of this interpretation and therefore facilitate experts to novices or intermediate comparison.
Our face and content validity data indicate that the TrEndo would be acceptable to experts and trainees as a training device (including at home). Future studies should investigate the training capacities of the TrEndo. As the TrEndo is user-friendly and mobile it also holds real potential as a home training device. We are currently investigating this autonomous training option.
It is important to realize that MAPs provide an indication of task efficiency and with the exception that no novices completed a knot in this study, knot quality was not evaluated. The comparison of quantitative measures of novice to expert does provide evaluation of novice performance and does confirm construct validity. To provide an integrated objective assessment of MIS skills, efficiency parameters should be combined with other valid metrics of MIS skill. Improved efficiency parameters may correlate with improved quality parameters; however, this should be assessed in future studies. The first step in comparing quality parameters with efficiency indicators may include tasks where it is relatively easy to score task “error” performance (e.g., a stretched surgical glove where subjects must pass a needle through defined targets without tearing). Inclusion of an inexpensive tensiometer might also provide information on quality assessment; however, current tensiometers are too expensive. Besides task quality and efficiency, surgeons also need a core knowledge base, clinical decision-making and communicative skills, and the ability to think and work under stress in a team setting when performing MIS on patients [52, 53]. Future steps should assess the additional value of (recorded TrEndo) efficiency parameters into an integrated assessment of MIS skills and the objective value of (TrEndo-based) efficiency parameters herein.
Conclusion
Face, content, and construct validities of the TrEndo were established. The TrEndo holds real potential as a (home) training device.
Disclosures
Pieter J. van Empel, Lennart B. van Rijssen, Joris P. Commandeur, Mathilde G. E. Verdam, Judith A. Huirne, Fedde Scheele, H. Jaap Bonjer, and W. Jeroen Meijerink have no conflicts of interest or financial ties to disclose.
Open Access
This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Mit der Zeitschrift Die Chirurgie erhalten Sie zusätzlich Online-Zugriff auf weitere 43 chirurgische Fachzeitschriften, CME-Fortbildungen, Webinare, Vorbereitungskursen zur Facharztprüfung und die digitale Enzyklopädie e.Medpedia.
Bis 30. April 2024 bestellen und im ersten Jahr nur 199 € zahlen!
Ein Frauenanteil von mindestens einem Drittel im ärztlichen Op.-Team war in einer großen retrospektiven Studie aus Kanada mit einer signifikanten Reduktion der postoperativen Morbidität assoziiert.
Sie sei „ethisch geboten“, meint Gesundheitsminister Karl Lauterbach: mehr Transparenz über die Qualität von Klinikbehandlungen. Um sie abzubilden, lässt er gegen den Widerstand vieler Länder einen virtuellen Klinik-Atlas freischalten.
Eine Studie zum Nutzen der druckentlastenden Kraniektomie nach schwerer tiefer supratentorieller Hirnblutung deutet einen Nutzen der Operation an. Für überlebende Patienten ist das dennoch nur eine bedingt gute Nachricht.
Gesundheitsminister Lauterbach hat die vom Bundeskabinett beschlossene Klinikreform verteidigt. Kritik an den Plänen kommt vom Marburger Bund. Und in den Ländern wird über den Gang zum Vermittlungsausschuss spekuliert.
Update Chirurgie
Bestellen Sie unseren Fach-Newsletterund bleiben Sie gut informiert.
Das Karpaltunnelsyndrom ist die häufigste Kompressionsneuropathie peripherer Nerven. Obwohl die Anamnese mit dem nächtlichen Einschlafen der Hand (Brachialgia parästhetica nocturna) sehr typisch ist, ist eine klinisch-neurologische Untersuchung und Elektroneurografie in manchen Fällen auch eine Neurosonografie erforderlich. Im Anfangsstadium sind konservative Maßnahmen (Handgelenksschiene, Ergotherapie) empfehlenswert. Bei nicht Ansprechen der konservativen Therapie oder Auftreten von neurologischen Ausfällen ist eine Dekompression des N. medianus am Karpaltunnel indiziert.
Das Webinar beschäftigt sich mit Fragen und Antworten zu Diagnostik und Klassifikation sowie Möglichkeiten des Ausschlusses von Zusatzverletzungen. Die Referenten erläutern, welche Frakturen konservativ behandelt werden können und wie. Das Webinar beantwortet die Frage nach aktuellen operativen Therapiekonzepten: Welcher Zugang, welches Osteosynthesematerial? Auf was muss bei der Nachbehandlung der distalen Radiusfraktur geachtet werden?
Inhalte des Webinars zur S1-Leitlinie „Empfehlungen zur Therapie der akuten Appendizitis bei Erwachsenen“ sind die Darstellung des Projektes und des Erstellungswegs zur S1-Leitlinie, die Erläuterung der klinischen Relevanz der Klassifikation EAES 2015, die wissenschaftliche Begründung der wichtigsten Empfehlungen und die Darstellung stadiengerechter Therapieoptionen.
