Skip to main content
Erschienen in: Knee Surgery, Sports Traumatology, Arthroscopy 2/2017

16.04.2016 | Sports Traumatology

Validation of the updated ArthroS simulator: face and construct validity of a passive haptic virtual reality simulator with novel performance metrics

verfasst von: Patrick Garfjeld Roberts, Paul Guyver, Mathew Baldwin, Kash Akhtar, Abtin Alvand, Andrew J. Price, Jonathan L. Rees

Erschienen in: Knee Surgery, Sports Traumatology, Arthroscopy | Ausgabe 2/2017

Einloggen, um Zugang zu erhalten

Abstract

Purpose

To assess the construct and face validity of ArthroS, a passive haptic VR simulator. A secondary aim was to evaluate the novel performance metrics produced by this simulator.

Methods

Two groups of 30 participants, each divided into novice, intermediate or expert based on arthroscopic experience, completed three separate tasks on either the knee or shoulder module of the simulator. Performance was recorded using 12 automatically generated performance metrics and video footage of the arthroscopic procedures. The videos were blindly assessed using a validated global rating scale (GRS). Participants completed a survey about the simulator’s realism and training utility.

Results

This new simulator demonstrated construct validity of its tasks when evaluated against a GRS (p ≤ 0.003 in all cases). Regarding it’s automatically generated performance metrics, established outputs such as time taken (p ≤ 0.001) and instrument path length (p ≤ 0.007) also demonstrated good construct validity. However, two-thirds of the proposed ‘novel metrics’ the simulator reports could not distinguish participants based on arthroscopic experience. Face validity assessment rated the simulator as a realistic and useful tool for trainees, but the passive haptic feedback (a key feature of this simulator) is rated as less realistic.

Conclusion

The ArthroS simulator has good task construct validity based on established objective outputs, but some of the novel performance metrics could not distinguish between surgical experience. The passive haptic feedback of the simulator also needs improvement. If simulators could offer automated and validated performance feedback, this would facilitate improvements in the delivery of training by allowing trainees to practise and self-assess.
Literatur
1.
Zurück zum Zitat Aggarwal R, Tully A, Grantcharov T, Larsen CR, Miskry T, Farthing A, Darzi A (2006) Virtual reality simulation training can improve technical skills during laparoscopic salpingectomy for ectopic pregnancy. Br J Obstet Gynaec 113:1382–1387CrossRef Aggarwal R, Tully A, Grantcharov T, Larsen CR, Miskry T, Farthing A, Darzi A (2006) Virtual reality simulation training can improve technical skills during laparoscopic salpingectomy for ectopic pregnancy. Br J Obstet Gynaec 113:1382–1387CrossRef
2.
Zurück zum Zitat Akhtar K, Standfield NJ, Gupte CM, Tuijthof GJM (2015) Chapter 7: Virtual reality simulators. In: Karahan M, Kerkhoffs GMMJ, Randelli P, Tuijthof GJM (eds) Effective training in arthroscopic skills, 1st edn. Springer, Berlin, pp 71–80 Akhtar K, Standfield NJ, Gupte CM, Tuijthof GJM (2015) Chapter 7: Virtual reality simulators. In: Karahan M, Kerkhoffs GMMJ, Randelli P, Tuijthof GJM (eds) Effective training in arthroscopic skills, 1st edn. Springer, Berlin, pp 71–80
3.
Zurück zum Zitat Alvand A, Auplish S, Khan T, Gill HS, Rees JL (2011) Identifying orthopaedic surgeons of the future: the inability of some medical students to achieve competence in basic arthroscopic tasks despite training: a randomised study. J Bone Joint Surg Br 93:1586–1591CrossRefPubMed Alvand A, Auplish S, Khan T, Gill HS, Rees JL (2011) Identifying orthopaedic surgeons of the future: the inability of some medical students to achieve competence in basic arthroscopic tasks despite training: a randomised study. J Bone Joint Surg Br 93:1586–1591CrossRefPubMed
5.
Zurück zum Zitat Angelo RL, Ryu RKN, Pedowitz RA, Gallagher AG (2015) Metric development for an arthroscopic Bankart procedure: assessment of face and content validity. Arthroscopy 31:1430–1440CrossRefPubMed Angelo RL, Ryu RKN, Pedowitz RA, Gallagher AG (2015) Metric development for an arthroscopic Bankart procedure: assessment of face and content validity. Arthroscopy 31:1430–1440CrossRefPubMed
7.
Zurück zum Zitat Brown D (2013) The role of simulation in the learning of surgical skills. Ann R Coll Surg Eng (Suppl) 95:292–295CrossRef Brown D (2013) The role of simulation in the learning of surgical skills. Ann R Coll Surg Eng (Suppl) 95:292–295CrossRef
8.
Zurück zum Zitat Cannon WD, Nicandri GT, Reinig K, Mevis H, Wittstein J (2014) Evaluation of skill level between trainees and community orthopaedic surgeons using a virtual reality arthroscopic knee simulator. J Bone Joint Surg Am 96:e57. doi:10.2106/JBJS.M.00779 CrossRefPubMed Cannon WD, Nicandri GT, Reinig K, Mevis H, Wittstein J (2014) Evaluation of skill level between trainees and community orthopaedic surgeons using a virtual reality arthroscopic knee simulator. J Bone Joint Surg Am 96:e57. doi:10.​2106/​JBJS.​M.​00779 CrossRefPubMed
9.
Zurück zum Zitat Coughlin RP, Pauyo T, Sutton JC, Coughlin LP, Bergeron SG (2015) A validated orthopaedic surgical simulation model for training and evaluation of basic arthroscopic skills. J Bone Joint Surg Am 97:1465–1471CrossRefPubMed Coughlin RP, Pauyo T, Sutton JC, Coughlin LP, Bergeron SG (2015) A validated orthopaedic surgical simulation model for training and evaluation of basic arthroscopic skills. J Bone Joint Surg Am 97:1465–1471CrossRefPubMed
10.
Zurück zum Zitat Darzi A, Smith S, Taffinder N (1999) Assessing operative skill: needs to become more objective. Br Med J 318:887–888CrossRef Darzi A, Smith S, Taffinder N (1999) Assessing operative skill: needs to become more objective. Br Med J 318:887–888CrossRef
11.
Zurück zum Zitat Datta V, Mackay S, Mandalia M, Darzi A (2001) The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg 193:479–485CrossRefPubMed Datta V, Mackay S, Mandalia M, Darzi A (2001) The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg 193:479–485CrossRefPubMed
12.
Zurück zum Zitat Ferguson JY, Alvand A, Price AJ, Rees JL (2015) Chapter 8: Theory on simulator validation. In: Karahan M, Kerkhoffs GMMJ, Randelli P, Tuijthof GJM (eds) Effective training in arthroscopic skills, 1st edn. Springer, Berlin, pp 81–94 Ferguson JY, Alvand A, Price AJ, Rees JL (2015) Chapter 8: Theory on simulator validation. In: Karahan M, Kerkhoffs GMMJ, Randelli P, Tuijthof GJM (eds) Effective training in arthroscopic skills, 1st edn. Springer, Berlin, pp 81–94
13.
Zurück zum Zitat Fucentese SF, Rahm S, Wieser K, Spillmann J, Harders M, Koch PP (2015) Evaluation of a virtual-reality-based simulator using passive haptic feedback for knee arthroscopy. Knee Surg Sports Traumatol Arthrosc 23:1077–1085CrossRefPubMed Fucentese SF, Rahm S, Wieser K, Spillmann J, Harders M, Koch PP (2015) Evaluation of a virtual-reality-based simulator using passive haptic feedback for knee arthroscopy. Knee Surg Sports Traumatol Arthrosc 23:1077–1085CrossRefPubMed
14.
Zurück zum Zitat Gélinas-Phaneuf N, Choudhury N, Al-Habib AR, Cabral A, Nadeau E, Mora V, Pazos V, Debergue P, DiRaddo R, Del Maestro RF (2014) Assessing performance in brain tumor resection using a novel virtual reality simulator. Int J Comput Assist Radiol Surg 9:1–9. doi:10.1007/s11548-013-0905-8 CrossRefPubMed Gélinas-Phaneuf N, Choudhury N, Al-Habib AR, Cabral A, Nadeau E, Mora V, Pazos V, Debergue P, DiRaddo R, Del Maestro RF (2014) Assessing performance in brain tumor resection using a novel virtual reality simulator. Int J Comput Assist Radiol Surg 9:1–9. doi:10.​1007/​s11548-013-0905-8 CrossRefPubMed
16.
Zurück zum Zitat Howells NR, Brinsden MD, Gill RS, Carr AJ, Rees JL (2008) Motion analysis: a validated method for showing skill levels in arthroscopy. Arthroscopy 24:335–342CrossRefPubMed Howells NR, Brinsden MD, Gill RS, Carr AJ, Rees JL (2008) Motion analysis: a validated method for showing skill levels in arthroscopy. Arthroscopy 24:335–342CrossRefPubMed
17.
Zurück zum Zitat Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL (2008) Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. J Bone Joint Surg Br 90:494–499CrossRefPubMed Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL (2008) Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. J Bone Joint Surg Br 90:494–499CrossRefPubMed
18.
Zurück zum Zitat Hunter S, McLaren P (1993) Specialist medical training and the Calman report. Br Med J 306:1281–1282CrossRef Hunter S, McLaren P (1993) Specialist medical training and the Calman report. Br Med J 306:1281–1282CrossRef
19.
Zurück zum Zitat Jacobsen ME, Andersen MJ, Hansen CO, Konge L (2015) Testing basic competency in knee arthroscopy using a virtual reality simulator: exploring validity and reliability. J Bone Joint Surg Am 97:775–781CrossRefPubMed Jacobsen ME, Andersen MJ, Hansen CO, Konge L (2015) Testing basic competency in knee arthroscopy using a virtual reality simulator: exploring validity and reliability. J Bone Joint Surg Am 97:775–781CrossRefPubMed
21.
Zurück zum Zitat Koehler RJ, Amsdell S, Arendt EA, Bisson LJ, Bramen JP, Butler A, Cosgarea AJ, Harner CD, Garrett WE, Olson T, Warme WJ, Nicandri GT (2013) The arthroscopic surgical skill evaluation tool (ASSET). Am J Sports Med 41:1229–1237CrossRefPubMedPubMedCentral Koehler RJ, Amsdell S, Arendt EA, Bisson LJ, Bramen JP, Butler A, Cosgarea AJ, Harner CD, Garrett WE, Olson T, Warme WJ, Nicandri GT (2013) The arthroscopic surgical skill evaluation tool (ASSET). Am J Sports Med 41:1229–1237CrossRefPubMedPubMedCentral
22.
Zurück zum Zitat Lopez G, Wright R, Martin D, Jung J, Bracey D, Gupta R (2015) A cost-effective junior resident training and assessment simulator for orthopaedic surgical skills via fundamentals of orthopaedic surgery: AAOS exhibit selection. J Bone Joint Surg Am 97:659–666CrossRefPubMed Lopez G, Wright R, Martin D, Jung J, Bracey D, Gupta R (2015) A cost-effective junior resident training and assessment simulator for orthopaedic surgical skills via fundamentals of orthopaedic surgery: AAOS exhibit selection. J Bone Joint Surg Am 97:659–666CrossRefPubMed
23.
Zurück zum Zitat Marsh H (2015) Better not look down…. Ann R Coll Surg Eng (Suppl) 97:339–342CrossRef Marsh H (2015) Better not look down…. Ann R Coll Surg Eng (Suppl) 97:339–342CrossRef
24.
Zurück zum Zitat Milburn JA, Khera G, Hornby ST, Malone PSC, Fitzgerald JEF (2012) Introduction, availability and role of simulation in surgical education and training: review of current evidence and recommendations from the Association of Surgeons in Training. Int J Surg 10:393–398CrossRefPubMed Milburn JA, Khera G, Hornby ST, Malone PSC, Fitzgerald JEF (2012) Introduction, availability and role of simulation in surgical education and training: review of current evidence and recommendations from the Association of Surgeons in Training. Int J Surg 10:393–398CrossRefPubMed
26.
Zurück zum Zitat Nousiainen MT, McQueen SA, Ferguson P, Alman B, Kraemer W, Safir O, Reznick R, Sonnadara R (2015) Simulation for teaching orthopaedic residents in a competency-based curriculum: do the benefits justify the increased costs? Clin Orthop Relat Res. doi:10.1007/s11999-015-4512-6 Nousiainen MT, McQueen SA, Ferguson P, Alman B, Kraemer W, Safir O, Reznick R, Sonnadara R (2015) Simulation for teaching orthopaedic residents in a competency-based curriculum: do the benefits justify the increased costs? Clin Orthop Relat Res. doi:10.​1007/​s11999-015-4512-6
27.
Zurück zum Zitat Palter VN, Grantcharov TP (2014) Individualized deliberate practice on a virtual reality simulator improves technical performance of surgical novices in the operating room. Ann Surg 259:443–448CrossRefPubMed Palter VN, Grantcharov TP (2014) Individualized deliberate practice on a virtual reality simulator improves technical performance of surgical novices in the operating room. Ann Surg 259:443–448CrossRefPubMed
28.
Zurück zum Zitat Panait L, Akkary E, Bell RL, Roberts KE, Dudrick SJ, Duffy AJ (2009) The role of haptic feedback in laparoscopic simulation training. J Surg Res 156:312–316CrossRefPubMed Panait L, Akkary E, Bell RL, Roberts KE, Dudrick SJ, Duffy AJ (2009) The role of haptic feedback in laparoscopic simulation training. J Surg Res 156:312–316CrossRefPubMed
29.
Zurück zum Zitat Pedowitz RA, Nicandri GT, Angelo RL, Ryu RKN, Gallagher AG (2015) Objective assessment of knot-tying proficiency with the fundamentals of arthroscopic surgery training program workstation and knot tester. Arthroscopy 31(10):1872–1879CrossRefPubMed Pedowitz RA, Nicandri GT, Angelo RL, Ryu RKN, Gallagher AG (2015) Objective assessment of knot-tying proficiency with the fundamentals of arthroscopic surgery training program workstation and knot tester. Arthroscopy 31(10):1872–1879CrossRefPubMed
30.
Zurück zum Zitat Rasmussen J (1983) Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE Trans Syst Man Cybern 13:257–266CrossRef Rasmussen J (1983) Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE Trans Syst Man Cybern 13:257–266CrossRef
31.
Zurück zum Zitat Shah J, Darzi A (2002) Virtual reality flexible cystoscopy: a validation study. Br J Urol 90:828–832CrossRef Shah J, Darzi A (2002) Virtual reality flexible cystoscopy: a validation study. Br J Urol 90:828–832CrossRef
32.
Zurück zum Zitat Stunt JJ, Kerkhoffs GMMJ, Horeman T, van Dijk CN, Tuijthof GJM (2014) Validation of the PASSPORT V2 training environment for arthroscopic skills. Knee Surg Sports Traumatol Arthrosc. doi:10.1007/s00167-014-3213-0 Stunt JJ, Kerkhoffs GMMJ, Horeman T, van Dijk CN, Tuijthof GJM (2014) Validation of the PASSPORT V2 training environment for arthroscopic skills. Knee Surg Sports Traumatol Arthrosc. doi:10.​1007/​s00167-014-3213-0
33.
Zurück zum Zitat Stunt JJ, Kerkhoffs GMMJ, van Dijk CN, Tuijthof GJM (2015) Validation of the ArthroS virtual reality simulator for arthroscopic skills. Knee Surg Sports Traumatol Arthrosc 23:3436–3442CrossRefPubMed Stunt JJ, Kerkhoffs GMMJ, van Dijk CN, Tuijthof GJM (2015) Validation of the ArthroS virtual reality simulator for arthroscopic skills. Knee Surg Sports Traumatol Arthrosc 23:3436–3442CrossRefPubMed
34.
Zurück zum Zitat Tashiro Y, Miura H, Nakanishi Y, Okazaki K, Iwamoto Y (2009) Evaluation of skills in arthroscopic training based on trajectory and force data. Clin Orthop Relat Res 467:546–552CrossRefPubMed Tashiro Y, Miura H, Nakanishi Y, Okazaki K, Iwamoto Y (2009) Evaluation of skills in arthroscopic training based on trajectory and force data. Clin Orthop Relat Res 467:546–552CrossRefPubMed
35.
Zurück zum Zitat Tuijthof GJM, van Sterkenburg MN, Sierevelt IN, van Oldenrijk J, Van Dijk CN, Kerkhoffs GMMJ (2010) First validation of the PASSPORT training environment for arthroscopic skills. Knee Surg Sports Traumatol Arthrosc 18:218–224CrossRefPubMed Tuijthof GJM, van Sterkenburg MN, Sierevelt IN, van Oldenrijk J, Van Dijk CN, Kerkhoffs GMMJ (2010) First validation of the PASSPORT training environment for arthroscopic skills. Knee Surg Sports Traumatol Arthrosc 18:218–224CrossRefPubMed
37.
Zurück zum Zitat Weiner IB, Craighead WE (eds) (2010) The corsini encyclopedia of psychology. Wiley, Hoboken Weiner IB, Craighead WE (eds) (2010) The corsini encyclopedia of psychology. Wiley, Hoboken
38.
Zurück zum Zitat Wentink M, Stassen LPS, Alwayn I, Hosman RJAW, Stassen HG (2003) Rasmussen’s model of human behavior in laparoscopy training. Surg Endosc 17:1241–1246CrossRefPubMed Wentink M, Stassen LPS, Alwayn I, Hosman RJAW, Stassen HG (2003) Rasmussen’s model of human behavior in laparoscopy training. Surg Endosc 17:1241–1246CrossRefPubMed
39.
Zurück zum Zitat Ziegler R, Fischer G, Müller W, Göbel M (1995) Virtual reality arthroscopy training simulator. Comput Biol Med 25:193–203CrossRefPubMed Ziegler R, Fischer G, Müller W, Göbel M (1995) Virtual reality arthroscopy training simulator. Comput Biol Med 25:193–203CrossRefPubMed
Metadaten
Titel
Validation of the updated ArthroS simulator: face and construct validity of a passive haptic virtual reality simulator with novel performance metrics
verfasst von
Patrick Garfjeld Roberts
Paul Guyver
Mathew Baldwin
Kash Akhtar
Abtin Alvand
Andrew J. Price
Jonathan L. Rees
Publikationsdatum
16.04.2016
Verlag
Springer Berlin Heidelberg
Erschienen in
Knee Surgery, Sports Traumatology, Arthroscopy / Ausgabe 2/2017
Print ISSN: 0942-2056
Elektronische ISSN: 1433-7347
DOI
https://doi.org/10.1007/s00167-016-4114-1

Weitere Artikel der Ausgabe 2/2017

Knee Surgery, Sports Traumatology, Arthroscopy 2/2017 Zur Ausgabe

Arthropedia

Grundlagenwissen der Arthroskopie und Gelenkchirurgie. Erweitert durch Fallbeispiele, Videos und Abbildungen. 
» Jetzt entdecken

Notfall-TEP der Hüfte ist auch bei 90-Jährigen machbar

26.04.2024 Hüft-TEP Nachrichten

Ob bei einer Notfalloperation nach Schenkelhalsfraktur eine Hemiarthroplastik oder eine totale Endoprothese (TEP) eingebaut wird, sollte nicht allein vom Alter der Patientinnen und Patienten abhängen. Auch über 90-Jährige können von der TEP profitieren.

Arthroskopie kann Knieprothese nicht hinauszögern

25.04.2024 Gonarthrose Nachrichten

Ein arthroskopischer Eingriff bei Kniearthrose macht im Hinblick darauf, ob und wann ein Gelenkersatz fällig wird, offenbar keinen Unterschied.

Therapiestart mit Blutdrucksenkern erhöht Frakturrisiko

25.04.2024 Hypertonie Nachrichten

Beginnen ältere Männer im Pflegeheim eine Antihypertensiva-Therapie, dann ist die Frakturrate in den folgenden 30 Tagen mehr als verdoppelt. Besonders häufig stürzen Demenzkranke und Männer, die erstmals Blutdrucksenker nehmen. Dafür spricht eine Analyse unter US-Veteranen.

Ärztliche Empathie hilft gegen Rückenschmerzen

23.04.2024 Leitsymptom Rückenschmerzen Nachrichten

Personen mit chronischen Rückenschmerzen, die von einfühlsamen Ärzten und Ärztinnen betreut werden, berichten über weniger Beschwerden und eine bessere Lebensqualität.

Update Orthopädie und Unfallchirurgie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.