Skip to main content
Erschienen in: Canadian Journal of Anesthesia/Journal canadien d'anesthésie 12/2013

01.12.2013 | Special Article

Assessment of procedural skills training and performance in anesthesia using cumulative sum analysis (cusum)

verfasst von: Tim Starkie, BMBS, Elizabeth J. Drake, BM

Erschienen in: Canadian Journal of Anesthesia/Journal canadien d'anesthésie | Ausgabe 12/2013

Einloggen, um Zugang zu erhalten

Abstract

Purpose

The current methods (work based assessments and logbooks) used to assess procedural competency and performance have well-documented deficiencies. Cumulative sum analysis (cusum), a statistical method that generates performance graphs over time, is an alternative tool that is not currently widely used. The purpose of this review is to investigate its current role in anesthetic procedural skills training and performance.

Source

A literature search of MEDLINE®, EMBASE™, BNI, CINAHL®, the Cochrane Library, NHS Evidence, and the Trip database was performed in October 2012. All papers using cusum to investigate performance in anesthetic procedural skills were included. Their references were searched manually to identify any additional studies.

Principal findings

Thirteen papers were identified. The procedural skills they investigated could be split broadly into three groups: ultrasound skills, airway and cannulation, and regional anesthesia. All papers had small sample sizes (< 30), with most researching novice trainee performance. Wide ranges were seen in the number of procedures required to achieve cusum-defined procedural competency. These were due to differences in definitions of success/failure of a procedure, the acceptable and unacceptable failure rates used for the initial cusum calculation, and individual trainee performance.

Conclusion

Cusum can be used to assess procedural competency, but several problems need to be overcome before it can become a universally accepted method. It is ideally placed to be used as a quality control tool for a trained individual and could also be used to assess the impact of new training methods or equipment on performance.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Cooper GM, McClure JH. Anesthesia chapter from saving mothers lives; reviewing maternal deaths to make pregnancy safer. Br J Anaesth 2008; 100: 17-22.PubMedCrossRef Cooper GM, McClure JH. Anesthesia chapter from saving mothers lives; reviewing maternal deaths to make pregnancy safer. Br J Anaesth 2008; 100: 17-22.PubMedCrossRef
2.
Zurück zum Zitat Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach 2007; 29: 648-54.PubMedCrossRef Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach 2007; 29: 648-54.PubMedCrossRef
3.
Zurück zum Zitat Bould MD, Crabtree NA. Are logbooks of training in anaesthesia a valuable exercise? Br J Hosp Med (Lond) 2008; 69: 236. Bould MD, Crabtree NA. Are logbooks of training in anaesthesia a valuable exercise? Br J Hosp Med (Lond) 2008; 69: 236.
4.
Zurück zum Zitat de Oliveira Filho GR. The construction of learning curves for basic skills in anesthetic procedures: an application for the cumulative sum method. Anesth Analg 2002; 95: 411-6. de Oliveira Filho GR. The construction of learning curves for basic skills in anesthetic procedures: an application for the cumulative sum method. Anesth Analg 2002; 95: 411-6.
5.
Zurück zum Zitat Bould MD, Crabtree NA, Haik VN. Assessment of procedural skills in anaesthesia. Br J Anaesth 2009; 103: 472-83.PubMedCrossRef Bould MD, Crabtree NA, Haik VN. Assessment of procedural skills in anaesthesia. Br J Anaesth 2009; 103: 472-83.PubMedCrossRef
6.
Zurück zum Zitat Reznik RK, MacRae H. Teaching surgical skills - changes in the wind. N Engl J Med 2006; 335: 2664-9. Reznik RK, MacRae H. Teaching surgical skills - changes in the wind. N Engl J Med 2006; 335: 2664-9.
7.
Zurück zum Zitat Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997; 84: 273-8.PubMedCrossRef Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997; 84: 273-8.PubMedCrossRef
8.
Zurück zum Zitat Reznick R, Regehr G, MacRae H, Martin J, McCulloch W. Testing technical skill via an innovative “bench station” examination. Am J Surg 1997; 173: 226-30.PubMedCrossRef Reznick R, Regehr G, MacRae H, Martin J, McCulloch W. Testing technical skill via an innovative “bench station” examination. Am J Surg 1997; 173: 226-30.PubMedCrossRef
9.
Zurück zum Zitat Fried GM, Feldman LS, Vassiliou MC, et al. Proving the value of simulation in laparoscopic surgery. Ann Surg 2004; 240: 518-28.PubMedCrossRef Fried GM, Feldman LS, Vassiliou MC, et al. Proving the value of simulation in laparoscopic surgery. Ann Surg 2004; 240: 518-28.PubMedCrossRef
10.
Zurück zum Zitat Taffinder N, Sutton C, Fishwick RJ, McManus IC, Darzi A. Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery: results from randomised controlled studies using the MIST VR laparoscopic simulator. Stud Health Technol Inform 1998; 50: 124-30.PubMed Taffinder N, Sutton C, Fishwick RJ, McManus IC, Darzi A. Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery: results from randomised controlled studies using the MIST VR laparoscopic simulator. Stud Health Technol Inform 1998; 50: 124-30.PubMed
11.
Zurück zum Zitat Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg 2001; 193: 479-85.PubMedCrossRef Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg 2001; 193: 479-85.PubMedCrossRef
12.
Zurück zum Zitat Darzi A, Mackay S. Assessment of surgical competence. Qual Health Care 2001; 10: Suppl 2: ii64-9. Darzi A, Mackay S. Assessment of surgical competence. Qual Health Care 2001; 10: Suppl 2: ii64-9.
13.
Zurück zum Zitat Wass V, Van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet 2001; 357: 945-9.PubMedCrossRef Wass V, Van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet 2001; 357: 945-9.PubMedCrossRef
14.
Zurück zum Zitat de Oliveira Filho GR, Helayel PE, da Conceicao DB, Garzel IS, Pavei P, Ceccon MS. Learning curves and mathematical models for interventional ultrasound basic skills. Anesth Analg 2008; 106: 568-73. de Oliveira Filho GR, Helayel PE, da Conceicao DB, Garzel IS, Pavei P, Ceccon MS. Learning curves and mathematical models for interventional ultrasound basic skills. Anesth Analg 2008; 106: 568-73.
15.
Zurück zum Zitat Bolsin S, Colson M. The use of the cusum technique in the assessment of trainee competence in new procedures. Int J Qual Health Care 2000; 12: 433-8.PubMedCrossRef Bolsin S, Colson M. The use of the cusum technique in the assessment of trainee competence in new procedures. Int J Qual Health Care 2000; 12: 433-8.PubMedCrossRef
16.
Zurück zum Zitat Kestin IG. A statistical approach to measuring the competence of anaesthetic trainees at practical procedures. Br J Anaesth 1995; 75: 805-9.PubMedCrossRef Kestin IG. A statistical approach to measuring the competence of anaesthetic trainees at practical procedures. Br J Anaesth 1995; 75: 805-9.PubMedCrossRef
17.
Zurück zum Zitat Naik VN, Devito I, Halpern SH. Cusum analysis is a useful tool to assess resident proficiency at insertion of labour epidurals. Can J Anesth 2003; 50: 694-8.PubMedCrossRef Naik VN, Devito I, Halpern SH. Cusum analysis is a useful tool to assess resident proficiency at insertion of labour epidurals. Can J Anesth 2003; 50: 694-8.PubMedCrossRef
18.
Zurück zum Zitat Biau DJ, Resche-Rigon M, Godiris-Petit G, Nizard RS, Porcher R. Quality control of surgical and interventional procedures: a review of the CUSUM. Qual Saf Health Care 2007; 16: 203-7.PubMedCrossRef Biau DJ, Resche-Rigon M, Godiris-Petit G, Nizard RS, Porcher R. Quality control of surgical and interventional procedures: a review of the CUSUM. Qual Saf Health Care 2007; 16: 203-7.PubMedCrossRef
19.
Zurück zum Zitat Sivaprakasam J, Purva M. CUSUM analysis to assess competence: what failure rate is acceptable? Clin Teach 2010; 7: 257-61.PubMedCrossRef Sivaprakasam J, Purva M. CUSUM analysis to assess competence: what failure rate is acceptable? Clin Teach 2010; 7: 257-61.PubMedCrossRef
20.
Zurück zum Zitat Hermanides J, Hallmann MW, Stevens MF, Lirk P. Failed epidural: causes and management. Br J Anaesth 2012; 109: 144-54.PubMedCrossRef Hermanides J, Hallmann MW, Stevens MF, Lirk P. Failed epidural: causes and management. Br J Anaesth 2012; 109: 144-54.PubMedCrossRef
21.
Zurück zum Zitat Arendt K, Segal S. Why epidurals do not always work. Rev Obstet Gynecol 2008; 1: 49-55.PubMed Arendt K, Segal S. Why epidurals do not always work. Rev Obstet Gynecol 2008; 1: 49-55.PubMed
22.
Zurück zum Zitat Charuluxananan S, Kyokong O, Premsamran P. Comparison of 25 and 27 gauge needle in spinal anesthesia learning curve for anesthesia residency training. J Med Assoc Thai 2005; 88: 1569-73.PubMed Charuluxananan S, Kyokong O, Premsamran P. Comparison of 25 and 27 gauge needle in spinal anesthesia learning curve for anesthesia residency training. J Med Assoc Thai 2005; 88: 1569-73.PubMed
23.
Zurück zum Zitat Schuepfer G, Johr M. Psoas compartment block (PCB) in children: Part II - generation of an institutional learning curve with a new technique. Paediatr Anaesth 2005; 15: 465-9.PubMedCrossRef Schuepfer G, Johr M. Psoas compartment block (PCB) in children: Part II - generation of an institutional learning curve with a new technique. Paediatr Anaesth 2005; 15: 465-9.PubMedCrossRef
24.
Zurück zum Zitat Komatsu R, Kasuya Y, Yogo H, et al. Learning curves for bag-and-mask ventilation and orotracheal intubation: an application of the cumulative sum method. Anesthesiology 2010; 112: 1525-31.PubMedCrossRef Komatsu R, Kasuya Y, Yogo H, et al. Learning curves for bag-and-mask ventilation and orotracheal intubation: an application of the cumulative sum method. Anesthesiology 2010; 112: 1525-31.PubMedCrossRef
25.
Zurück zum Zitat Dalal PG, Dalal GB, Pott L, Bezinover D, Prozesky J, Bosseau Murray W. Learning curves of novice anesthesiology residents performing simulated fibreoptic upper airway endoscopy. Can. J Anesth 2011; 58: 802-9. Dalal PG, Dalal GB, Pott L, Bezinover D, Prozesky J, Bosseau Murray W. Learning curves of novice anesthesiology residents performing simulated fibreoptic upper airway endoscopy. Can. J Anesth 2011; 58: 802-9.
26.
Zurück zum Zitat Correa JB, Dellazzana JE, Sturm A, Leite DM, de Oliveira Filho GR, Xavier RG. Using the cusum curve to evaluate the training of orotracheal intubation with the Truview EVO2 laryngoscope (Portuguese). Rev Bras Anestesiol 2009; 59: 321-31.PubMedCrossRef Correa JB, Dellazzana JE, Sturm A, Leite DM, de Oliveira Filho GR, Xavier RG. Using the cusum curve to evaluate the training of orotracheal intubation with the Truview EVO2 laryngoscope (Portuguese). Rev Bras Anestesiol 2009; 59: 321-31.PubMedCrossRef
27.
Zurück zum Zitat Margarido CB, Arzola C, Balki M, Carvalho JC. Anesthesiologists’ learning curves for ultrasound assessment of the lumbar spine. Can J Anesth 2010; 57: 120-6.PubMedCrossRef Margarido CB, Arzola C, Balki M, Carvalho JC. Anesthesiologists’ learning curves for ultrasound assessment of the lumbar spine. Can J Anesth 2010; 57: 120-6.PubMedCrossRef
28.
Zurück zum Zitat Barrington MJ, Wong DM, Slater B, Ivanusic JJ, Owens M. Ultrasound-guided regional anesthesia: how much practice do novices require before achieving competency in ultrasound needle visualization using a cadaver model. Reg Anesth Pain Med 2012; 37: 334-9.PubMedCrossRef Barrington MJ, Wong DM, Slater B, Ivanusic JJ, Owens M. Ultrasound-guided regional anesthesia: how much practice do novices require before achieving competency in ultrasound needle visualization using a cadaver model. Reg Anesth Pain Med 2012; 37: 334-9.PubMedCrossRef
29.
Zurück zum Zitat Niazi AU, Haldipur N, Prasad AG, Chan VW. Ultrasound-guided regional anesthesia performance in the early learning period: effect of simulation training. Reg Anesth Pain Med 2012; 37: 51-4.PubMedCrossRef Niazi AU, Haldipur N, Prasad AG, Chan VW. Ultrasound-guided regional anesthesia performance in the early learning period: effect of simulation training. Reg Anesth Pain Med 2012; 37: 51-4.PubMedCrossRef
30.
Zurück zum Zitat Halpern SH, Banerjee A, Stocche R, Glanc P. The use of ultrasound for lumbar spinous process identification: a pilot study. Can J Anesth 2010; 57: 817-22.PubMedCrossRef Halpern SH, Banerjee A, Stocche R, Glanc P. The use of ultrasound for lumbar spinous process identification: a pilot study. Can J Anesth 2010; 57: 817-22.PubMedCrossRef
31.
Zurück zum Zitat Vereijken B, Whiting HT. In defence of discovery learning. Can J Sport Sci 1990; 15: 99-106.PubMed Vereijken B, Whiting HT. In defence of discovery learning. Can J Sport Sci 1990; 15: 99-106.PubMed
32.
Zurück zum Zitat Tsui BC. “Credentials” in ultrasound-guided regional blocks. Reg Anesth Pain Med 2006; 31: 587-8.PubMed Tsui BC. “Credentials” in ultrasound-guided regional blocks. Reg Anesth Pain Med 2006; 31: 587-8.PubMed
33.
Zurück zum Zitat Issenberg SB, McGaghie WC, Petrusa ER, Lee GD, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 2005; 27: 10-28.PubMedCrossRef Issenberg SB, McGaghie WC, Petrusa ER, Lee GD, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 2005; 27: 10-28.PubMedCrossRef
34.
Zurück zum Zitat Runcie CJ. Assessing the performance of a consultant anaesthetist by control chart methodology. Anaesthesia 2009; 64: 293-6.PubMedCrossRef Runcie CJ. Assessing the performance of a consultant anaesthetist by control chart methodology. Anaesthesia 2009; 64: 293-6.PubMedCrossRef
35.
Zurück zum Zitat Lim TO, Soraya A, Ding LM, Morad Z. Assessing doctors’ competence: application of CUSUM technique in monitoring doctors’ performance. Int J Qual Health Care 2002; 14: 251-8.PubMedCrossRef Lim TO, Soraya A, Ding LM, Morad Z. Assessing doctors’ competence: application of CUSUM technique in monitoring doctors’ performance. Int J Qual Health Care 2002; 14: 251-8.PubMedCrossRef
36.
Zurück zum Zitat Williams SM, Parry BR, Schlup MM. Quality control: an application of the cusum. BMJ 1992; 304: 1359-61.PubMedCrossRef Williams SM, Parry BR, Schlup MM. Quality control: an application of the cusum. BMJ 1992; 304: 1359-61.PubMedCrossRef
37.
Zurück zum Zitat Panni MK, Camann WR, Tsen LC. Resident training in obstetric anesthesia in the United States. Int J Obstet Anesth 2006; 15: 284-9.PubMedCrossRef Panni MK, Camann WR, Tsen LC. Resident training in obstetric anesthesia in the United States. Int J Obstet Anesth 2006; 15: 284-9.PubMedCrossRef
38.
Zurück zum Zitat Grigg OA, Farewell VT, Spiegelhalter DJ. Use of risk-adjusted CUSUM and RSPRT charts for monitoring in medical contexts. Stat Methods Med Res 2003; 12: 147-70.PubMed Grigg OA, Farewell VT, Spiegelhalter DJ. Use of risk-adjusted CUSUM and RSPRT charts for monitoring in medical contexts. Stat Methods Med Res 2003; 12: 147-70.PubMed
Metadaten
Titel
Assessment of procedural skills training and performance in anesthesia using cumulative sum analysis (cusum)
verfasst von
Tim Starkie, BMBS
Elizabeth J. Drake, BM
Publikationsdatum
01.12.2013
Verlag
Springer US
Erschienen in
Canadian Journal of Anesthesia/Journal canadien d'anesthésie / Ausgabe 12/2013
Print ISSN: 0832-610X
Elektronische ISSN: 1496-8975
DOI
https://doi.org/10.1007/s12630-013-0045-1

Weitere Artikel der Ausgabe 12/2013

Canadian Journal of Anesthesia/Journal canadien d'anesthésie 12/2013 Zur Ausgabe

Perioperative Cardiovascular Rounds

Hemodynamic instability and fluid responsiveness

Delir bei kritisch Kranken – Antipsychotika versus Placebo

16.05.2024 Delir Nachrichten

Um die Langzeitfolgen eines Delirs bei kritisch Kranken zu mildern, wird vielerorts auf eine Akuttherapie mit Antipsychotika gesetzt. Eine US-amerikanische Forschungsgruppe äußert jetzt erhebliche Vorbehalte gegen dieses Vorgehen. Denn es gibt neue Daten zum Langzeiteffekt von Haloperidol bzw. Ziprasidon versus Placebo.

Eingreifen von Umstehenden rettet vor Erstickungstod

15.05.2024 Fremdkörperaspiration Nachrichten

Wer sich an einem Essensrest verschluckt und um Luft ringt, benötigt vor allem rasche Hilfe. Dass Umstehende nur in jedem zweiten Erstickungsnotfall bereit waren, diese zu leisten, ist das ernüchternde Ergebnis einer Beobachtungsstudie aus Japan. Doch es gibt auch eine gute Nachricht.

Darf man die Behandlung eines Neonazis ablehnen?

08.05.2024 Gesellschaft Nachrichten

In einer Leseranfrage in der Zeitschrift Journal of the American Academy of Dermatology möchte ein anonymer Dermatologe bzw. eine anonyme Dermatologin wissen, ob er oder sie einen Patienten behandeln muss, der eine rassistische Tätowierung trägt.

Ein Drittel der jungen Ärztinnen und Ärzte erwägt abzuwandern

07.05.2024 Klinik aktuell Nachrichten

Extreme Arbeitsverdichtung und kaum Supervision: Dr. Andrea Martini, Sprecherin des Bündnisses Junge Ärztinnen und Ärzte (BJÄ) über den Frust des ärztlichen Nachwuchses und die Vorteile des Rucksack-Modells.

Update AINS

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.