nach oben

Canadian Journal of Anesthesia/Journal canadien d'anesthésie

Erschienen in:

01.02.2012 | Review Article/Brief Review

Review article: Assessment in anesthesiology education

verfasst von: John R. Boulet, PhD, David Murray, MD

Erschienen in: Canadian Journal of Anesthesia/Journal canadien d'anesthésie | Ausgabe 2/2012

Einloggen, um Zugang zu erhalten

Abstract

Purpose

The purpose of this article is to provide a review of the competencies and associated assessment techniques relevant to the practice of anesthesiology. Although many of the competencies are difficult to define and measure specifically, advances in assessment techniques have provided more opportunities to gather meaningful performance data.

Principal findings

Establishing the competence of anesthesiologists demands a host of measures, including standardized tests and less-structured peer evaluations. Simulation-based assessment will play an increasingly important role both in certification and in maintenance of certification for anesthesiologists.

Conclusions

While there are many psychometric challenges associated with the assessments pertinent to the education of anesthesiologists, technological advances combined with an increased awareness of sound measurement principles will yield more meaningful competency measures that can be used to improve the practice of anesthesiology.

American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for Educational and Psychological Testing. 1999. Washington, DC, American Educational Research Association.

Hofer TP, Hayward RA. Are bad outcomes from questionable clinical decisions preventable medical errors? A case of cascade iatrogenesis. Ann Intern Med 2002; 137(5 Part 1): 327-33.PubMed

Frank JR. The CanMEDS 2005 Physician Competency Framework. Better standards. Better physicians. Ottawa, Canada. The Royal College of Physicians and Surgeons of Canada. Available from URL: http://meds.queensu.ca/medicine/obgyn/pdf/CanMEDS2005.booklet.pdf (accessed August 2011).

Accreditation Council for Graduate Medical Education. ACGME Outcome Project. Available from: http://www.acgme.org/acWebsite/home/home.asp (accessed August 2011).

Glavin R, Flin R. Review article: The influence of psychology and human factors on education in anesthesiology. Can J Anesth 2012; 59: this issue. DOI:10.1007/s12630-011-9634-z.

Bould MD, Naik VN, Hamstra SJ. Review article: New directions in medical education related to anesthesiology and perioperative medicine. Can J Anesth 2012; 59: this issue. DOI:10.1007/s12630-011-9633-0.

Boulet JR, Murray DJ. Simulation-based assessment in anesthesiology: requirements for practical implementation. Anesthesiology 2010; 112: 1041-52.PubMedCrossRef

Davis MH. OSCE: the Dundee experience. Med Teach 2003; 25: 255-61.PubMedCrossRef

Norcini JJ, McKinley DW. Assessment methods in medical education. Teaching and Teacher Education 2007; 23: 239-50.CrossRef

Epstein RM. Assessment in medical education. N Engl J Med 2007; 356: 387-96.PubMedCrossRef

10.

Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. JAMA 2009; 302: 1316-26.PubMedCrossRef

11.

Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990; 65(9 Suppl): S63-7.PubMedCrossRef

12.

Edler AA, Fanning RG, Chen MI, et al. Patient simulation: a literary synthesis of assessment tools in anesthesiology. J Educ Eval Health Prof 2009; 6: 3.PubMedCrossRef

13.

Boulet JR, Smee SM, Dillon GF, Gimpel JR. The use of standardized patient assessments for certification and licensure decisions. Simul Healthc 2009; 4: 35-42.PubMedCrossRef

14.

Dillon GF, Boulet JR, Hawkins RE, Swanson DB. Simulations in the United States Medical Licensing Examination (USMLE). Qual Saf Health Care 2004; 13 Suppl 1: i41-5.PubMedCrossRef

15.

Nestel D, Kneebone R, Black S. Simulated patients and the development of procedural and operative skills. Med Teach 2006; 28: 390-1.PubMed

16.

Amin Z, Boulet JR, Cook DA, et al. Technology-enabled assessment of health professions education: consensus statement and recommendations from the Ottawa 2010 conference. Med Teach 2011; 33: 364-9.PubMedCrossRef

17.

Bould MD, Crabtree NA, Naik VN. Assessment of procedural skills in anaesthesia. Br J Anaesth 2009; 103: 472-83.PubMedCrossRef

18.

Leblanc VR. Review article: Simulation in anesthesia: state of the science and looking forward. Can J Anesth 2012; 59: this issue. DOI:10.1007/s12630-011-9638-8.

19.

Norcini J, Anderson B, Bollela V, et al. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011; 33: 206-14.PubMedCrossRef

20.

Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006; 119: 166.e7.CrossRef

21.

Devitt JH, Kurrek MM, Cohen MM, Cleave-Hogg D. The validity of performance assessments using simulation. Anesthesiology 2001; 95: 36-42.PubMedCrossRef

22.

Boulet JR. Generalizability theory: basics. In: Everitt BS, Howell D (Eds). Encyclopedia of Statistics in Behavioral Science. Chichester: John Wiley & Sons, Ltd; 2005. 704-11.

23.

Boulet JR, Swanson DB. Psychometric challenges of using simulations for high-stakes assessment. In: Dunn WF (Ed). Simulators in Critical Care Education and Beyond. Des Plains, IL: Society of Critical Care Medicine; 2004: 119-30.

24.

Clauser BE, Margolis MJ, Swanson DB. Issues of validity and reliability for assessments in medical education. In: Holmboe ES, Hawkins RE (Eds). Practical Guide to the Evaluation of Clinical Competence, 1st ed. Philadelphia, PA: Mosby/Elsevier; 2008: 10-23

25.

Raymond MR. Job analysis and the specification of content for licensure and certification examinations. Applied Measurement in Education 2001; 14: 369-415.CrossRef

26.

Kearney RA, Sullivan P, Skakun E. Performance on ABA-ASA in-training examination predicts success for RCPSC certification. American Board of Anesthesiology-American Society of Anesthesiologists. Royal College of Physicians and Surgeons of Canada. Can J Anesth 2000; 47: 914-8.PubMedCrossRef

27.

Gale TC, Roberts MJ, Sice PJ, et al. Predictive validity of a selection centre testing non-technical skills for recruitment to training in anaesthesia. Br J Anaesth 2010; 105: 603-9.PubMedCrossRef

28.

Murray DJ, Boulet JR, Avidan M, et al. Performance of residents and anesthesiologists in a simulation-based skill assessment. Anesthesiology 2007; 107: 705-13.PubMedCrossRef

29.

McClintock JC, Gravlee GP. Predicting success on the certification examinations of the American Board of Anesthesiology. Anesthesiology 2010; 112: 212-9.PubMedCrossRef

30.

The American Board of Anesthesiology. Maintenance of certfication in anesthesiology (MOCA). Available from: http://www.theaba.org/Home/anesthesiology_maintenance (accessed July 2011).

31.

Ottestad E, Boulet JR, Lighthall GK. Evaluating the management of septic shock using patient simulation. Crit Care Med 2007; 35: 769-75.PubMedCrossRef

32.

Wilkinson JR, Crossley JG, Wragg A, Mills P, Cowan G, Wade W. Implementing workplace-based assessment across the medical specialties in the United Kingdom. Med Educ 2008; 42: 364-73.PubMedCrossRef

33.

Miller A, Archer J. Impact of workplace based assessment on doctors’ education and performance: a systematic review. BMJ 2010; 341: c5064.PubMedCrossRef

34.

Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach 2007; 29: 855-71.PubMedCrossRef

35.

Berkenstadt H, Erez D, Munz Y, Simon D, Ziv A. Training and assessment of trauma management: the role of simulation-based medical education. Anesthesiol Clin 2007; 25: 65-74.PubMedCrossRef

36.

Hunt EA, Walker AR, Shaffner DH, Miller MR, Pronovost PJ. Simulation of in-hospital pediatric medical emergencies and cardiopulmonary arrests: highlighting the importance of the first 5 minutes. Pediatrics 2008; 121: e34-43.PubMedCrossRef

37.

Cooper JB, Taqueti VR. A brief history of the development of mannequin simulators for clinical education and training. Qual Saf Health Care 2004; 13 Suppl 1: i11-8.PubMedCrossRef

38.

Flin R, Patey R, Glavin R, Maran N. Anaesthetists’ non-technical skills. Br J Anaesth 2010; 105: 38-44.PubMedCrossRef

39.

Schuwirth LW, Southgate L, Page GG, et al. When enough is enough: a conceptual basis for fair and defensible practice performance assessment. Med Educ 2002; 36: 925-30.PubMedCrossRef

40.

Downing SM, Haladyna TM. Validity threats: overcoming interference with proposed interpretations of assessment data. Med Educ 2004; 38: 327-33.PubMedCrossRef

41.

Wass V, Van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet 2001; 357: 945-9.PubMedCrossRef

42.

Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002; 287: 226-35.PubMedCrossRef

43.

Dillon GF, Clyman SG, Clauser BE, Margolis MJ. The introduction of computer-based case simulations into the United States Medical Licensing Examination. Acad Med 2002; 77(10 Suppl): S94-6.PubMedCrossRef

44.

Moore DE Jr, Pennington FC. Practice-based learning and improvement. J Contin Educ Health Prof 2003; 23 Suppl 1: S73-80.PubMedCrossRef

45.

Lynch DC, Swing SR, Horowitz SD, Holt K, Messer JV. Assessing practice-based learning and improvement. Teach Learn Med 2004; 16: 85-92.PubMedCrossRef

46.

Carraccio C, Englander R. Evaluating competence using a portfolio: a literature review and web-based application to the ACGME competencies. Teach Learn Med 2004; 16: 381-7.PubMedCrossRef

47.

Harden RM. Trends and the future of postgraduate medical education. Emerg Med J 2006; 23: 798-802.PubMedCrossRef

48.

Graham J, Hocking G, Giles E. Anaesthesia non-technical skills: can anaesthetists be trained to reliably use this behavioural marker system in 1 day? Br J Anaesth 2010; 104: 440-5.PubMedCrossRef

49.

Canas M, Moreno R, Rhodes A, Grounds RM. Patient safety in anesthesia. Minerva Anestesiol 2010; 76: 753-7.PubMed

50.

Evans RG, Edwards A, Evans S, Elwyn B, Elwyn G. Assessing the practising physician using patient surveys: a systematic review of instruments and feedback methods. Fam Pract 2007; 24: 117-27.PubMedCrossRef

51.

Lurie SJ, Mooney CJ, Lyness JM. Measurement of the general competencies of the Accreditation Council for Graduate Medical Education: a systematic review. Acad Med 2009; 84: 301-9.PubMedCrossRef

52.

Duffy FD, Gordon GH, Whelan G, et al. Assessing competence in communication and interpersonal skills: the Kalamazoo II report. Acad Med 2004; 79: 495-507.PubMedCrossRef

53.

Symons AB, Swanson A, McGuigan D, Orrange S, Akl EA. A tool for self-assessment of communication skills and professionalism in residents. BMC Med Educ 2009; 9: 1.PubMedCrossRef

54.

Ong LM, de Haes JC, Hoos AM, Lammes FB. Doctor-patient communication: a review of the literature. Soc Sci Med 1995; 40: 903-18.PubMedCrossRef

55.

Nicolai J, Demmel R. The impact of gender stereotypes on the evaluation of general practitioners’ communication skills: an experimental study using transcripts of physician-patient encounters. Patient Educ Couns 2007; 69: 200-5.PubMedCrossRef

56.

Schirmer JM, Mauksch L, Lang F, et al. Assessing communication competence: a review of current tools. Fam Med 2005; 37: 184-92.PubMed

57.

Mercer SJ, Moneypenny MJ, Guha A. Communication and simulation for anaesthetists. Anaesthesia 2009; 64: 1259-60.PubMedCrossRef

58.

Park CS. Simulation and quality improvement in anesthesiology. Anesthesiol Clin 2011; 29: 13-28.PubMedCrossRef

59.

Whelan GP, McKinley DW, Boulet JR, Macrae J, Kamholz S. Validation of the doctor-patient communication component of the Educational Commission for Foreign Medical Graduates Clinical Skills Assessment. Med Educ 2001; 35: 757-61.PubMedCrossRef

60.

Smith AF, Greaves JD. Beyond competence: defining and promoting excellence in anaesthesia. Anaesthesia 2010; 65: 184-91.PubMedCrossRef

61.

Lynch DC, Surdyk PM, Eiser AR. Assessing professionalism: a review of the literature. Med Teach 2004; 26: 366-73.PubMedCrossRef

62.

Wilkinson TJ, Wade WB, Knock LD. A blueprint to assess professionalism: results of a systematic review. Acad Med 2009; 84: 551-8.PubMedCrossRef

63.

Veloski JJ, Fields SK, Boex JR, Blank LL. Measuring professionalism: a review of studies with instruments reported in the literature between 1982 and 2002. Acad Med 2005; 80: 366-70.PubMedCrossRef

64.

Hodges BD, Ginsburg S, Cruess R, et al. Assessment of professionalism: recommendations from the Ottawa 2010 Conference. Med Teach 2011; 33: 354-63.PubMedCrossRef

65.

Norcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: a method for assessing clinical skills. Ann Intern Med 2003; 138: 476-81.PubMed

66.

van Zanten M, Boulet JR, Norcini JJ, McKinley D. Using a standardised patient assessment to measure professional attributes. Med Educ 2005; 39: 20-9.PubMedCrossRef

67.

Norcini JJ. Peer assessment of competence. Med Educ 2003; 37: 539-43.PubMedCrossRef

68.

Massagli TL, Carline JD. Reliability of a 360-degree evaluation to assess resident competence. Am J Phys Med Rehabil 2007; 86: 845-52.PubMedCrossRef

69.

Heffron MG, Simspon D, Kochar MS. Competency-based physician education, recertification, and licensure. WMJ 2007; 106: 215-8.PubMed

70.

Hingle ST, Robinson S, Colliver JA, Rosher RB, McCann-Stone N. Systems-based practice assessed with a performance-based examination simulated and scored by standardized participants in the health care system: feasibility and psychometric properties. Teach Learn Med 2011; 23: 148-54.PubMedCrossRef

71.

Gaba DM. Crisis resource management and teamwork training in anaesthesia. Br J Anaesth 2010; 105: 3-6.PubMedCrossRef

72.

Wright MC, Phillips-Bute BG, Petrusa ER, Griffin KL, Hobbs GW, Taekman JM. Assessing teamwork in medical education and practice: relating behavioural teamwork ratings and clinical performance. Med Teach 2009; 31: 30-8.PubMedCrossRef

73.

Volk MS, Ward J, Irias N, Navedo A, Pollart J, Weinstock PH. Using Medical Simulation to Teach Crisis Resource Management and Decision-Making Skills to Otolaryngology Housestaff. Otolaryngol Head Neck Surg 2011; 145: 35-42.PubMedCrossRef

74.

Morgan PJ, Pittini R, Regehr G, Marrs C, Haley MF. Evaluating teamwork in a simulated obstetric environment. Anesthesiology 2007; 106: 907-15.PubMedCrossRef

75.

Case SM, Swanson DB. Constructing Written Test Questions for the Basic and Clinical Sciences, Third Edition (revised). Philadelphia, PA: National Board of Medical Examiners; 2010 .

76.

Edler AA. The use of simulation education in competency assessment: more questions than answers. Anesthesiology 2008; 108: 167.PubMedCrossRef

77.

Weller JM, Jolly B, Robinson B. Generalisability of behavioural skills in simulated anaesthetic emergencies. Anaesth Intensive Care 2008; 36: 185-9.PubMed

78.

Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M. OSCE checklists do not capture increasing levels of expertise. Acad Med 1999; 74: 1129-34.PubMedCrossRef

79.

Hodges B, McNaughton N, Regehr G, Tiberius R, Hanson M. The challenge of creating new OSCE measures to capture the characteristics of expertise. Med Educ 2002; 36: 742-8.PubMedCrossRef

80.

Scavone BM, Sproviero MT, McCarthy RJ, et al. Development of an objective scoring system for measurement of resident performance on the human patient simulator. Anesthesiology 2006; 105: 260-6.PubMedCrossRef

81.

De Champlain AF. Ensuring that the competent are truly competent: an overview of common methods and procedures used to set standards on high-stakes examinations. J Vet Med Educ 2004; 31: 61-5.PubMedCrossRef

82.

Downing SM, Tekian A, Yudkowsky R. Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teach Learn Med 2006; 18: 50-7.PubMedCrossRef

83.

Boulet JR, Murray D, Kras J, Woodhouse J. Setting performance standards for mannequin-based acute-care scenarios: an examinee-centered approach. Simul Healthc 2008; 3: 72-81.PubMedCrossRef

Titel: Review article: Assessment in anesthesiology education
verfasst von: John R. Boulet, PhD
David Murray, MD
Publikationsdatum: 01.02.2012
Verlag: Springer-Verlag
Erschienen in: Canadian Journal of Anesthesia/Journal canadien d'anesthésie / Ausgabe 2/2012
Print ISSN: 0832-610X
Elektronische ISSN: 1496-8975
DOI: https://doi.org/10.1007/s12630-011-9637-9

Neu im Fachgebiet AINS

Blutdrucksenkung schon im Rettungswagen bei akutem Schlaganfall?

31.05.2024 Apoplex Nachrichten

Der optimale Ansatz für die Blutdruckkontrolle bei Patientinnen und Patienten mit akutem Schlaganfall ist noch nicht gefunden. Ob sich eine frühzeitige Therapie der Hypertonie noch während des Transports in die Klinik lohnt, hat jetzt eine Studie aus China untersucht.

Nicht Creutzfeldt Jakob, sondern Abführtee-Vergiftung

29.05.2024 Hyponatriämie Nachrichten

Eine ältere Frau trinkt regelmäßig Sennesblättertee gegen ihre Verstopfung. Der scheint plötzlich gut zu wirken. Auf Durchfall und Erbrechen folgt allerdings eine Hyponatriämie. Nach deren Korrektur kommt es plötzlich zu progredienten Kognitions- und Verhaltensstörungen.

Häusliche Gewalt in der orthopädischen Notaufnahme oft nicht erkannt

28.05.2024 Häusliche Gewalt Nachrichten

In der Notaufnahme wird die Chance, Opfer von häuslicher Gewalt zu identifizieren, von Orthopäden und Orthopädinnen offenbar zu wenig genutzt. Darauf deuten die Ergebnisse einer Fragebogenstudie an der Sahlgrenska-Universität in Schweden hin.

Update AINS

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.

hier abonnieren

Live-Webinar "Urologie und Sexualmedizin in der Praxis"

Springer Medizin

Review article: Assessment in anesthesiology education

Abstract

Purpose

Principal findings

Conclusions

Neu im Fachgebiet AINS

Blutdrucksenkung schon im Rettungswagen bei akutem Schlaganfall?

Ähnliche Überlebensraten nach Reanimation während des Transports bzw. vor Ort

Nicht Creutzfeldt Jakob, sondern Abführtee-Vergiftung

Häusliche Gewalt in der orthopädischen Notaufnahme oft nicht erkannt

Update AINS

Live-Webinar "Urologie und Sexualmedizin in der Praxis"

Springer Medizin

Abstract

Purpose

Principal findings

Conclusions

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Weitere Artikel der Ausgabe 2/2012

Special theme issue on advances in education in anesthesiology

Different algorithms for glycemic control will yield different results

Review article: New directions in medical education related to anesthesiology and perioperative medicine

Review article: Closing the research gap at the interface of learning and clinical practice

A flexible gel pad as an effective medium for scanning irregular surface anatomy

Atlas of Common Pain Syndromes, 3rd Edition

Neu im Fachgebiet AINS

Blutdrucksenkung schon im Rettungswagen bei akutem Schlaganfall?

Ähnliche Überlebensraten nach Reanimation während des Transports bzw. vor Ort

Nicht Creutzfeldt Jakob, sondern Abführtee-Vergiftung

Häusliche Gewalt in der orthopädischen Notaufnahme oft nicht erkannt

Update AINS