Skip to main content
Erschienen in: Journal of General Internal Medicine 7/2015

01.07.2015 | Original Research

The Quality of Written Feedback by Attendings of Internal Medicine Residents

verfasst von: Jeffrey L Jackson, MD MPH, Cynthia Kay, MD MSc, Wilkins C Jackson, HS, Michael Frank, MD

Erschienen in: Journal of General Internal Medicine | Ausgabe 7/2015

Einloggen, um Zugang zu erhalten

ABSTRACT

BACKGROUND

Attending evaluations are commonly used to evaluate residents.

OBJECTIVES

Evaluate the quality of written feedback of internal medicine residents.

DESIGN

Retrospective.

PARTICIPANTS

Internal medicine residents and faculty at the Medical College of Wisconsin from 2004 to 2012.

MAIN MEASURES

From monthly evaluations of residents by attendings, a randomly selected sample of 500 written comments by attendings were qualitatively coded and rated as high-, moderate-, or low-quality feedback by two independent coders with good inter-rater reliability (kappa: 0.94). Small group exercises with residents and attendings also coded the utterances as high, moderate, or low quality and developed criteria for this categorization. In-service examination scores were correlated with written feedback.

KEY RESULTS

There were 228 internal medicine residents who had 6,603 evaluations by 334 attendings. Among 500 randomly selected written comments, there were 2,056 unique utterances: 29 % were coded as nonspecific statements, 20 % were comments about resident personality, 16 % about patient care, 14 % interpersonal communication, 7 % medical knowledge, 6 % professionalism, and 4 % each on practice-based learning and systems-based practice. Based on criteria developed by group exercises, the majority of written comments were rated as moderate quality (65 %); 22 % were rated as high quality and 13 % as low quality. Attendings who provided high-quality feedback rated residents significantly lower in all six of the Accreditation Council for Graduate Medical Education (ACGME) competencies (p <0.0005 for all), and had a greater range of scores. Negative comments on medical knowledge were associated with lower in-service examination scores.

CONCLUSIONS

Most attending written evaluation was of moderate or low quality. Attendings who provided high-quality feedback appeared to be more discriminating, providing significantly lower ratings of residents in all six ACGME core competencies, and across a greater range. Attendings’ negative written comments on medical knowledge correlated with lower in-service training scores.
Literatur
1.
4.
Zurück zum Zitat van der Ridder JMM, Stokking KM, McGaghie WC, ten Cate OT. What is feedback in clinical education? Medical Education. 2008;42(2):189–97.PubMedCrossRef van der Ridder JMM, Stokking KM, McGaghie WC, ten Cate OT. What is feedback in clinical education? Medical Education. 2008;42(2):189–97.PubMedCrossRef
5.
Zurück zum Zitat Kluger AN, DeNisi A. The effects of feedback intervention on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996;119:254–84.CrossRef Kluger AN, DeNisi A. The effects of feedback intervention on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996;119:254–84.CrossRef
6.
Zurück zum Zitat Berbano EP, Browning R, Pangaro L, Jackson JL. The impact of the Stanford Faculty Development Program on ambulatory teaching behavior. J Gen Intern Med. 2006;21:430–4.PubMedCentralPubMedCrossRef Berbano EP, Browning R, Pangaro L, Jackson JL. The impact of the Stanford Faculty Development Program on ambulatory teaching behavior. J Gen Intern Med. 2006;21:430–4.PubMedCentralPubMedCrossRef
7.
Zurück zum Zitat Jackson JL, O'Malley PG, Salerno SM, Kroenke K. The teacher and learner interactive assessment system (TeLIAS): a new tool to assess teaching behaviors in the ambulatory setting. Teach Learn Med. 2002;14:249–56.PubMedCrossRef Jackson JL, O'Malley PG, Salerno SM, Kroenke K. The teacher and learner interactive assessment system (TeLIAS): a new tool to assess teaching behaviors in the ambulatory setting. Teach Learn Med. 2002;14:249–56.PubMedCrossRef
8.
Zurück zum Zitat Salerno SM, O'Malley PG, Pangaro LN, Wheeler GA, Moores LK, Jackson JL. Faculty development seminars based on the one-minute preceptor improve feedback in the ambulatory setting. J Gen Intern Med. 2002;17:779–87.PubMedCentralPubMedCrossRef Salerno SM, O'Malley PG, Pangaro LN, Wheeler GA, Moores LK, Jackson JL. Faculty development seminars based on the one-minute preceptor improve feedback in the ambulatory setting. J Gen Intern Med. 2002;17:779–87.PubMedCentralPubMedCrossRef
9.
Zurück zum Zitat Salerno SM, Jackson JL, O'Malley PG. Interactive faculty development seminars improve the quality of written feedback in ambulatory teaching. J Gen Intern Med. 2003;18:831–4.PubMedCentralPubMedCrossRef Salerno SM, Jackson JL, O'Malley PG. Interactive faculty development seminars improve the quality of written feedback in ambulatory teaching. J Gen Intern Med. 2003;18:831–4.PubMedCentralPubMedCrossRef
10.
Zurück zum Zitat Sender-Liberman A, Liberman M, Steinert Y, McLeod P, Meterissian S. Surgery residents and attending surgeons have different perspectives of feedback. Med Teach. 2005;27(5):470–2.PubMedCrossRef Sender-Liberman A, Liberman M, Steinert Y, McLeod P, Meterissian S. Surgery residents and attending surgeons have different perspectives of feedback. Med Teach. 2005;27(5):470–2.PubMedCrossRef
11.
Zurück zum Zitat Archer JC. State of the science in health professional education: effective feedback. Medical Education. 2010;44(1):101–8.PubMedCrossRef Archer JC. State of the science in health professional education: effective feedback. Medical Education. 2010;44(1):101–8.PubMedCrossRef
12.
Zurück zum Zitat Jensen AR, Wright AS, Kim S, Horvath KD, Calhoun KE. Educational feedback in the operating room: a gap between resident and faculty perceptions. Am J Surg. 2012;204:248–55.PubMedCrossRef Jensen AR, Wright AS, Kim S, Horvath KD, Calhoun KE. Educational feedback in the operating room: a gap between resident and faculty perceptions. Am J Surg. 2012;204:248–55.PubMedCrossRef
13.
Zurück zum Zitat Bing-You RG, Towbridge RL. Why medical educators may be failing at feedback. JAMA. 2009;302(12):1330–1.PubMedCrossRef Bing-You RG, Towbridge RL. Why medical educators may be failing at feedback. JAMA. 2009;302(12):1330–1.PubMedCrossRef
14.
Zurück zum Zitat Gil DH, Heins M, Jones PB. Perceptions of medical school faculty members and students on clinical clerkship feedback. J Med Educ. 1984;59:856–64.PubMed Gil DH, Heins M, Jones PB. Perceptions of medical school faculty members and students on clinical clerkship feedback. J Med Educ. 1984;59:856–64.PubMed
15.
16.
Zurück zum Zitat Bahar-Ozvaris S, Aslan D, Sahin-Hodoglugil N, Sayek I. A faculty development program evaluation: from needs assessment to long-term effects, of the teaching skills improvement program. Teach Learn Med. 2004;16:368–75.PubMedCrossRef Bahar-Ozvaris S, Aslan D, Sahin-Hodoglugil N, Sayek I. A faculty development program evaluation: from needs assessment to long-term effects, of the teaching skills improvement program. Teach Learn Med. 2004;16:368–75.PubMedCrossRef
17.
Zurück zum Zitat Moss HA, Derman PB, Clement RC. Medical student perspective: working toward specific and actionable clinical clerkship feedback. Med Teach. 2012;34:665–7.PubMedCrossRef Moss HA, Derman PB, Clement RC. Medical student perspective: working toward specific and actionable clinical clerkship feedback. Med Teach. 2012;34:665–7.PubMedCrossRef
18.
Zurück zum Zitat Frye AW, Hollingsworth MA, Wymer A, Hinds MA. Dimensions of feedback in clinical teaching: a descriptive study. Acad Med. 1996;71:S79–81.PubMedCrossRef Frye AW, Hollingsworth MA, Wymer A, Hinds MA. Dimensions of feedback in clinical teaching: a descriptive study. Acad Med. 1996;71:S79–81.PubMedCrossRef
19.
Zurück zum Zitat Kogan JR, Conforti LN, Bernabeo EC, Durning SJ, Hauer KE, Holmboe ES. Faculty staff perceptions of feedback to residents after direct observation of clinical skills. Med Educ. 2012;46:201–15.PubMedCrossRef Kogan JR, Conforti LN, Bernabeo EC, Durning SJ, Hauer KE, Holmboe ES. Faculty staff perceptions of feedback to residents after direct observation of clinical skills. Med Educ. 2012;46:201–15.PubMedCrossRef
20.
Zurück zum Zitat Delva D, Sargeant J, Miller S, et al. Encouraging residents to seek feedback. Med Teach. 2013;35:e1625–31.PubMedCrossRef Delva D, Sargeant J, Miller S, et al. Encouraging residents to seek feedback. Med Teach. 2013;35:e1625–31.PubMedCrossRef
21.
Zurück zum Zitat Ende J, Pomerantz A, Erickson F. Preceptors' strategies for correcting residents in an ambulatory care medicine setting: a qualitative analysis. Acad Med. 1995;70:224–9.PubMedCrossRef Ende J, Pomerantz A, Erickson F. Preceptors' strategies for correcting residents in an ambulatory care medicine setting: a qualitative analysis. Acad Med. 1995;70:224–9.PubMedCrossRef
22.
24.
Zurück zum Zitat Skeff KM, Stratos GA, Berman J, Bergen MR. Improving clinical teaching. Evaluation of a national dissemination program. Arch Intern Med. 1992;152:1156–61.PubMedCrossRef Skeff KM, Stratos GA, Berman J, Bergen MR. Improving clinical teaching. Evaluation of a national dissemination program. Arch Intern Med. 1992;152:1156–61.PubMedCrossRef
25.
Zurück zum Zitat ACGME Program requirements for graduate medical education in internal medicine. Accreditation council for graduate medical education . 7-1-2013. 12-22-2014. ACGME Program requirements for graduate medical education in internal medicine. Accreditation council for graduate medical education . 7-1-2013. 12-22-2014.
26.
Zurück zum Zitat Kay C, Jackson JL, Frank M. The relationship between internal medicine residency graduate performance on the ABIM certifying examination, yearly in-service training examinations, and the USMLE Step 1 Examination. Acad Med 2014. Kay C, Jackson JL, Frank M. The relationship between internal medicine residency graduate performance on the ABIM certifying examination, yearly in-service training examinations, and the USMLE Step 1 Examination. Acad Med 2014.
27.
Zurück zum Zitat Davis DA, Mazmanian PE, Fordis M, Van HR, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296:1094–1102.PubMedCrossRef Davis DA, Mazmanian PE, Fordis M, Van HR, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296:1094–1102.PubMedCrossRef
28.
Zurück zum Zitat Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions education: a systematic review. Adv Health Sci Educ Theory Pract. 2009;14:595–621.PubMedCrossRef Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions education: a systematic review. Adv Health Sci Educ Theory Pract. 2009;14:595–621.PubMedCrossRef
29.
30.
Zurück zum Zitat Haber RJ, Avins AL. Do ratings on the American Board of Internal Medicine Resident Evaluation Form detect differences in clinical competence? J Gen Intern Med. 1994;9:140–5.PubMedCrossRef Haber RJ, Avins AL. Do ratings on the American Board of Internal Medicine Resident Evaluation Form detect differences in clinical competence? J Gen Intern Med. 1994;9:140–5.PubMedCrossRef
31.
Zurück zum Zitat Thompson WG, Lipkin M Jr, Gilbert DA, Guzzo RA, Roberson L. Evaluating evaluation: assessment of the American Board of Internal Medicine Resident Evaluation Form. J Gen Intern Med. 1990;5:214–7.PubMedCrossRef Thompson WG, Lipkin M Jr, Gilbert DA, Guzzo RA, Roberson L. Evaluating evaluation: assessment of the American Board of Internal Medicine Resident Evaluation Form. J Gen Intern Med. 1990;5:214–7.PubMedCrossRef
32.
Zurück zum Zitat Berbano EP, Browning R, Pangaro L, Jackson JL. The impact of the Stanford Faculty Development Program on ambulatory teaching behavior. J Gen Intern Med. 2006;21:430–4.PubMedCentralPubMedCrossRef Berbano EP, Browning R, Pangaro L, Jackson JL. The impact of the Stanford Faculty Development Program on ambulatory teaching behavior. J Gen Intern Med. 2006;21:430–4.PubMedCentralPubMedCrossRef
33.
Zurück zum Zitat Hawkins RE, Sumption KF, Gaglione MM, Holmboe ES. The in-training examination in internal medicine: resident perceptions and lack of correlation between resident scores and faculty predictions of resident performance. Am J Med. 1999;106:206–10.PubMedCrossRef Hawkins RE, Sumption KF, Gaglione MM, Holmboe ES. The in-training examination in internal medicine: resident perceptions and lack of correlation between resident scores and faculty predictions of resident performance. Am J Med. 1999;106:206–10.PubMedCrossRef
34.
Zurück zum Zitat Durning SJ, Pangaro LN, Lawrence LL, Waechter D, McManigle J, Jackson JL. The feasibility, reliability, and validity of a program director's (supervisor's) evaluation form for medical school graduates. Acad Med. 2005;80:964–8.PubMedCrossRef Durning SJ, Pangaro LN, Lawrence LL, Waechter D, McManigle J, Jackson JL. The feasibility, reliability, and validity of a program director's (supervisor's) evaluation form for medical school graduates. Acad Med. 2005;80:964–8.PubMedCrossRef
35.
Zurück zum Zitat Durning SJ, Cation LJ, Jackson JL. The reliability and validity of the American Board of Internal Medicine Monthly Evaluation Form. Acad Med. 2003;78:1175–82.PubMedCrossRef Durning SJ, Cation LJ, Jackson JL. The reliability and validity of the American Board of Internal Medicine Monthly Evaluation Form. Acad Med. 2003;78:1175–82.PubMedCrossRef
36.
Zurück zum Zitat Caverzagie KJ, Iobst WF, Aagaard EM, et al. The internal medicine reporting milestones and the next accreditation system. Ann Intern Med. 2013;158:557–9.PubMedCrossRef Caverzagie KJ, Iobst WF, Aagaard EM, et al. The internal medicine reporting milestones and the next accreditation system. Ann Intern Med. 2013;158:557–9.PubMedCrossRef
37.
Zurück zum Zitat Holmboe ES, Fiebach NH, Galaty LA, Huot S. Effectiveness of a focused educational intervention on resident evaluations from faculty a randomized controlled trial. J Gen Intern Med. 2001;16:427–34.PubMedCentralPubMedCrossRef Holmboe ES, Fiebach NH, Galaty LA, Huot S. Effectiveness of a focused educational intervention on resident evaluations from faculty a randomized controlled trial. J Gen Intern Med. 2001;16:427–34.PubMedCentralPubMedCrossRef
38.
Zurück zum Zitat Laughlin T, Brennan A, Brailovsky C. Effect of field notes on confidence and perceived competence: survey of faculty and residents. Can Fam Physician. 2012;58:e352–6.PubMedCentralPubMed Laughlin T, Brennan A, Brailovsky C. Effect of field notes on confidence and perceived competence: survey of faculty and residents. Can Fam Physician. 2012;58:e352–6.PubMedCentralPubMed
39.
Zurück zum Zitat Mann K, van der Vleuten C, Eva K, et al. Tensions in informed self-assessment: how the desire for feedback and reticence to collect and use it can conflict. Acad Med. 2011;86:1120–7. Mann K, van der Vleuten C, Eva K, et al. Tensions in informed self-assessment: how the desire for feedback and reticence to collect and use it can conflict. Acad Med. 2011;86:1120–7.
Metadaten
Titel
The Quality of Written Feedback by Attendings of Internal Medicine Residents
verfasst von
Jeffrey L Jackson, MD MPH
Cynthia Kay, MD MSc
Wilkins C Jackson, HS
Michael Frank, MD
Publikationsdatum
01.07.2015
Verlag
Springer US
Erschienen in
Journal of General Internal Medicine / Ausgabe 7/2015
Print ISSN: 0884-8734
Elektronische ISSN: 1525-1497
DOI
https://doi.org/10.1007/s11606-015-3237-2

Weitere Artikel der Ausgabe 7/2015

Journal of General Internal Medicine 7/2015 Zur Ausgabe

Leitlinien kompakt für die Innere Medizin

Mit medbee Pocketcards sicher entscheiden.

Seit 2022 gehört die medbee GmbH zum Springer Medizin Verlag

Update Innere Medizin

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.