Skip to main content
SpringerLink
Log in

Understanding resident ratings of teaching in the workplace: a multi-centre study

  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Providing clinical teachers with feedback about their teaching skills is a powerful tool to improve teaching. Evaluations are mostly based on questionnaires completed by residents. We investigated to what extent characteristics of residents, clinical teachers, and the clinical environment influenced these evaluations, and the relation between residents’ scores and their teachers’ self-scores. The evaluation and feedback for effective clinical teaching questionnaire (EFFECT) was used to (self)assess clinical teachers from 12 disciplines (15 departments, four hospitals). Items were scored on a five-point Likert scale. Main outcome measures were residents’ mean overall scores (MOSs), specific scale scores (MSSs), and clinical teachers’ self-evaluation scores. Multilevel regression analysis was used to identify predictors. Residents’ scores and self-evaluations were compared. Residents filled in 1,013 questionnaires, evaluating 230 clinical teachers. We received 160 self-evaluations. ‘Planning Teaching’ and ‘Personal Support’ (4.52, SD .61 and 4.53, SD .59) were rated highest, ‘Feedback Content’ (CanMEDS related) (4.12, SD .71) was rated lowest. Teachers in affiliated hospitals showed highest MOS and MSS. Medical specialty did not influence MOS. Female clinical teachers were rated higher for most MSS, achieving statistical significance. Residents in year 1–2 were most positive about their teachers. Residents’ gender did not affect the mean scores, except for role modeling. At group level, self-evaluations and residents’ ratings correlated highly (Kendall’s τ 0.859). Resident evaluations of clinical teachers are influenced by teacher’s gender, year of residency training, type of hospital, and to a lesser extent teachers’ gender. Clinical teachers and residents agree on strong and weak points of clinical teaching.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Arah, O. A., Heineman, M. J., & Lombarts, K. M. (2012). Factors influencing residents’ evaluations of clinical faculty member teaching qualities and role model status. Medical Education, 46(4), 381–389.

    Article  Google Scholar 

  • Bakken, L. L. (2005). Who are physician-scientists’ role models? Gender makes a difference. Academic Medicine, 80(5), 502–506.

    Article  Google Scholar 

  • Bandiera, G., Lee, S., & Tiberius, R. (2005). Creating Effective Learning in Today’s Emergency Departments: How Accomplished Teachers Get It Done. Annals of Emergency Medicine, 45(3), 253–261.

    Article  Google Scholar 

  • Beckman, T. J. (2004). How reliable are assessments of clinical teaching? A review of the published instruments. Journal of General Internal Medicine, 19(9), 971–977.

    Article  Google Scholar 

  • Beckman, T. J., Cook, D. A., & Mandrekar, J. N. (2005). What is the validity evidence for assessments of clinical teaching? Journal of General Internal Medicine, 20(12), 1159–1164.

    Article  Google Scholar 

  • Beckman, T. J., & Mandrekar, J. N. (2005). The interpersonal, cognitive and efficiency domains of clinical teaching: Construct validity of a multi-dimensional scale. Medical Education, 39(12), 1221–1229.

    Article  Google Scholar 

  • Billet, S. (2001). Learning in the workplace: Strategies for effective practice. Crows Nest: Allen & Unwin.

    Google Scholar 

  • Boor, K., Teunissen, P. W., Scherpbier, A. J., van der Vleuten, C. P., van de Lande, J., & Scheele, F. (2008). Residents’ perceptions of the ideal clinical teacher-A qualitative study. European Journal of Obstetrics, Gynecology, and Reproductive Biology, 140(2), 152–157.

    Article  Google Scholar 

  • Bruijn, M., Busari, J. O., & Wolf, B. H. (2006). Quality of clinical supervision as perceived by specialist registrars in a university and district teaching hospital. Medical Education, 40(10), 1002–1008.

    Article  Google Scholar 

  • Centra, J. A., & Gaubatz, N. B. (2000). Is there gender bias in student evaluations of teaching? The Journal of Higher Education, 71(1), 17–33.

    Article  Google Scholar 

  • Chitsabesan, P., Corbett, S., Walker, L., Spencer, J., & Barton, J. R. (2006). Describing clinical teachers’ characteristics and behaviours using critical incidents and repertory grids. Medical Education, 40(7), 645–653.

    Article  Google Scholar 

  • Claridge, J. A., Calland, J. F., Chandrasekhara, V., Young, J. S., Sanfey, H., & Schirmer, B. D. (2003). Comparing resident measurements to attending surgeon self-perceptions of surgical educators. American Journal of Surgery, 185(4), 323–327.

    Article  Google Scholar 

  • Colthart, I., Bagnall, G., Evans, A., Allbutt, H., Haig, A., Illing, J., et al. (2008). The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10. Medical Teacher, 30(2), 124–145.

    Article  Google Scholar 

  • Davis, D. A., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., & Perrier, L. (2006). Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA, 296(9), 1094–1102.

    Article  Google Scholar 

  • Day, S. C., Norcini, J. J., Shea, J. A., & Benson, J. A, Jr. (1989). Gender differences in the clinical competence of residents in internal medicine. Journal of General Internal Medicine, 4(4), 309–312.

    Article  Google Scholar 

  • De Groot, J., Brunet, A., Kaplan, A. S., & Bagby, M. (2003). A comparison of evaluations of male and female psychiatry supervisors. Academic Psychiatry: The Journal of the American Association of Directors of Psychiatric Residency Training and the Association for Academic Psychiatry, 27(1), 39–43.

    Article  Google Scholar 

  • Dolmans, D. H., Wolfhagen, H. A., Gerver, W. J., De, G. W., & Scherpbier, A. J. (2004). Providing physicians with feedback on how they supervise students during patient contacts. Medical Teacher, 26(5), 409–414.

    Article  Google Scholar 

  • Engbers, R., Paul, M. J., Fluit, C. R. M. G., & Bolhuis, S. (2013). Towards organizational development for sustainable high-quality medical teaching. Perspectives on Medical Education, 2(1), 28–40.

    Article  Google Scholar 

  • Eva, K. W., & Regehr, G. (2005). Self-assessment in the health professions: A reformulation and research agenda. Academic Medicine, 80(10 Suppl), S46–S54.

    Article  Google Scholar 

  • Eva, K. W., & Regehr, G. (2007). Knowing when to look it up: A new conception of self-assessment ability. Academic Medicine, 82(10 Suppl), S81–S84.

    Article  Google Scholar 

  • Fluit, C., Bolhuis, S., Grol, R., Ham, M., Feskens, R., Laan, R., et al. (2012). Evaluation and feedback for effective clinical teaching in postgraduate medical education: Validation of an assessment instrument incorporating the CanMEDS roles. Medical Teacher, 34(11), 893–901.

    Article  Google Scholar 

  • Fluit, C. R., Bolhuis, S., Grol, R., Laan, R., & Wensing, M. (2010). Assessing the quality of clinical teachers: A systematic review of content and quality of questionnaires for assessing clinical teachers. Journal of General Internal Medicine, 25(12), 137–145.

    Article  Google Scholar 

  • Fluit, C. R., Bolhuis, S., Klaassen, T., Visser, Md, Grol, R., Laan, R., et al. (2013). Residents provide feedback to their clinical teachers: Reflection through dialogue. Medical Teacher, 35(9), e1485–e1492.

    Article  Google Scholar 

  • Harden, R. M., & Crosby, J. (2000). AMEE Guide No 20: The good teacher is more than a lecturer—The twelve roles of the teacher. Medical Teacher, 22(4), 334–347.

    Article  Google Scholar 

  • Hesketh, E. A. (2001). A framework for developing excellence as a clinical educator. Medical Education, 35(6), 555–564.

    Article  Google Scholar 

  • Irby, D. M., Ramsey, P. G., Gillmore, G. M., & Schaad, D. (1991). Characteristics of effective clinical teachers of ambulatory care medicine. Academic Medicine, 66(1), 54–55.

    Article  Google Scholar 

  • Kilminster, S. M., & Jolly, B. C. (2000). Effective supervision in clinical practice settings: A literature review. Medical Education, 34(10), 827–840.

    Article  Google Scholar 

  • Lombarts, K. M., Heineman, M. J., & Arah, O. A. (2010). Good clinical teachers likely to be specialist role models: Results from a multicenter cross-sectional survey. PLoS ONE, 5(12), e15202.

    Article  Google Scholar 

  • McOwen, K. S., Bellini, L. M., Guerra, C. E., & Shea, J. A. (2007). Evaluation of clinical faculty: Gender andminority implications. Academic Medicine, 82(10 Suppl), S94–S96.

    Article  Google Scholar 

  • Parsell, G., & Bligh, J. (2001). Recent perspectives on clinical teaching. Medical Education, 35(4), 409–414.

    Article  Google Scholar 

  • Paukert, J. L., & Richards, B. F. (2000). How medical students and residents describe the roles and characteristics of their influential clinical teachers. Academic Medicine, 75(8), 843–845.

    Article  Google Scholar 

  • Rotem, A., Godwin, P., & Du, J. (1995). Learning in hospital settings. Teaching and Learning in Medicine, 7(4), 211–217.

    Article  Google Scholar 

  • Sargeant, J. (2008). Toward a common understanding of self-assessment. The Journal of Continuing Education in the Health Professions, 28(1), 1–4.

    Article  Google Scholar 

  • Sargeant, J., Mann, K., Van der Vleuten, C. P., & Metsemakers, J. (2008). “Directed” self-assessment: Practice and feedback within a social context. Journal of Continuing Education in the Health Professions, 28(1), 47–54.

    Article  Google Scholar 

  • Sargeant, J. M., Mann, K. V., Van der Vleuten, C. P., & Metsemakers, J. F. (2009). Reflection: A link between receiving and using assessment feedback. Advances in Health Sciences Education, 14(3), 399–410.

    Article  Google Scholar 

  • Schum, T. R., Koss, R., Yindra, K. J., & Nelson, D. B. (1993). Students’ and residents’ ratings of teaching effectiveness in a Department of Pediatrics. Teaching and Learning in Medicine, 5(3), 128–132.

    Article  Google Scholar 

  • Shea, J. A., & Bellini, L. M. (2002). Evaluations of clinical faculty: The impact of level of learner and time of year. Teaching and Learning in Medicine, 14(2), 87–91.

    Article  Google Scholar 

  • Snell, L., Tallett, S., Haist, S., Hays, R., Norcini, J., Prince, K., et al. (2000). A review of the evaluation of clinical teaching: New perspectives and challenges. Medical Education, 34(10), 862–870.

    Article  Google Scholar 

  • Stalmeijer, R. E., Dolmans, D. H., Wolfhagen, I. H., Muijtjens, A. M., & Scherpbier, A. J. (2008). The development of an instrument for evaluating clinical teachers: Involving stakeholders to determine content validity. Medical Teacher, 30(8), e272–e277.

    Article  Google Scholar 

  • Stalmeijer, R. E., Dolmans, D. H., Wolfhagen, I. H., Peters, W. G., van Coppenolle, L., & Scherpbier, A. J. (2010). Combined student ratings and self-assessment provide useful feedback for clinical teachers. Advances in Health Sciences Education, 15(3), 315–328.

    Article  Google Scholar 

  • Steiner, I. P., Yoon, P. W., Kelly, K. D., Diner, B. M., Blitz, S., Donoff, M. G., et al. (2005). The influence of residents training level on their evaluation of clinical teaching faculty. Teaching and Learning in Medicine, 17(1), 42–48.

    Article  Google Scholar 

  • Streiner, D. L., & Norman, G. R. (2003). Health measurement scales. Oxford: Oxford University Press.

    Google Scholar 

  • Sutkin, G., Wagner, E., Harris, I., & Schiffer, R. (2008). What makes a good clinical teacher in medicine? A review of the literature. Academic Medicine, 83(5), 452–466.

    Article  Google Scholar 

  • Ullian, J. A., Bland, C. J., & Simpson, D. E. (1994). An alternative approach to defining the role of the clinical teacher. Academic Medicine, 69(10), 832–838.

    Article  Google Scholar 

  • Ward, M., Gruppen, L., & Regehr, G. (2002). Measuring self-assessment: Current state of the art. Advances in Health Sciences Education, 7(1), 63–80.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank staff and residents at the participating departments. They also want to thank Rikkert Stuve (The Text Consultant) for editing the final manuscript. This study was funded by the Department of Evaluation, Quality and Innovation of Medical Education of the Academic Educational Institute, Radboud University Nijmegen Medical Centre.

Conflict on interest

The authors report no conflict of interest.

Author information

Authors and Affiliations

  1. Academic Educational Institute, Radboud University Medical Center Nijmegen, 306 IWOO, PO Box 9101, 6500 HB, Nijmegen, The Netherlands

    Cornelia R. M. G. Fluit, Sanneke Bolhuis & Roland Laan

  2. Institute for Educational Measurement (CITO), Arnhem, The Netherlands

    Remco Feskens

  3. Scientific Institute for Quality of Healthcare, Radboud University Medical Center Nijmegen, 306 IWOO, PO Box 9101, 6500 HB, Nijmegen, The Netherlands

    Richard Grol & Michel Wensing

Authors
  1. Cornelia R. M. G. Fluit
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Remco Feskens
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sanneke Bolhuis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Richard Grol
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michel Wensing
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Roland Laan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cornelia R. M. G. Fluit.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fluit, C.R.M.G., Feskens, R., Bolhuis, S. et al. Understanding resident ratings of teaching in the workplace: a multi-centre study. Adv in Health Sci Educ 20, 691–707 (2015). https://doi.org/10.1007/s10459-014-9559-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-014-9559-8

Keywords

Search

Navigation