BACKGROUND
METHODS
Identification of Studies
Data Extraction
RESULTS
Author | First publ. Ref.no | Add. Publ. Ref.no | Instrument | Setting | Disciplines | Teachers (N) | Evaluators (N) | Evaluators (type) | Country |
---|---|---|---|---|---|---|---|---|---|
Afonso 2005 | 39 | - | I | IntM,CCU | 30 | 83 | S,R | USA | |
Beckman 2003 | 40 | 41,42,43 | MTEF | I | IntM | 10 | 3 | P | USA |
Bergen 1993 | 44 | - | CTORS | I | IntM | 40 | - | TO | USA |
Cohen 1996 | 50 | 73 | TES | I | Surg | 43 | - | S,R,F | USA |
Copeland 2000 | 51 | 45,46,47,48,49,88 | CTEI | I/O | IntM/Ped/Surg Anes/Path/Radiol | 711 | - | S,R,F | USA |
Cox 2001 | 52 | - | - | I/OR | Surg | 20 | 49 | R | USA |
de Oliveira 2008 | 53 | - | - | I | Anes | 39 | 19 | R | Brasil |
Dolmans 2004 | 54 | - | - | I | Ped | 13 | - | C | Netherlands |
Donelly 1989 | 55 | - | - | ? | IntM | 300 | 100 | C | USA |
Donner-Banzhof 2003 | 56 | - | - | GP | GP | 80 | 80 | R | Germany |
Guyatt 1993 | 57 | - | - | I/A | IntM | 41 | - | C,I,R | USA |
Hayward 1995 | 58 | - | - | O | IntM | 15 | - | R | USA |
Hekelman 1993 | 59 | 60 | - | GP | GP | 16 | 2 | TO | USA |
Hewson 1990 | 61 | - | WICT | I | IntM | 9 | 28 | R | USA |
Irby 1981 | 62 | 63,75,76 | CTAF | I | Gyn | 230 | 320 | S | USA |
James 2002 | 64 | 65 | MedEdIQ | O | IntM/Ped/GP | 156 | 131 | C | USA |
Lewis 1990 | 66 | - | - | GP | GP | 10 | 24 | R | USA |
Litzelman 1998 | 67 | 68,69,71 | SFDP | W | GenM | 178 | 374 | C | USA |
Love 1982 | 70 | - | - | Pharmacy | IntM/Ped/Surg ED/AmbC | 39 | 66 | C | USA |
McLeod 1991 | 72 | - | - | - | IntM | 35 | 50 | S | USA |
Mullan 1993 | 74 | - | - | I | Ped | - | - | C | USA |
Schum 1993 | 79 | 78 | OTE | I | Ped | 186 | 375 | S,C,R | USA |
Shellenberger 1982 | 80 | - | PEQ | GP | GP | - | - | C | USA |
Silber 2006 | 38 | - | - | I | IntM/Surg | 11 | 57 | R | USA |
Smith 2004 | 33 | - | - | I | IntM | 99 | 731 | R | USA |
Solomon 1997 | 81 | - | - | I/O | GenM | 147 | - | C | USA |
Spickard 1996 | 82 | - | - | I | IntM | 44 | 91 | C | USA |
Stalmeijer 2008 | 84 | 83 | - | - | All | - | - | C | Netherlands |
Steiner 2000 | 85 | 86 | - | I | ED | 29 | - | R | USA |
Tortolani 1991 | 87 | 77 | - | I | Surg | 62 | 23 | R | USA |
Williams 1998 | 89 | - | - | I | IntM | 203 | 29 | R | USA |
Zuberi 2007 | 90 | - | SETOC | O | IntM/Ped/ Surg/Gyn/ GP/Opt/Oto | 87 | 224 | C | USA/ Pakistan |
Content of the Questionnaires
Measurement Characteristics of the Instruments
Validity source evidence | ||||||||
---|---|---|---|---|---|---|---|---|
Content | Response process | Internal structure | Relation to other variables | |||||
instrument | Source of items | Content validity | Evaluations (N) | Items (N) | Likert scale | Feasibility | ||
Afonso | 1 | 1,2,4 | 199 | 18 | 5 | - | FA Cronbach’s α | Anonymous and open evaluations compared |
Beckman | 1,2,3 | 1,2,3,4 | 30 | 28 | 5 | 1,2,3 | Cronbach’s α Kendall’s Tau | Follow-up compared scores of residents and peers |
Bergen | 1,2 | 1,2,3,4 | - | 21 | 5 | - | Interrater agreement | - |
Cohen | - | 1,2 | 3750 | 4 | 5 | 3 | ICC | - |
Copeland | 1,2,3 | 1,2,3,4,5,6 | 8048 | 15 | 5 | - | FA Cronbach’s α G-coefficient | Follow-up compared scores of residents and peers Compared to OTS |
Cox | 2,4 | 1,2,4,5 | 753 | 20 | 5 | - | Cronbach’s α | - |
de Oliveira | 4 | 1,2,4,5,6 | 954 | 11 | 4 | - | Cronbach’s α Inter Item C G-Study | Compared to overall perception of quality |
Dolmans | 2,4,5 | 1,3,4,5,6 | - | 18 | 5 | - | - | - |
Donelly | - | 1,2 | 952 | 12 | 7 | - | - | Hypotheses formulated in advance |
Donner-Banzhof | 1,2,3,4 | 1,2,4,5 | 80 | 41 | 4 | - | Cronbach’s α Pearson r | - |
Guyatt | 2,4 | - | - | 14 | 5 | 1 | FA Intra domain correlation | - |
Hayward | 1,4 | 1,2,3,4,5,6 | 142 | 18 | 5 | 3 | FA Cronbach’s α G-Study | - |
Hekelman | 2,4 | 2,3,5,6 | 160 | 17 | - | - | Cronbach’s α ICC | - |
Hewson | 2,4 | 1,2,4,5 | - | 46 | 5 | - | Cronbach’s α | - |
Irby | 1,2 | 1,2,3,4 | 1567 | 9 | 5 | 3 | Spearman Brown Pearson r | Qualitative comparison with other instruments |
James | 2,4,5 | 1,2,5 | 156 | 58 | 6 | - | FA Cronbach’s α | Scores compared with grades of students |
Lewis | 1,2,4 | 1,2,4,5 | - | 16 | VAS | - | ICC interrater correlation | Compared qualitative evaluation data |
Litzelman | 1,2,5 | 1,2,3,4 | 1581 | 25 | 5 | 1 | FA Cronbach’s α Inter Item C | - |
Love | 1,2 | 1,2,3,4 | 281 | 9 | 5 | 1,3 | Pearson r | Residents and attendings compared |
McLeod | 1,4 | 1,4,5 | - | 25 | 6 | - | FA Kruskall-Wallis Wilcoxon Rank | |
Mullan | 4 | 1,4,5 | - | 23 | - | - | Standardized alpha | Compared to OTS |
Schum | 1,2 | 1,2,4 | 749 | 10 | 7 | - | FA Cronbach’s α | Compared to OTS |
Shellenberger | 1,2,4 | 1,2,4,5 | - | 34 | 4 | - | FA Cronbach’s α | - |
Silber | 4 | 1,2,4,5,6 | 226 | 22 | 5 | - | Product-moment correlation | - |
Smith | 2,4 | 1,2,3,4,5,6 | 731 | 32 | 5 | 1,3 | Cronbach’s α Inter Item C Inter-rater reliability SEM | Hypotheses formulated in advance/ Scores compared with grades of students |
Solomon | - | 1 | 2185 | 13 | 4 | - | ICC Spearman Brown SEM Inter-rater reliability | - |
Spickard | 1 | 1,2,3 | - | 9 | 9 | - | FA Cronbach’s α | - |
Stalmeijer | 1,2,3,5 | 1,2,3,4,5,6 | 30 | - | - | - | - | |
Steiner | 1 | 1,2 | 48 | 4 | 5 | - | - | Compared with other instruments |
Tortolani | - | 1,2 | - | 10 | 5 | - | FA Pearson r | - |
Williams | - | 1,2 | 203 | 1 | 5 | - | ICC | Correlation with leadership |
Zuberi | 1,2,3,5 | 1,2,3,4,5,6 | - | 15 | 7 | - | Cronbach’s α Inter Item C G-coefficient | ROC curves |