Background
Methods
Ethics statement
MSF system in the Netherlands
MSF instrument and development
Content generation and content validity
Pilot field testing
Final MSF system
Study design, population and setting
Statistical analysis
Results
Study participants
Mean ratings and missing data
Factors and items | Mean score [SD] | Factor loadings on primary factor | Internal consistency reliability | Corrected item-total correlations |
---|---|---|---|---|
Collaboration and self-insight (42% of variance)
| 8.47 [1.09] | 0.900 | ||
Communicates effectively with other health care professionals | 8.23 [1.07] | 0.581 | 0.655 | |
Collaborates with physician colleagues | 8.56 [0.91] | 0.841 | 0.795 | |
Accepts feedback provided | 8.38 [1.06] | 0.748 | 0.711 | |
Recognizes his/her own limitations | 8.13 [1.19] | 0.643 | 0.702 | |
Participates effectively as a member of the health care team | 8.40 [1.21] | 0.631 | 0.700 | |
Exhibits professional behavior towards physician colleagues | 8.61 [0.85] | 0.779 | 0.750 | |
If a member of my own family needed care I would recommend this physician | 8.59 [0.84] | 0.760 | 0.761 | |
Clinical performance (8% of variance)
| 8.40 [0.79] | 0.900 | ||
Performs technical procedures skillfully | 8.45 [0.96] | 0.638 | 0.615 | |
Selects diagnostic tests appropriately | 8.38 [0.93] | 0.739 | 0.758 | |
Critically assesses diagnostic information | 8.44 [0.93] | 0.763 | 0.823 | |
Makes the correct diagnosis following consultation | 8.43 [0.88] | 0.780 | 0.822 | |
Selects appropriate treatments | 8.41 [0.93] | 0.650 | 0.791 | |
Accepts responsibility for own professional actions | 8.4 [0.91] | 0.452 | 0.629 | |
Coordination and continuity
(5% of variance)
| 8.47 [0.73] | 0.851 | ||
Handles transfer of care appropriately | 8.37 [1.03] | 0.727 | 0.697 | |
Maintains confidentiality of patients and their families | 8.57 [0.89] | 0.660 | 0.601 | |
Provides a clear understanding about who is responsible for the continuing care of patients | 8.35 [0.94] | 0.632 | 0.717 | |
Co-ordinates care effectively for patients with other health care professionals and physicians | 8.44 [0.89] | 0.609 | 0.684 | |
Maintains quality medical records | 8.13 [1.23] | 0.456 | 0.547 | |
Manages patients with complex problems | 8.46 [0.88] | 0.632 | 0.669 | |
Practice based learning and improvement
(4% of variance) | 8.12 [1.13] | 0.813 | ||
Contributes to quality improvement programs and practice guidelines | 8.22 [1.26] | 0.652 | 0.726 | |
Teaches adequately medical colleagues and co-workers | 7.97 [1.42] | 0.652 | 0.728 | |
Participates adequately in research activities | 7.67 [1.52] | 0.599 | 0.739 | |
Critically evaluates the medical literature | 7.96 [1.36] | 0.655 | 0.725 | |
Emergency medicine
(4% of variance)
| 8.38 [0.85] | 0.767 | ||
Gives priority to urgent requests | 8.46 [0.93] | 0.660 | 0.634 | |
Handles emergency situations effectively | 8.49 [0.91] | 0.703 | 0.631 | |
Manages own stress effectively | 8.22 [1.12] | 0.564 | 0.549 | |
Time-management and responsibility
(4% of variance)
| 8.69 [1.30] | 0.770 | ||
Handles requests for consultation in a timely manner | 8.40 [1.00] | 0.749 | 0.645 | |
Advises referring physician if referral request is outside the scope of his/her practice | 8.53 [0.87] | 0.550 | 0.576 | |
Assumes appropriate responsibility for patients | 8.67 [0.69] | 0.527 | 0.539 | |
Provides timely information to referring physicians about mutual patients | 8.28 [0.13] | 0.690 | 0.608 |
Factors and items | Mean score [SD] | Factor loadings on primary factor | Internal consistency reliability | Corrected item-total correlations |
---|---|---|---|---|
Relationship with other health care professionals (57% of variance)
| 8.07 [1.11] | .925 | ||
Is able to verbally communicate effectively with other health care professionals | 8.10 [1.21] | 0.691 | 0.757 | |
Is courteous to co-workers | 8.35 [1.11] | 0.760 | 0.763 | |
Respects the professional knowledge and skills of co-workers | 8.31 [1.10] | 0.782 | 0.765 | |
Collaborates well with co-workers | 8.31 [1.10] | 0.811 | 0.848 | |
Is accessible for appropriate communication about patients | 8.37 [1.06] | 0.611 | 0.728 | |
Participates effectively as a member of the health care team | 8.28 [1.13] | 0.660 | 0.766 | |
This physician presents him/herself in a professional manner | 8.59 [0.92] | 0.574 | 0.740 | |
Communication with patients (7% of variance)
| 8.03 [1.07] | .900 | ||
Communicates effectively with patients | 8.21 [1.15] | 0.830 | 0.794 | |
Communicates effectively with families | 8.11 [1.24] | 0.818 | 0.764 | |
Shows compassion to patients and their families | 8.36 [1.07] | 0.721 | 0.812 | |
Is courteous to patients and their families | 8.56 [0.89] | 0.656 | 0.772 | |
Respects the rights of patients to make informed decisions | 8.46 [0.96] | 0.578 | 0.706 | |
Is reasonably accessible to patients | 8.28 [1.08] | 0.579 | 0.700 | |
Patient care (6% of variance)
| 8.29 [1.06] | .830 | ||
Accepts responsibility for patient care | 8.64 [0.72] | 0.748 | 0.720 | |
Maintains confidentiality of patients | 8.69 [0.77] | 0.711 | 0.613 | |
Accepts responsibility for professional actions | 8.64 [0.86] | 0.781 | 0.773 | |
Responds appropriately in emergency situations | 8.40 [1.11] | 0.643 | 0.586 |
Factor and items | Mean score[SD] | Factor loadings | Internal consistency reliability | Corrected item-total correlations |
---|---|---|---|---|
Patient-centeredness (60% of variance)
| .959 | |||
Explained my illness or concern to me clearly | 8.30 [1.32] | 0.792 | 0.792 | |
Spends enough time with me | 8.29 [1.38] | 0.825 | 0.781 | |
Shows interest in my problems | 8.25 [1.43] | 0.840 | 0.806 | |
Answers my questions well | 8.33 [1.32] | 0.873 | 0.827 | |
Treats me with respect | 8.54 [1.02] | 0.810 | 0.763 | |
Shows compassion | 8.10 [1.53] | 0.804 | 0.770 | |
I would go back to this physician | 8.50 [1.23] | 0.819 | 0.803 | |
I would recommend this physician to others | 8.43 [1.34] | 0.828 | 0.823 | |
Explains my treatment choices or options | 8.11 [1.48] | 0.771 | 0.769 | |
Tells me how and when to take my medicine | 8.00 [1.59] | 0.646 | 0.722 | |
Explains clearly different steps of my treatment plan (including risks and benefits) | 8.06 [1.53] | 0.763 | 0.779 | |
Asks details about my personal history, when appropriate | 7.72 [1.83] | 0.664 | 0.659 | |
Explains my physical exam clearly | 8.13 [1.49] | 0.799 | 0.750 | |
Asks permission for some treatments or exams | 8.03 [1.60] | 0.689 | 0.697 | |
Explains clearly what could be done in unsuspected circumstances, such as fever, illness or changes in my complaints | 7.84 [1.70] | 0.757 | 0.754 | |
Tells me what to do if my problems do not get better | 7.90 [1.23] | 0.785 | 0.787 | |
Makes sure that my other caregivers are well informed | 7.94 [1.68] | 0.670 | 0.650 |
Dimension structure and reliability of the dimensions
Collaboration | Clinical performance | Practice based learning and improvement | Coordination and continuity | Responsibility and time-management | Emergency medicine | |
---|---|---|---|---|---|---|
Collaboration | 1.000 | 0.499** | 0.451** | 0.295** | 0.459 | 0.437 |
Clinical performance | 1.000 | 0.551 | 0.432** | 0.383 | 0.408 | |
Practice based learning and improvement | 1.000 | 0.357** | 0.445 | 0.400** | ||
Coordination and continuity | 1.000 | 0.338 | 0.343** | |||
Responsibility and time-management | 1.000 | 0.328** | ||||
Emergency medicine | 1.000 |
Relationship with healthcare professionals | Communications with patients | Professionalism | |
---|---|---|---|
Relationship with healthcare professionals | 1.000 | 0.667** | 0.537** |
Communications with patients | 1.000 | 0.574** | |
Patient care | 1.000 |
Sociodemographic variables influencing rating
Overall rating peers | Overall rating co-workers | Overall rating patients | ||||
---|---|---|---|---|---|---|
Parameter estimated coefficient* [SE] | P-value | Parameter estimated coefficient* [SE] | P-value | Parameter estimated coefficient* [SE] | P-value | |
Physicians'
characteristics
| ||||||
Male (reference: female) | -.0137 [.065] | .832 | -.019 [.091] | .838 | -.049 [.091] | .591 |
Years of experience | -.008 [.004] |
.043
| -.012 [.005] |
.029
| .003 [.005] | .598 |
Surgery | ref. | ref. | ref. | |||
Internal medicine | .069 [.064] | .287 | .139 [.094] | .140 | .096 [.088] | .280 |
Raters' characteristics
| ||||||
Female | ref. | ref. | ref. | |||
Male | -.002 [.051] | .974 | -.200 [.071] |
.005
| -.055 [.062] | .378 |
Age | .005 [.002] |
.002
| ||||
Relation
| ||||||
Membership of the same specialist group | .153 [.049] |
.002
| ||||
Working together: < 6 months | -.115 [.179] | .523 | .046 [.185] | .804 | ||
Working together: > 6 months and < 1 year | .042 [.111] | .702 | -.036 [.122] | .770 | ||
Working together > 1 year | ref. | ref. |
Relationship between the different ratings
Self rating | Medical colleagues' ratings | Co-workers' ratings | Patient ratings' | |
---|---|---|---|---|
Self rating | 1.000 | 0.062 | 0.082 | 0.067 |
Medical colleagues' ratings | 1.000 | 0.352* | 0.214* | |
Co-workers' ratings | 1.000 | 0.220* | ||
Patient ratings' | 1.000 |
Determining the minimum sample size required
Reliability coefficient of 0.60 | Reliability coefficient of 0.70 | Reliability coefficient of 0.80 | |
---|---|---|---|
Peers | 4 | 5 | 5 |
Co-workers | 4 | 5 | 6 |
Patients | 9 | 11 | 12 |
True variance | Residual variance | |
---|---|---|
Peers | 0.06 | 0.42 |
Co-workers | 0.16 | 0.45 |
Patients | 0.093 | 1.07 |