Establishing Crosswalks Between Common Measures of Burnout in US Physicians

In the US, burnout is more common in physicians than in workers in other fields,¹ and is characterized by work-related feelings of exhaustion and depersonalization or interpersonal disengagement.^{2, 3} Physician burnout is associated with poor physician health outcomes, reduced quality of care, and at least 4.6 billion dollars in excess health system costs annually.^4‐6 In an effort to curb physician burnout,^{7, 8} health systems across the nation are integrating measures of burnout into routine organizational assessments to monitor system functioning and evaluate the effectiveness of practice changes designed to improve physician well-being.^9‐11 This practice is recommended in the National Academy of Medicine’s consensus report on clinician burnout and regarded by healthcare leaders as a basic first step to addressing the problem.^{7, 10‐14}

With the widespread adoption of physician burnout assessment within US healthcare systems has come the problem of comparing outcomes across different burnout measures. With several validated options available that vary in length and cost, a number of different measures are currently in use in the US,^{9, 10} including the Maslach Burnout Inventory-Human Services Survey for Medical Personnel (MBI),¹⁵ Stanford Professional Fulfillment Index (PFI),¹⁶ and the Mini-Z Single-Item Burnout (MZSIB) scale.¹⁷ When two different burnout measures are used across organizations or within an organization over time, the scores are not comparable unless they are placed onto the same metric, or “linked,” in psychometric analyses. To date, no studies to our knowledge have linked common measures of physician burnout onto a single metric, which would allow healthcare organizations to compare burnout scores/rates across different measures.

The primary aim of this study was to link the PFI and MZSIB to the MBI metric and create crosswalks that map scores from the PFI and MZSIB to corresponding scores on the MBI. Using the crosswalks, we aimed to describe the closest cut-point equivalences for scales linked to the same metric. Our secondary aim was to examine the psychometric properties of scales linked to the same metric, including each scale’s reliability and associations with relevant adverse outcomes.

Linking refers to the statistical process of placing two or more measures with different content and/or construct severity levels onto the same scale.¹⁸ Through this process, a relationship is established between the linked measures, such that for each score on Burnout Measure A, an equivalent score (within standard error) on Burnout Measure B is established.

Healthcare organizations across the US are monitoring physician burnout as an indicator of health system performance.⁹ Common applications of physician burnout measurement as a performance indicator are to make inferences regarding the quality of physicians’ medical practice environments, workforce sustainability, and healthcare quality.⁹ Yet, comparisons of performance over time, across organizations, or across studies are not possible when different burnout measures have been employed. In this study, we used IRT linking to place common burnout measures—the PFI and MZSIB—onto the metric of the MBI, and created crosswalks that map raw scores on the PFI-WE, PFI-ID, and MZSIB scales to corresponding MBI subscale scores. For scales linked to the same metric, we identified the closest cut-point equivalencies across all linked metrics and compared the reliability across linked outcome metrics.

By linking the PFI, MZSIB, and MBI to the same metric, the crosswalks we produced allow investigators using these measures to make several useful comparisons.²⁵ First, investigators can compare summary sample scores across the PFI, MZSIB, and MBI. That is, using the crosswalks produced in this study, group-level emotional exhaustion scores can be compared across the MBI-EE, PFI-WE, and MZSIB scales, and group-level depersonalization scores can be compared across the MBI-DP or PFI-ID scales.²⁵ Second, investigators can use the crosswalks to calculate emotional exhaustion/depersonalization rates across metrics by substituting respondents’ raw (total) scores on the PFI or MZSIB with the corresponding MBI t-score. The corresponding MBI t-scores can then be used to calculate the percent of physicians scoring at or above a selected MBI cut-point. The substituted MBI scores can be further analyzed in descriptive and inferential analyses.²⁵ The crosswalks can also be used to calculate emotional exhaustion/depersonalization rates across metrics using only aggregated data. In Supplemental Appendix 10, we demonstrate how to calculate emotional exhaustion/depersonalization rates on the MBI metric using frequency tables of physicians’ raw scores on the PFI. The crosswalks can facilitate comparisons of burnout scores/rates across organizations using different measures, within organizations using different measures over time, and to published regional/national benchmarks. The use of our crosswalks to convert burnout scores from different measures to a common metric may also improve comparative effectiveness and meta-analysis research by reducing error associated with the use of different scales across studies.^{25, 37}

Our reliability assessment provides important information regarding the psychometric performance of each measure, each of which has its own strengths and weaknesses that should be considered within the intended purpose of an organization’s assessment.⁹ For example, the MBI-EE scale provides >0.90 reliability to assess a wide range of emotional exhaustion levels, but at the cost of additional items. With less than half the items of the MBI-EE, the PFI-WE scale offers >0.80 reliability to assess a similar range of above-average emotional exhaustion levels as the MBI-EE scale, but has less precision at below average emotional exhaustion levels than the MBI-EE scale. Similarly, with only one item, the MZSIB offers the least response burden but has less precision to assess emotional exhaustion than the MBI-EE and PFI-WE scales (an expected result given the MZSIB was originally designed as a brief screening tool, not an outcome assessment). However, this level of precision may be sufficient, for example, if the intended purpose of assessment is for screening followed by additional assessment, or to predict the risk of occupational outcomes of depression symptoms, distress, or intent to leave one’s practice at a group-level. The PFI-ID scale offers the most reliable assessment of depersonalization across the widest range of depersonalization levels, with one additional item compared to MBI-DP scale. We should note that, to our knowledge, this is the first assessment of the MZSIB’s reliability (as internal consistency reliability is not applicable to single-item scales and test-retest reliability has not yet been investigated for this measure).

All scales showed significant correlations with important, adverse outcomes, including physician depression, distress, and intent to leave. The association between each measure and each adverse outcome underscores the importance of including measures of physician burnout in institutional assessments.

To our knowledge, this is the first study to crosswalk common measures of burnout among US physicians. Strengths of this study include the use of a single-group linking design (permitting the direct comparison of physicians’ actual MBI scores to those predicted by linking to determine the accuracy of our results) and the use and agreement of two different linking methods.

However, this study has several limitations. First, because the MBI-EE and MBI-DP metrics to which the PFI and MZSIB are linked were derived from a prior IRT analysis of 2014 MBI data from the Shanafelt et al. (2015) national physician burnout prevalence study,^{29, 30} the mean of each MBI anchor scale is fixed to the mean EE and DP scores of US physicians in 2014. Therefore, when interpreting a score on a target scale relative to its SDs above/below the mean score on its MBI anchor scale, it should be known that the comparison is relative to the underlying mean MBI score of US physicians in 2014. Despite this limitation, the crosswalks remain valid assuming that the MBI subscales function equivalently across the 2014 US physician sample and US general physician population. Second, although our findings provide support for the invariance of our crosswalks across early and late responder groups and, therefore, provide potential support for the representativeness of our sample, this support relies on the assumption that late responders are an adequate proxy for non-respondents. Nevertheless, several studies have demonstrated no significant differences in burnout estimates across respondent and non-respondent groups, despite the low response rates that are common in physician survey research.^{1, 38} Third, we chose to highlight the closest cut-point equivalencies across linked measures using commonly used cut-points on each metric. Because raw scores on each target metric are linked to continuous scores on each anchor metric, the closest cut-point equivalencies across metrics are an approximation. Although we identified the closest cut-point equivalency for scores ≥27 and ≥10 on the respective MBI-EE and MBI-DP scales, investigators can use crosswalks published in Brady et al. ²⁹ in conjunction with the crosswalks presented herein to identify cut-point equivalencies on the PFI and MZSIB at other MBI raw score cut-points.

As US healthcare organizations are increasingly measuring physician burnout as an indicator of health system performance, there is a need to compare burnout outcomes across different assessments. Our findings allow healthcare organizations using the PFI or MZSIB to compare group-level scores to historical, regional, or national MBI scores (and vice-versa).