Participants
University of Utah Health (U of U Health) annually surveys all academic faculty and staff employed within the health sciences campus to measure employee engagement. This survey is administered by Dialogue™, formerly known as Waggl™, which is a digital, organizational engagement survey company [
16]. U of U Health contracts with Dialogue™ for administration of the survey, use of their questionnaire items, access to their reporting software, and utilization of the population bank of their qualitative assessment responses. The present cross-sectional study focuses on the 791 physicians and advance practice clinicians who completed the survey either in January or April 2019. Participation was voluntary, and all data were confidential. Although Dialogue™ tracks responses by employee identification number, identifying information was not available to any U of U Health employee. The ethical approval and informed consent to participate was waived by the U of U Health Internal Review Board (IRB# 00,124,369). All methods were carried out in accordance with relevant guidelines and regulations.
Measurement
The engagement survey consisted of 11 quantitative items and 1 qualitative item to measure employee satisfaction, opportunities for professional development and advancement, job-related resources, workplace communication, and well-being. The complete list of items is in supplemental Table
1. Eight of the survey items were from the Dialogue™ question bank. These items were derived from the field of employee engagement research and selected through expert consensus. They have been used broadly throughout the healthcare industry [
17]. The remaining three items, specifically those measuring work control, workplace stress and burnout, were adapted and modified from the Mini-Z worklife survey in order to fit the direction of the agreement scale of the instrument. The Mini-Z survey has demonstrated moderate reliability, with Cronbach’s alpha of 0.8 for the complete measure, and good internal validity [
18]. Additionally, the single-item measuring burnout is highly correlated with the emotional exhaustion scale of the Maslach Burnout Inventory [
19]. The 5-point Likert scale of the Dialogue™ instrument had the following anchors: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree and 5 = strongly agree. Responses for each quantitative item were dichotomized by agreement with “strongly agree” and “agree” being recategorized into a “Yes” response and “neutral,” “disagree” and “strongly disagree” being recategorized into a “No” response. Participants who responded to the quantitative items were included in all analyses, regardless if they completed the survey in its entirety.
Table 1
Characteristics of participants (overall and stratified by degree)
Total | 791b | 158 | 633 | |
Age 30–49 n (%) | | | | 0.44 |
No | 281 (35.5) | 52 (32.9) | 229 (36.2) | |
Yes | 510 (64.5) | 106 (67.1) | 404 (63.8) | |
Gender n (%) | | | | < 0.001 |
Female | 356 (45.4) | 118 (77.6) | 238 (37.6) | |
Male | 429 (54.7) | 34 (22.4) | 395 (62.4) | |
Race Ethnicity n (%) | | | | 0.51d |
White | 619 (78.3) | 132 (83.5) | 487 (76.9) | |
Asian | 55 (7.0) | 8 ( 5.1) | 47 ( 7.4) | |
Hispanic | 14 (1.8) | 2 ( 1.3) | 12 ( 1.9) | |
Other | 13 (1.6) | 1 ( 0.6) | 12 ( 1.9) | |
Prefer not to say | 74 (9.4) | 14 ( 8.9) | 60 ( 9.5) | |
Everyone else | 16 (2.0) | 1 ( 0.6) | 15 ( 2.4) | |
Appointment Type n (%) | | | | < 0.001 |
Mostly Clinical | 709 (89.6) | 155 (98.1) | 554 (87.5) | |
Mostly Other | 36 (4.6) | 3 ( 1.9) | 33 ( 5.2) | |
Mostly Research | 46 (5.8) | 0 ( 0.0) | 46 ( 7.3) | |
Degree n (%) | | | | - |
APC | 158 (20.0) | - | - | |
MD | 633 (80.0) | - | - | |
Demographic information was collected via population from human resources records when participants completed the survey. Available demographic variables included age, sex, race and ethnicity, faculty appointment type (research or clinical), and academic degree. Data from the sole qualitative item, “What would make you feel more appreciated at work? How would this be impactful?” is not included in the present study.
Statistical analyses
Descriptive demographic information was summarized overall with counts and percentages because all variables were categorical. With provider type (i.e., physician vs advance practice clinician) being the primary exposure variable of interest, demographics were also stratified by this variable and characteristic comparisons between physicians and advance practice clinicians were made using a Chi-Square test for sufficiently large sample sizes, and Fisher’s Exact test for small sample sizes. For analysis of the 11-dichotimzed items, item agreement percentages were presented overall and stratified by demographics of interest (i.e., sex and degree status) while being compared with a Chi-Square test. Additionally, odds ratios with 95% confidence intervals (CIs) were presented to assess the odds of agreement for each item between the stratified groups.
While individual items alone provided useful insights, of greater interest were underlying trends seen across combinations of these items. As such, an exploratory factor analysis (EFA), with iterated principal axis factor extraction and squared multiple correlations on the diagonal of the correlation matrix, was conducted on the 11 quantitative items on their original scale. This was done to confirm the validity of grouping certain items into domains. Orthogonal varimax and oblique promax rotations were examined, and upon findings of high correlations between factors as well as findings yielding more of a simple structure (see Supplemental Figs.
1a-1b) the promax rotation was used for final results. Rotated factor pattern loadings (correlations between items and factors) were provided to determine the item domains. Item communalities (proportion of variance of each item contributed by the factors) as well as inter-factor correlations were also provided. Diagnostics were assessed to confirm an optimal number of factors and overall factor solution. As a sensitivity analysis, the EFA was repeated while changing the extraction method to maximum likelihood and minimum residual to confirm stable loadings of the domains.
To measure the internal consistency of item responses, McDonald’s omega was used [
20]. This was preferred over the commonly used Cronbach’s alpha due to its ability to handle multiple latent dimensions in the item responses (as opposed to one item-wide dimension “unidimensionality” for alpha), correlation between errors (alpha assumes independence between errors), violation of tau-equivalence (different factor loadings of the items while alpha assumes all are equal), and the overall outperformance of omega over alpha under such situations [
21‐
29]. Thus, with factors underlying the items, as well as correlations between those factors, total omega and hierarchical omega were employed to consider these phenomena. In addition, the algebraic greatest lower bound (GLBa) was used as a companion, which has been shown to be a reliable estimate in the presence of non-normal or skewed data [
28,
30‐
33].
An additional analysis consisted of comparing the sample percentage of burnout (those who answered “strongly disagree” or “disagree” to the item “Burnout is not a problem for me”) to the national percentage using a one-sample z hypothesis test for proportions and a 5% significance level.
As a sub-analysis, domains from the EFA were converted into weighted factor scores. Because domains were shown to be correlated, all demographic predictors were fit simultaneously in multivariate linear regression with all domains as outcomes. Thus, models were fit each with a different outcome and involving all the same predictors. Coefficients, however, across all models covaried. To confirm selection of predictors, a multivariate analysis of variance (MANOVA) Pillai test was conducted to determine which predictors were jointly contributing to all outcomes significantly. Adjusted beta-hats (\(\widehat{\beta }\)ADJ) were calculated for predictors, which reported the average change in domain outcome with each one-unit increase in predictor. Significance of predictors was reported with p-values. Model diagnostics were assessed for predictor/outcome sets to ensure optimal fit. To capture uncertainty of estimates, while owing to the fact multiple sets of coefficients were present that covaried, 95% confidence ellipses were plotted to capture uncertainty in two dimensions (two outcomes were plotted at a time, and all combinations were assessed). The ellipse captured the area within which one could be 95% confident that the true joint domain outcome was contained. With the predictors of gender and degree being of interest, all comparisons considered these predictors while holding all others constant at mean levels.
All other hypothesis tests (besides the one sample z-test) were two-sided with a significance level of 5%. All analyses were performed in SAS, version 9.4 (SAS Institute Inc).