Effects of work conditions on provider mental well-being and quality of care: a mixed-methods intervention study in the emergency department

The study setting is a 24-h interdisciplinary ED of a tertiary referral hospital in a major city in Southern Germany. The academic hospital provides major services and medical specialties for an administrative region of almost two million inhabitants. The ED serves adult patients with mean yearly visits of over 85,000. It is organized in three sections according to patient’s chief complaints: i.e., ten separate bays for non-surgical patients, five separate bays for surgical patients, two resuscitation bays, and an observation and clinical decision unit with 24 beds. The ED is regularly staffed with junior and senior physicians from internal medicine, trauma surgery, and neurology, as well as further specialists on call.

The study team approached two hospitals for participation, whereof this ED’s department head, hospital administration, and hospital’s worker council agreed to take part. The study team established first contact directly with the head of the ED. The head discussed the proposal with the head of nursing and introduced it in two team meetings of ED physicians and nurses, respectively. After positive feedback from ED staff, the study was presented at the hospital board meeting and received approval. The specific motivation and decision for participation were not specified prior to study start. However, the ED was described as a high strain work environment with particular challenges for nurses, physicians, and patient care. All professionals working in the ED, i.e., nurses, physicians, and administrative staff, were eligible for participation. At baseline, ED staff consisted of 101 nurses (including assistant nurses), 44 physicians, and 20 administrators. External providers and on-call consultants were not included.

Proposed intervention effects on (a) work system factors, (b) provider mental well-being, and (c) quality of care were identified using the following methods:

ED providers received surveys at baseline and follow-up that were distributed through internal mail. Pre-stamped envelopes were provided for direct return of questionnaires to the study team. Deadline for survey completion was four weeks and estimated average time for filling in questionnaires was 25 min. Matching across time was ensured through personalized study codes.

(1a) Work system factors were measured with a validated self-report tool for work analysis in hospitals that has been previously applied to ED work settings [23, 24]. Following SEIPS framework, several work factors were surveyed. Task-related factors included scales on patient stressors (i.e., dealing with difficult patients; three items), job control (i.e., autonomous decision making and personal discretion; four items), participation opportunities (i.e., influence on work-related decisions; four items), work overload (i.e., job duties exceeding work time; three items). Organizational factors consisted of personnel resources (i.e., adequate staffing; three items), information problems (i.e., availability and clarity of work-related information; three items), uncertainty (i.e., job insecurity; three items), social support (i.e., support by colleagues and supervisors; three items), and supervisor feedback (i.e., feedback on performance and task behaviors by senior leaders; two items). Self-reported mean weekly overtime (in hours and minutes), profession (ED physician, ED nurse, ED administrator) and professional tenure (in years) were further retrieved.

(1b) Provider well-being included two key burnout dimensions, i.e., emotional exhaustion and depersonalization (four items each) [9] and a screening tool for depressive symptoms (two items) [25]. Both tools are validated and have been previously applied in healthcare as well as ED provider samples [8, 24]. Conventional cut-off criteria were used to determine providers with elevated levels of emotional exhaustion (scale mean > 3.5) [26] and depressive symptoms (scale sum score ≥ 3) [25]. Job satisfaction and turnover intentions were measured with one item, respectively.

(1c) Provider perceptions of quality of care were measured with a three-item scale on the frequency of medical errors [27]. Respondents were asked to indicate whether they had experienced a near miss, minor error, or serious error during the past year. For each type of error, a short definition was provided. Additionally, overall patient safety was further assessed with one item (“Please rate the degree of patient safety in your department from your point of view”) [28].

Observation sessions of ED nurses’ and physicians’ workflows were allocated randomly across three ED sections and professions. Randomization and sessions were limited to provider day shifts between 10:00 am and 5:00 pm on 40 pre-defined days of data collection. Trained observers shadowed providers for 90-min sessions using a standardized participant observation approach that has been previously applied to ED settings [6, 29].

(2a) Work system factors were represented by observed interruption rates. An established tool to identify workflow interruptions was applied [6, 29]. Referencing the SEIPS model, we distinguished between interruptions initiated by patients and their relatives (task-related factors), and those by ED colleagues of the same or another profession (organizational factors). Furthermore, duration of personal breaks (e.g., time for personal rests, short respites from work, or regular pauses during the shift) during observation sessions was coded (in % of observed time) [30].

(2b) Provider well-being was surveyed immediately after each observation session with a short survey on current cognitive, emotional, and physical aspects of work stress in each observed provider [31].

All ED patients undergoing consultation or treatment on days of on-site data collection were eligible. After information and verbal consent, patients filled in the survey. Patients’ relatives were allowed to fill in the survey by proxy if they accompanied patients throughout their ED stay. Patients with incapability to communicate due to illness severity or other physical and mental constraints were not surveyed. The patient questionnaire was handed out by members of the study team, preferably at the end of patients’ ED treatment. All study team members received prior training in how to approach and interview ED patients.

(3c) Patient perceptions of ED quality of care were obtained with a standardized patient survey (Munich Patient Inventory) with additional translations in English, Russian, and Turkish language. This questionnaire assesses patient-perceived quality of care [32] and has been tested for reliability and validity in different clinical settings [33, 34], including the ED setting [6]. It contains scales on the quality of interaction (example item: “My problems and complaints are taken seriously by ED providers”), patient information (e.g., “I am comprehensively informed about the course of therapy and treatment”), ED organization (e.g., “I know who of the ED providers is responsible for me”), and waiting time (e.g., “My waiting time until the first consultation with an ED physician was adequate”). Additionally, patients’ overall satisfaction with care was obtained with one question (“Overall, how do you evaluate care in this ED?”).

An approximate measure for daily ED workload was computed with day-level data on patient numbers, patient acuity (ESI, Emergency Severity Index score), and staffing levels. Data was extracted from ED administrative records and staff rosters.

To complement quantitative results, we used ED stakeholder interviews for qualitative process evaluation [35, 36]. Nine stakeholder interviews were conducted at study half-time and follow-up. A semi-structured interview guideline assessed intervention implementation, facilitators and barriers, and providers’ mental models [35]. Questions were derived from Nielsen & Randell (2013)‘s framework for comprehensive process evaluation of organizational-level interventions [35]. The German interview guideline can be obtained from the corresponding author. Interviewees were recruited through convenience sampling. All interviewed stakeholders had a job tenure > 5 years in the ED, except one junior physician. Four physicians (three with leadership responsibilities), four nurses (one in leadership position) and one ED administrator were interviewed. Overall, there were four female and five male interviewees. The department head, nursing supervisor, and administration head were interviewed twice to comprise intervention implementation at ED management level at half-time as well as follow-up. Interviews lasted between 30 and 60 min. We did not apply a prior estimate of expected sample size for data saturation since project resources did not allow for more stakeholder interviews.

The intervention started after feedback of baseline survey results to ED staff during regular internal meetings as well as through internal mail. Ten 90-min meetings, termed health circles [15], were held at three-week intervals over a period of seven months. Three to seven ED nurses and physicians participated in each meeting. Principally, all ED providers were invited to take part. Decision for participation was completely at the discretion of each ED provider. Practically, the majority of participants consisted of staff members who were on duty on days of respective meetings; for nurses, usually before or after their shift; for physicians, mostly in breaks during their shift. Therefore, participants varied considerably across individual meetings. Participation in health circles was considered work time. Additionally, ED nursing management and the hospitals’ workers council (German: ‘Personalrat’; employee representatives being elected by hospital employees) attended. All meetings were moderated by the study team. In the first meeting, the concept of health circles was introduced. Potential advantages of this approach (i.e., tailored to local needs, employee involvement, intervention process adapted to context) as well as potential problems (e.g., insufficient implementation of solutions, organizational constraints, time-consuming and long-lasting process of structural re-organization) were discussed. Afterwards, participants identified adverse ED work conditions in a systematic process facilitated by moderators: Participants classified problematic work conditions according to their practical importance and potential for change and formed an agenda of issues for improvement for subsequent meetings. Each of the following meetings focused on the development of measures for one of the identified work problems. In these guided health circles, participants collectively analyzed and discussed potential solutions for work problems and defined an action plan with concrete measures, persons responsible, and deadlines [19]. Two meetings were designated to evaluate implementation status of measures and to discuss potential adjustments. Each health circle meeting was documented in written form and made available to the entire ED staff through the intranet information system. Additionally, participating nurses and physicians were assigned to report on health circle meeting outcomes in their regular team meetings.

After each health circle meeting, participants provided short feedback on their satisfaction with and effectiveness of the respective meeting. We used a self-developed tool that measured five aspects: (1) participant’s satisfaction with meeting outputs (“I am satisfied with the results of today’s health circle”), (2) meeting atmosphere (“I am satisfied with today’s group atmosphere”), (3) opportunities to actively develop solutions for work problems (“In this health circle I can actively contribute to improvements of our work situation”), (4) motivation to improve work conditions (“Today’s health circle meeting motivates me to improve processes and contents of my work”), and (5) participant’s expectations of actual implementation of developed solutions in everyday work (“The developed solutions can be implemented in our daily workflow”). All questions were answered on five-point Likert-Scale ranging from 1=“no, not at all” to “5 = yes, exactly”.

In addition to health circles, three meetings of a steering committee were held during the intervention period. These meetings comprised ED management (ED head, head nurse), the hospitals’ work council and health promotion department, and head of ED administration. The steering committee discussed measures developed by ED staff which could not be implemented immediately, needed approval from ED management, or affected inter-departmental or hospital-wide coordination and decisions [15]. In each meeting, the committee reviewed the action plan, monitored project status, and discussed measures developed in health circle meetings with regard to their implementation in routine work organization and processes.

First, to identify proposed intervention effects, changes in provider survey results between baseline and follow-up were calculated using paired t-tests. To assess strength of changes, Cohen’s effect sizes were calculated and classified as weak (0.2–0.4), moderate (0.5–0.7), or strong effects (> 0.8) [37]. Second, ITS data of hourly interruption rates from work observation sessions, work stress reports, and patient perceptions of care were aggregated to mean scores at the day-level. Data was analyzed with segmented regression analysis with 40 available data points, i.e., 20 each pre- and post-intervention [21]. A daily ED workload measure was calculated from mean daily staffing levels and number and acuity of patients as indicated by ESI scores [38]. Autoregressive integrated moving average (ARIMA) models were estimated and controlled for ED workload [39]. In all steps, listwise deletion was used for missing data. All quantitative analyses were conducted with SPSS 24 (IBM, Chicago). Interview data was analyzed applying content analysis. All nine interviews were audio-taped and transcribed verbatim to cluster recurrent main themes [40].

In the first health circle meeting, participants identified six adverse ED work conditions: (1) lack of personal breaks (i.e., regular work breaks and personal pauses were often omitted due to high workload); (2) work agreements (i.e., unspecified agreements on patient care and tasks that hinder fast and efficient care); (3) high work pressure environment: point of triage (i.e., ED’s location where incoming patients are assessed for the severity of their symptoms based on a standardized process), (4) leadership, (5) staff information (i.e., insufficient information on current reorganizational projects), and (6) staff shortages (i.e., sustained understaffing of ED personnel). Thirteen respective measures were developed focusing on improvements in task-related and organizational work factors (see Table 1). Implementation fidelity varied by the time of follow-up: Eight solutions were implemented or in progress. Five were deemed unfeasible by the steering committee due to financial constraints and personnel shortages and were not pursued.

At baseline, 170 provider surveys were distributed whereof 76 were returned (response rate: 44.7%). At follow-up, 73 out of 157 surveys were returned (46.5%). Forty-one ED providers participated at both waves (29 ED nurses, 5 ED physicians, 7 ED administrators). Tests for panel attrition (between those who returned a complete survey both times and those who only answered at baseline or follow-up) indicated that the final sample reported higher professional tenure, higher ratings of work overload, and higher depersonalization (see Additional file 1: Table S2).

Overall, 160 observation sessions (80 each at baseline and follow-up) were conducted, resulting in 240 observation hours: 99 with ED nurses and 61 with ED physicians. One hundred fifty-six work stress reports were collected after observations (76 at baseline, 80 at follow-up).

Altogether, 1418 ED patients were surveyed; 694 at baseline (survey response rate: 69.2%) and 724 at follow-up (81.2%).

At baseline, respondents reported high levels of patient stressors, work overload, information problems, and uncertainty. Participation opportunities, personnel resources, and supervisor feedback were rated as below average (see Table 2). Considering mean changes over time, job control significantly increased (p = 0.01). Mean weekly overtime significantly decreased from 7.3 to 5.8 h (p = 0.01). Supervisor feedback improved although this change was not significant (p = .058). However, social support deteriorated at follow-up (p < 0.01). Considering Cohen’s delta, effect sizes for changes in work factors were rather weak ranging from Δ = .31 (social support) to Δ = .42 (overtime) (see Table 2).

No significant changes were observed in mean daily ED workload, workflow interruptions by patients, and respondents’ time spent in personal breaks (see Table 3). Intra-professional interruptions (e.g., nurse interrupts nurse) decreased before the intervention (β = − 0.1, p = 0.04), whereas inter-professional interruptions (e.g., nurse interrupts physician) significantly increased after the intervention (β = 0.1, p = 0.03). Mean interruptions by relatives decreased after the intervention (β = − 0.7, p = 0.03).

At baseline, 61% of respondents reported high emotional exhaustion and 22% depressive symptoms above cut-off. At follow-up, the proportion of ED respondents with reported emotional exhaustion (75.6%) and depressive symptoms increased (34.1%). Both trends were not statistically significant. Depersonalization significantly increased over time (p = 0.01; see Table 2). Respondents further reported less job satisfaction (p = 0.01) and higher turnover intentions at follow-up (p < 0.01). However, mean daily work stress did not change significantly (see Table 3).

Respondents’ reports of the frequency of medical errors and overall ED patient safety remained stable over time (see Table 2).

However, significant changes in patient-perceived quality of care were observed (see Table 3). At follow-up, patient evaluations of ED organization improved (β = 0.02, p = 0.02). Further, ratings of waiting time declined before the intervention (β = − 0.02, p = 0.01), but improved significantly after the intervention (β = 0.03, p = 0.01). Concerning interaction with ED providers, a negative pre-intervention trend was observed (β = − 0.01, p = 0.045), however, no significant subsequent changes were identified. Patient’s overall satisfaction with ED care remained stable at a high level. Daily ED workload negatively predicted overall patient satisfaction (β = 0.1, p = 0.02), such that patients were less satisfied with overall quality of care on days with less favorable patient/provider-ratios.

Forty-one surveys were collected after health circle meetings. Participants indicated high satisfaction with meeting outputs (mean, M = 4.2; standard deviation, SD = 1.0), meeting atmosphere (M = 4.7, SD = 0.5), and opportunities to actively develop solutions for work problems (M = 4.1, SD = 0.7). Participants also reported high motivation to improve work conditions (M = 4.0, SD = 1.0). However, developed measures were deemed only partially realizable in everyday work (M = 3.3, SD = 1.1).

We established a mixed-methods ITS study design that allows robust inferences concerning prospective changes of outcomes between pre- and post-intervention assessments [21]. However, pragmatic improvement studies in dynamic clinical settings comprise multiple limitations. First, although our study setting features a typical urban ED setting and relies on elaborated analyses, it lacks a control group. This limits inferences concerning causation as well as secular trends. We describe a realist approach that aimed to change ED work factors. This comprehensive approach targeting several task- and organizational-level aspects over one year does not allow for attribution of effects to single interventions or steps. Our participatory approach consists of several interventions of collective intertwined initiatives that occur in the course of a multitude of everyday concurrent events in patient care [17]. Therefore, we cannot attribute effects of specific measures to primary and secondary outcomes nor specify time lags of measures being effective. Second, our results strongly depend on local contextual factors and the process of intervention implementation [35]. In the study period, other process changes occurred, i.e., reorganization of the triage process and preparation of constructional expansion. ED providers thus might have perceived limited capacity to engage with intervention measures on top of high daily workloads. Nevertheless, our approach combined quantitative results with qualitative information from stakeholder interviews to shed light on these potentially relevant facilitators and barriers in intervention implementation [36, 46]. Third, we acknowledge the rather small longitudinal sample of provider surveys. Although high commitment in data collection and recruitment was undertaken, follow-up bias occurred. This might partially be due to high staff turnover rates which are generally characteristic of EDs [3]. To offset limitations of provider ratings which are prone to subjective bias, we further used objective and independent methods such as expert observation sessions, patient surveys, and register data to measure study outcomes. Yet, observations were confined to day-time shifts which limit inferences concerning provider workflow routines during evening or night shifts [49]. The patient survey tool has been previously applied in various populations including ED patients where it proved its reliability [6]. Specific investigations into the validity of the instrument for ED patient surveys are not yet reported. Fourth, our timing of follow-up assessment needs careful consideration [50]. For practical reasons, we used a one-year time lag. However, ongoing improvements might have failed to reach their full impact on work system factors and provider well-being at the time of our follow-up measurement [17, 51]. Finally, we acknowledge that ED work systems comprise multiple factors and that system interventions should comprehensively address various components to improve provider and patient outcomes [13, 17, 42, 52]. Yet, although preliminary evidence points to positive effects of comprehensive organizational-level interventions for provider outcomes [17], simultaneous effects for patient care need to be elicited. In our study, solutions concerning other domains such as changes in technologies, tools, or environmental factors, were considered by ED providers in meetings but not prioritized for implementation. This refers particularly to resource constraints such as understaffing or structural provisions of the physical environment. Although we carefully introduced the intervention approach to providers and sought to manage stakeholder expectations in the beginning of the study, post-hoc, we cannot infer on specific anticipations or ‘implicit theories’ [36] of involved stakeholders, e.g., employee assumptions that hospital management would provide additional resources or prioritization of staff shortages.

Implementation of organizational-level interventions is time-consuming and evaluation of intervention effects is challenging [53]. We used a multi-disciplinary intervention that included both ED nurses and physicians in collaborative meetings and implementation of solutions. Yet, this partly resulted in perceptions of imbalanced involvement, efforts, and contributions of both professions. Future attempts should therefore seek opportunities to implement interventions that take account of the multi-disciplinary nature of ED work as well as consider unique expectations and needs of each profession in the course of participatory work design in clinical care [3]. Concerning methodological aspects, future studies should consider applying cluster-randomized and controlled designs across various ED settings as well as realist evaluation for intervention evaluation [46]. Varying follow-up measurement intervals should be considered to capture potentially time-delayed intervention effects [50]. With regard to intervention content, future studies should expand the scope of assessed work system factors as well as their differential effects on processes, provider and patient outcomes [14]. Furthermore, ED practitioners could expand or adapt existing tools and practices in performance management to include continuous improvement of work system factors [18]. Finally, although our intervention approach was well-accepted by ED providers, partial improvements in work conditions and patient evaluations of care were accompanied by deteriorations in provider mental well-being. Future studies of work system interventions should thus elucidate beneficial concomitants of provider well-being before and during intervention implementation in high stress care environments.