Operating Room Performance Optimization Metrics: a Systematic Review

The OR comprises a complex environment with multi-layered social interactions, unpredictability and a low tolerance for mistakes [3]. Irregularities in the workflow are often triggered by a combination of factors such as demanding caseloads, pressure to perform complex tasks and conflicting priorities. This can result in increased mental strain and stress amongst the healthcare professionals [4].

Irregularities in workflow on the OR also impact patients. Approximately 60% of the patients visit the OR at some point during their hospital stay [5]. Undergoing hospital admission and an operation makes many people experience emotions such as nervousness, agitation and uncertainty. Irregularities in the process can worsen this [6].

Accounting for about 35% to 40% of the costs, the OR is a large contributor to a hospital’s finances, also being one of the most costly units [7‐9]. Over the past years, healthcare costs have increased and diminishing returns have prompted healthcare administrators to alleviate institutional costs through reductions in budget allocations [10].

Partly driven by the increasing demands for care on the one hand, and constrained resources on the other, a technological evolution has taken place over the last decades. This has played an important role in the development of surgery and resulted in dramatic changes in working conditions within the OR [11]. But healthcare professionals are not always prepared for this transformation of their work. Healthcare professionals are reported to lack preparation for radical (technical) changes in their work [12].

Despite the growing influx of new healthcare professionals, the sector experiences a major exodus of healthcare professionals. Causes are the heavy workload and a lack of autonomy. The limited autonomy of healthcare professionals in their daily work is appointed as a long-standing issue [12]. Research from the Dutch doctors organisation De Jonge Dokter has interviewed 622 young doctors about their work. About 50% of the interviewees has thought about quitting their job due to high work pressure, emotional pressure and working culture [13].

The impact of the workflow of the OR on patients, pressure on healthcare professionals that work on the OR, the vast changing work environment and economical constraints make that optimizing the OR is high on the academic agenda. However, the high expectations of patients, interactions between different professionals, unpredictability and complex surgical case scheduling make managing and changing the system difficult. Attempts to resort to commonly used industrial principles to increase factors such as efficiency have been demonstrated to easily fail due to these (and possibly other) particular characteristics of the OR [7]: human factors have too great of an impact to standardize and automate certain OR processes. Another complicating factor is the divergent perspectives on OR performance optimization.

The metrics used to quantify OR performance optimization reported in literature are diverse [14]. Many articles focus on the efficiency aspect of OR performance optimization, some focus more on the quality aspects. For example, the work of Bellini et al. [7] speaks of the optimization related factor efficiency in the sense of more precise scheduling and limiting waste of resources. Costa Jr. et al. (2015) speak of both efficiency and optimization, hereby focussing on resources and time management. Sandbaek et al. [16] refer to OR efficiency as maximizing throughput and OR utilization while minimizing overtime and waiting time, without additional resources. Tanaka et al. (2011) assesses OR performance using indicators such as the number of operations, the procedural fees per OR, the total utilization time per OR and total fees per OR. Rothstein & Raval [3] refer to the metrics of OR efficiency based on the Canadian Paediatric Wait Times Project: off-hours surgery, same-day cancellation rate, first case start-time accuracy, OR use, percentage of unplanned closures, case duration accuracy, turnover time and excess staffing costs. Alternatively, Arakelian, Gunningberg and Larsson (2008) emphasize that apart from cost-effectiveness, work in the OR should be organized to fulfil the demands on patient safety and high-quality care. From their perspective, OR departments must create efficient ways of planning and processing the work, while at the same time maintaining the quality of care. These authors also show that there are diverging perspectives among OR personnel on what efficiency and productivity entail.

The previous paragraphs illustrate that when speaking about OR performance optimization, different terminology is being used. Furthermore, although many studies focus on how to optimize or monitor certain aspects of the OR, studies on the impact of these changes on the quality and efficiency of the hospital as a whole appear to be lacking. This may lead to uncertain optimisation strategies that are difficult to substantiate with supporting evidence [15]. Table 1 summarizes both quality and efficiency aspects of OR workflow and strives to align the methods and metrics to assess OR performance in terms of 1. Patient safety 2. Quality of care 3. Cost-effectiveness and 4. Healthcare professional well-being.

An inventory of the topics of the articles was made by filtering out 1. the focus/aim of the study, 2. the method used and 3. the conclusion. Optimization strategies in other hospital departments might be transferable to the OR as well. Therefore, to gain insight in the distribution of optimization strategies on the different departments of the hospital, the articles were analysed by labelling the operational department the research is focussed on, which topic was investigated, and which method was used. After creating this overview, only data about the OR was used. To obtain OR performance metrics a second analysis was conducted: OR performance characteristics from the articles were split into aspects with their corresponding metrics.

Figure 1 shows the search engines, search terms and number of papers found.

Based on Table 1, the characteristics of OR performance (efficiency and quality) have been split up into aspects and were then further specified into metrics (Table 3).

Table 4 shows the categories used to analyse the articles, the corresponding labels and their names. In Appendix 1, Supp. Tab. 1, the topic category and their corresponding sources are presented.

Appendix 2, Supp. Figure 1, shows a sunburst graph that illustrates the distribution of the labels per department (D_x). To give an overview that represents the distribution of departments in a hospital, only the data from the second search criteria (Workflow AND Optimization AND Hospital) was included in this graph. The inner circle contains the different departments, namely the OR, ER (emergency room), outpatient clinic, patient clinic and the hospital in general. The middle circle shows the corresponding methods, the outer circle shows the topics. Most articles focus on the OR (D_1, N = 16). The ER receives less attention (D_2, N = 2). There were no articles that were labelled for outpatient clinic (D_3).

In Appendix 2, Supp. Figure 2 zooms in on methods and corresponding topics of just the OR. This graph includes all OR data from both search criteria and shows the methods, topics and number of articles in each category. In Fig. 2, a selection of the OR sunburst graph is displayed. This selection contains the most frequent combination of period, method and topic (N ≥ 3). Most articles have the aim to optimize scheduling (N = 7), workflow tracking (N = 5) and patient flow (N = 4) by computational means such as machine learning.

All data was then stratified by the means of a bar chart. Figure 3 shows the data while looking at different methods per topic. Computational methods (M_8) are used most frequently (N = 41).

The methods that were used the least are experiments with the medical staff (M_7, N = 2) and system engineering (M_10, N = 2). The most investigated topics are patient flow (T_10, N = 18), OR scheduling (T_3, N = 17) and workflow tracking systems (T_5, N = 15).

Optimizing the system for a certain metric while also considering the other metrics should be part of the optimization strategy. Practical execution of this strategy is a roadmap with design steps in which the metrics are incorporated. In the following paragraphs this idea will be illustrated with Fig. 5 and an example scenario for an optimization goal.

Figure 5 shows an example of the main steps of a research approach (aim, method, data collection, results, conclusion) with an emphasis on the phase between method and data collection. The approach is based on the Plan-Do-Check-Act method of Deming [28]. In a fictitious scenario the aim of the project is to improve the well-being of healthcare professionals on the OR. This is the first step of the model in Fig. 5. In second step it is determined that the method used to achieve the aim will be increasing the metric autonomy of the healthcare professionals. A questionnaire amongst the staff involved shows that the healthcare professionals would like to have more autonomy over when they work. A more open work schedule is therefore suggested.

In the third step of the model the most important metrics that could be affected by this change are listed by the researchers:

In the fourth step the selected metrics are combined in logical sets as system optimisation metrics with assigned weights and acceptance ranges. As an example, we could look at optimizing OR utilization. In this case the constant consists of the metric off time per OR per month and the utility time per OR per month (see Eq. 1).

Ranges of the constant are then given scores and weights to calculate the optimal value (Table 5). For example, if T* were to be greater than 1, there would be more off time per OR than utility time. That is an undesirable scenario. This range is therefore given a score of -1 and a weight of 2.

When T* is low there is a high utility rate of the OR’s. When T* is high there is a low utility rate. A more complete overview would be created when also plotting financial metrics and metrics concerning the well-being of the medical staff.

After data has been collected in step 5 of the model, the results of step 6 can be compared in step 7. One can then evaluate whether the intended innovation will improve the system in such a way that it is worth the investment. And if not, consider carefully based on the metric overviews where adjustments to the intervention are required.

This study has limitations. A major focus of this paper is the importance of seeing the whole picture when doing research. We have given examples of possibilities to bring this way of doing research into practice. However, despite the broad view of this study, we did not cover all aspects of healthcare. We looked at just the OR. Following our own philosophy, we want to stress that an even broader scope is relevant for successful optimization in healthcare. There is an intricate interplay between the different departments of a hospital. Increasing the efficiency of the OR might, for example, cause trouble in the timetable of the PACU (Post Anaesthesia Care Unit).

An informative topic to investigate further is to test the effect of awareness of metrics when optimizing OR metrics in practice. The hypothesis here is that more awareness of OR performance metrics and their correlations amongst researchers could lead to better optimizing strategies. In this context, the model in Fig. 5 might also be tested. Does it increase awareness? Do researchers use different approaches with the model than without? Does this lead to better outcomes?

Another direction is the continuous measuring of OR performance metrics to be able to monitor unintended interactions, in ways that not put a burden on healthcare professionals (i.e., increasing administrative tasks). Furthermore, technology can speed up and smoothen processes within the OR, but the impact on perioperative processes might not have been considered. An interesting way to put these thoughts to practice is investigating how the increase of technology on the OR has influenced the work of healthcare professionals such as OR nurses and supporting department.