Indicators of intensive care unit capacity strain: a systematic review

The core aim of this review was to systematically evaluate the literature to identify proposed indicators of ICU capacity strain. The specific objectives included: (1) to generate an inventory of quality and performance indicators associated with strain on ICU capacity; and (2) to categorize these indicators of ICU capacity strain at the patient-level, ICU-level, and health system-level across attributes of quality indicators (i.e., importance; scientific acceptability; usability; feasibility) and the Donabedian framework (i.e., structure – where healthcare is delivered; process – how healthcare is delivered; outcome – the effects of the delivery of healthcare).

We developed a comprehensive search strategy in consultation with a research librarian (RF) that was peer-reviewed by a second research librarian [13]. We searched the following electronic databases between August 11 and 24, 2015: Ovid MEDLINE (1946-), Ovid EMBASE (1988-), CINAHL Plus with Full Text via EBSCOhost (1937-), the Cochrane Library (inception-), including the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials (CENTRAL), and Web of Science Core Collection (1900-). We ran update searches in MEDLINE and EMBASE on February 1, 2017. Our search strategy combined the following concepts: (1) intensive care, critical care, critical illness, multi-organ dysfunction, multi-organ failure; (2) quality indicator, quality measure, performance indicator, quality improvement, quality assurance, quality control, performance improvement, best practice, processes of care, complications, adverse event, medication error, safety, effectiveness, efficiency, appropriateness, outcomes assessment, outcome, audit; and (3) strain, capacity, occupancy, census, resource, operations management, acuity, rationing, queuing, avoidable, unplanned, readmission, nighttime, discharge, absenteeism, burnout, workload, discrete event simulation (Additional file 2). Grey literature sources were searched for operations management reports, and selected conference proceedings (Additional file 3). Bibliographic records were exported to an EndNote X7 (Thomson Reuters, Philadelphia, PA, USA) database for screening.

Studies were included if they mentioned all of the following themes: (1) intensive care (i.e., intended to refer to patients (adults, children, and neonates) who are critically ill or at risk for an acute clinical deterioration that may necessitate support in an ICU setting); (2) quality or performance indicator (i.e., any measurable variable intended to evaluate the structure, process, or outcome of care provided to patients); (3) capacity strain (i.e., any measurable variable intended to evaluate the untoward impact at the patient-level, ICU-level, or heath-system level stress on ICU capacity due to the mismatch in demand and supply in our healthcare system). We considered studies published in English and after 1990. Finally, selected levels of evidence included all studies types (i.e., abstracts and full texts) reporting original primary and/or secondary data, as well as administrative reports to government or healthcare agencies.

We used a two-stage process for study selection [14]. First, two reviewers (OGR and SMB) independently screened the titles and abstracts (when available) of search results to determine whether a study fulfilled the general inclusion criteria. Disagreements were resolved by discussion. The full-text versions of all citations classified as “include” by either reviewer were retrieved for in-depth review. The same two reviewers (OGR and SMB independently assessed the eligibility of each full-text manuscript for final inclusion into the review. Any disagreement was resolved by discussion.

Two independent reviewers (OGR and SMB) extracted data using standardized, piloted, data extraction report forms. All strain indicators were identified, abstracted, and agreed upon by the two independent authors (OGR and SMB). The following data were abstracted from each citation: author identification, year of publication, title, journal of publication, study design, identified strain indicator(s), along with the operation definition and the relevance of each strain indicator.

Each strain indicator was characterized on its importance, scientific acceptability, usability and feasibility, as similarly performed previously by the authors [14]. Initially, 20% of retrieved citations (n = 20) had strain indicators described in duplicate to ensure consistency. Due to significant redundancy in strain indicators, the remainder of citations was extracted by a single reviewed (OGR) [14]. If there was uncertainty, strain indicators were reviewed in duplicate and consensus on its characteristics were achieved through discussion. Each strain indicator was stratified (yes/no) according to whether study authors described it as being importance (i.e., ICU quality, patient-centeredness, healthcare costs); described its scientific basis and rationale; and whether it was operationally usable and feasible (i.e., easy to indicator or implement; able to be integrated into an electronic health record [EHR]).

We assessed the internal validity of included studies using the Newcastle-Ottawa Quality Assessment Scale (NOS), where studies are scored 0 to 9 based on quality [15]. Studies were rated high quality if they had a total score of 6–9, moderate quality with a score of 4–5, and poor quality if they had a score of 3 or fewer [14].

The primary analysis was descriptive. Strain indicators were categorized according to the Donabedian framework by stratifying whether each indicator represented a structure, process, or outcome related to ICU capacity strain [16]. Strain indicators were further evaluated by OGR and SMB using the four criteria proposed by the US Strategic Framework Board for a National Quality Measurement and Reporting System, as outlined above [17].

Our initial search strategy identified 5297 citations, of which 51 articles satisfied eligibility criteria and were included (Fig. 1). Of these, 40 were full-text articles and 11 were presented in abstract form only (Additional File 4). The majority were cohort studies (n = 39; 76.5%), five (9.8%) were case-control, three (5.8%) were cross-sectional, two (3.9%) were modeling studies, one (2%) was a correlational study, and one (2%) was a quality improvement project (Table 1). Most studies were research in nature (n = 43, 84.3%), while the remainder where quality improvement projects (n = 8, 15.7%).

The mean Newcastle-Ottawa quality score was 8.5 (range 4–9). The majority of full text studies were high quality (n = 36, 90%), while a few were moderate quality (n = 4, 10%); with no observational studies rated as poor quality. Of the remaining studies identified (n = 11, 21.6%), quality assessment was not possible due to insufficient data as studies were only available in abstract form.

A total of 16 potential strain indicators were identified 110 times in our included citations, and classified as structure (n = 4, 25%), process (n = 7, 44%) and outcome (n = 5, 31%) indicators, respectively (Table 2). The most commonly identified strain indicators were ICU acuity (n = 21; 19.1% [process]), ICU readmission (n = 18; 16.4% [outcome]), after-hours discharge (n = 15; 13.6% [process]) and ICU census (n = 13; 11.8% [structure]). There was substantial heterogeneity in the operational definitions used to define strain indicators across studies (Table 2). Strain indicators were also stratified across the six dimension of healthcare quality: safety (n = 4; 25%); effectiveness (n = 3; 19%); patient-centeredness (n = 1; 6%); timeliness (n = 3; 19%); efficiency (n = 4; 25%); and equitability (n = 1; 6%) [18].

Features of these potential strain indicators, as discussed by study authors, generally focused on importance as key ICU performance or quality indicators (n = 95; 74%), followed by scientific acceptability (n = 23; 24%) and then by usability and feasibility (n = 11; 12%) (Table 3). Importance was further stratified across specific elements of capacity strain, including importance to patient-related outcomes (n = 47; 55%), importance as an indicator of ICU operations and organizational planning (n = 42; 44%), and importance to healthcare costs (n = 6; 6%). None of the studies presented data regarding the reliability or validity of the performance of the indicators.

First, we identified 16 potential indicators of strained ICU capacity that encompassed all three domains of the Donabedian framework and quality dimensions. Second, we found that the operational definitions for strain indicators across studies were substantially heterogeneous. For example, we found 110 examples of indicators in our retrieved studies; and due to varied definitions, these were subsequently consolidated into related themes. Third, few studies had evaluated the scientific acceptability, usability or feasibility of the proposed strain indicators. Similarly, none of the studies identified specifically described the reliability or validity of the performance of these potential indicators of strain. Fourth, the most common strain indicators identified included indicators of ICU acuity, ICU readmission, after-hours discharge, and ICU census/occupancy. Notably, these strain indicators have considerable overlap with commonly recommended key indicators of ICU performance (KPI) [19, 20]. Finally, we also identified several indicators that while perhaps also analogous with some ICU KPIs, may also be suitable to characterize strained capacity conditions. These specific structure (e.g., queuing; nurse-to-patient ratios), process (e.g., bed turnover; workload; refusal rate) and outcome (e.g., healthcare professional burnout; surgery postponements) indicators could be evaluated over both the short-term and intermediate-term to provide holistic data on contributors to and effects of strained ICU capacity.

Population growth, advances in medical science and improved capability to support critically ill patients have all translated into a sustained and rising demand for critical care services [21]. This increased demand; however, cannot be universally accompanied by an increased supply of ICU resources, which are costly [22]. The costs of expanding ICU extend beyond the “ICU bed” per se and necessitate considerable investment in human resources, specialized equipment (i.e., mechanical ventilators), and supplies to sustainably operate. Moreover, the supply side of ICU services is not standardized and are highly inconsistent across jurisdictions for reasons that are not based on evidence-informed scientific assessment of need [23, 24]. Regardless of the reasons, mismatches between demand and supply for ICU services are increasingly encountered [7]. This mismatch on any given day in any given ICU will create strain on that ICU’s capacity to accommodate the next critically ill patient. Strained ICUs are at risk of providing suboptimal quality of care, and risk higher rates of adverse events, premature discharges, unplanned readmissions and death [4, 25‐28]. Arguably, sustained strain contributes to inefficient and inequitable utilization of finite ICU resources. While opening additional ICU beds may seem the simplest response, this is not necessarily sustainable, and likely only a short-term reprieve [29]. Rather, in order to explain the breadth of effect strain may have on patients, professionals and operations, a constellation of evidence-informed quality indicators are likely to be necessary.

Many of the strain indicators identified in our review are highly correlated with commonly cited ICU KPI or quality indicators [19, 20]. However, we contend many of these indicators have not been specifically evaluated in the context of strained operating conditions and capacity. Moreover, many will exhibit context-specific variation and require local or regional evaluation. Commonly described indicators of strain capacity, particularly occupancy/bed availability, acuity, and numbers of new admissions have repeatedly shown association with adverse outcomes, deviations in care processes, and changes in resource use [3, 5‐7, 9, 10, 30]. However, it remains uncertain whether these alone may completely capture the spectrum of effects for how strain may manifest and exert its effects across heterogeneous ICU settings. Rather, they may put focus on queuing rather than on demand and the capacity to manage high flow and increased patient turnover, such as in specialized ICUs (i.e., cardiac surgery) [31]. Complementary indicators of strain may be important in such circumstances.

ICU readmission has been recognized as a potential indicator of strained ICU capacity [10]. While ICU readmission has been endorsed as a standard KPI for ICUs [19, 20]; there is uncertainty to its validity [32]. In a retrospective cohort study, Wagner et al. found greater strain, defined by average census, number of new ICU admissions, and ICU acuity, was associated with a shorter ICU stay and a small but significant increased risk for ICU readmission following discharge [10]. However, among those readmitted, there is uncertainty whether these events were avoidable or attributable to premature triage in response to strained conditions. Recent observations suggest the majority of ICU readmissions are not preventable or attributable to causal actions or omissions of the ICU team [33]. Further evaluation of ICU readmission as a potential indicator of strain should aim to integrate additional indicators of strain and adjudication of whether ICU readmission was avoidable.

While our rigorous review identified 51 studies that characterized a spectrum of indicators of strained capacity across a range of quality dimensions, some of which are likely routinely measured or should be implemented by ICUs, our findings should be considered in the context of the following limitations. The identified indicators across studies were variably defined, were often not the intended primary exposure, and the operational characteristics were incomplete described (e.g., elements identified in the US Strategic Framework Board for a National Quality Measurement and Reporting System) (Table 3). Further refinement and streamlining of definitions will be needed prior to operationalizing these potential identified strain indicators. Furthermore, as for the abovementioned reasons, pooled analyses across “strain indicators” were not feasible.

The challenge for healthcare professionals is to clearly understand when and to what extent strain is negatively impacting their decision-making, the quality of care provided and the performance of a given ICU, and to readily identify and respond to factors most responsible. Healthcare professionals should be particularly mindful of the influence strained ICU capacity may have to modify behaviours and care processes [5‐7]. There is uncertainty on how best to measure strained capacity. While simple objective indicators such as ICU census/occupancy are strongly perceived to indicate strain [34], the spectrum of how strain can manifest is likely far most complex and it is unlikely that any single indicator will satisfy all the potential domains from which strain may originate. Moreover, as a consequence, there may be limited appreciation for the collateral effect of strained capacity, such as a negative impact on ambulance offloads, emergency department crowding, and postponements of elective surgery.

Simple, translatable and easily reportable indicators are needed to identify and quantify strained capacity. Widespread implementation of electronic health records (EHR) and data repositories/registries have enabled the routine calculation and reporting of standard ICU KPIs [3]. Such infrastructure could be re-orientated to generate a concise dashboard or index consisting of strain indicators across multiple quality domains (Table 4). For example, 13 of our proposed indicators could readily be integrated into a strain dashboard. Additional potential indicators, including healthcare professional burnout (i.e., attribution, absenteeism, overtime), workplace satisfaction, patient-family satisfaction would require additional resources to integrate. We contend this is a fundamental step towards developing innovative quality improvement interventions aimed to improve safety (i.e., care processes, adverse events, nosocomial infections), effectiveness (i.e., SMR), timeliness (e.g., access, queuing), efficiency (e.g., flow, avoidable ICU days), and to better equip ICUs to anticipate and manage strained capacity. Future work should aim to evaluate the feasibility of implementing a constellation of strain indicators, evaluate the association of strain indicators on patient-centered and health resource outcomes, including impact on additional KPIs, establish threshold for optional strain indicator definitions and benchmarks (Table 5), recognizing some may be geographically and context-specific. Finally, future work should consider the development of indicators that try to capture the match between the demand and capacity, with the goals of having ICUs operate in an ideal range to limit both strain and healthcare inefficiencies.