Delphi study to derive expert consensus on a set of criteria to evaluate discharge readiness for adult ICU patients to be discharged to a general ward—European perspective

A growing group of elderly, more fragile and multimorbid patients will increase the future need for intensive care capacities. To meet this demand, adding more intensive care unit (ICU) beds is often not an option due to financial limitations and the lack of specialized and highly skilled care givers to staff the beds. Therefore, optimizing the utilization of given ICU resources is a high priority for hospital management to avoid bottleneck situations. Capacity strain is often created by pending discharges and flow delays which could account to 15–25% of the total ICU length of stay (LOS) [1, 2]. On the other hand, high census levels can cause premature discharges resulting in unstable patients at lower levels of care together with an overestimation of the receiving ward capacities. That ultimately results in readmissions and even increases capacity strain, LOS, mortality and costs [3‐5]. Optimal patient outflow can be achieved by identifying those patients early that are stable enough to transition to the next lower level of care and keeping boarding time to a minimum through close cooperation with the receiving unit [1, 5, 6]. Discharging the right patient at the right time reduces LOS, readmission rates, and costs, where an inappropriate discharge will achieve the opposite and increase risk of mortality [7‐9]. Until today, discharge readiness assessment is often rushed, subjective and untransparent, and based on a limited amount of available aggregated data and decision criteria. The need for a comprehensive proposal of discharge criteria for adult ICUs that is widely applicable in daily clinical practice throughout Europe has been phrased in a variety of studies [5, 9‐13], and is even more relevant today in light of the ongoing COVID-pandemic and extremely strained ICU capacities. But what does it take to identify stable patients timely, minimize their boarding time and ensure safe and efficient care transitions? Basically, the set of criteria for an objective evaluation of patient discharge readiness should satisfy two purposes: First, patient specific criteria such as patient status, interventions and medications, diagnosis and prognosis should indicate a stable state of the patient for at least the next 48 h that allows safe discharge with a minimized risk of readmission. Second, the set of criteria should incorporate system-specific criteria such as nursing workload related criteria at the discharging and the receiving unit, and institutional factors such as available technical infrastructure, skill sets, patient/nurse ratios, protocols and processes.

The absence of such a holistic discharge readiness evaluation tool was the motivation behind this study. The study group condensed evidence-based criteria and recommendations, structured and referenced them in a table format as a first proposal for a set of discharge criteria. This proposal was then subject to a 5-level Delphi study involving a multi-professional panel of European intensive care experts to reach consensus on a standard set of discharge criteria. Providing a consented and standardizable set of criteria for use in daily clinical practice should provide a holistic view to the interdisciplinary care team on individual patient discharge readiness and organizational capabilities. Further, it should guarantee equity in care provision by improving objectivity and comparability in clinical decision making, and increase quality of care transitions and efficient use of ICU capacities in the interest of the patient and the society [14].

With the study aim to reach consensus on a standardized set of ICU discharge criteria, the research group selected a modified online Delphi process. The Delphi method was chosen as it is a suitable research tool, specifically in areas where there is limited scientific evidence, lack of agreement, incomplete knowledge or uncertainty and conclusions are heavily relying on expert opinion [15‐18]. In this case, it should help to build on the limited scientific evidence for established and well-defined ICU discharge criteria that was identified by the previously performed scoping literature review [19]. Further, the Delphi technique had four main characteristics that suited the objective of this study: anonymity between participants through a non-face-to-face format, iteration with controlled feedback of group opinion, statistical aggregation of group responses and expert input [18]. The study was realized via an online voting platform designed for Delphi studies (welphi.com) to include a geographically spread panel of experts and to allow participation in an asynchronous, online, participatory and interactive way, at comparatively low cost and time investment.

Generally, the Delphi study was conducted in three stages: 1.) Scoping literature review, criteria preselection and panelist recruitment, 2.) Online Delphi process (detailed process description in online supplement b, doc. 2), 3.) Conclusion on final results.

In the 1^st stage, the investigators derived a preselection of criteria from the earlier performed scoping literature review [19] that was then reviewed by selected experts for suitability and comprehensiveness. Items, values and ranges were added or edited where applicable. Scientific evidence was referenced per criterion and a first proposal of an ICU discharge criteria checklist was structured by criteria categories (online supplement a, doc. 1, tab. S1). Potential experts were identified based on a combination of proven research activities (topic-related publications in Pubmed and on ResearchGate, topic related congress presentations (ESICM, ISICEM) and through peer recommendation of being a practice specialist in the specific field. In addition, the investigators aimed for an expert panel representing a diversity of European countries and healthcare systems as well as a balance in professions. In this context, the selected and approached experts mainly work in Western European and North American countries with high standards of care, although in different healthcare system settings. Email invitations with background and introduction to the study were sent out to the potential panelists. Upon acceptance to participate, 28 enrolled participants (17 ICU doctors, thereof 2 female, and 11 ICU nurses, thereof 6 female), representing 12 different healthcare systems, received a pre-read document (online supplements a, doc. 1) and their personalized sign-in credentials to access the online Delphi tool.

In the online Delphi process (2^nd stage), expert group consensus was built through an iterative process that used systematic progression through five rounds of voting on questions, statements, or criteria in this case [20]. The individual rounds of the process were set-up based on available literature on Delphi studies in similar settings [15, 17, 18, 21, 22], process reviews with appointed methodologists and the technical support team of the online Delphi platform. The enrolled panelists voted on every criterion for inclusion in a standard set of discharge criteria that is necessary and suitable to evaluate individual patient’s discharge readiness for adult patients in any type of European intensive care setting and not specific to any individual disease process or specialty. The investigators did not actively participate in the online Delphi process, but reviewed, summarized and discussed the results after each round in order to set up the following rounds. Consensus was defined through agreement on proposed criteria by ≥ 90% of the panelists [22, 23]. For the first three rounds, exclusion of criteria from subsequent rounds was defined by reaching < 75% of agreement per criterion with the cut-off value oriented on comparable research [23]. In round 1, agreement for inclusion for round 2 was defined per criterion through not editing / leaving as is, editing, or adding a criterion to the list. The remark “removal” per criterion was counted as disagreement and a criterion would have been removed from the list if ≥ 25% of the experts voted for removal in round 1. In round 2, agreement for further inclusion of a criterion in the list was defined as answering either very relevant or relevant per criterion on the provided Likert scale (5 values: “very relevant”, “relevant”, “cannot judge”, “not relevant”, “completely irrelevant”). Criteria that met consensus of ≥ 90% through answers of “very relevant” or “relevant” were already approved to enter round 4. Criteria that reached agreement between 75 – 89% on being “very relevant” or “relevant”, went into round 3, where experts were confronted with the voting results on group level compared to their own results and outliers had the chance to change their vote towards the groups opinion. In round 4, only those criteria having reached consensus in round 2 and the additional criteria finally having reached consensus of ≥ 90% in round 3, went through further finetuning on criteria importance rank, criteria evaluation time frames, specific values, and calculation method as well who from the care team could best evaluate per criterion if it has been met. For round 5, the panelists received a statistical summary report of the group voting on criteria phrasing, importance ranking, evaluation time windows, criteria calculation, and preferred decision makers per criterion. Further, they were asked to agree / disagree on the final criteria phrasing, value calculation method and criteria importance ranking. The aspects “criteria evaluation time frame” and “who could best evaluate if a specific criterion has been met” where excluded from further voting due to the heterogeneity of the round 4 results (online supplements c, results round 4). Based on the voting results and provided comments, the investigators derived the final list of discharge criteria, shared the results with the expert panel and prepared the study manuscript for publication. An overview of the set-up of the five rounds is given in Fig. 1.

In the 3rd stage, based on the voting results of the 5th round and the provided comments, the team of investigators concluded on the final list of 28 ICU discharge criteria with their binary decision metric (values for Fit for ICU discharge and Needs further intensive care therapy / monitoring). Where appropriate—the preferred value calculation method to evaluate discharge was proposed, and a criterion importance ranking was provided with exceptions when the mandatory to be met—requirement could be overruled.

Of the 54 experts (33 medical doctors and 21 nurses) approached, 17 doctors (2 female) and 11 nurses (6 female) agreed to participate. One participant left the panel after the second round. The final panel represented 12 different healthcare systems (11 European, one Canadian) (online supplements c, fig. S1) and brought in experience from different ICU types and sizes and professional backgrounds: More than half of the participants worked in mixed ICUs (mixed ICUs (18), medical ICUs (2), surgical ICUs (3), other (1), no info (4)). Different ICU sizes were equally represented (4–12 bed ICUs (8), > 12 – 24 bed ICUs (7), > 24 bed ICUs (8), no info (5)). Of the entire panel, 16 participants were either senior clinicians (3), directors or heads of department (7) or professors (6, thereof 2 in nursing). More than half of the panelists (17) had > 10 years of work experience in a critical care environment, whereof 9 panelists have worked there even > 20 years (further demographic details and graphical illustration in online supplements c, figs. S2, S3, S4 and S5). Participation with completion of the entire survey per round ranged from 78–92% completion rate over the five rounds of voting (R1: 92%, R2: 85%, R3: 78%, R4: 85%, R5: 85%; for further details see online supplements c, tab. S2). The online Delphi process was conducted between June 24^th, 2020 and March 21^st 2021. Results per round are shown in Fig. 2.

The detailed results per round with voting split, provided and edited comments are given in online supplements c with results description per round (doc.3) and results tables per round (doc. 4, tab. S3: results round 1, tab. S4: results round 2, tab. S5: results round 3, tab. S6: results round 4, tab. S7: results round 5, tab. S8: results round 5, tab. S9: phrasing comparison round 5 and final list, tab. S10: final ICU discharge criteria list). And the end of the Delphi process, 28 ICU discharge criteria reached final consensus, thereof 23 criteria reached consensus ≥ 90% and 5 criteria reached consensus of 75 – 88% (tab. S10). For 5 criteria a deletion from the list was agreed with ≥ 92% consensus, because these criteria became obsolete as aspects were finally covered through other rephrased criteria (tab. S8). Based on comments received in the 5^th round and discussion among the team of investigators, 8 criteria got changed in phrasing for simplification for the final criterial list although they have all met already the consensus threshold (tab. S9). The set of proposed ICU discharge criteria for adult patients in any type of ICU covers patient-specific as well as organization-specific discharge criteria to evaluate discharge readiness of the individual patient but also to assess organizational capabilities to allow patient discharge to the next lower level of care (general ward in this study context). 24 patient-specific criteria reflect on the different organ systems (respiratory system, cardiovascular system, central nervous system, urogenital system), pain, fluid loss and drainages, medication and nutrition, patient diagnosis, prognosis, and patient preferences. 8 out of the 24 patient-specific criteria evaluate as part of their criteria phrasing whether currently available capacities and competencies as well as available technical infrastructure at the receiving unit are in place to safely discharge the patient. 4 criteria focus on the institution specific boundary conditions that allow or don’t allow patient discharge (institution’s specific admission criteria of receiving unit, safety standards such as isolation or support measures for out of hours discharges, and current capacity, acuity, and workload levels at the receiving unit). For every criterion a binary decision metric was defined with values for “Fit for discharge” and “Needs further intensive care therapy / monitoring”. For consistency and ease of use in daily clinical practice, all criteria have been phrased in a way that they can be answered with the values “yes” or “no” in the binary decision metric. For 6 criteria concerning the patient’s vital parameters, a value calculation method has been defined: For “blood oxygenation”, the worst value must be above the defined threshold and the trend must be stable over a defined time frame. For “respiratory rate”, “heart rate” and “mean arterial pressure”, the worst value must be within the acceptable range and the trend must be stable over a defined time frame. For “cardiac rhythm” and “hemoglobin” the trend must be stable over a defined time frame. For a definition of an appropriate value evaluation time frame per criterion that indicates stability, thus preventing patient readmission within 48 h after ICU discharge, agreement within one of the proposed time frame categories was low. 42% has been reached as a maximum agreement level in only one criterion, followed by 2 criteria reaching 38% and 8 criteria reaching 33% agreement in one category. All other criteria reached lower agreement levels per proposed evaluation time frame category. Based on the heterogeneity of the answers, the investigators decided to take this aspect out from any further voting round as they didn’t expect reaching a significantly higher agreement level within the following two rounds. As a result, no specific criterion evaluation time frames can be proposed. However, as the phrasing of the value calculation method refers to a defined time frame, for implementation in daily clinical routine, this time frame needs to be specified once by the clinical decision makers and in context of the institution specific workflows of vital parameter measurement, documentation, and visualization. The votes distribution per criterion could serve as a first orientation.

Also, the question in the fourth round, who of the stakeholder group can evaluate best if the patient meets a criterion, resulted in very heterogenous results. Only for 9 out of 33 criteria an agreement of ≥ 70% was reached on one stakeholder. A general insight from the panelists’ comments and the voting distribution was, that a lot of criteria need to be evaluated by an interdisciplinary team, that often should even involve a clinician or nurse from the receiving unit. In the performed survey, there was no selection option for the interdisciplinary team. That is why panelists commented via the comments field on the need to have an interdisciplinary team (13 related comments). Based on these results, it was decided to not further iterate on the decision maker. The investigators would rather recommend for criteria implementation in daily clinical practice to define the best decision maker(s) per criterion based on institution specific roles and collaboration aspects. Based on the comments, for some criteria certain roles can equally assess discharge readiness, often depending more on competency and experience level than on the specific role. An interdisciplinary team including representatives from the receiving unit should be consulted for criteria that reflect on patient specific criteria in context with capabilities of the receiving unit. For clinical practice implementation, it is also important to know the importance rank of each criterion, meaning if it is mandatory to meet the criterion or not and if there are exceptions in place that allow to overrule the criterion. After some simplification of the ranking, 17 out of 28 criteria reached > 90% consensus on “mandatory to be met”. Eleven criteria reached 73%—89% consensus on “mandatory to be met”. End-of-life care and agreed treatment limits where the exceptions mentioned the most (in 12 criteria), when “mandatory to be met” can be overruled. Looking at the voting alternatives, the highest vote for “not mandatory to be met” with 26% was reached for the criterion “Patient’s preference is to stop intensive care therapy and to leave the ICU”.

Finally, over a course of 5 rounds of voting on a basis of 40 initial ICU discharge criteria, where several criteria have been added, edited and deleted along the way through the panel’s consensus, the study resulted in 28 in-depth defined ICU discharge criteria for adult patients, that should be applicable to any type of ICU (Table 1).

The objective of this Delphi study to reach consensus on a standardized set of ICU discharge criteria has been achieved. In the voting process, we involved 28 clinicians and nurses with a dedicated expertise and research interest in ICU transfer processes, coming from divers geographical and professional backgrounds. That allowed us to get a comprehensive evaluation of the provided first proposal of ICU discharge criteria that we derived from our previous literature research.

From the beginning, patient as well as process-related conditions were in focus of the consensus process. As phrased in earlier studies, only the two perspectives together could give a holistic view on discharge readiness for an individual patient to a particular next lower level of care [5, 9, 12]. Patient-specific criteria should indicate a stable state of the patient for at least the next 48 h that allows safe discharge with a minimized readmission risk [24, 25]. Organization-specific aspects affect the quality and success of care transitions from an ICU to a lower level of care and should reflect on current ICU capacity strain driven by census level and nursing workload as well as the capabilities of the next lower level of care, communication and handover practices between departments, as well as an early definition of patient-centric care goals that are aligned throughout the patient pathway.

The first round of voting was started with two blocks of criteria: the patient-specific block with 30 criteria and the organization-specific block with 10 criteria. The study was successfully ended after the 5th round with 24 patient-specific and 4 organization-specific criteria. Patient-specific criteria were represented by a holistic view on the different organ systems and therapeutic interventions as well as patient’s autonomy, continuous care needs, the patient’s wish, and therapeutic susceptibility. Organization-specific criteria were aligned institution specific admission and discharge criteria, discharge timing and accompanying safety measures, available technology, and care capacities at the next lower level of care. Throughout the various rounds of voting, it became clear that patient- and organization-specific discharge readiness is deeply intertwined as among the patient-specific criteria, 12 criteria also reflect on the capabilities of the receiving unit in the way they were finally phrased.

In daily clinical practice, there is an absence of evidence-based and well-defined ICU discharge criteria that reflect a holistic assessment on the patient’s fit for discharge status as well as on the organizational capabilities that allow a safe and timely transition to the next lower level of care. A critical care expert panel consented via a modified online Delphi process on a final list of 28 criteria to evaluate discharge readiness in adult ICU patients for a care transition to a general ward environment. The set of criteria covers patient-specific aspects, such as a holistic view on organ systems, therapeutic interventions as well as patient’s autonomy, continuous care needs, patient’s preferences, and therapeutic susceptibility. The consented organization-specific criteria focus on the underlying framework conditions, like discharge timing, safety measures, available care capacities, skill sets and technology. First clinical practice implementation studies are recommended to further define criteria evaluation time frames, the role of the different stakeholders in the decision process and the criteria importance ranking. Future research shall focus on validation of the criteria set, utility, criteria reporting and decision support automation, and visualization.

In a broader perspective, applying clearly defined ICU discharge criteria may reduce decision making subjectivity, improve patient safety and workflows in daily care transitions, support efficient use of limited ICU resources and equity of care, but also prevent avoidable patient deterioration and overburdening of lower levels of care. That means, patients and organizations could benefit from the implementation of such discharge criteria as a clinical decision support in daily clinical practice.