A systematic review and meta-analysis of enrollment into ARDS and sepsis trials published between 2009 and 2019 in major journals

Enrollment problems in randomized controlled trials (RCTs) lead to reduced precision regarding interventions’ effects [1], increased study costs [2], and slows the adoption of evidence into practice [3]. Research involving critically ill patients poses unique challenges to trial enrollment, including narrow recruitment windows and reliance on surrogate decision-makers for obtaining informed consent [4, 5]. Moreover, critical care trials are often underpowered to detect a clinically meaningful difference in mortality due to, at least in part, enrollment difficulties [6] Additionally, several recently published high profile critical care trials were terminated early due to difficulties with enrollment [7‐9].

Underpowered trials, and those terminated strictly due to feasibility issues, raise ethical concerns as they expose participants to any risks involved in participation with a reduced likelihood of social benefit [10]. In an effort to address issues of trial inefficiencies, it has been proposed that alternative recruitment strategies [11] and trial designs [12] are needed. Accurate estimation of enrollment rate is essential to inform trial planning and to serve as a benchmark for evaluating the utility of novel trial designs. However, reporting of trial recruitment and enrollment metrics varies, and little is known about trial factors that influence the enrollment rate.

We conducted a systematic review of trials that evaluated interventions for common critical care syndromes, including acute respiratory distress syndrome (ARDS), acute lung injury (ALI), and sepsis, to assess enrollment rate and contributing trial design factors. We focused this review on trials published in the last decade to ensure the results reflect contemporary critical care science and recruitment practices to best inform trial design and conduct in the near future.

We conducted a systematic review and meta-analysis, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (Additional file 1: Appendix A) [13]. The protocol was not eligible for registration in the International Prospective Register of Systematic Reviews since the outcomes of interest were not health-related [14]. This study did not meet the criteria for human subjects research as defined by United States federal policy 45 Code of Federal Regulations 46 and thus was exempt from review by the institutional review board.

In this systematic review of 94 contemporary ARDS/ALI and sepsis trials, we found an overall enrollment rate of less than 1 participant per month per site, and almost 1 in 10 trials failed to meet the target sample size due to slow enrollment. These findings have important implications for trial planning and provide evidence to support the general perception among critical care trialists that conducting trials among critically ill patients is arduous, and novel interventions to improve enrollment, trial efficiencies, and alternative designs are needed [12].

Our overall findings are consistent with a previous study that found a median enrollment rate of 0.90 patients per site per month among 23 multicenter RCTs of non-surgical interventions for adult critically ill patients [20]. Another prior review found that \(\ge \hspace{0.17em}\)95% of target enrollment was not achieved in 15% of adult critical care trials due to recruitment or logistical issues [6]. Although this proportion was lower in our focused review of ALI/ARDS and sepsis trials, both are likely to be underestimates since trials that fail to achieve enrollment targets may be more likely to remain unpublished or be published in lower-impact journals, which were not included in either review. Furthermore, studies that are underpowered or stopped early due to enrollment difficulties expose enrolled participants to potential risk with reduced likelihood of benefit, thus raising ethical concerns in terms of not providing societal benefit [10].

The overall enrollment rate among sepsis trials was nearly twice that in ARDS/ALI trials, which is likely reflective of important clinical differences between these syndromes that may impact research recruitment processes. For example, patients with ARDS/ALI are likely to be mechanically ventilated and therefore reliant on surrogate decision-makers for research participation decisions. Surrogates often feel overwhelmed by the acuity of the patient’s illness and substituted judgment process, which influences their decision-making processes for research participation [21]. Use of ethical and effective behavioral economic nudges during recruitment may help to minimize the surrogate decision-making process and promote informed trial participation [22, 23].

We found several trial-related factors that were associated with better enrollment rates that may help to inform future trial planning. Single-center and non-industry funded trials had higher enrollment rates, which may indicate potential participants’ greater trust and confidence in the medical and research teams, important factors in surrogate decision-makers’ research participation decisions [24]. Single-center trials may recruit more rapidly in part due to increased motivation of recruiters who likely know or may even be the principal investigator (PI) of the trial. In contrast, multi-center trials face a higher burden to provide consistent recruitment oversight and troubleshooting for site PIs who are crucial to recruitment efforts but also likely to have competing demands on their time. Additionally, there may be differences in the motivation of those recruiting based on the funding source. While adding sites may not be the most efficient solution for trials struggling with slow enrollment, this decision must be balanced with the limited generalizability of evidence from single-center trials. Geographic differences in enrollment rates may partly be explained by differences in the use of single vs multicenter trials and varying requirements for regulatory controls. Interestingly, neither the timing of consent (prospective required versus retrospective permitted), nor type of intervention was associated with enrollment rate in this sample of clinical trials.

Previous qualitative studies have identified factors important to enrollment decisions made by surrogates of critically ill patients. In addition to the importance of trust [21, 25], those who elected to enroll the patient cited that they had a desire to help future patients, were following the perceived wish of the patient, had a generally favorable view of research, and perceived the trial as safe. Those who declined enrollment cited the acuity of the patient’s illness, a fear of “rocking the boat”, and concern with potential interference with the medical team’s decisions [4]. The experiences in COVID-19 ICU trial enrollment suggest there are likely additional influential factors on ICU surrogates’ decision-making for research participation. For example, the CoDEX trial reported an enrollment rate of 3.6 participants per month per site [26], the HENIVOT trial 13.8 participants per month per site [27], and the INSPIRATION trial 15 participants per month per site [28]. These enrollment rates are significantly higher than what we found in this review of ICU trials conducted before the pandemic, and may be attributable to the allocation of increased resources for recruitment, a feeling of desperation among COVID patients and surrogates motivated to try anything for this unknown and life-threatening disease, or the high volume of COVID patients in the ICU. While research conducted during the COVID-19 pandemic represents an extreme example that is not likely to generalize to most other critical care trials, there may be lessons to glean nonetheless about other important influences on enrollment processes and decisions for participation in critical care research.

Despite multi-center trials being common in this sample, recruitment still required a median of three years to complete. Although the number of eligible patients at each site was inconsistently reported and thus was not included in our analysis, given the ubiquity of ARDS/ALI and sepsis in the ICU [29], it is unlikely that a lack of eligible participants is the primary limiting factor. Indeed, a multicenter study of research recruitment outcomes among critically ill patients found that over half of the opportunities to recruit an eligible patient failed, most commonly due to the research team being unavailable or an inability to contact a surrogate decision-maker [5]. Increasing the number of personnel involved in screening and approaching eligible participants may be an efficient way of increasing the trial enrollment rate, but the increased cost of doing so must also be considered.

Several limitations must be considered when interpreting the results. First, this report only includes trials that enrolled patients with ARDS/ALI or sepsis and were published in select journals. While this intentionally narrow scope may limit the generalizability of the results, we focused on the most common critical illness syndromes and high-impact journals as this is the evidence base with the greatest impact on clinical practice. It is likely that trials that experience very slow enrollment or other difficulties achieving their enrollment targets are more likely to be published in lower-impact journals or go unpublished. Therefore, our results likely represent an overestimation of enrollment rates and an underestimation of the problem. Second, due to a lack of reporting of site-level recruitment metrics, our analysis assumes that each site was actively enrolling throughout the duration of the trial. This assumption is likely to be inaccurate for some studies and may have led to an underestimate in the overall point estimates in this review. Third, site- and provider-level factors could play an important role in enrollment, however, important details about such factors are rarely reported in publications. Future research focused on collecting and analyzing site- and provider-level factors may improve our understanding of factors contributing to successful trial enrollment. Fourth, due to the small numbers in some subgroups, our subgroup analyses may be underpowered. Lastly, we could not determine the overall proportion of eligible participants approached who were enrolled (consent rate) due to significant heterogeneity in reporting. Indeed, adherence to CONSORT reporting guidelines has been notably poor among critical care trials [30], which often experience unique recruitment circumstances (e.g., clinicians declining participation on behalf of the patient or reliance on surrogate decision-makers). Adoption of a standard CONSORT diagram is needed to facilitate an accurate evaluation of consent rates.

Our review of 94 recent high-impact ARDS/ALI and sepsis trials revealed that enrollment challenges remain an important source of delay in new evidence generation and threaten the scientific validity of trial findings. Novel strategies that surmount modifiable enrollment barriers are urgently needed to improve the efficiency of critical care trials to enable rapid implementation of evidence-based interventions into clinical practice.