Improving outcomes after pediatric cardiac arrest – the ICU-Resuscitation Project: study protocol for a randomized controlled trial

The primary aim of this ongoing multicenter trial, Improving Outcomes from Pediatric Cardiac Arrest – the ICU-Resuscitation Project (ICU-RESUS), is to assess whether structured debriefings for the ICU care environment combined with point-of-care bedside practice will improve the rate of survival with favorable neurologic outcome in children receiving CPR in the ICU.

Three types of designs were considered for this cluster randomized trial: a parallel design, a traditional stepped-wedge (TSW) design [20], and a hybrid stepped-wedge (HSW) design (Fig. 1). In a parallel design, each hospital is randomly assigned to either an intervention or control group for the duration of the trial. In a TSW design, each hospital begins enrolling in the control group and transitions to enrolling in the intervention group at a randomly assigned time. The design ultimately selected for this trial is an HSW design, which is a hybrid between the parallel and stepped-wedge designs. As in a parallel design, two randomly selected hospitals are permanently assigned to enroll in the control group, and two others are permanently assigned to enroll in the intervention group. The remaining six hospitals, as in a stepped-wedge design, initially enroll in the control group but are randomly assigned a time to transition to enrolling in the intervention group. However, it is important to note that the influence of the intervention on hospital staff and thereby on patient outcomes is not fully effective on the day that the intervention is initiated. Therefore, each of these six hospitals has a brief transition period during which outcome data will not be included in the analysis. To our knowledge, ICU-RESUS is the first implementation of an HSW design, one with several advantages in the context of the trial.

An HSW design, like a TSW design, offers a logistically feasible way to roll out a seemingly desirable intervention for participating hospitals at staggered intervals while also allowing a robust comparison between treatment and control periods. In particular, the design enables valid comparisons in the midst of underlying temporal trends in the outcome. In contrast, a simple before-and-after design, as in the single-center pilot study, cannot adequately account for temporal trends, and any observed “intervention effect” cannot be disentangled from other influencing factors that may have changed over time. In addition to the advantages of a TSW design, an HSW design has been reported, in many instances, to have increased power compared with a TSW design or a parallel group design [21].

The intervention is a two-part ongoing training program for ICU staff. The program includes structured debriefings for the entire care environment [19] (i.e., any care provider from several disciplines: physicians, nurses, respiratory therapists, extracorporeal support specialists) and point-of-care practice with a manikin [22].

A 1-h CPR debriefing is held approximately monthly (at least nine times per year) by the unit, and all ICU staff are encouraged to attend. Providers are “invited” to the sessions through emails and departmental flyers. Ideally, a recent resuscitation from within the unit is selected, but a de-identified case from another unit or hospital may be used if needed. The purpose of these debriefings is to carefully review aspects of care that went well and aspects that could be improved [18]. The case introduction includes relevant patient history and details of the patient characteristics and medical management preceding CPR. The debriefing then includes event details such as precode cardiac rhythm, interruptions in compressions, defibrillation attempts with pre- and postshock rhythm, ventilation, compression rate and depth, and release velocity (leaning). When available, case presentations emphasize patient-centric physiologic CPR targets using arterial catheter pressure tracings and end-tidal CO₂ waveforms. Cases may also include important aspects of pediatric postarrest care such as recognition of seizures [23‐25], prevention of fevers [26], and avoidance of hypotension [27]. These sessions are protected under quality improvement guidelines at the individual institutions.

Point-of-care bedside practice allows ICU staff to practice chest compression delivery on a manikin that is set up on a cart so that it can easily be brought to the clinicians throughout the unit. Manikins are equipped to monitor and provide real-time feedback on the quality of delivery (e.g., rate, depth, and release velocity). Both infant and child manikins are available for staff to practice on a model that is relevant to their patients for the shift. Point-of-care practice is frequent (≥ 48 per unit each month), but brief (< 2 minutes) in duration. This component of training was in place during both the control and intervention periods of the single-center pilot study of team debriefings and has been described previously [22, 28‐30].

ICU-RESUS is supported by the National Heart, Lung, and Blood Institute (NHLBI) and conducted by the Eunice Kennedy Shriver National Institute of Child Health and Human Development-funded Collaborative Pediatric Critical Care Research Network (CPCCRN). Hospitals were eligible for participation if they had not implemented either of the elements of the ICU-RESUS intervention and were committed to not implementing the intervention until scheduled by the trial randomization. The ten participating hospitals are geographically diverse tertiary care pediatric hospitals with patients from a mixture of urban, suburban, and rural areas of the United States [31]. These hospitals include 18 distinct units from which subjects will be enrolled.

All eligible subjects at participating institutions are enrolled under a waiver of consent. The central institutional review board at the University of Utah granted a waiver of consent based on the following factors:

Subjects are considered for eligibility if they (1) are ≥ 37 weeks gestational age and ≤ 18 years of age and (2) receive CPR in the ICU. Subjects are excluded if they, prior to the arrest, (1) had a preexisting terminal illness and were not expected to survive the hospitalization (e.g., patients transferred to the ICU for end-of-life care); (2) had a lack of commitment to aggressive ICU therapies; (3) were brain dead; or (4) had an out-of-hospital cardiac arrest associated with the current hospitalization. CPR events lasting < 1 minute will be excluded from analysis if a site investigator determines CPR to have been unnecessary.

We anticipate enrolling 1540 index CPR events from 18 clinical units at 10 participating hospitals during a 4.5-year enrollment period. A schedule of enrollment, interventions, and assessment is provided in Fig. 2.

The role of randomization in an HSW trial is to determine which clusters will permanently enroll in the control group, which will enroll permanently in the intervention group, and the order in which the others will transition from control to intervention (Fig. 1c). Although the analysis will control for each individual unit, randomization occurs at the hospital level as opposed to the unit level to prevent the possibility of contamination. If units within a hospital were allowed to transition at different times, the intervention in one unit might inadvertently influence outcomes in the other, nontransitioned unit. One way that contamination could happen is through staff members from a transitioned unit temporarily working a shift in the nontransitioned unit. Therefore, when multiple units from a hospital are participating in the trial, all units are assigned the same transition date, reducing the possibility of contamination.

All possible permutations of the hospitals were considered for the order in which hospitals would transition from control to intervention. However, some permutations lead to a transition schedule in which the smallest hospitals transition early and the largest hospitals transition late, causing more subjects to be enrolled in the control group than in the intervention group. The imbalance for each permutation was estimated on the basis of hospital-specific enrollment expectations, and permutations with excessive imbalance were excluded. The permutation selected for implementation was then chosen at random from permutations with adequate balance with respect to expected enrollment between intervention and control groups.

The timing of the intervention for each hospital was established by the data coordinating center prior to enrollment. At a participating hospital randomly selected to transition from control to intervention, investigators and trial staff are informed of their selection 3 months prior to the assigned transition date. This time frame was selected to provide hospitals with sufficient notice to prepare for implementation while not divulging assignment so far in advance as to potentially affect a hospital’s resolve to stay the course. The scheduled hospital is allowed a 4-week window to begin the transition by holding an inaugural debriefing. The exact dates marking the beginning and end of the transition period, a period during which newly enrolled subjects are excluded from analysis, is determined by the date on which the first and fourth debriefings are held. Debriefings are held monthly, making the transition period approximately 3 months.

As in the single center pilot study, the primary outcome is survival to hospital discharge with favorable neurologic outcome. Neurologic outcome is measured with the Pediatric Cerebral Performance Category scale (PCPC [32]) (Table 1) as per standard Utstein-style reporting guidelines [33]. Survivors are considered to have a favorable neurologic outcome if they have no worse than moderate disability at hospital discharge (PCPC score of 1, 2, or 3) or if their PCPC is no worse than at baseline. Baseline PCPC is retrospectively assessed in enrolled patients and is typically based on their status prior to the event leading to the current hospitalization. However, baseline PCPC is evaluated no more than 30 days prior to CPR for subjects who have been in the hospital for more than 90 days. Subjects who are born during the same hospitalization in which the CPR occurred will have their baseline PCPC assessed based on their status prior to the decompensation associated with the CPR event. PCPC assessment is performed by unblinded research coordinators on the basis of detailed objective criteria [32].

In the single-center pilot, the intervention was associated with an increase in survival with favorable neurologic outcome from 29% to 50% (21 percentage points). We anticipate that we will see improvement to a rate similar to that achieved in the single-center pilot, but preliminary data from another study show that CPCCRN hospitals have a higher average baseline rate of survival with favorable neurologic outcome than the single-center pilot [34]. Therefore, our goal was to select a design and sample size that would provide approximately 80% power (at a significance level of α = 0.05) to detect a more conservative increase from 40% to 51% in survival with favorable neurologic outcome. An important factor for sample size calculation in cluster-randomized trials is the intracluster correlation (ICC), a measure of the similarity in outcomes among patients in the same cluster compared with outcomes among patients in different clusters [35]. The ICC was assumed to be 3% on the basis of pilot data from the multicenter Therapeutic Hypothermia after Pediatric Cardiac Arrest In-Hospital Trial [36].

The power and sample size estimates used for planning this trial were based on the method introduced by Hussey and Hughes [37], though generalized to allow for transition periods and implemented in PASS version 15 software [38]. This method uses a Wald Z-test to evaluate a null hypothesis of no effect against a two-sided alternative. We considered several trial design configurations, including parallel group, TSW, and HSW designs. We also investigated the impact of transition duration. With an ICC of 3%, an HSW design was found to be optimal (Table 2) [21, 39]. Long transition periods in HSW and especially in TSW designs increased the trial duration necessary to achieve > 80% power (Table 2). Therefore, an HSW design was selected with short (3-month) transitions.

To estimate power for the HSW design, we assume that two hospitals, each with two units, are randomly assigned to begin with the intervention, and similarly that four units at two hospitals are assigned to remain as control hospitals for the duration of the study. Enrollment was assumed to be evenly distributed across the 18 units and also across the 18 time periods. A 3-month transition period was included for each of the ten transitioning units in the design-pattern matrix (Table 3). Units are the clusters used for sample size calculations and planned analyses. However, as noted above, randomization occurs at the hospital level to minimize the possibility of intervention contamination between units within a hospital. We found that analyzing 1256 index events (4 per unit per period) would provide 77% power to detect an 11-percentage-point increase in the rate of survival with favorable neurologic outcome from a baseline rate of 40%. Power increases to 84% when analyzing 1570 index events (5 per unit per period). On the basis of concavity of the power curve, analyzing 1391 subjects provides > 80% power. On the basis of historical CPR rates in participating units, we anticipate enrolling 1540 index events over the 4.5-year enrollment period. An estimated 48 index events will occur during the transition periods, allowing analysis of the remaining 1492 events and providing > 80% power.

Data collected for this trial are entered into OpenClinica, a web-based electronic data capture system, and stored on a secure server at the data coordinating center. An automated system for validating data against a set of predefined rules will query clinical centers regarding data that are invalid, illogical, or incomplete. Data elements critical to the primary aim of this trial are monitored remotely by the data coordinating center to confirm the accuracy of entered data compared with the source documents. Remote monitoring is performed on initial subjects enrolled at each clinical center and then on a subset of additional subjects enrolled throughout the trial.

The primary analysis is to assess the effect of the intervention on survival with favorable neurologic outcome. This analysis is based on subjects enrolled during the control period or the intervention period and excludes subjects enrolled during the transition period. Secondary events (after the first eligible CPR event) during the hospitalization are excluded. The effect of the intervention will be assessed with the following test of hypothesis:

Survival with favorable neurologic outcome will be the outcome for a multivariable logistic regression model. The primary predictor in the model will be treatment group (intervention vs. control). Fixed covariates in the model will be age category (≤ 6 months, > 6 months), Pediatric Risk of Mortality score (PRISM; a measure of illness severity in the 2–6 h prior to CPR) [40], first documented rhythm, illness category (medical cardiac, surgical cardiac, medical noncardiac, surgical noncardiac), and a piecewise linear spline of time. Age, illness category and severity, and first documented rhythm are known predictors of mortality and are included to improve the model fit and potentially increase power. In contrast, survival with favorable neurologic outcome is not expected to change over time. However, without adjusting for time, an improvement over time in survival with favorable neurologic outcome that is unrelated to the intervention would be erroneously interpreted as evidence for the intervention owing to the increased enrollment in the intervention group toward the end of the trial. Including time as a covariate is critical for TSW and HSW designs in order to have valid inference in case of potential temporal trends. Although time is often modeled as a categorical variable in TSW trials, we chose to model time as a piecewise linear spline because we expect it to provide better model fit with fewer estimated parameters. For the spline, knots will divide the enrollment period into epochs of equal duration. The number of knots will be chosen to minimize the Akaike information criterion. There are 18 distinct units at the 10 participating hospitals. Unit will be included as a random effect to account for variability in outcomes between units and increase the generalizability of results.

The data needed for this analysis are readily available, and automated data validation ensures that critical measurements are entered. However, in a very small number of subjects, the window for the PRISM score will be entirely prior to hospitalization. In order to obtain a best estimate of prearrest severity of illness, measurements more than 30 minutes prior to the arrest may be used in this case. In the very rare case that a subject experiences a CPR event within the first 30 minutes of hospitalization, all values will be assumed to be normal prior to the decompensation associated with the CPR event.

This trial compares the effect of the ICU-RESUS intervention to current practice, but participating units had varied quality improvement programs prior to this trial. Although participating units are committed to implementing the ICU-RESUS intervention as scheduled by randomization, they are free to continue, modify, or supplement existing programs with other interventions. The programs in place in each unit will be tracked throughout the trial to allow a description of the control group and an exploratory investigation of additional elements of a successful program.

The NHLBI appointed an independent data and safety monitoring board (DSMB) to monitor study progress, adherence to the protocol, participant safety, and efficacy. The DSMB has initially planned to meet after the first, second, and third years of enrollment. Formal efficacy monitoring is not planned until after the second year, owing to the sigmoidal growth of the information fraction for TSW and HSW designs and because very early trends may not be representative. In particular, TSW and HSW designs have many more subjects enrolled in the control group than in the intervention group at interim time points. However, efficacy monitoring remains important to allow early stopping and dissemination of results in case of a very large intervention effect, as was observed in the single-center pilot study [19]. Efficacy monitoring will be conducted with the Lan-DeMets approach for flexible alpha spending [41], using conservative O’Brien-Fleming boundaries for early stopping [42].