Data analysis plan
The data analysis will be conducted following the intention-to-treat (ITT) principle which includes 3 rules. The first rule establishes the landmark event for entry into the study, which is the randomization of hospitals. All randomized hospitals will be considered enrolled, and data of eligible patients treated by enrolled hospitals will be analyzed. The second rule establishes patient-level data collection, which is independent of treatment assignment. The third rule defines the pre-specified time period for counting study events, which varies for each outcome.
We will follow the order principle of analyses, which states that analyses involving the primary outcome measure must be preceded by analyses of higher-order outcomes when those outcomes are censored. In our study, all-cause death events are censored. To follow the order principle of analyses, we will compare the all-cause mortality incidence rate within the first 30 days of admission across the four treatment groups. Incidence rate is calculated as the number of all-cause deaths divided by the number of included patients and expressed per 1000 (or 10,000 patients) per 30 days of follow-up. We will calculate the incidence-rate difference between the four treatment groups, using the usual care group as the reference. The four treatment arms are rapid testing and de-escalation, rapid testing only, de-escalation only, and usual care (i.e., neither rapid testing nor de-escalation).
The primary outcome of days on ESA will be analyzed using a linear mixed-effects model, which will include the treatment arm as a fixed effect and a random-intercept at the randomized hospital level. We will adjust for baseline covariates between the intervention and control groups. These covariates include patient age, sex, race, CURB65, Elixhauser index, serum Na, serum glucose, hemoglobin, MRSA nasal swab (yes/no), WBC count, albumin, baseline oxygenation for CAP patients (O2 sat off O2 or supplemental oxygen delivery device type (vent > bipap > high flow, etc.), max or median respiratory rate on day 1, min SBP on day 1, vasopressors, and comorbidities (chronic pulmonary disease, lymphoma, metastatic cancer, obesity), as well as hospital characteristics in the year prior to study initiation (duration of antibiotics and ESA, AKI, CDI, 14-day mortality, and 30-day mortality, all among CAP patients who would have met study criteria). The primary comparison is to compare the duration of exposure to ESA therapy in the four treatment groups. Interaction between the two interventions will be tested as a secondary analysis. We will consider a two-sided p-value < 0.05 statistically significant. As we use a priori stratification (by the type of hospital), we will examine differential intervention effects by each pre-defined stratum. The primary outcome data will be available for every patient since it is recorded in the EHR. In case of missing secondary data, we will assume it is missing at random and a sensitivity analysis will be performed with data imputed via Multiple Imputation by Chained Equations (MICE).
To address the non-adherence limitation, we will conduct two secondary analyses [
17]. First, we will perform an as-treated analysis, comparing outcomes among those who receive the treatment versus those who receive control, regardless of randomization. Second, we will calculate complier average causal effect (CACE) estimation, which uses randomization as an instrument to account for unobserved confounding and provides a randomization-respecting estimate [
18]. CACE estimates the intention-to-treat effect in the subgroup of participants who always comply with their treatment allocation. In addition, we will use per-protocol analysis, comparing those who comply with their random allocation in the treatment group with all the controls. We will use an inverse probability weighting approach to minimize selection bias [
19]. All the secondary analyses will use the framework of mixed-effects (hierarchical, two-level) models as described above for the primary ITT analyses.
All continuous secondary outcomes will be analyzed using the same method as the primary outcome. For each secondary outcome, we will adjust for the rate of that outcome in the 12 months prior to the study. Binary outcomes will be analyzed using generalized linear mixed-effects models. All analyses will be conducted using SAS 9.3 (Cary, NC), R-studio (Boston, MA), and STATA (College Station, TX). Our trial will follow the Consolidated Standards of Reporting Trials guidelines, extended for cluster-randomized controlled trials [
20].
Cost analysis
We will conduct a cost analysis from the health system perspective, including only medical costs related to providing care and excluding costs associated with patient time spent seeking care, caregiver time, transportation, or productivity loss. Total direct medical costs of the hospitalization/inpatient encounter will be collected through the Cleveland Clinic finance system. Costs are grouped into different categories, including inpatient, outpatient, laboratory testing, and pharmacy. For the intervention groups, we will also include the cost of rapid diagnostic testing and time cost of the pharmacists, calculated as hourly salary multiplied by the number of minutes pharmacists spent discussing de-escalation with providers. Due to the right-skewed nature of cost data, we will use bootstrap methods to compare median costs between groups [
21]. We will draw 1000 bootstrap samples with replacement from each study group and calculate the mean cost difference and 95% confidence interval. Finally, we will employ a generalized linear mixed-effects model (with a log-link function and gamma distribution) to analyze the interventions' effect on total cost, accounting for differences in baseline patient-level characteristics.
Assessment of safety
Unanticipated problems related to the intervention will be logged and reported to the Institutional Review Board (IRB) at the time of continuing renewal. Major deviations will be reported to the IRB upon discovery.
Periodically during the trial, the study team will:
-
▪ Review the research protocol
-
▪ Evaluate the trial’s progress
-
▪ Consider external factors, such as scientific or therapeutic developments that may affect the study’s safety or ethics
-
▪ Review center performance, make recommendations, and assist in resolving problems
-
▪ Protect the safety of study participants
-
▪ Conduct interim analysis, if appropriate
-
▪ Ensure confidentiality of trial data and results of monitoring
-
▪ Address any problems with study conduct, enrollment, sample size, or data collection
Any protocol changes that are deemed significant by the investigator team will be reviewed and approved by the IRB at Cleveland Clinic. This will also be shared by AHRQ. All protocol changes will be documented with version control in place.
Interim analyses
Safety interim analysis will be conducted every 6 months. The all-cause death incidence rate within the first 30 days of admission is calculated as the number of all-cause deaths divided by the number of eligible patients (according to inclusion and exclusion criteria) and expressed per 1000 (or 10,000 patients) per 30 days of follow-up. We will use the all-cause death incidence rate within the first 30 days of admission calculated during the 1-year period before the study onset as a baseline estimation. We will calculate incidence-rate ratios and incidence-rate differences between the four treatment groups, using the usual care group as the reference. The four treatment arms are rapid testing and de-escalation, rapid testing only, de-escalation only, and usual care (neither rapid testing nor de-escalation). If the all-cause death incidence-rate ratio or incidence-rate difference between the usual care group and any of the three intervention groups is statistically significant at a p-value < 0.05, the study’s Steering Committee will be notified.
Futility interim analysis will be conducted after half the data have been collected (patient n = 6250). At that time, we will calculate conditional power: the probability of statistical significance at the study’s completion given the date obtained so far. We will calculate the futility index as 1 − conditional power. The study will be stopped if the futility index is above 0.8 (conditional power falls below 0.2). To calculate conditional power, we will calculate current (at n ≥ 6250) z-statistic, using the PASS Probability Calculator. PASS 2022 implemented conditional power calculation using Jennison and Turnbull (year 2000; pages 205–208), the general upper one-sided conditional power at stage k for rejecting a null hypothesis about a parameter θ at the end of the study, given the observed test statistic, Zk.
Dissemination plans
The project’s design emphasizes the intention and plan to utilize the knowledge and products gained to improve patient care. The trial was registered at ClinicalTrials.gov before enrolling the first patient. As the study progresses, we will update the trial progress and recruitment status (not yet enrolling–enrolling–enrollment completed). To encourage the translation of the study’s results into practice, we will disseminate the findings widely through conference presentations (e.g., CHEST annual conference, IDWeek, Society of Hospital Medicine) and peer-reviewed publications. The large scale and pragmatic design of our study make it unique. The knowledge obtained from this study will likely inform CAP guidelines on the potential of rapid molecular diagnostic assays to reduce ESA use and the safety and efficacy of antimicrobial de-escalation following negative cultures. Additionally, study team members will collaborate with their respective professional societies. Dr. Klompas, who has served on various pneumonia guideline panels, will help disseminate the findings at the Infectious Disease Society of America (IDSA) and Society for Healthcare Epidemiology of America (SHEA). Dr. Haessler, a board of trustees member for SHEA, will assist in disseminating the findings at SHEA meetings and conferences. Authorship on any future publications utilizing trial data will be evaluated individually, contingent on the contributor’s involvement following ICJME guidelines.