Using a continuum of hybrid effectiveness-implementation studies to put research-tested colorectal screening interventions into practice

Multiple prior publications provide detailed descriptions of the Hybrid 1 and 2 studies presented here; the Hybrid 3 study is on-going [7‐9]. A brief description of each study is provided below. We also describe the two new domains, program design and program implementation as these were distinctly different between the three studies.

The Hybrid 1 study (Systems of Support to Increase Colorectal Cancer Screening, SOS, R01 CA121125) was a 4-arm patient-level randomized trial (4675 patients). It was conducted from 2008 to 2011 in an integrated health care system that provides both health insurance and health care (primary and specialty care, including colonoscopies) [7, 10]. The research staff used automated data (electronic health record [EHR] data, laboratory data, and procedural CPT codes) to identify age-eligible patients not current for CRC screening and evaluate outcomes. The intervention was embedded in the health care system and used an EHR-linked registry to deliver a centralized program to encourage CRC screening uptake. Usual care (Arm 1) received annual birthday reminders of overdue screening tests including CRC screening. In addition to usual care, participants randomized to active interventions received stepped intensity interventions: information on CRC screening choices, mailed fecal kits, postage-paid return envelopes, and a mailed reminder if the kit was not returned (Arm 2); mailings plus brief telephone assistance if screening was not completed after the mailings (Arm 3); mailings plus brief assistance plus nurse navigation for those still not completing screening (Arm 4). The research team managed the database with a vendor service mailing the intervention components; clinic medical assistants and nurses worked from their regular clinic but had protected time (the study paid for about 4–8 h a week of their time) to provide telephone assistance and navigation to patients across the organization [7]. The primary outcome was CRC screening adherence over 2 years in each of the progressive intensity arms compared to usual care (Arm 1). Secondary implementation outcomes included the reach of the intervention, cost-effectiveness from the health plan’s perspective, qualitative assessments from the patients’ perspective of how the intervention could be improved, and the intervention’s fit within the organization’s other quality improvement efforts to increase CRC screening [10‐14]. The mailed program alone doubled CRC screening uptake, with incremental increases in the stepped intensity groups [10]. Mailed interventions and calls from medical assistants or nurses from other clinics were acceptable to patients if the medical assistant or nurse had access to their EHR record and could communicate with the patient’s physician [13].

The Hybrid 2 study (Strategies and Opportunities to STOP Colon Cancer in Priority Populations, UH3 AT007782), was a pragmatic cluster-randomized trial conducted from 2014 to 2016 at 26 safety net clinics with over 40,000 patients in the first year who were age-eligible and overdue for CRC screening [8, 15]. EHR and laboratory data were used to identify eligible patients and to evaluate effectiveness outcomes. The intervention was adapted in part from the Hybrid 1 SOS study, in that it used EHR data to identify individuals overdue for CRC screening and a registry to automatically generate mailings. However, the research team, EHR vendor, and clinic staff collaboratively designed and implemented the program for use in community clinic settings and by diverse patients. Adaptations included translating patient letters into Spanish and other languages, using wordless fecal immunochemical test (FIT) pictographic instructions [16], and embedding the registry into the EHR (Reporting Workbench) to generate lists of patients overdue for CRC screening and to create mailings. An advisory committee made up of organizational and clinic program leaders, CRC screening experts, and patients reviewed all materials and suggested changes or enhancements. Clinic staff generated lists and completed mailings, with the research team only providing initial training, monthly clinic staff “champion” meetings to augment training and troubleshoot issues with the EHR-embedded tool, and facilitation of Plan-Do-Study-Act (PDSA) cycles to help the clinics improve program implementation [17]. The primary outcomes were FIT completion (effectiveness) and implementation outcomes based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework (e.g., percent of clinics participating, percent of letters mailed by each clinic, percent of clinics continuing the program) and the Consolidated Framework for Implementation Research (CFIR) to assess contextual factors related to implementation (e.g. barriers and facilitators). Additional implementation outcomes included program cost-effectiveness. The intervention overall led to significantly higher rates of FIT completion compared to usual care clinics [15], but the net difference was modest (net increase 3.4%), mainly because implementation varied greatly across intervention clinics, with the health center-level percent of eligible intervention participants mailed a FIT ranging from 6.5 to 68.2% [15]. Among the patients who were mailed FITs, completion rates were close to the Hybrid 1 SOS study, 21% [15]. Mixed methods summative analyses of barriers and facilitators to implementation are underway, but qualitative rapid cycle assessments during the study suggested that contributing factors were delays in start-up, personnel changes and staffing shortages, and challenges with the technologies and lab ordering processes [18].

The Hybrid 3 study began in 2015 and is an on-going evaluation of two different mailed FIT programs being implemented by two Medicaid/Medicare insurance plans in 2 states (BeneFIT, UDP-14-0012) [9], with age-eligible health plan members identified and outcomes evaluated using claims data (12,000 patients in year 1). The investigators initiated the relationship with the health plans with the goal of addressing implementation barriers experienced in the Hybrid 2 study, specifically that clinics often had insufficient staff or resources to consistently implement the program. One program is working collaboratively with clinics to integrate it into their usual workflow (e.g., a vendor mails the same FIT kit type used in clinical care, and completed tests are mailed back to the clinics, where staff place test orders and deliver follow-up care). The other is a centralized program whose vendor mails the FITs, which are sent to a centralized laboratory for processing, then sends the results to the providers and directly contacts patients with abnormal results. Mixed methods outcomes include percent of age-eligible patients receiving mailings, acceptability of and satisfaction with the program by the health plans, implementation barriers and facilitators, effectiveness (FIT and colonoscopy completion rates), and program costs. Contextual factors related to implementation and maintenance of the programs are assessed using the CFIR framework.

Table 2 compares each of the three reported hybrid studies across key design characteristics and shows differences with progression from Hybrid 1 to Hybrid 3 designs. SOS, the Type 1 hybrid study, was a person-level randomized controlled trial that invited almost all age-eligible individuals’ overdue for CRC screening and had few exclusions (CRC, inflammatory bowel disease, life-threatening health conditions). Outcomes were primarily effectiveness, but mixed methods were used to understand screening barriers and facilitators and to further refine study interventions in subsequent study years [13]. Cost-effectiveness evaluations were performed from a health care organization perspective to assist with decisions about adoption of the overall program and its specific components (such as the incremental cost-effectiveness of adding phone assistance and navigation) [12]. Challenges included the requirement of verbal consent by the institutional review board, with only 38% of those invited agreeing to participate [19]. Non-white and Hispanic individuals, and individuals with lower levels of education, were less likely to participate than non-Hispanic white individuals and patients with higher levels of education.

STOP CRC, the Hybrid 2 study, sought to adapt and implement the SOS program within community health center clinics that care for populations with the lowest CRC screening rates. STOP CRC had almost no exclusions and consent was waived, making it possible to reach nearly 100% of the population that needed the intervention. This study measured both overall effectiveness of the program compared to control clinics, (i.e., FIT uptake) and variation in effectiveness across health centers [18]. Implementation was measured as the proportion of eligible patients who were mailed a FIT (quantitative) at each health center and their barriers and facilitators to implementation (qualitative) [16] and maintenance (RE-AIM). Early in the study, it became obvious that implementation fidelity was the primary challenge, with some clinics delaying start-up until almost the end of the participant accrual interval. These delays and implementation variability led to reductions in program effectiveness.

In BeneFIT, the Hybrid 3 study, we are evaluating mailed FIT programs designed to decrease the burden of implementation delivery on clinics by enlisting Medicaid/Medicare health insurance plans [9]. BeneFIT is a quasi-experimental, naturalistic study that compares two different implementation strategies developed by the health plans, without any usual care comparison group. The investigators serve as consultants to the health plans for both program design and delivery and are conducting a formal program evaluation. The health plans are responsible for conducting their programs, and both contracted with vendors to implement the mailings. The program is measuring implementation using mixed methods: quantitative (e.g., proportion of age-eligible enrollees mailed a FIT and delivered reminders, supporting activities by clinics, program costs) and qualitative interviews of key stakeholders to assess motivations, barriers, and facilitators of the program. Effectiveness outcomes are limited to what is easily measurable (e.g., FIT and colonoscopy completion rates using claims data). The Hybrid 3 study has no usual care comparison, as we knew already from the Hybrid 2 study that no mailings led to lower FIT completion rates.

Program design and delivery, the two new domains proposed for the hybrid studies, were distinctly different between the three studies. In Hybrid 1, the research team designed and built the EHR-linked mailed fecal testing program, trained clinical staff, and paid for their time, which helped to ensure fidelity to and completion of the intervention components. In Hybrid 2, the research team collaborated with the EHR vendor and health centers to design and build the EHR-embedded mailed FIT program, but the program itself was delivered by the clinic staff, with the research team assisting only with training and quality improvement activities. The timing, dose, and components of the intervention varied across health centers. For example, some health centers mailed letters and kits only once during the intervention year, while others completed monthly mailings. Some health centers combined the introductory letter mailing with the kit mailing and some required that the completed FIT be returned to the clinic, both were associated with decreased program effectiveness. The research provided clinics with small incentives but did not pay for program implementation. In Hybrid 3, the research team serves as consultants to the health plans and assists with evaluation but provides no training or implementation assistance. Hybrid 3 was paid for entirely by the health plans. They received an incentive for participation, but this was mainly to offset costs related to research, data capture, qualitative assessments, and time spent in meetings with the research staff.

Collaboration with the target setting is often a critical component of implementation studies and helps to ensure a smoother implementation process. In the Hybrid 1 study, the researchers designed program elements and dose, while considering the resources available in that specific setting. In the Hybrid 2 study, researchers and healthcare centers collaboratively adapted the Hybrid 1 study components based on the new settings’ resources. In the Hybrid 3 study, the health plans implementing the program adapted the program components to fit their resources, with the researchers serving as consultants only.

How an intervention is delivered is also a critical aspect of its transition from research to clinical practice. In our studies, the “agents” who delivered the intervention, as defined by Proctor et al. [5], shifted from tight to almost no research control. In the Hybrid 1 study, the research team delivered the mailings, and they also trained and monitored the clinical staff (nurses who made the follow-up calls). In the Hybrid 2 study, the research team assisted with training and implementation supports (PDSA cycles). In the Hybrid 3 study, the research team provided consultation services only, with the health plans delivering the program on their own, as would be the case once the program was institutionalized.

Our studies demonstrate that research-tested interventions for increasing CRC screening are rarely ready “right out of the box” for broad dissemination and may explain in part the long transition from research to practice. Even when the intervention is found to be effective, cost-effective, feasible, and acceptable, transitioning the intervention to day-to-day practice is a stepped and nuanced process, and one size may not fit all. Furthermore, contextual factors may lead to diminution of intervention effectiveness, resulting in additional research questions and further program adaptations.

Our analysis of the Hybrid 1, 2, and 3 SOS, STOP CRC, and BeneFIT studies has limitations. We describe the hybrid continuum of research to practice studies using only one area of research, CRC screening, and this continuum may be different than in other areas of health services implementation research. We also note that others may disagree with the hybrid category we assigned each study.

Our studies began prior to the hybrid design framework publication, and while our knowledge of the model may have influenced the design of our Hybrid Type 2 and 3 studies, we did not plan our progression a priori based on the framework. The progression of the three studies over a ten-year period was more organic, with each study answering questions posed by the prior study, and implementation progressively becoming more important than effectiveness, as the body of evidence on effectiveness grew. Lastly, the hybrid framework was intended to speed research into practice, which our studies may or may not demonstrate. Mailed FIT programs are burgeoning, but implementing sites face many implementation challenges and potential solutions that we and others continue to explore.