High-quality evidence regarding the risks and benefits of treatments is lacking across a multitude of clinical specialties [
1‐
7], and the clinical research enterprise is not focused on generating the kind of information that drives national guidelines for clinical care. Similarly, traditional clinical trials are often unable to provide high quality evidence in a timely or cost-effective manner [
8,
9]. High quality evidence is typically generated by conducting randomized controlled trials (RCTs) using costly stand-alone, non-reusable systems, which may be separate from clinical care, designed with rigorous inclusion and exclusion criteria, and conducted under ideal conditions [
10]. These design features are a double-edged sword: they help ensure that findings reflect true variation as a result of an intervention, but have been criticized as being so specific as to not be generalizable to broader populations and settings. Many are calling for a change, including the National Academies of Medicine (formerly the Institute of Medicine), to a system in which data for research are gathered during routine clinical care to enable continual learning, i.e., a “learning healthcare system” [
11].
Pragmatic clinical trials (PCTs) are trials that use data collected in the electronic health record (EHR) as part of routine care, or are “embedded” in routine care, and are a foundational component of such a system. By their nature, PCTs are designed to show real-word effectiveness in broad, generalizable patient groups (as opposed to the more restricted protocols and populations found in exploratory randomized trials) [
12]. These PCTs have the potential to significantly decrease the evidence gap and inform real-world practice with digital health data collected at the point of care. They involve critical partnerships between health care systems and academic investigators to embed clinically meaningful research questions into the infrastructure of the health system to generate real-world generalizable results in an efficient manner.
While embedded PCTs hold great promise and much work has been done to describe the continuum of pragmatic versus explanatory trials for considerations in the design phase of PCTs [
13,
14], information summarizing real-world experiences and best practices with PCTs is scant; there is no clear framework for deciding when PCTs would be optimal. The current ethical, regulatory and logistical systems were created primarily with more “traditional” RCTs in mind [
15], and policy makers are still working to understand the unique challenges associated with PCTs. To speed this learning process, the Common Fund of the National Institutes of Health (NIH) created the Health Care Systems Research Collaboratory (Collaboratory) in 2012. The Collaboratory’s mission is to strengthen the national capacity to implement cost-effective, large-scale PCTs, by supporting the design and execution of a series of pragmatic trials, or demonstration projects (Table
1), with the intent to learn how best to design, conduct, and disseminate the results of these PCTs. To date, the Collaboratory has funded the ten demonstration projects described here and worked with the Project Principal Investigators (PIs) to overcome challenges and barriers to designing, conducting, and disseminating results from their PCTs. In addition, an NIH funding opportunity announcement (RFA-RM-16-019) was developed as a part of the Common Fund initiative to support more projects over the next several years. These projects address questions of public health importance, include a large, generalizable population of patients, and engage healthcare delivery organizations as research partners [
16]. Members of the Collaboratory Core Groups (described in more detail below) worked with the PIs to facilitate the PCTs and have reported on solutions to the challenges encountered. These include a special issue on the ethics of research in usual care settings [
17,
18], a special issue on the ethical and regulatory complexities of pragmatic clinical trials [
15,
19‐
29], journal articles on initiating and implementing patient-reported outcomes measures [
30], electronic health records, phenotyping, and informatics [
31‐
33], stakeholder engagement and health care systems interactions [
34‐
37], as well as biostatistical lessons learned on cluster and constrained randomization [
38,
39]. Members of the Collaboratory have also developed a Living Textbook on the design, conduct, and dissemination of PCTs (
http://rethinkingclinicaltrials.org/).
Table 1
NIH Collaboratory Demonstration Projects
2012 | | | |
Pain Program for Active Coping and Training ( PPACT; NCT02113592) | Help patients adopt self-management skills for chronic pain, limit use of opioid medications, and identify factors amenable to treatment in the primary care setting [ 2] | Patients with chronic pain on long-term opioid therapy (~1000+ patients) in three staff model health plans | Cluster randomization by primary care provider |
Strategies and Opportunities to Stop Colorectal Cancer in Priority Populations ( STOP CRC; NCT01742065) | Improve the rates of colorectal-cancer screening by mailing fecal immunochemical testing tests to patients at Federally Qualified Health Centers | Individuals eligible for screening per the US Preventive Task Force guidelines (~35,000 patients) in 26 Federally Qualified Health Center clinics | Cluster randomization by Federally Qualified Health Center clinic |
Suicide Prevention Outreach Trial ( SPOT; NCT02326883) | Compare outcomes in patients who receive care-management or online skills training for suicide prevention versus usual care in three healthcare systems | Individuals at elevated risk for suicide on a depressions scale (~19,500 patients) in three large healthcare systems | Individual randomization |
Time to Reduce Mortality in End-Stage Renal Disease ( TiME; NCT02019225) | Determine whether increasing the durations of hemodialysis sessions reduces mortality and hospitalization rates for patients receiving maintenance hemodialysis care | Adults who have initiated treatment with maintenance hemodialysis within the past 120 days (~6800 patients) in 266 dialysis facilities operated by two dialysis provider organizations | Cluster randomization by dialysis facility |
Lumbar Image Reporting with Epidemiology ( LIRE; NCT02015455) | Determine if inserting epidemiological benchmarks (essentially representing the normal range) into lumbar spine imaging reports reduces subsequent spine-related tests and treatments | 100 clinics (~230,000 patients) in four health systems | Cluster randomization by clinic. Stepped-wedge one-way crossover design |
Active Bathing to Eliminate Infection ( ABATE; NCT02063867) | Determine if using antiseptic bathing for all patients plus nasal ointments for patients harboring methicillin-resistant Staphylococcus aureus (MRSA) reduces multidrug-resistant organisms and bloodstream infections | Patients in adult medical, surgical, oncology, and step-down units. (~600,000 patients) in 53 hospitals | Cluster randomization by hospital |
Blood Pressure Medication Timing Study (BPMedTime) | To compare adverse cardiovascular events in patients who are instructed to take their currently prescribed once-daily antihypertensive medications at bedtime compared with patients who continue to take their once-daily antihypertensive medications in the morning or afternoon | Initial plan was to test this question in 1000 patients, but to have enough power, it was determined that 5000 patients were needed | This trial was not transitioned to the implementation phase |
2014 | | | |
Pragmatic Trial of Video Education in Nursing Homes ( PROVEN; NCT02612688) | Determine if showing advance care planning videos in nursing homes affects the rates of hospitalization among frail, demented residents | 2 nursing home health systems (360 nursing homes) serving long-stay (>12 months) patients with advanced comorbid conditions (~15,000 patients) | Cluster randomization by nursing home |
Improving Chronic Disease management with Pieces ( ICD-Pieces; NCT02587936) | Improve care for patients with chronic kidney disease, diabetes, and hypertension by using a novel technology platform (Pieces) that uses the electronic health record to identify patients and by assigning practice facilitators within primary care practices or community medical homes | Patients with multiple co-morbid conditions (chronic kidney disease, diabetes, and hypertension; ~11,000) in 124 primary care practices in four healthcare systems | Stratified cluster randomization of clinical practices within health systems |
Trauma Survivors Outcomes and Support ( TSOS; NCT01625416) | Coordinate care and improve outcomes for trauma survivors with post-traumatic stress disorder(PTSD) and comorbidity | Expect 40 patients at each center (~960 patients) in 24 US trauma centers | Cluster randomization by trauma center. Stepped-wedge one-way crossover design |
Although there have been numerous publications describing nuanced challenges and solutions encountered in the conduct of specific Demonstration Projects, this is the first to widen the focus and describe the NIH Collaboratory more generally. We build on the knowledge created by the Cores to review important generalizable lessons learned over the first four years since the Collaboratory was established. We also discuss the remaining opportunities and challenges for the Collaboratory and its forthcoming projects, and for the medical community more broadly regarding PCTs to power a learning health system.