Developing a framework for the ethical design and conduct of pragmatic trials in healthcare: a mixed methods research protocol

The SUPPORT trial [22] sought to determine the optimal level of supplemental oxygen on incidence of retinopathy of prematurity and mortality in preterm infants. Infants were randomized to high and low oxygen saturations, both of which fell within the range used routinely in practice. Lower oxygen levels were found to reduce retinopathy but increase mortality, leading the authors to urge caution in their use. In response to an anonymous complaint by parents, the US Office for Human Research Protections launched an investigation into the trial [23]. It determined that investigators failed to adequately inform parents of “reasonably foreseeable risks” as consent materials did not list visual impairment and death as research risks. This incited considerable debate in the literature [24‐27] centered almost exclusively on considerations of risk and risk communication and reflecting a preoccupation with US regulations. A central ethical issue raised by the SUPPORT trial is: Should usual care interventions be considered part of research or clinical practice? Other issues raised are: What constitutes usual care? Can a study be considered “minimal risk” if its endpoints include serious impairments such as blindness or death? Do parents need to be informed that their child is participating in a trial? At what level of detail do the study interventions need to be disclosed to participants? What potential benefits and harms ought to be disclosed? What type of research ethics review is appropriate?

The FLUID trial is a pragmatic cluster randomized cross-over trial comparing resuscitation with Ringer’s Lactate versus normal saline on death and hospital readmissions in hospitalized patients, with outcomes assessed using health administrative data [28]. Both fluids are considered usual care interventions that have been available for decades and are administered to many hospitalized patients. While fluid administration is an individual-level intervention, the study becomes feasible only when interventions are implemented as hospital policies (i.e. at the cluster level) with a waiver of patient informed consent. In particular, it is essential to have only one type of study fluid available throughout the hospital to minimize the risk of contamination due to a patient receiving both types of study fluids in different areas of the hospital. It would also be logistically challenging and very costly to recruit and randomize individual patients within all areas of participating hospitals. In contrast, the availability of routinely collected data available for all patients permits the conduct of a very cost-efficient trial involving hospitals across the province, without the need to recruit patients for data collection. However, the FLUID trial raises several ethical issues: What is an appropriate justification for adopting cluster randomization? When individual-level interventions are implemented institution-wide as a policy in a cluster randomized trial, may one proceed without patient consent? Do patients need to be notified about the trial; if so, how? Is consent required for the use of routinely collected participant data?

The Surgical Checklist Trial was a stepped wedge trial randomizing surgical units in two hospitals in Norway to evaluate the impact of the World Health Organization Surgical Safety Checklist on morbidity, mortality, and length of hospital stay [29]. The stepped wedge design is characterized by the fact that clusters (here, surgical units) cross gradually and in random sequence from the control to the intervention condition, with all clusters exposed to the intervention by the end of the trial. As stated by the investigators, the 19-item checklist consists of an oral confirmation by surgical teams of the completion of the basic steps for ensuring safe delivery of anesthesia, prophylaxis against infection, effective teamwork, and other essential practices in surgery. Compliance with the checklist was assessed prospectively by nurses. All patient outcomes were collected from hospital administrative databases. The regional research ethics committee advised the investigators that the study was considered clinical service improvement and that research ethics approval and patient informed consent were not required. The Surgical Checklist Trial raises several ethical issues: Should trials evaluating quality and service improvement be considered research? What are the ethical implications if health system leaders conduct (potentially less robust) quality improvement studies rather than rigorous pragmatic randomized controlled trials or refrain from conducting important studies due to perceived ethical barriers? Should health professionals targeted by study interventions be considered research participants? If study participation poses no more than minimal risk, is patient or provider consent required? What is the harm–benefit balance of exposing all surgical units to the intervention by the end of the study?

The objective of this study is to conduct interviews with a small group of pragmatic trial experts and stakeholders (trialists, ethicists, methodologists, chairs of research ethics committees, health system leaders, quality improvement experts, and patient representatives on research study teams) to generate a thorough understanding of types of ethical issues arising in the practice of pragmatic trials from a variety of perspectives. Informants will be selected using a purposive sampling strategy, augmented through snowball sampling. Potential interviewees will be selected across a broad range of jurisdictions and clinical areas to reflect a range of experiences, including lower- and middle-income countries. The sample size will be determined to ensure representation from all targeted stakeholders and by when saturation is reached (i.e. when new interviews cease to provide fresh information) [30‐34]. Based on our prior experience with similar studies [35], we anticipate that 24–40 interviews will be required. Interviews are expected to last 1 h and will be audio recorded. A semi-structured interview guide will allow participants to respond freely, to illustrate concepts, and to present perspectives that the interviewer can probe further. To monitor the progress of the interviews and permit follow-up of issues that may emerge from the data, interviewing, transcription, and analysis will proceed concurrently. The interview guide may evolve as a typology of ethical issues begins to emerge. Recordings will be transcribed and verified before analysis. Data will be imported into a qualitative software package (NVivo 11) to facilitate thematic coding, evaluation, and analysis. The results will be used to formulate a typology of ethical issues arising from pragmatic trials, to be addressed in the conceptual work. It will also inform data extraction and questionnaire items for studies 2–5.

The objectives of this study are to select and review a random sample of recently completed and ongoing trials that have more pragmatic (than explanatory) aims, to describe ethical characteristics, identify ethical challenges reported, the circumstances under which they arise, and how they are being addressed. We anticipate challenges in identifying a sample of “pragmatic” trials given wide variation in definitions and inconsistent and unreliable reporting of trial design. We will develop objective and reproducible criteria to characterize a trial as having pragmatic aims, as well as the conditions under which those criteria apply. We will work with an information scientist to develop a sensitive and specific electronic search strategy to identify a sample of trials. To develop and validate an electronic search strategy, we will use a multi-pronged approach to identify a gold standard set of trials meeting our criteria for testing the search strategy, including: (1) pragmatic trials conducted by the investigator team and our extensive networks; (2) pragmatic trials identified in the key informant interviews; (3) databases of funded pragmatic trials in Canada, the USA, UK, France, and Australia; (4) a database of pragmatic trials maintained by the PRECIS group [36]; (5) demonstration projects by the National Institutes of Health Collaboratory [37] and PCORnet [38]; and (6) trials included as exemplars in recent publications about pragmatic trials. The proportion of these pragmatic trials retrieved by the search strategy will be calculated and used to refine the search strategy if necessary as done in our previous work [39]. Once validated, the search strategy will be used to select a random sample of 300 pragmatic trials. As the ethical landscape may have been changing in recent years, we will include completed trials as well as study protocols for ongoing trials. Eligibility criteria, to be refined, will include trials or study protocols published in the past five years by investigators in Canada, UK, USA, France, and Australia, including trials conducted in lower- and middle-income countries. Items for extraction will be generated based on team discussion and key informant interviews. After pilot testing, two reviewers will independently extract data from each trial report. Discrepancies between reviewers will be identified and resolved by discussion with a third reviewer if required. Prevalence of ethical issues and practices arising from pragmatic design features, study design, and type of study interventions will be described overall and within subgroups of interest where feasible (e.g. over time, between countries, study sponsors). Methodological and reporting quality of the included trials will be collected and evaluated against major methodological and reporting criteria as done in our previous work [40, 41]. The preliminary ethics framework will be updated as extractions proceed.

The objective of this study is to gather more detailed information about practices and experiences identified in our review of pragmatic trials. After pilot testing, we will administer a survey consisting of open- and closed-ended items to corresponding authors of the sample of 300 trials. The primary mode of survey administration will be web-based, but alternatives (paper, telephone) will be considered to increase the response rate. A series of contacts (pre-notifications, notifications, and reminders) based on Dillman’s recommendations for the implementation of mail and Internet surveys will be used [42]. The survey will be used to characterize the ethical conduct, review, and reporting of pragmatic trials from the perspectives of trialists. We will offer respondents a $30 gift certificate or donation on behalf of the respondent in appreciation for their time. We will use questionnaire personalization—a previously published methodology developed by our team—to gather more detailed information about aspects of the published trial [43]. The anticipated response rate, based on previous experience with this population and methodology, is 65% [44]. Results from the survey will be compared to results from the published trial to assess adequacy of reporting of ethical issues and to describe implications of design choices (e.g. impact of alternative consent models on study recruitment and risks of selection bias). Potential non-response bias will be assessed by comparing characteristics of respondents and non-respondents using information in trial reports. We will describe the use of gatekeepers (i.e. individuals or bodies that represent the interests of community members, communities, or organizations [45]), use of consent waivers or alternative consent models (e.g. “streamlined consent” [12]), and details about information conveyed to participants in each study arm. We will describe the type of ethics review required, perceived impact on the timing of implementation, ethical and scientific quality of the trial, and uniformity of process and decisions in multicenter trials. We will explore the possibility of requesting informed consent documents and research ethics application forms and protocols for a subset of trials. Results will inform the ethics framework for analysis in the conceptual work.

The objective of this study is to gather information on the views, practices, and experiences of research ethics committee chairs in Canada, the USA, UK, France, and Australia. We will aim to select all research ethics committees that review clinical trials in Canada (approximately 200), the UK (approximately 100), Australia [46] (approximately 200), France (approximately 39) [47], and a random sample of 200 from over 9000 Institutional Review Board Organizations in the USA [48]. We chose these five countries primarily based on logistical considerations: our team members have connections with research ethics organizations in these countries which will help facilitate participation. Canada does not maintain a list of research ethics committees. We will use a strategy previously developed by our team to identify eligible committees [49]. It involves integrating internet searches with a list of Institutional Review Board Organizations maintained by the US Office of Human Research Protections. Given that biomedical and non-biomedical Institutional Review Boards are not differentiated in this list, we will use stratification by National Institutes of Health funding levels to increase the efficiency of identifying Boards with relevant experience reviewing clinical research.

Questionnaire items will be informed by the preliminary ethics framework, key informant interviews, and results from the trialist survey. Questionnaires will consist of open- and closed-ended items and include a series of scenarios. After pilot testing, a series of contacts (notifications and reminders) based on Dillman’s recommendations for Internet surveys will be used [42]. We will likely encounter challenges in ensuring an adequate response rate; based on previous experience with surveying this population, we expect a response rate of approximately 35% [49]. We will deliberately keep questionnaires short and adhere to recent recommendations for improving response rates [42]. Results will be summarized using descriptive statistics and compared across subgroups (e.g. country, size and type of committee, years of experience). Where feasible, potential non-response bias will be assessed by comparing characteristics of respondents and non-respondents using information on research ethics committee websites. Questionnaires will be prepared in both English and French. Open-ended responses will be analyzed thematically. Results will inform the ethics framework for analysis in the conceptual work and knowledge translation activities to research ethics committees.

The objective of this study is to gather information on the views and experiences of trial participants or prospective trial participants (e.g. patients), gatekeepers (organizational leadership, medical directors), and communities. Our research team members are involved in 15–20 pragmatic trials at any one time. We will identify one ongoing or recently completed pragmatic trial in each country, ensuring a range of types of pragmatic trials. We will conduct focus groups with eligible trial participants (with permission from the responsible research ethics committees and the chief investigator). We anticipate five focus groups with patients and five with gatekeepers. Focus groups will be 1–2 h in length and involve six participants per group. A semi-structured discussion guide will be used to gather information on participants’ experiences with the trial including recruitment, informed consent, perceived benefits and harms, any privacy concerns, and satisfaction with the trial. Among eligible prospective participants, we will explore potential reasons for non-participation. Discussions will be recorded, transcribed verbatim, and verified by the facilitator before analysis in NVivo 11 [50]. To monitor progress and permit follow-up of issues that may emerge from the data, discussions, transcription, and analysis will proceed concurrently [51]. For focus groups not conducted in French, interview guides will be developed in English by the study team and then translated into French by bilingual members of the research team. To verify the accuracy of the translation, all guides will be independently back-translated. All focus group discussions will be transcribed in the language in which the group was conducted. Non-English transcripts will be translated into English and then independently back-translated to the original language. A Canadian pragmatic trial will be used to design a survey of community members targeted by the trial (e.g. diabetes patients, hospital patients). Quantitative and qualitative analyses will be used to summarize results from the community survey. Results will be used to inform the ethical analysis in the conceptual work.