Background
In contrast to traditional randomized clinical trials (RCTs) that are designed to evaluate the efficacy and safety of a drug or intervention under ideal conditions, pragmatic trials aim to compare the effectiveness, benefits, and harms of interventions under real-world conditions [
1]. Pragmatic trials embedded within usual clinical care delivery, or “learning health systems,” are designed to maximize the external validity of the knowledge they generate [
2‐
4]. By facilitating rapid enrollment of large numbers of patients, pragmatic trials can markedly reduce recruitment challenges and high costs that characterize many traditional RCTs [
5‐
15]. However, the benefits of trials embedded in clinical care must be considered in the context of the loss of patient and physician autonomy that they may impose especially if entire hospitals or clinics are randomized as a cluster to a specific treatment approach. Moreover, such trials may be conducted with waivers of informed consent. Several ethicists have noted that standard informed consent procedures may not be applicable in pragmatic trials comparing interventions that fall within the “standard of care” [
16‐
19] because such trials do not increase risk to patients beyond the risk of usual care. However, such trials alter the process by which an individual’s care is selected, thereby limiting patients’ and physicians’ decisional autonomy [
20].
As promising as pragmatic trials may be for improving the quality and costs of care [
2], little empirical work has addressed these ethical concerns [
17,
21‐
23]. In a recent qualitative study, we examined how patients and physicians value physician decisional autonomy and traditional informed consent in the context of pragmatic trials [
24]. In that study, patients receiving maintenance dialysis generally accepted restrictions on physician decisional autonomy and, when necessary to achieve the goals of the study, endorsed simple notification rather than formal consent. Physicians generally expressed similar views, though some expressed greater reservations about forgoing opt-in consent [
24]. Informed by these qualitative results, we designed the current study to quantify how strongly patients and physicians value physicians’ treatment autonomy relative to other desirable study features such as minimization of participant burden and opportunity for informed consent. We conducted a survey-based discrete choice study among patients with end-stage renal disease (ESRD) receiving maintenance hemodialysis and with physicians to determine the relative influence of these study attributes on willingness to participate in a pragmatic dialysis trial.
Methods
Setting and participants
Patients receiving maintenance hemodialysis for ESRD were recruited from 11 outpatient dialysis facilities in the greater Philadelphia region between June 2015 and March 2016. Exclusion criteria included age <18 years, initiation of dialysis within the previous 90 days, inability to speak English, and inadequate visual or cognitive function as determined by each facility’s charge nurse. Potential participants were approached during a dialysis session by a trained research coordinator. After providing informed consent, participants completed the survey using a tablet computer. Patient participants received compensation of US$10 after survey completion.
Physician members of the American Society of Nephrology (ASN) were invited to participate via email between July 2015 and February 2016. Eligible physicians were practicing nephrologists treating adult patients in outpatient dialysis units in the USA. Potentially eligible physicians were randomly selected from the ASN membership list and sent email invitations in two separate waves starting in July and December 2015, respectively, followed by two reminder emails over a 4-week period. Physician participants received a US$50 gift card for completing the study. The study was approved by DaVita Clinical Research (#18-2015) and the Institutional Review Board of the University of Pennsylvania (#819593).
Survey instrument and study design
The survey consisted of a presentation of scenarios, each describing a hypothetical cluster-randomized clinical trial comparing two approaches to blood pressure management for patients receiving maintenance hemodialysis. The basic trial design was held constant across the scenarios but trial attributes were varied. The survey content was informed by our previous qualitative study in which semi-structured interviews were used to understand views of patients and physicians about protocolized treatment approaches for clinical trials [
24]. We chose two of the three attributes evaluated in this study based on themes from these interviews: (1) the effect of protocolized care on physician treatment autonomy (treatment autonomy); and (2) information exchange between patients and physicians as it relates to the mechanism of consent (consent approach). Although the third attribute, participant burden, did not emerge as a major theme in the qualitative study, most patients and physicians interviewed did express concern about the burden of participation, making it appropriate for further evaluation. For the treatment autonomy attribute, we varied the physician’s role in selecting a blood pressure target for the patient and choosing which anti-hypertensive medications to use. For the participant burden attribute we varied the testing required for the trial from routine non-invasive blood pressure measurement to monthly blood draws and an additional research visit separate from dialysis treatment. For the consent approach attribute we varied the trial’s enrollment mechanism (opt-in consent, opt-out consent, notification-only, no-notification) (Additional file
1).
A full factorial design using these three attributes produced 16 scenarios (2 autonomy levels × 2 burden levels × 4 consent levels): however, we restricted the number of scenarios to 8 per participant by randomly assigning each to receive one of six groups of 8 scenarios to minimize participant fatigue [
25]. In each group, two of the four consent approaches were tested across the two levels of autonomy and two levels of burden. The 6 groups entailed all possible pairwise comparisons of the four consent attributes. We pilot-tested the computerized presentations with 5 patients and 14 physicians to assess acceptability of the format and language, and modified them accordingly. The final survey was written at an 8th-grade literacy level and supplemented with cartoon illustrations to promote readability.
The scenarios for physicians contained more medical terminology than those directed to patients (Additional file
1: Appendix A) but had comparable content. Physicians were informed that oversight of this large, adequately powered hypothetical trial would be provided by Institutional Review Boards and an independent Data and Safety Monitoring Board, and that patients would be provided with the following statement:
Patients will not be included in the study if their doctor thinks they should not participate.
The sequence for the eight scenarios in each group was determined by electronic randomization to mitigate potential ordering effects. After reviewing each scenario, participants used a 4-point scale to indicate willingness to have their dialysis facility participate in the clinical trial (definitely participate, probably participate, probably not participate, definitely not participate). An a priori decision was made to dichotomize willingness to participate into either a “Yes” (definitely or probably participate) or “No” (probably not or definitely not participate) for the primary analysis.
Because of limited prior research exposure among this patient population [
24], patients were asked to view a 3-minute animated video entitled “Approaches to Research in Medical Practice” from the Research on Medical Practices (ROMP) Ethics Study (rompethics.iths.org) prior to presentation of the clinical trial scenarios. Patients then completed a 5-item knowledge test about randomization and standard-of-care research, with the correct answers revealed afterwards (Additional file
1: Appendix B).
After completion of the scenarios, patients completed the Research Attitude Questionnaire [
26], a 7-item measure that assesses patients’ support for and value of biomedical research (theoretical scores range from 7 to 35, with higher scores indicating more favorable views). Patients also completed the 9-item Revised Health Care System Distrust Scale [
27] to capture their trust in the values and competence of the healthcare system (theoretical scores range 9 to 45, with higher scores indicating more distrust). Finally, we collected demographic information about patients (age, gender, race, ethnicity, employment status, income, comorbidities, hospitalizations in the past year, and years receiving dialysis therapy) and physicians (age, gender, race, ethnicity, years in practice, practice setting, and past experience with clinical trials).
Statistical analysis
The primary outcome, willingness to have the dialysis facility participate in the trial, was analyzed using hierarchical random-effects logistic regression with an exchangeable working correlation structure and robust error variance estimators to account for the clustering of the responses by each participant [
28,
29]. Prior to model building, we examined all continuous variables using locally weighted scatterplot smoothing to determine the appropriateness of entering such variables as linear terms. No variable required transformation. The associations of participant characteristics with willingness to participate were assessed using
t tests and Chi-square tests for continuous and categorical variables, respectively. We decided a priori to include all three trial attributes (exposures) in the baseline models for both patients and physicians. Other variables were added to the model if they had a
p value <0.20 for the relationship between the variable and the primary outcome in a model that also included the three exposure variables. A priori we decided to also run an adjusted model without including either the healthcare system distrust or research attitude score. These data are not presented here as the results did not differ from the primary model. For physicians, none of the demographic variables met the inclusion criterion, thus the final model includes only the three trial attributes. We additionally explored potential interactions among all three trial attributes by adding two-way interaction terms one at a time to the final models for both patient and physician participants.
We based our sample size estimate on the number of trial attributes tested, the number of levels of each of these attributes, and the desire to detect an odds ratio (OR) of 0.67 for a main effect of an attribute or an OR of 0.5 for an interaction (between any two trial attributes) on the willingness to participate. We targeted 200 patients and 200 physicians for enrollment, yielding > 80% power to declare significance at
p = 0.05. This calculation assumed that the “design effect,” [
30] reflecting the number of responses per participant and the correlation of responses within participants, would be no greater than 6. The design effect is calculated as:
$$ 1+\uprho \left(\mathrm{k}-1\right), $$
where
ρ is the intraclass correlation and
k is the number of scenarios per participant [
31].
For all analyses, two-sided p values <0.05 were considered statistically significant. Statistical analyses were conducted in Stata (v14.1, StataCorp, College Station, TX, USA) and R Studio (v1.0.136, RStudio Inc., Boston, MA, USA) using the R language (v1.0.136, R Foundation for Statistical Computing, Vienna, Austria) and the R package ggplot2 (v2.2.1, Springer-Verlag, New York, NY, USA).
Discussion
In this study of patients with dialysis-dependent end-stage renal disease and physicians, we did not detect an effect of curtailed treatment autonomy or research burden on willingness to participate in a hypothetical pragmatic trial, but did find an effect of the approach to consent. Both patients and physicians indicated levels of comfort that were similar for opt-out, participant notification, and traditional opt-in consent, but were less willing to participate when participants would receive no notification about the trial. However, even with no notification, two thirds of patients indicated willingness to have their facilities participate.
Patients and physicians are accustomed to interactions in which decisions about treatment approaches are either shared between the patient and the physician or directed by the physician. In traditional RCTs, therapeutic approaches are determined by chance rather than by choice [
34], and such restrictions may dissuade patients or physicians from participating in the study. However, in cluster-randomized pragmatic trials patients may not have an option to avoid trials that restrict treatment autonomy because entire hospitals or clinics may be assigned to provide the same treatment approach for all eligible patients. Even when randomization is at the individual patient level, for some pragmatic trials the requirement for informed consent is waived and thus access to alternative treatments is eliminated. Interestingly, for patients and physicians in the current study, limitations on treatment autonomy did not reduce willingness to have their dialysis facility participate in the trial. Moreover, patients were as likely to participate in a trial that relied on opt-out consent or a notification of enrollment as in a trial that used opt-in informed consent, and two thirds of patients were willing to participate even in the absence of any notification. This finding is particularly notable in light of increased distrust in medical care and biomedical research among African Americans [
35], who constituted the majority of participants in this ESRD cohort. Indeed, we found that race was not associated with willingness to participate in the hypothetical pragmatic trials.
Strikingly, the hypothetical pragmatic trials presented in our surveys were more acceptable to patients than physicians. This may be explained, at least in part, by the desire of some physicians to rely on their own experience to determine the best treatment for an individual patient [
24]. However, notably, one third of physicians expressed willingness to have their dialysis facility participate in a trial conducted without any patient notification. Another interesting finding is that, like patients, physicians’ willingness to participate was not influenced by degree of burden to patients resulting from the trial. The absence of an effect of this attribute might be due to the small incremental burdens for this group of patients of extra testing and blood collection relative to thrice-weekly hemodialysis treatments. It is also possible that effects smaller than we were powered to detect might be evident in a larger sample.
Overall, our findings suggest that a pragmatic clinical trial comparing standard-of-care interventions for blood pressure management is generally acceptable to patients receiving maintenance dialysis and to their treating physicians. Among those we evaluated, the only influential attribute for patients and physicians was the consent approach, a finding that is relevant to ongoing discussions about the use of alternatives to the “traditional” opt-in consent in pragmatic clinical trials [
36]. In particular, so long as studies were transparent and included notification to patients, they were generally acceptable to both patients and physicians. Understanding the views of multiple stakeholders is important for informing these discussions. A strength of this study is that we assessed the views of both patents and physicians, and found generally similar results for these two groups.
The findings of our study need to be considered in light of several limitations. The most important is that responses of the patients and physicians to hypothetical RCTs may not reflect the decisions they would make if they were considering participation in actual trials. However, structured vignettes have been shown to be a valid methodology in other clinical settings for producing results that accurately reflect those from more expensive and time-intensive non-hypothetical experimental procedures [
37], and hypothetical preferences are the single best predictor of future enrollment decisions [
38]. A second limitation is that the valuations of these trial attributes and willingness to participate may not reflect those of patients receiving hemodialysis in other geographic areas or with other diseases, or physicians from other specialties. Likewise, results may vary among trials testing different types of interventions. We chose to focus on hemodialysis patients and physicians because of the potential for dialysis care delivery as a setting for pragmatic trials [
39]. The findings from this study should inform future studies in different clinical contexts. An additional limitation is the low survey response rate by physicians that occurred despite pilot testing the study instrument and offering a US$50 financial incentive conditional on completing the survey. Physicians are historically difficult to engage in survey research [
40], and we acknowledge the possibility of non-response bias in the physician component of this study. Somewhat reassuringly, the age and sex of our respondents mirrored those of a random sample of physician members of the ASN. Finally, the hypothetical trials used in this study tested only three trial attributes, and it is possible that additional attributes (e.g., risk to patients) may influence participation decisions. However, we informed our attribute selection from interviews with patients and nephrologists [
24], and chose to constrain the number of attributes to avoid overburdening participants.
Acknowledgements
The authors are grateful to assistance from Kristiina Negron, MS, and Ashley Kraybill, MPH, and to the patients and staff members at the participating DaVita dialysis facilities.