You are here

Article Text

Article menu
Sleep
Original article
Ethics, consent and blinding: lessons from a placebo/sham controlled CPAP crossover trial
Free
  1. Yasaman Djavadkhani1,
  2. Nathaniel S Marshall1,2,
  3. Angela L D'Rozario1,3,
  4. Megan R Crawford1,4,
  5. Brendon J Yee1,5,
  6. Ronald R Grunstein1,5,
  7. Craig L Phillips1,6
  1. 1NHMRC Centre for Integrated Research and Understanding of Sleep (CIRUS), Woolcock Institute of Medical Research, University of Sydney, Sydney, New South Wales, Australia
  2. 2Sydney Nursing School, University of Sydney, Sydney, New South Wales, Australia
  3. 3Sydney Local Health District, Sydney, New South Wales, Australia
  4. 4Sleep Disorder Services and Research Center, Rush University Medical Center, Chicago, Illinois, USA
  5. 5Department of Respiratory and Sleep Medicine, Royal Prince Alfred Hospital, Sydney, New South Wales, Australia
  6. 6Department of Respiratory and Sleep Medicine, Royal North Shore Hospital, Sydney, New South Wales, Australia
  1. Correspondence to Dr Yasaman Djavadkhani, NHMRC Centre for Integrated Research and Understanding of Sleep (CIRUS), Woolcock Institute, 431 Glebe Pt Rd, Glebe, NSW 2037, Australia; yasmina.djavadkhani{at}sydney.edu.au

Abstract

Introduction Performing rigorously designed clinical trials in device-based treatments is challenging. Continuous positive airway pressure (CPAP) is the most effective device-based treatment for obstructive sleep apnoea. We performed a randomised crossover trial of CPAP versus placebo therapy and did not disclose the presence of placebo. We assessed rates of staff unblinding, the likelihood of patient unblinding and obtained patient perceptions on lack of full disclosure.

Methods All patients (n=30) underwent a semi-structured exit interview. Prior to full disclosure patients were asked questions to ascertain whether they suspected one therapy was ineffective. The use of placebo was then disclosed and additional questions were administered to indicate the likelihood of unblinding had full disclosure occurred during consent. Staff unblinding was determined by means of a questionnaire that was completed after each patient encounter.

Results While the lack of full disclosure prevented patient unblinding during the trial, patients revealed a clear preference for active CPAP. After disclosing the presence of placebo, 73% (n=22) felt they would have been unblinded had they known at the start of the trial. Only one patient described unease about the lack of full disclosure. Staff thought they were unblinded in 6% (n=16/282) of encounters. They correctly identified the treatment device in 69% of cases (n=11/16, p<0.001).

Conclusions Successful patient blinding was achieved, however this was probably reliant on the lack of full disclosure. Staff unblinding occurred and highlights the difficulty with investigator blinding in device-based trials. Ethical challenges in this type of study are likely to compromise research feasibility.

Trial registration number This clinical trial is registered with the Australian and New Zealand Clinical Trials Registry at http://www.anzctr.org.au (ACTRN 12605000066684).

  • Sleep apnoea

https://doi.org/10.1136/thoraxjnl-2014-206354

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Key messages

    What is the key question?

    • Is it possible to blind patients and investigators to treatment allocation in randomised sham continuous positive airway pressure controlled crossover trials?

    What is the bottom line?

    • Patient blinding may be possible with lack of full disclosure but investigator blinding is unlikely to be achieved.

    Why read on?

    • This is the first study to raise important practical, scientific and ethical issues for any non-implantable medical device-based crossover trials where the maintenance of blinding depends on deliberately withholding full disclosure of the sham device.

    Introduction

    In trials using pharmacotherapy, the use of an inert tablet is usually an appropriate control for placebo effects when used in conjunction with blinding of patients and investigators. However, under the usual conditions of full disclosure, blinding of the patient is more challenging when using a non-pharmacological treatment such as a non-implantable medical device. The proportion of new treatments that are device based is increasing relative to drugs and other modalities. There is also concern about the differential standards for efficacy and safety applied to drugs and devices even when used to treat the same conditions. This has resulted in greater scrutiny of the evidence base for the effectiveness and safety of devices with the resultant need to design and encompass matching placebo devices in randomised controlled trials. However, when devices have clear and immediate physical effects, it becomes challenging to successfully blind participants and investigators under conditions where full disclosure is mandatory.

    One example of a non-implantable medical device is continuous positive airway pressure (CPAP) which is the standard treatment for obstructive sleep apnoea syndrome (OSA). It acts as a pneumatic splint of the upper airway during sleep by delivering air pressure from a pump to a mask worn on the face.

    A sham form of a CPAP device can be used as a placebo comparison for active CPAP. An active CPAP device ordinarily delivers pressures anywhere between 4 and 20 cm H2O. In a sham CPAP device, the exhalation port of the CPAP mask is increased in size, and a resistor is added between the pump and the tubing. In this way a pressure of less than 1 cm H2O is delivered to the mask whilst maintaining the same appearance and noise of an active CPAP machine.1 However, because air pressure is the mechanism of action, sham devices feel different due to a markedly lower mask air pressure compared with the therapeutic device.

    In chronic conditions such as OSA, randomised crossover trials offer an efficient way to test interventions because of their relative statistical efficiency. Patients are exposed to sham and active CPAP interventions to compare device effectiveness within patients. However, this approach has raised concerns due to the difficulty of preserving blinding. If patients were told in advance (full disclosure) that one treatment was inert it would unblind the trial because patients could immediately tell which treatment had the lower pressure. One author’s (NM) experience from a previous sham CPAP crossover trial was that patients immediately noticed the pressure differences after crossover. Though they were asked not to discuss this with study personnel, they would often mention their experience of pressure change thereby inadvertently unblinding study personnel.2 Several research groups that have recently conducted crossover trials have made it clear in their manuscripts that they did not fully disclose to patients that they would receive ineffective (placebo) treatment.2–7 This is due to concern that the resultant unblinding would render the trial scientifically uninterpretable, a concern first raised by Karlawish and Pack8 more than a decade ago. However, withholding information conflicts with the concept of true informed consent.9 A summary of disclosure patterns of published crossover trials using sham CPAP is presented in table 1.

    View this table:
    Table 1

    Summary of level of disclosure in published randomised crossover trials using sham continuous positive airway pressure (CPAP)

    Despite these concerns, no studies to date have attempted to evaluate the success of blinding in randomised crossover trials with sham CPAP. We conducted a placebo-controlled crossover trial of CPAP in which the existence of a placebo was not disclosed.6 Using data from patient interview questions and from staff questionnaires during the trial, we sought to determine whether staff unblinding occurred, whether patients thought they would have been unblinded had they known there would have been a placebo used in the trial, and patient perceptions on lack of full disclosure.

    Methods

    This is an auxiliary study of a published randomised crossover trial comparing the effects of 2 months of CPAP with sham CPAP on lipid metabolism in patients with moderate to severe OSA (apnoea hypopnoea index ≥25/h sleep).6 The active and placebo CPAP devices (Remstar Auto; Philips Respironics, Murrysville, Pennsylvania, USA) were identical. All other details regarding the study protocol may be found in the original report.6

    The patient information sheet disclosed that patients would be using two CPAP machines that ‘will deliver pressure in a different way’. They were also told that one of the aims of the study was to determine ‘whether the way in which a CPAP machine delivers pressure is important in determining which machine you prefer to use’.

    Our local ethics committee was concerned about the lack of full disclosure and its effects on informed consent. However, they also recognised the additional scientific problem that would be introduced by the trial becoming unblinded. They agreed to approve the study, inclusive of withholding knowledge of the placebo device from patients, provided further investigation was performed to assess the impact of this withheld knowledge. Full disclosure was made at an exit interview with each patient before study discharge. In this interview, the reasoning for not fully disclosing the nature of the placebo device during the consenting process was explained.

    Final approval was sought from the Ethics Committee (RPAH Zone) of the Sydney South West Area Health Service.

    Patient exit interview

    Patients underwent a semi-structured exit interview at the time of completion or withdrawal from the study. All interviews were undertaken by the same investigator (NM) who remained nominally blinded to treatment allocation. Online supplementary appendix A lists the scripted prompts and questions that were used by the interviewer with patients. Patients were initially asked numerous questions about their treatment experience/preferences. This was designed to elicit from the patient whether they suspected the existence of a placebo or non-efficacious treatment. Subsequently, an unblinded investigator (ALD) took over the interview and debriefed the patients on the true nature of the study. They asked the patients whether they felt they would have been unblinded if there had been full disclosure at the start of the study, and how they felt about not having been told that there was a placebo treatment used in the trial. Patients were asked what their bed partners thought about the relative performance of each machine.

    Staff questionnaires

    Staff members were asked to complete the questionnaires after any type of encounter with the patient to determine whether they had been unblinded. Encounters included events such as venepuncture. The questionnaires were not completed after every single patient encounter as we had intended, as study personnel were often busy. The exact denominator, or number of staff–patient encounters, is unknown. If staff thought they were definitely unblinded, they were reassigned so as to no longer have contact with the patient.

    Statistical analysis

    We used descriptive statistics, frequencies and percentages to describe our data. χ2 tests were used to test whether staff treatment allocation guesses were statistically correct more often than 50% of the time. Mixed model analysis of variance was used (SAS V9.3) to test whether adherence rates differed between treatments and whether the order in which treatment was received affected adherence. Patient numbers were used as random effects, and treatment, order and order by treatment interaction were fixed effects.

    Results

    In the original study, 38 patients were randomised and 29 completed the trial. Thirty-four patients started treatment but three withdrew almost immediately after initiating treatment. Staff questionnaires were obtained for the remaining 31 patients. Of these, 30 patients experienced both treatment arms and subsequently underwent the exit interview. One patient withdrew prior to completion of the second arm. No patients suspected the presence of a placebo during the trial.

    Patient exit interview

    The patient perceptions of the two treatment arms are described in table 2. Before being told that there was a placebo, the majority of patients identified the treatment arm with active CPAP as the preferred treatment, felt that it was better for their sleep and preferred to use it in the long term. More patients thought their bed partner would report that CPAP was more effective than placebo.

    View this table:
    Table 2

    Patient perceptions of the two treatment arms, active continuous positive airway pressure (CPAP) and sham CPAP, at the exit interview (n=30)

    After telling patients that there was a placebo, 73% (n=22, p=0.02) stated they felt they would have been able to determine which device was the placebo during the trial if full disclosure had occurred during the consenting process.

    Only one patient stated that he felt slightly uncomfortable that full disclosure did not occur. All other patients reported that they understood why full disclosure had not occurred and that withholding this information was warranted. The interviewer also noted that very few patients remembered the contents of the informed consent documents they had signed and many had not retained these even though the trial was less than 6 months in duration. Some patients could not recall that there had been such a document.

    Staff questionnaires

    Staff questionnaires were completed for 31 patients. There were 282 staff–patient encounters documented. The number of staff encounters recorded per patient averaged 9 (SD 3, range 3–15 per patient). Figure 1 illustrates the results of the staff questionnaires.

    Figure 1
    Figure 1

    Staff perceptions on patient treatment assignment after each staff–patient encounter.

    Staff thought they were definitely unblinded in 6% (n=16/282) of recorded encounters and then mostly correctly identified the treatment (n=11/16, 69%, p<0.01). Staff thought they might have been unblinded in 22% (n=61/282) of recorded encounters and they typically guessed correctly (n=44/61, 72%, p<0.01). Of the 55 correct guesses/unblinding episodes, 21 occurred in the first arm and 34 occurred in the second arm (p=0.11).

    Adherence

    Adherence was compared in those that started with active CPAP then crossed over to sham CPAP, and vice versa (table 3). CPAP adherence was highest in those who started with active CPAP and reduced significantly on commencing sham CPAP (5.6 vs 3.5 h). In those who started with sham CPAP, adherence was low and remained low after commencing active CPAP (3.3 vs 3.2 h).

    View this table:
    Table 3

    Continuous positive airway pressure (CPAP) adherence rates in each arm before and after crossover

    Discussion

    In this study, we sought to determine whether staff unblinding occurred in our sham CPAP crossover trial. We also assessed the likelihood of patient unblinding had full disclosure occurred during the consenting process. We purposefully did not disclose the presence of a placebo in an effort to preserve blinding. Informing patients that the study aimed at testing ‘two different deliveries of pressure’ rather than telling them that one treatment would be ineffective meant that patients should not have been able to have predetermined perceptions of reduced benefit in either arm. Our results demonstrate that the vast majority (72%) of patients felt that they would have been able to identify the placebo treatment had they been informed at the start of the trial. Prior to unblinding the patient, although no patient suspected that sham CPAP was used when directly prompted, the majority of patients were able to identify active CPAP as the more effective treatment. Examination of the staff–patient encounters reveals that unblinding occurs amongst staff. When staff members thought that they had been unblinded they were usually correct. Any degree of unblinding is undesirable and this study highlights the practical difficulties in preserving double blinding in a sham CPAP crossover trial. We believe that staff blinding would be equally as problematic in parallel studies of sham CPAP.

    We found that adherence was influenced by type of treatment and by order of treatment. First, adherence was lower on sham CPAP regardless of order of administration. In our trial, this was to be expected given the clear differences in patient preferences. However, those that commenced on sham CPAP followed by active CPAP continued to have lower adherence, potentially due to their discouraging initial experience. This may imply that adherence is predictably affected by order of treatment interaction, also noted by other investigators.4 ,10 This highlights a shortcoming of crossover trials.

    Amongst the crossover studies that did not disclose the presence of placebo, all but one study demonstrated a clear discrepancy between adherence rates in each arm, with lower rates in the sham CPAP arm.2–7 The only study that showed equivalent rates of adherence between arms was a study performed in patients with mild OSA.2 These patients had no clear preference for active CPAP, presumably due to milder symptoms and reduced symptomatic benefit. Overall, in these studies in which there was lack of full disclosure there was lower use of sham CPAP devices.

    In contrast to the majority of crossover trials, Weinstock et al10 was the only group that we are aware of that clearly disclosed the existence of a placebo device at the time of consent (Susan Redline, personal communication). This study does not appear to have had significant issues with dropouts or dismal compliance on the sham arm as we may have predicted. However, they did find significantly lower adherence in the sham arm, particularly if it was provided on the second arm. One interpretation of this apparent success may be that patients did not remember the contents of the informed consent documents. Additionally, it is difficult to make conclusions about the effect of full disclosure based on only one study.

    Interestingly, patients did not object to the lack of full disclosure when it was revealed to them. It may be because such a high proportion felt that their behaviour would have been influenced by this knowledge. From our interviews it appears that informed consent documents were not valued by patients. They often did not remember what was in them, or that they existed. They often did not retain their provided copies. This suggests that these documents may not be serving their intended function. Even though our study had not intended to investigate patient perceptions of informed consent documents in clinical trials, it was apparent through the interviews that patients in our trial derived very little if any value from them.

    Limitations include that we were unable to capture every single staff–patient encounter as they were numerous and the task relied on staff completion on every encounter. This might have led to preferential completion of the questionnaire after unblinding events. As such, the data might reflect spontaneous adverse event reporting data where unblinding events are more likely to be reported. In addition, after the exit interview and after full disclosure, we did not to verify that the 73% of patients who thought they could identify the active treatment actually could. A further limitation was that we never ascertained from the patients exposed to sham CPAP last, whether full disclosure would have resulted in them being less inclined to use it. This information would be important for ethics committees when considering future trials.

    The proportion of device-based treatment is on the rise. Rigorous research in this area differs to pharmacologic agents and is challenging with practical difficulties. Investigator blinding is difficult if not impossible to achieve in parallel and crossover design trials. We have found that with the use of sham CPAP in a crossover trial, the only solution to maintaining patient blinding and scientific integrity is to abstain from disclosing to patients the existence of a placebo. This in turn creates an ethical dilemma and is a challenge that warrants further attention and discussion.

    Acknowledgments

    We would like to thank Brendan Funnell for graphical design assistance, and also Kerri Melehan, Gislaine Gauthier and Dianne Richards for their involvement with the study. We would like to thank the patients for their valuable time, without which the study would not have been completed.

    References

    1. ↵
      1. Farré R,
      2. Hernández L,
      3. Montserrat JM, et al
      . Sham continuous positive airway pressure for placebo-controlled studies in sleep apnoea. Lancet 1999;353:1154. doi:10.1016/S0140-6736(99)01056-9
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Marshall NS,
      2. Neill AM,
      3. Campbell AJ, et al
      . Randomised controlled crossover trial of humidified continuous positive airway pressure in mild obstructive sleep apnoea. Thorax 2005;60:427–32. doi:10.1136/thx.2004.032078
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. Robinson GV,
      2. Smith DM,
      3. Langford BA, et al
      . Continuous positive airway pressure does not reduce blood pressure in nonsleepy hypertensive OSA patients. Eur Respir J 2006;27:1229–35. doi:10.1183/09031936.06.00062805
      OpenUrlAbstract/FREE Full Text
    4. ↵
      1. Coughlin SR,
      2. Mawdsley L,
      3. Mugarza JA, et al
      . Cardiovascular and metabolic effects of CPAP in obese males with OSA. Eur Respir J 2007;29:720–7. doi:10.1183/09031936.00043306
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Cross MD,
      2. Mills NL,
      3. Al-Abri M, et al
      . Continuous positive airway pressure improves vascular function in obstructive sleep apnoea/hypopnoea syndrome: a randomised controlled trial. Thorax 2008;63:578–83. doi:10.1136/thx.2007.081877
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Phillips CL,
      2. Yee BJ,
      3. Marshall NS, et al
      . Continuous positive airway pressure reduces postprandial lipidemia in obstructive sleep apnea: a randomized, placebo-controlled crossover trial. Am J Respir Crit Care Med 2011;184:355–61. doi:10.1164/rccm.201102-0316OC
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Jones A,
      2. Vennelle M,
      3. Connell M, et al
      . The effect of continuous positive airway pressure therapy on arterial stiffness and endothelial function in obstructive sleep apnea: a randomized controlled trial in patients without cardiovascular disease. Sleep Med 2013;14:1260–5. doi:10.1016/j.sleep.2013.08.786
      OpenUrlCrossRefPubMed
    8. ↵
      1. Karlawish JH,
      2. Pack AI
      . Addressing the ethical problems of randomized and placebo-controlled trials of CPAP. Am J Respir Crit Care Med 2001;163:809–10. doi:10.1164/ajrccm.163.4.ed0201d
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA 2013;310:2191–4. doi:10.1001/jama.2013.281053
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Weinstock TG,
      2. Wang X,
      3. Rueschman M, et al
      . A controlled trial of CPAP therapy on metabolic control in individuals with impaired glucose tolerance and sleep apnea. Sleep 2012;35:617–625B. doi:10.5665/sleep.1816
      OpenUrlPubMed
      1. Phillips CL,
      2. McEwen BJ,
      3. Morel-Kopp MC, et al
      . Effects of continuous positive airway pressure on coagulability in obstructive sleep apnoea: a randomised, placebo-controlled crossover study. Thorax 2012;67:639–44. doi:10.1136/thoraxjnl-2011-200874
      OpenUrlAbstract/FREE Full Text
      1. McEwen BJ,
      2. Phillips CL,
      3. Morel-Kopp MC, et al
      . Diurnal changes and levels of fibrin generation are not altered by continuous positive airway pressure (CPAP) in obstructive sleep apnoea (OSA). A randomised, placebo-controlled crossover study. Thromb Haemost 2012;108:701–9. doi:10.1160/TH12-04-0231
      OpenUrlCrossRefPubMed
      1. Arias MA,
      2. García-Río F,
      3. Alonso-Fernández A, et al
      . Pulmonary hypertension in obstructive sleep apnoea: effects of continuous positive airway pressure: a randomized, controlled cross-over study. Eur Heart J 2006;27:1106–13. doi:10.1093/eurheartj/ehi807
      OpenUrlAbstract/FREE Full Text
      1. Alonso-Fernandez A,
      2. García-Río F,
      3. Arias MA, et al
      . Obstructive sleep apnoea-hypoapnoea syndrome reversibly depresses cardiac response to exercise. Eur Heart J 2006;27:207–15. doi:10.1093/eurheartj/ehi621
      OpenUrlAbstract/FREE Full Text
      1. Arias MA,
      2. García-Río F,
      3. Alonso-Fernández A, et al
      . CPAP decreases plasma levels of soluble tumour necrosis factor-alpha receptor 1 in obstructive sleep apnoea. Eur Respir J 2008;32:1009–15. doi:10.1183/09031936.00007008
      OpenUrlAbstract/FREE Full Text
      1. Alonso-Fernandez A,
      2. García-Río F,
      3. Arias MA, et al
      . Effects of CPAP on oxidative stress and nitrate efficiency in sleep apnoea: a randomised trial. Thorax 2009;64:581–6. doi:10.1136/thx.2008.100537
      OpenUrlAbstract/FREE Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors RRG, NSM and CLP were responsible for the conception and design of the study. NSM, ALD, YD, CLP and MRC were responsible for the acquisition of data. NSM and CLP performed the statistical analyses. All authors contributed to the interpretation of data, drafting and revising of the article and final approval of article. YD is the guarantor.

    • Funding Funding for the original study was obtained from the Australian National Health and Medical Research Council (NHMRC) project grant 301936 (RRG). This analysis was funded by NHMRC Practitioner Fellowship APP1022730 (RRG), Career Development Fellowship 1061545 (CLP) and the NHMRC Centre for Integrated Research in Understanding of Sleep fellowship (YD).

    • Competing interests We received support from Philips Respironics, the CPAP manufacturer that provided the active and placebo/sham CPAP machines for use in this study.

    • Ethics approval Granted by Sydney South West Area Health Service, NSW, Australia (Royal Prince Alfred Hospital Zone) with protocol number X05-0128.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement The data from this study are available on request.