Skip to main content
main-content

01.12.2018 | Research | Ausgabe 1/2018 Open Access

Trials 1/2018

Impact of question order on prioritisation of outcomes in the development of a core outcome set: a randomised controlled trial

Zeitschrift:
Trials > Ausgabe 1/2018
Autoren:
Sara T. Brookes, Katy A. Chalmers, Kerry N. L. Avery, Karen Coulman, Jane M. Blazeby, on behalf of the ROMIO study group
Wichtige Hinweise

Electronic supplementary material

The online version of this article (doi:https://​doi.​org/​10.​1186/​s13063-017-2405-6) contains supplementary material, which is available to authorized users.

Abstract

Background

Core outcome set (COS) developers increasingly employ Delphi surveys to elicit stakeholders’ opinions of which outcomes to measure and report in trials of a particular condition or intervention. Research outside of Delphi surveys and COS development demonstrates that question order can affect response rates and lead to ‘context effects’, where prior questions determine an item’s meaning and influence responses. This study examined the impact of question order within a Delphi survey for a COS for oesophageal cancer surgery.

Methods

A randomised controlled trial was nested within the Delphi survey. Patients and health professionals were randomised to receive a survey including clinical and patient-reported outcomes (PROs), where the PRO section appeared first or last. Participants rated (1–9) the importance of 68 items for inclusion in a COS (ratings 7–9 considered ‘essential’). Analyses considered the impact of question order on: (1) survey response rates; (2) participants’ responses; and (3) items retained at end of the survey.

Results

In total, 116 patients and 71 professionals returned completed surveys. Question order did not affect response rates among patients, but fewer professionals responded when clinical items appeared first (difference = 31.3%, 95% confidence interval [CI] = 13.6–48.9%, P = 0.001). Question order led to different context effects within patients and professionals. While patients rated clinical items highly, irrespective of question order, more PROs were rated essential when appearing last rather than first (difference = 23.7%, 95% CI = 10.5–40.8%). Among professionals, the greatest impact was on clinical items; a higher percentage rated essential when appearing last (difference = 11.6%, 95% CI = 0.0–23.3%). An interaction between question order and the percentage of PRO/clinical items rated essential was observed for patients (P = 0.025) but not professionals (P = 0.357). Items retained for further consideration at the end of the survey were dependent on question order, with discordant items (retained by one question order group only) observed in patients (18/68 [26%]) and professionals (20/68 [29%]).

Conclusions

In the development of a COS, participants’ ratings of potential outcomes within a Delphi survey depend on the context (order) in which the outcomes are asked, consequently impacting on the final COS. Initial piloting is recommended with consideration of the randomisation of items in the survey to reduce potential bias.

Trial registration

The randomised controlled trial reported within this paper was nested within the development of a core outcome set to investigate processes in core outcome set development. Outcomes were not health-related and trial registration was not therefore applicable.
Zusatzmaterial
Additional file 1: Table S1. Nurses: percentage of items rated essential within the non-comparative and comparative context (a consistency effect). (DOCX 13 kb)
13063_2017_2405_MOESM1_ESM.docx
Additional file 2: Table S2. Surgeons: percentage of items rated essential within the non-comparative and comparative context (a contrast effect). (DOCX 13 kb)
13063_2017_2405_MOESM2_ESM.docx
Additional file 3: Table S3. Male patients: percentage of items rated essential within the non-comparative and comparative context (a consistency effect). (DOCX 12 kb)
13063_2017_2405_MOESM3_ESM.docx
Additional file 4: Table S4. Female patients: percentage of items rated essential within the non-comparative and comparative context (a consistency effect). (DOCX 12 kb)
13063_2017_2405_MOESM4_ESM.docx
Additional file 5: Table S5. Health professionals (aged ≤ 50 years): percentage of items rated essential within the non-comparative and comparative context (a contrast effect). (DOCX 12 kb)
13063_2017_2405_MOESM5_ESM.docx
Additional file 6: Table S6. Health professionals (aged > 50 years): percentage of items rated essential within the non-comparative and comparative context (a contrast effect). (DOCX 12 kb)
13063_2017_2405_MOESM6_ESM.docx
Literatur
Über diesen Artikel

Weitere Artikel der Ausgabe 1/2018

Trials 1/2018 Zur Ausgabe