Original ArticleFormatting modifications in GRADE evidence profiles improved guideline panelists comprehension and accessibility to information. A randomized trial
Section snippets
Background
Most clinical practice guidelines suffer from limitations in their methodological approach to assess and apply research evidence supporting their recommendations [1], [2]. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach provides a systematic, explicit, and transparent process for evaluating and reporting quality of research evidence and moving from evidence to recommendations [3], [4], [5], [6], [7]. Key features of the GRADE system are structured
Overview of design
We conducted a randomized controlled trial (RCT) with guideline panelists comparing two formats of the evidence profile (table A and table B) differing by four features (see Fig. 1). The content of the tested evidence profiles was based on two different clinical questions. We ran the trial during ongoing work on the AT9 guidelines. The Research Ethics Board of McMaster University approved the study.
The AT9 executive committee issued e-mail invitations to panelists to complete an online
Results
Of 116 panelists invited, 88 (76%) participated in the trial (31% females, 45% <45 years, 52% between 46 and 65 years, and 3% above 65 years of age). Of the 88 participants 24% were editors, 31% had no formal training in health research methodology, 43% had not previously participated in guideline panels, 48% had looked at five or fewer GRADE evidence profiles, and 81% were most comfortable with speaking and reading English. See Fig. 1 for flowchart. Stratification resulted in equal
Main findings
Our results show that guideline panelists preferred presentation of additional information in table cells over footnotes (Fig. 2) and absolute risk differences over absolute risk estimates (Fig. 4). These presentation features were associated with improved comprehension of key information and reduced time spent finding this information. Guideline panelists agreed that information about quality assessment and effect estimates in GRADE evidence profiles was easy to find, easy to use, and helpful
Acknowledgments
The authors thank Aravin Duraikannan for valuable help with the randomization procedure and Rachel Gutterman for help with organizing the conduct of the study.
References (16)
- et al.
GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables
J Clin Epidemiol
(2011) - et al.
GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
J Clin Epidemiol
(2011) - et al.
Methodology for antithrombotic and thrombolytic therapy guideline development: American College of Chest Physicians Evidence-based Clinical Practice Guidelines (8th edition)
Chest
(2008) - et al.
Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information
J Clin Epidemiol
(2010) - et al.
User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews
J Clin Epidemiol
(2010) - et al.
Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group
BMC Health Serv Res.
(2004) - et al.
Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive
BMJ
(2008) - et al.
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
BMJ
(2008)
Cited by (29)
A multimethods randomized trial found that plain language versions improved adults understanding of health recommendations
2024, Journal of Clinical EpidemiologyGRADE guidance 23: considering cost-effectiveness evidence in moving from evidence to health-related recommendations
2023, Journal of Clinical EpidemiologyA multimethods randomized trial found that plain language versions improved parents’ understanding of health recommendations
2023, Journal of Clinical EpidemiologyNo difference in knowledge obtained from infographic or plain language summary of a Cochrane systematic review: three randomized controlled trials
2018, Journal of Clinical EpidemiologyRE: Which estimates comprise optimal reporting in systematic reviews?
2017, Journal of Clinical Epidemiology