Understanding the applicability of results from primary care trials: lessons learned from applying PRECIS-2

Abstract

Objective

To compare two approaches for trial teams to apply PRECIS-2 to pragmatic trials: independent scoring and scoring following a group discussion.

Study Design and Setting

We recruited multidisciplinary teams who were conducting or had conducted trials in primary care in collaboration with the Pragmatic Clinical Trials Unit, Queen Mary University of London. Each team carried out two rounds of scoring on the nine PRECIS-2 domains: first independently using an online version of PRECIS-2 and second following a discussion.

Results

Seven teams took part in the study. Before the discussion, within-team agreement in scores was generally poor and not all raters were able to score all domains; agreement improved after the discussion. The PRECIS-2 wheels suggested that the trials were pragmatic, although some domains were more pragmatic than others.

Conclusion

PRECIS-2 can facilitate information exchange within trial teams. To apply PRECIS-2 successfully, we recommend a discussion between those with detailed understanding of what usual care is for the intervention, the trial's design including operational and technical aspects, and the PRECIS-2 domains. For some cluster-randomized trials, greater insight may be gained by plotting two PRECIS-2 wheels, one at the individual participant level and another at the cluster level.

Introduction

Many randomized controlled trials (RCTs) are carried out with the aim of informing health professionals, patients, and policy makers about whether an intervention should be adopted in practice. To facilitate this, it is recommended that trials taking a pragmatic design approach are carried out, testing the intervention under conditions as similar as possible to the conditions that would pertain if the intervention was rolled out in routine care [1], [2], [3], [4], [5], [6]. It may be that the intervention itself alters or replaces some aspects of usual care, but the principle remains that if the purpose of the trial is to directly inform clinical practice, aside from the intervention itself, other aspects of care should be as they usually would be in the real world. For brevity, we refer to these types of conditions as “usual care.” Recently, there has been a growing interest in the benefits from more pragmatic designs. This has led to the publication of literature on how to design and conduct pragmatic trials [7], [8], [9], [10], [11], the latest of these developments, PRECIS-2 allows researchers to plot a graph illustrating whether their design is more or less pragmatic across a number of domains.

PRECIS-2 is an updated version of PRECIS [9] with significant changes including revisions to the domains, the addition of a Likert scoring scale and a web site which can be used to support use of the tool (https://precis-2.org). PRECIS-2 (Fig. 1) has nine domains covering different aspects of a trial: eligibility, recruitment, setting, organization, flexibility of delivery, flexibility of adherence, follow-up, primary outcome, and primary analysis [12]. To apply PRECIS-2, each domain is scored from one to five—a score of one indicates an explanatory design with highly controlled or ideal conditions for the intervention and a score of five indicates a very pragmatic design, replicating usual care conditions for that domain. The tool was developed to be used at the design stage so that if the tool highlights that a trial design does not match the investigators' aims, they may choose to modify the design or reasons for the design may become more transparent. The tool can, however, also be used retrospectively as part as a critical appraisal of the generalizability of results from a trial or to illustrate a trial design to those using results.

Investigators who used the original PRECIS either discussed PRECIS wheels as a group to come to some consensus [13], [14], [15], [16], [17], [18] or used independent scoring by different team members [19], [20], [21]. Challenges in applying the tool were reported both by investigators who discussed PRECIS wheels and those that did not hold a discussion. The approach we used to apply PRECIS-2 was informed by this work [13], [14], [15], [16], [17], [18], [19], [20], [21]. In the earlier studies applying the original PRECIS, the tool was applied both in the design phase, as the authors intended PRECIS to be used, and also to trials which were already completed. Similarly, in the work described here, we consider trials at all stages in design.

Our primary aim in this work was to inform investigators how best to use the tool regardless of whether it was to be used for design or retrospectively. We were particularly interested in doing so for trials at the pragmatic end of the pragmatic–explanatory spectrum for which PRECIS-2 was designed and in exploring the difference between using the tool with and without discussion between group members.