Development of a new adapted QuinteT Recruitment Intervention (QRI-Two) for rapid application to RCTs underway with enrolment shortfalls—to identify previously hidden barriers and improve recruitment

Well-conducted pragmatic randomised controlled trials (RCTs) are needed to provide high-quality evidence to ensure optimal patient outcomes and efficient use of resources. Tools are available to support trial design [1] or access evidence [2], and strategic initiatives aim to improve conduct [3, 4], but completing RCTs within targets and budgets remains problematic [5‐7]. Only around one half of RCTs meet their enrolment target [6]. Failures with recruitment have been identified as the biggest threat to the completion of RCTs, and poor recruitment can compromise statistical power, delay acquisition of findings, and lead to costly extensions or premature closure [8, 9]. Only a small number of simple interventions such as incentives and follow-up messages have robust quantitative evidence of effectiveness [10, 11]. Meanwhile, systematic reviews of qualitative research have called for interventions that better reflect the complexity of the enrolment process, including the communication strategies of recruiters and the perspectives of patient participants [12, 13].

The QuinteT (Qualitative research integrated in Trials) Recruitment Intervention (QRI) is one such complex intervention [14]. It was first developed during recruitment to the UK ProtecT (Prostate cancer testing and Treatment) trial comparing active monitoring, surgery, and radiotherapy for localised prostate cancer [15, 16]. This initial approach was refined through a synthesis of evidence combining ProtecT and QRIs in five other similarly challenging RCTs, which also elucidated the fragile, protracted, and evolving nature of the recruitment process [17, 18]. The QRI protocol was published in 2016 to explain the design, theoretical underpinnings, and methods [14] and to facilitate its use by other groups [19]. The QRI aims to optimise recruitment in RCTs tackling important healthcare questions and anticipating recruitment challenges because of controversy, divergent clinical opinions, or inclusion of very different interventions (such as surgery v. radiation or invasive v. conservative/de-escalation options). A QRI involves close integrated working with the RCT team from the design stage onwards, aiming to prevent the development of anticipated barriers, and then prospectively identifying and understanding the challenges that arise along the recruitment pathway from patient screening to randomisation. Remedial actions are then targeted to optimise recruitment and ensure timely completion of accrual.

The QRI is a two-phase intervention [14] (Table 1). During phase I, a researcher collects and analyses data to understand the RCT’s recruitment processes and barriers prospectively through:

The analysis of these data from phase I of a QRI finds two types of recruitment barriers: ‘clear obstacles’ (anticipated, often organisational issues known to the RCT team) and ‘hidden challenges’ (difficulties relating to underlying aspects of the RCT design that cause discomfort for recruitment staff and/or patients, and that are often unknown to—or underestimated by—RCT teams) [17]. Each RCT will have its own array of clear obstacles (such as clinic space and time, staff availability) and hidden challenges (such as variable clinician equipoise, strong patient preferences) that interact to hamper recruitment. Phase I of the QRI aims to understand and provide evidence to reveal how these obstacles and challenges impede recruitment so that they can then be discussed with the RCT team and a collaborative plan of remedial actions to tackle the barriers can then be developed and implemented during phase II [14]. Phase II actions to improve recruitment are tailored to the RCT’s particular difficulties and may include, for example, additional support and training for recruiters in relation to equipoise and presentation of the RCT, increased collection of accrual monitoring data, and unpacking terminology to improve the clarity of patient information [20‐23].

An observational study of five completed QRIs found evidence of significantly increased recruitment rates in three RCTs and study completion to time and target in all five RCTs [24]. RCTs in oncology and surgery with invasive versus monitoring or placebo arms, or comparing surgical procedures (all with very difficult recruitment issues), have completed recruitment with support from QRIs and published outcomes [16, 25‐27]. Many other RCTs with QRIs have completed recruitment and are in follow-up, or continuing to enrol. A small number ceased recruitment, with evidence explaining their insurmountable difficulties [28, 29].

The QRI was originally designed to be integrated with RCTs anticipating recruitment problems so there would be time to collect and analyse data to identify and understand the recruitment barriers (phase I), and then implement actions to optimise recruitment in phase II. However, very many RCTs launch with targets to complete accrual but then encounter enrolment shortfalls, some serious enough to threaten premature closure, even with valiant efforts by the RCT team [7‐9]. The QRI protocol was applied to some RCTs underway with enrolment difficulties when requested, but there was mixed success, primarily because of the time needed to obtain governance approvals and undertake data collection and analysis, particularly when these RCTs already needed urgent remediation [30]. The QRI required adaptations to provide more rapid insights about recruitment barriers operating in ongoing RCTs, particularly the existence and manifestation of hidden challenges [17]. This paper presents the development, application, and finalisation of the QRI-Two intervention—designed to provide a rapid ‘diagnosis’ of recruitment barriers and proposed remedial strategies to improve recruitment in ongoing RCTs with enrolment shortfalls.

The aim of this study was to investigate how to adapt the published QRI to better fit the needs of RCTs already underway with enrolment shortfalls. The objectives were to:

The study design is illustrated in Fig. 1. A CReDECI 2 checklist is provided (Table 2) outlining the process of development and evaluation of this complex intervention [31].

The findings of this study include data from all forms of QRI or QRI-Two undertaken with ongoing RCTs between 2007 and 2019 (n=32 RCTs). A QRI or QRI-Two began after a direct request for help from the RCT team or a referral from a funding/oversight body because of concerns about serious recruitment difficulties, enrolment shortfalls, or imminent closure. Funding provided for QRI-Two work varied, ranging from expenses only to funding of researcher posts. In the sections below, the development, application, and finalisation of the QRI-Two are presented (see Fig. 1 for study outline).

In this study of 32 RCTs across Europe and the USA from 2007 to 2019, a new intervention, the QRI-Two, was developed to provide opportunities to improve recruitment in RCTs underway with enrolment shortfalls. The QRI-Two was shown to be able to identify an array of previously unknown or underestimated and potentially remediable recruitment barriers in all 26 actively recruiting RCTs struggling with enrolment studied here. RCT teams had previously tried their usual improvement strategies, but were still struggling, with some facing closure. The rapid QRI-analysis of existing study documents (RCT protocol, patient information) and screening/accrual data found previously hidden or underestimated challenges linked to the intellectual and emotional demands placed on recruiters and patients by the enrolment process. Evidence of, or clues to, the existence of these hidden challenges were indicated in RCT documents by vague eligibility criteria, inconsistent descriptions of interventions, or mis-interpretable/ambiguous terminology, or in variations in accrual data patterns between sites or over time. Triangulated clues and evidence could be linked to underlying difficulties such as discomfort with equipoise and decisions to exclude some eligible patients. New insights were also provided about other issues, such as why only small numbers of eligible patients were approached in some sites, and how patient preferences could emerge from unclear study presentation. The QRI-Two workshop provided a forum for open discussion of the RCT’s newly diagnosed barriers in the context of the RCT team’s knowledge of the RCT’s design and clinical nuances, with opportunities to consider whether barriers were insurmountable or remedial actions could be developed and implemented to improve recruitment. It was feasible to apply the QRI-Two rapidly and it was confirmed that the intervention was most relevant for actively enrolling RCTs experiencing shortfalls.

Poor RCT recruitment is a common difficulty that besets many RCTs, resulting in delayed and underpowered evidence, and wasted resources. Systematic reviews of the evidence base have identified some simple interventions but only a few with evidence of effectiveness [5‐7, 10‐13]. QRIs undertaken with RCTs anticipating severe recruitment challenges have exposed the fragile, protracted, and evolving nature of the RCT recruitment process, explaining why a complex intervention with a portfolio of actions is needed to optimise recruitment [17, 18]. The original published QRI, integrated with an RCT from the design stage, offers the prospect of preventing the development of some barriers and addressing recruitment difficulties as they emerge in RCTs anticipating serious challenges [14]. The QRI-Two builds on insights about recruitment derived from completed QRIs. The ethos underlying all QRIs is that it is essential to understand the actual obstacles and challenges hampering an RCT (not just anticipated barriers) so that appropriate data can be collected, issues and discomforts can be discussed, and targeted remedial actions implemented to improve recruitment. This approach is further supported by a systematic review focused on the role of clinicians in RCTs [12] and the findings of a study of 172 RCTs discontinued due to poor recruitment that concluded most had potentially remediable barriers related to issues related to patient eligibility and strong patient and recruiter views about interventions [7].

The QRI-Two offers the opportunity to identify previously hidden but potentially remediable recruitment barriers in RCTs underway with serious enrolment shortfalls. Recruitment barriers found in the actively enrolling RCTs in this analysis included unclear study presentation, difficulties with eligibility and equipoise, losses of eligible patients on the recruitment pathway, strong patient intervention preferences, and a reluctance by recruiters to approach eligible patients. After the diagnosis of recruitment barriers in a QRI-Two, there is then the question of what to do next. The original (integrated) QRI protocol has two phases, with phase I providing an understanding of the recruitment issues (particularly hidden challenges) and phase II comprising the implementation of strategies to prevent or address challenges and optimise recruitment [14]. The finalised QRI-Two protocol is similar in that it diagnoses the barriers and identifies potential remedial strategies to improve recruitment that RCT teams need to implement. In a small number of RCTs, the QRI-Two can find that the recruitment issues are insurmountable (for example because recruiters cannot find a position of equipoise, or following changes in practice), and this may lead to the RCT closing. However, more typically the QRI-Two identifies new or underestimated barriers with clearer insights that enable the development of targeted remedial actions.

In QRIs integrated from the start, analysis of interviews and audio-recordings of recruitment appointments have provided the evidence for the development of supportive training and other actions to address recruitment challenges [15, 20‐23, 36, 37]. These remedial actions could also be implemented in RCTs after the QRI-Two. Following the rapid diagnosis of recruitment barriers, time is needed for revision and ethical approval of RCT protocol and/or patient information materials, and then implementation of actions. Such actions need care to tackle such deep-seated barriers and to avoid the development of defensiveness in RCT teams after months or years of shortfalls. Such actions and their evaluation may require additional resources during the remainder of the recruitment period or with extensions. The balance of additional costs needs to be considered in the context of the magnitude of the costs of current problems with enrolment shortfalls leading to delayed evidence, underpowered RCTs, long extensions, and premature closures.

An interesting observation among these RCTs was the limited availability of data about the recruitment process/pathway. Most RCTs were collecting some screening/accrual data, although very few used these data to investigate recruitment barriers, tending to focus on monitoring accrual rates compared with targets and transferring to the data centre only items necessary to comply with the CONSORT guidelines (simply numbers ‘assessed for eligibility’ and ‘excluded’) [38]. These crude criteria on their own reveal little about sources of recruitment difficulties. In a QRI integrated from the start, data are collected about the recruitment process using the SEAR (Screening, Eligibility, Approached, and Randomised) framework [21]. In the QRI-Two workshops, the RCT teams began to appreciate how such data could be used to identify previously hidden recruitment challenges. Some were able to use their existing target-based data in new ways, but many RCT teams and data centres were unsure about how to make protocol modifications or find resources to collect and analyse such data. The lack of data collected about the recruitment process could be a major reason why barriers are so poorly understood. Without a concerted effort, recruitment will continue in many RCTs in a ‘data-free zone’, with little prospect of ever understanding their actual (evidence-based) recruitment challenges.

The QRI-Two protocol was finalised after development in 12 RCTs and application in 20 RCTs in Europe and the USA. The QRI-Two can be rapidly applied to RCTs underway with serious enrolment shortfalls or threatened with closure to diagnose the array of previously unrecognised or underestimated recruitment challenges hampering enrolment. This provides the opportunity for RCT teams to determine whether the challenges are insurmountable or could be addressed by developing and implementing targeted remedial actions to improve the recruitment process and complete enrolment. The effectiveness of the QRI-Two in improving recruitment requires further evaluation.