Promoting Recruitment using Information Management Efficiently (PRIME): a stepped-wedge, cluster randomised trial of a complex recruitment intervention embedded within the REstart or Stop Antithrombotics Randomised Trial

PRIME is a closed-cohort, stepped-wedge, cluster randomised trial investigating a complex intervention to boost recruitment in RESTART. PRIME is an embedded recruitment trial within the RESTART parallel-group randomised trial. The stepped-wedge design involved a sequential roll out of the PRIME complex intervention to clusters (active hospital sites in RESTART) over twelve 1-month time periods [8]. The PRIME trial is an example of a closed-cohort, stepped-wedge trial because all sites at the start of the trial were expected to continue until the end of the study and no new sites were to be added [9]. All sites began in the control state (no intervention). The month in which the sites started the PRIME complex intervention was randomly allocated so that groups of sites began the intervention sequentially at equally spaced time intervals (steps) as shown in Fig. 1.

The PRIME complex intervention involved a recruitment co-ordinator discussing ways to improve recruitment to RESTART at each site via teleconference with the principal investigator and/or the local team, focussing on the provision of software for each site with instructions on how to extract from national stroke audit data lists of their own patients who were potentially eligible for RESTART. There was then a second teleconference to review progress 6 months later. We sent each site a questionnaire before the recruitment review and then a follow-up questionnaire before the 6-month follow-up review. Full details of the PRIME complex intervention, along with inclusion/exclusion criteria and the sample size calculation can be found in the PRIME protocol publication [10].

We created bespoke stroke audit data exports for PRIME in collaboration with the Scottish Stroke Care Audit (SSCA) and the Sentinel Stroke National Audit Programme (SSNAP) teams, to reflect the RESTART eligibility criteria as closely as possible. We provided sites with instructions on how to produce these exports once they had received their recruitment review. Further details can be found in the PRIME protocol [10].

Each audit database produced two bespoke RESTART audit reports (short and long) which could be run separately. The short reports for both were most likely to have suitable patients for RESTART whilst the long reports widened the criteria, and contained further potentially eligible patients. Once the reports were generated the site staff checked that the patient met all the eligibility criteria, using the patient’s medical records and/or via their general practitioner.

The 6-month follow-up questionnaire asked sites to collect information on the number of patients randomised directly from the audit reports.

Active hospital sites in RESTART were invited to take part in PRIME in June 2015. There were 109 active hospital sites in the RESTART collaboration in the UK at that time. Twenty-four were excluded: two sites in Northern Ireland (where stroke audit data collection was not consistent); two sites in Scotland and two sites in England that piloted the bespoke stroke audit data exports; 14 sites recently activated to RESTART in 2015 (to ensure that all sites taking part had several months recruiting before receiving the intervention); and the remaining four excluded sites were unsuitable for other reasons. The PRIME recruitment co-ordinator telephoned the local RESTART co-ordinator at each of the remaining 85 sites in ascending order of site identification number to invite them to take part in a ‘recruitment review’, following up by email if required, until the sample size target of 72 sites agreed, as shown in the PRIME complex intervention flow chart (Fig. 2). Sites were not informed about the exact content of the recruitment review or that its timing would be randomly allocated. The recruitment co-ordinator and staff at each participating site remained blinded to the timing of the recruitment review until 2 months before the month allocated for each site’s review, when this had to be revealed in order to organise the review.

We used stratified block randomisation to randomly allocate the 72 sites into 12 groups of six sites each, with stratification by hospital location (Scotland vs. England/Wales) to ensure that the proportions of sites with access to each national stroke audit data source were approximately consistent across the 12 groups. A programmer independent of the trial created the computer-generated random allocation sequence and sent this list to the trial data manager who then informed the recruitment co-ordinator at the required times for them to arrange the review.

A timeline cluster diagram is shown in Fig. 3 to clarify the timing of trial processes and blinding.

Full details of the statistical methodology can be found in the published Statistical Analysis Plan [11]. Briefly, the primary outcome of recruitment rate per site per month was analysed using a negative binomial generalised linear mixed model (GLMM), including the fixed-effect explanatory variables: number of months since start of the embedded PRIME study (represented by a linear term in the model to adjust for any background changes in trial recruitment rate), season (December/January versus all other months), site location (Scotland versus England/Wales), and an indicator variable for whether the complex intervention was due to have been implemented or not (according to the planned randomisation schedule). Site was also included as a random effect in the model.

The primary analysis followed an ‘as-randomised’ intention-to-treat principle, which meant that we analysed the data according to the allocated timing of the recruitment review, rather than the time that the recruitment review actually occurred. In addition, for the primary analysis, all 72 sites were included regardless of any site withdrawals or site suspensions or compliance with the PRIME trial procedures. For sites that were suspended (i.e. not permitted to recruit patients due to the principal investigator leaving or another reason), their missing observations were imputed as ‘0’ patients randomised. This was believed to be an appropriate imputation since it is likely, based on the reasons for suspension (e.g. due to the principal investigator being off sick), that many of these sites who were suspended would have failed to randomise any patients if they remained in the trial, and also ‘0’ was by far the most frequently observed number of patients randomised per month. However, to check the robustness of this assumption and the impact of any deviations from the protocol, a per-protocol secondary analysis was performed after excluding any sites experiencing permanent suspensions or sites failing to receive either the initial recruitment review or the 6-month follow-up review.

The median delay between the scheduled randomised time of review (first day of the month allocated) and the actual time the recruitment review was delivered was 11.5 days (interquartile range (IQR) 8 to 19.75 days; range 0–71 days). In the 6-month period before any site received the PRIME complex intervention (March 2015 to August 2015), the total number of patients randomised was 0 for 47 sites (65%), 1 for 18 sites (25%), 2 for 6 sites (8%), and 3 for 1 site (1%).

Every time the recruitment review was conducted, we recorded whether there were any issues which impacted the PRIME complex intervention delivery. For 10 sites (14%), the sound quality for the teleconference was quite poor and for two sites (3%) it was very poor. This was due to a mixture of reasons including, sites using speaker phones and separate phones being used in the same room causing echo. For nine sites (12%) the principal investigator was absent from the review, and for three recruitment reviews (4%) the research co-ordinator was ill which may have affected delivery. For the 48 remaining sites (67%), we delivered the recruitment review with no issues.

Table 1 shows responses from the pre-recruitment review questionnaire. Eighteen sites (32%) were already using stroke audit data to recruit to RESTART; the most popular information source to identify eligible patients was screening logs (40 sites, 71%).

A complete summary of baseline site characteristics and responses to the pre-review questionnaire stratified by randomised group are shown in Additional file 1.

The change in the cumulative randomisation total (primary outcome) over time is shown in Fig. 5, with shading to indicate the number of sites that had actually received the recruitment review (i.e. started the intervention) up to that point. Separate plots for each of the randomised groups of sites are shown in Additional file 2.

Table 2 shows the results of the primary analysis on the primary outcome. The adjusted rate ratio for the primary outcome (PRIME complex intervention versus control condition) was estimated to be 1.06 (95% confidence interval 0.55 to 2.03, p = 0.87) after adjusting for site, December/January months, site location (Scotland versus England and Wales), and time in months since the start of the embedded study.

The results were very similar and conclusions unchanged after performing other pre-specified sensitivity analyses (see Additional file 3). In particular, after restricting the analysis to the intervention roll-out period only (September 2015 to July 2016), the adjusted rate ratio became 1.07 (0.54 to 2.13, p = 0.84). Fitting a second-order spline function to model any secular changes over time improved the precision of estimation and reduced the width of the confidence interval slightly, but there was no change to the conclusions (RR 1.06, 95% CI 0.58 to 1.92, p = 0.86).

Fifty-one out of 72 sites (71%) returned the 6-month follow-up questionnaire, and 30 of these (59%) were known to have generated and used the audit data extract reports.

Out of the 30 sites using the audit reports, five sites (17%) ‘strongly agreed’ that the reports were useful, 15 sites ‘agreed’ that the reports were useful (50%), eight sites were ‘neutral’ (27%), and two sites ‘disagreed’ that the reports were useful.

Sixteen sites (53%) indicated that they either had a problem running or using the reports or listed disadvantages to using the reports. The most common disadvantage given was lack of time (seven sites, 23%); and a further three sites (10%) had problems with resource issues: for example, getting consultants or other staff to run the reports or review medical notes. Five sites (17%) complained that the procedure was not identifying many extra patients beyond those already screened as inpatients. Four sites (13%) specified that the reports were picking up a high number of ineligible patients and patients who had died. Two other sites (7%) were concerned about accuracy of the reports, indicating that ‘patients who are already in the trial are not in the list’, and ‘[some patients] weren’t a bleed at all’.

One site that used the audit reports made a suggestion for improvement: they wrote that they would have liked to have had ‘more information regarding understanding the report once produced, and how to link ID numbers to actual patients’.

Twenty-one sites answered ‘no’ to the question of whether they extracted and used the audit data extract reports. The most common barrier cited was resource/staffing/time issues (10/21, 48%); followed by use of other screening methods to identify RESTART patients (6/21, 29%); and problems running the reports or accessing the SSNAP database (6/21, 29%).

Where sites did create audit reports, these were run to an average of 3 years back (range 0.5–6 years) (see Table 3). Seventeen sites (68%) ran the audit reports once; seven sites ran the reports twice (28%); and one site ran the reports three times.

Summary statistics for the number of patients identified by the reports, numbers invited back to clinic and numbers responding are shown in Table 3. Sites had the option to record their answers split by audit report (i.e. short and long), or if preferred, they had the option to record the combined total using the ‘Both’ category.

The number of eligible patients invited to a screening visit, declining to come to a screening visit, and numbers randomised is shown in Table 4.

Under the ‘Short’ report category, two sites each reported one patient coming back to clinic; both patients were randomised. Under the ‘Long’ report category, one site reported a single patient coming back to clinic, but the patient was not randomised by the time of the 6-month questionnaire. Under the ‘Both’ category, the site with two patients coming back to clinic randomised one of them, and the site with three patients coming back to clinic managed to randomise all three of them. Therefore, the total number of patients randomised was six (half of whom were randomised through a single site). Although it should be noted, for this particular site, the data collected on the 6-month questionnaire was not from the bespoke audit data reports – they extracted a list of all ICH patients from the national audit data.

A strength of our embedded recruitment trial is that it has a cluster randomised design. Stepped-wedge or parallel-group cluster randomised designs are preferable to alternative non-randomised designs, such as before-and-after studies, because they are less susceptible to confounding bias due to temporal trends [15]. In cases where the potential harms or burden due to an intervention are known to be low, then the stepped-wedge design should be considered because all clusters receive a potentially efficacious intervention, and the trial may be more efficient compared to a parallel-group cluster trial. For recruitment trials embedded in multi-centre trials, it may be expected that there is a high level of heterogeneity in recruitment rates between sites due to substantial differences in site size and, therefore, a stepped-wedge trial design may confer greater power.

A limitation of our study is that the PRIME complex intervention was only implemented by a single recruitment co-ordinator which may limit the generalisability of results to other trials and settings. Even so, a detailed protocol was followed, so it is unlikely that this was a major limitation. The most important factor limiting generalisability was the fact that PRIME was embedded within a single host trial. If the intervention was implemented in other trials and settings the results may have been different.

In contrast, the START programme [16] was designed to test two recruitment interventions consisting of (1) optimised patient information sheets and (2) multimedia approaches to encourage patient involvement in research, across multiple host trials, which has provided more generalisable results compared to confining a recruitment intervention to a single host trial. The START investigators showed that an optimised patient information sheet did not significantly improve recruitment or retention rates into the REFORM trial [17].

A second but less important limitation relates to the 6-month follow-up questionnaire results. There was a limited response rate from sites to the first page of the 6-month questionnaire asking for number of patients randomised from the audit reports, which suggests that there may have been randomisations that resulted from the audit reports that were not recorded. Moreover, some sites reported that they were still working through the lists at the time of the 6-month review. Therefore, the self-reported results of the numbers randomised from the audit reports may only represent partial information. It should also be noted that for at least a few sites, the numbers given were from other lists of patients (apart from the audit reports) and some sites accessed the audit data directly without using the bespoke audit reports, albeit this could still be regarded as an indirect consequence of the intervention.

Although our study showed no significant effect of the PRIME complex intervention, this does not necessarily mean that similar recruitment interventions involving searching electronic patient records will not be effective in other trials or settings. Indeed, in primary care, using electronic patient records has been shown to boost recruitment to ongoing RCTs [18, 19]. Further work is needed to establish the effectiveness of using electronic patient records in secondary care. Alternative ways to increase recruitment may also need to be investigated, particularly those which minimise impact on clinician time and resources.

PRIME was registered with the Northern Ireland Hub for Trials Methodology Research SWAT repository (SWAT22, http://​bit.​ly/​2a4n7Yb) on 23 December 2015 and was submitted to the Online Resource for Recruitment research in Clinical trials (ORRCA, www.​orrca.​org.​uk).

The host trial, RESTART, aimed to recruit 720 participants over 2 years, from May 2013 to May 2015 based on recent epidemiological data [3, 4]. Due to under-recruitment, RESTART’s funder granted an extension of recruitment until 31 May 2018. The trial end date is February 2019. PRIME’s first recruitment review took place in September 2015 and the last review completed in August 2016. The first 6-month follow-up review took place in March 2016 and the last one completed in February 2017. After the final 6-month follow-up was received there was a period of 3 months to analyse the data, meaning that PRIME completed in May 2017.