Strengths and limitations
We systematically developed optimised patient information materials on the basis of an established and published process [
23‐
26], according to a published protocol [
16], and we report our findings in line with best practice guidance for reporting recruitment SWATs [
17]. This SWAT was fully powered and used an a priori sample size calculation, unlike most SWATs, including those conducted as part of the MRC START project, which set sample sizes on the basis of convenience [
27]. In line with the statistical analysis plan, we undertook the analysis according to the initial randomisation. However, although in the SWAT all the initial invitations were correctly sent as per random allocation to the intervention or control PIB, the ECLS team sent out further reminders in two practices to patients who had not responded, in the intervention as well as in the control arm of the SWAT, which were not optimised or randomised. The use of these reminders was a capacity decision, and at the time of the SWAT, not all patients were sent reminders; this was because sending out more fresh invitations through newly recruited GPs led to a better recruitment return than reminding non-responders. However, because the reminder letters were not randomised, their use may have diluted the effect of the recruitment intervention, although this effect is mitigated because we aimed to identify differences in proportions between the intervention and control groups, rather than absolute levels. This highlights some of the issues with undertaking recruitment SWATs, including difficulties in aligning the SWAT and host trials [
28].
A limitation of our study is that we were unable to gather data to assess any moderators of the effect of the intervention, such as age, gender, ethnicity or socioeconomic status, which may have provided additional information on the impact of the intervention in different groups. It was not the aim of the study to undertake qualitative interviews with patients sent the trial information, so we were not able to explore the wider impact of optimised patient materials beyond recruitment rates. There are also a number of different ways in which the intervention PIB was optimised (we were evaluating a particular way of producing a PIB, rather than any single change to the PIB), thus in the absence of a process evaluation or a series of trials of individual PIB changes, it is difficult to determine whether any single change or different combination of changes may have been more effective. The user-testing process may have had a positive impact in its own right by improving readability and ease of comprehension and therefore may have led to better engagement with the trial; however, we did not assess this.
In comparison with the materials used in many clinical trials, the original PIB was of high quality, with use of colour photographs and developed by a highly experienced trials team. Additionally, the original PIB was proofread by an ECLS co-investigator who was a START co-investigator. Thus it may not have been representative of the typical PIB developed by trial teams. This may have limited the potential additional benefit of the user-testing process and may explain the lack of difference between the intervention and control groups. However, our present findings with an OR of 1.016 (95% CI, 0.660 to 1.564) are in line with those of the three other SWATs undertaken as part of START, where the ORs of the user-tested versus control leaflets were as follows: 1.01 (95% CI, 0.71–1.45) [
29], 1.12 (95% CI, 0.78 to 1.61) [
18] and 1.63 (95% CI, 1.00 to 2.67) [
18]. Therefore, all current SWATs to date have found little or no effect of the intervention PIB compared with the control PIBs, suggesting that proofreading of the control PIB by a START co-investigator did not significantly impact the control leaflet in this current SWAT. The latest Cochrane review also undertook a meta-analysis of these SWATs with an overall risk difference estimate of 1% (95% CI, − 1% to 3%). The START ECLS risk difference is 0% (95% CI, − 3% to 3%) and so is entirely consistent with the other three. Trialists now routinely involve patients and the public to assist with developing information leaflets for patients, which may reduce the relative benefits of user testing.
In this SWAT ‘harm’ could include reduced recruitment in the intervention PIB group. We therefore evaluated a two-tailed hypothesis for the primary and secondary outcomes, which accepted that sending the recruitment intervention to potential participants could cause benefit or loss to recruitment for the host trial. Although patients not being recruited represent a loss to the host trial, for the patient, not being enrolled in the trial may not be harmful, because the patient may have made an informed decision not to participate. The results demonstrate that the recruitment intervention was not effective for increasing response and randomisation rates.
Comparison with existing literature
This SWAT adds to the small but emerging literature on the effects of modified information on trial recruitment. In the Cochrane review of recruitment interventions [
9], three trials explored the impact of supplementary written material on recruitment and found little evidence of benefit. As part of the MRC-START programme, this article reports the fourth SWAT evaluating the effects of optimised participant information materials on trial recruitment in different trial contexts [
18,
29]. This will enable us to determine the effectiveness of the intervention within each individual host trial and across different trial contexts and patient populations, using a meta-analysis. We are taking this approach because recruitment interventions may have different impacts according to the specific contexts, trial interventions and patient populations. In this specific SWAT, we tested the intervention in the context of a screening trial. Previous SWATs of the same intervention have been undertaken in a falls prevention trial [
29], and in two trials delivering telehealth interventions for patients with cardiovascular disease and depression [
18]. These trials have shown small increases in the numbers of patients positively responding and enrolling; however, such increases were not statistically significant. This SWAT shows similar results in a small but statistically non-significant increase in response and recruitment rates. In this SWAT, the proportions of patients responding in both intervention (19.7% response; 15.8% enrolled) and control (18.2% response; 15.6% enrolled) groups were higher than in our previous trials; for example, the Healthlines Depression recruitment SWAT achieved recruitment rates of 6.3% in the intervention group versus 4% in the control group (OR = 1.63; 95% CI, 1.00 to 2.67) [
18]. This may have been a consequence of the use of reminder letters in both the intervention and control groups. All current SWATs of this enhanced PIB intervention have been compared with original PIBs developed by highly experienced host trial teams based within UK Clinical Research Collaboration-accredited Clinical Trials Units, which have some of the most experienced teams delivering trials in the United Kingdom. It may be that the optimised leaflets may be found to be more effective if compared with leaflets developed by less experienced trial teams in the United Kingdom or elsewhere.