Background
Randomised controlled trials (RCTs) are considered the gold standard in research design for evaluating interventions [
1,
2]. However, issues surrounding recruitment and retention are common, which often lead to delays, extra costs, protocol changes or the altogether abandonment of trials [
3]. Poor or slow recruitment, whereby the target sample number is not reached within the anticipated timeframe, can lead to smaller sample sizes, which can give rise to sampling bias, limiting statistical power, and may increase the possibility of a type 2 error [
4,
5]. For example, within the UK, the National Institute for Health Research (NIHR) invests substantial funding in health intervention research; in 2014 and 2015, this amounted to £237.6 million, with a considerable proportion allocated to RCTs [
6]. Yet in Raftery et al.’s [
3] review of 125 NIHR-funded RCTs, the authors found that 43.1% of studies submitted a time extension request citing recruitment issues. Walters et al. [
6] reported that only 56% of 151 studies conducted under NIHR funding in the UK between 2004 and 2016 reached their recruitment target. Due to the time, cost, and resource implications of recruitment issues, the identification of more effective recruitment methods has become a priority [
7,
8].
Study within a trial
A developing area of trial methodology, which can be used to investigate recruitment strategies, involves embedding a study within a trial (SWAT). SWATs are a means of evaluating methodology and formally assessing how effectively a study is running. Treweek and colleagues [
9] defined a SWAT as a ‘self-contained study that has been embedded within a host trial with the aim of evaluating or exploring alternative ways of delivery or organising a particular trial process’ (p. 143). The experimental design of SWATs means they are valuable for testing hypotheses and investigating whether traditional approaches adopted historically can be refined or adapted as research practices evolve. As Shah points out in the paper by Gheorghiade et al. [
10], it is ironic that we use trials to generate evidence-based treatments, whereas we rely largely on anecdotal approaches as to what we think works to inform their conduct, rather than developing empirical evidence to identify best practice.
The value of SWATs is increasingly being recognised, with several developments moving to raise their profile and encourage their routine use in RCTs [
11]. These include online research platforms specifically designed to address recruitment and retention challenges within RCTs, such as Trials Forge [
12] and Clinical triAls [
13]. These online resources primarily aim to share knowledge from existing studies and build a platform to collate the findings and inform new hypotheses. Another initiative, the Prioritising Recruitment in Randomised Trials [PRioRiTy;
14] study has identified leading priorities concerning recruitment and retention into RCTs, using a priority setting partnership approach involving the public, carers and healthcare professionals. Amongst the ten leading questions identified were the need to identify how best to design and deliver information on RCTs to the public and the advantages and disadvantages of using technology during recruitment. This focus is timely as the dramatic influx of technology into everyday life provides opportunities to enhance practice using alternative media to present trial information in an efficient, interactive, and potentially low-cost way, for example through video clips, social media and/or informational websites.
Currently sharing an information sheet or form is the conventional format for providing study information to potential participants. Traditionally, this information sheet or participant information sheet (PIS) is a printed text document that aims to provide potential participants with comprehensive details of the study. These often become long and complex to ensure accordance with ethical guidance [
15] and may not be visually appealing [
16]. Previous research has shown that after receiving a PIS, participants may still not be fully informed, often failing to understand key aspects of the study, such as the right to withdraw [
17] and potential side effects [
18]. Thus, it is likely that the traditional PIS could be a barrier to study participation for certain demographic groups. In light of these potential limitations of the PIS, the Health Research Authority in the UK has advocated for the investigation of alternative media for participant material [
19].
Indeed, there has been an increase in the use of multimedia implemented within trial design (e.g. websites, social media, audio and video clips). It has been suggested that video clips may be an acceptable means of promoting understanding of participation in RCTs with a small but growing number of studies assessing their utility. Meropol et al. [
20] looked at the use of a web-based platform to explain key features of a cancer RCT and found that video clips (when compared to text) increased patient knowledge and decreased attitudinal barriers. However, a recent Cochrane review investigating recruitment strategies, which included the effects of video clips versus standard information in three oncology trials, found that there was insufficient evidence to draw conclusions about the effects of clips [
21].
More recently, Jolly et al. [
22] conducted a recruitment SWAT in a RCT for people with mild symptoms of chronic obstructive pulmonary disease to compare the use of a multimedia resource and standard PIS materials. The multimedia resource shared information about the RCT and included video clips of patients discussing their experiences of participation. No group differences in enrolment were found, although the authors suggested that the multimedia format may not have been best matched to the information preferences of the participants (largely men over 70 years) and may be better suited to populations with higher general consumption of multimedia. Related to this, the TRECA study (Trials Engagement in Children and Adolescents) [
23] has conducted user testing to explore the use of a multimedia platform in healthcare trials with children, adolescents and their parents [
24]. Participants commented positively about the interactive resource, with varied preferences of information medium; some preferred animations and video clips whereas others preferred text. These resources are due to be embedded and tested in six healthcare trials.
Research using SWATs to investigate the use of multimedia in recruitment processes could identify more effective and efficient recruitment methods in RCTs. As the utility of multimedia may differ depending on the target group, there is a need to investigate its inclusion during recruitment of diverse patient and public populations. SWATs themselves create a vital opportunity to evaluate methodology in real world trials; however, reporting of SWATs remains limited. Through sharing the challenges faced and the key learning points from the present study, we hope to provide information to help support the routine inclusion of SWATs in RCTs.
Aim
This study aimed to use a SWAT to explore whether the inclusion of a video clip alongside a standard PIS improved the rate of recruitment, compared to a standard PIS alone. We further aimed to report our experience of embedding a SWAT during an active RCT and the challenges encountered, and offer reflections for consideration by other researchers.
Discussion
Contrary to our expectations, the inclusion of a video during recruitment did not result in an improvement in recruitment rates in this RCT of a parenting intervention. However, the interpretation of the results should be made cautiously given the small sample size and the exploratory nature of the study. This finding is consistent with both the findings of Jolly et al. [
22] and Campbell et al. [
28]. Campbell et al. [
28] simulated recruitment of parents from low-income backgrounds and found no difference in rates of agreement to enrol when comparing various formats of consent including text, narrated video clips and PowerPoint presentations. Participants reported no greater preference for video clip over text, although when accounting for educational attainment, those who read at an 8th grade level or below did show improved comprehension for an enhanced text version of the information sheet. However, interestingly even though the preference was specified, there were no differences seen in rates of enrolment. Thus, it is possible that media such as a video may be beneficial in enhancing understanding of research participation, without that understanding necessarily translating into increased recruitment. Indeed, it is conceivable that enhanced understanding of what research participation involves may lead fewer participants to enrol, if, for example, the demands of participation exceed the time participants may be able to commit to the study.
Moreover, participants included in this SWAT were generally highly educated, particularly in those interviewed, and so have an assumed higher reading level. Therefore, it would be plausible that the enhanced PIS used was accessible and understandable. By extension, the video may not have had as much impact on understanding within this population, compared to a potential participant population who have a lower reading ability or based on previous research [
28]. Equally this may have been a matter of preference, which may explain the low number of views of the video. One parent specified they were unable to watch the video without headphones in case of waking their child and would have therefore preferred a PIS.
Another possible explanation for these findings may be linked to the format and use of the video clip. The metadata indicated that only one third watched the clip in full, assuming no participant watched it twice. Although studies on patient engagement with clips do not typically report metadata, research on the use of educational videos in the context of distance learning has found that the rate of in-video dropout is positively correlated with video duration, with a predicted drop-out rate of 53% for a 5-min video [
29]. Thus, it may be instructive to examine whether a shorter clip is more beneficial. It is also possible that tailored videos that provide content that can be selected based on participants’ questions such as in Meropol et al.’s [
20] study may be more effective than general videos. Additionally, it is possible that video is least accessible to those who stand to benefit most, if socioeconomic factors linked to literacy are also associated with more limited access to smartphone models, data plans, and/or availability of a computer in the home.
An unexpected, but notable finding came from the researcher experience which suggested that email contact prior to the recruitment phone call, regardless of group allocation, helped promote participants’ engagement with the study. This finding is contrary to the Cochrane Review by Mapstone et al. [
30] who reported that pre-warning participants was not beneficial to enrolment, examining methods including sending a letter a week before [
31], a postcard prior to sending a questionnaire [
32] and leaving a voicemail [
33]. However, none of these studies investigated the use of email to pre-warn participants, and our finding may be attributed to a shift in type of contact preference towards email. Specifically, as Samuels et al. [
34] found that when asking subjects how they would like to be contacted for future research studies 58% opted for an email whereas only 16% opted for a phone call.
Challenges, limitations and reflections on SWAT
We encountered a number of challenges in implementing the SWAT during an active RCT.
Whilst it is an advantage that the SWAT was responsive to recruitment challenges as they arose, the timeframe to implement the study significantly undermined the amount of data that could be collected. From initial idea to implementing the SWAT, the entire process took 13 months leaving just 4 months of data collection. This involved collaboration with the PPI group, submitting a substantial ethics amendment, creating the video clip and gaining local approvals from each of the trial sites. Thus, the sample size was small and this limited inferences that could be gleaned from the study. Future research would benefit from including SWAT procedure in the original ethics application and holding PPI consultations in the first instance to avoid delays. Indeed, in the UK, the NIHR Health Technology Assessment (HTA) programme now encourages the inclusion of SWAT protocols in funding proposals for host trials. To this end, the SWAT template provided by Smith et al. [
35] may provide a helpful template that can be referenced as part of the host trial’s protocol. This would allow the opportunity to monitor the effectiveness of recruitment strategies from the start and for trials to determine the relative benefits of different strategies. However, there are also likely to be cases when SWATs will provide opportunities to test innovations that respond to unexpected challenges as they arise. Thus, efforts to increase the efficiency of ethical review and local approvals may be crucial to realising the benefit of SWATs.
The SWAT also added time into the recruitment process and an additional step, an email contact, that had not been completed previously. This had the effect of a cumulative increase in workloads in the context of demanding recruitment targets and deadlines. Combined with the randomisation allocation which changed weekly, this may have increased the likelihood of human error. This can be seen in the randomisation phase where six participants were contacted with the incorrect study allocation. Embedding the SWAT at the beginning of the trial could provide further time for training; alternatively, automating aspects of the recruitment process could also minimise the scope for human error.
It was difficult to control for a number of extraneous factors. For example, the number of contacts was conflated by the temporal factor of recruitment deadlines, such that the frequency of contacts increased in line with monthly deadlines. Ideally, the number of times the study team contacted participants would be standardised; however, this is not practical in the context of target-driven recruitment. Similarly, it was difficult to prioritise the interviews with participants about their SWAT experience at the risk of overburdening participants and without undermining the time allocated to outcome assessment. Thus, less interview data were collected than expected, particularly in the video condition, and this may have limited insight. Moreover, whilst participants were generally positive about the information they received through the SWAT, it would have been helpful to find out why they may not have watched the full duration of the video. Due to the limited sample size, we were unable to control for the initial contact type participants received during screening. For example, potential participants either completed screening in person with a member of the wider team or received questionnaires in the post. Whilst these were not notably different between groups, further research may benefit from controlling for extraneous factors of this nature. Moreover, participants were highly educated and often women; it is difficult to generalise these findings to other types of participants who may have different experiences which would influence their engagement with written material or video content.
The optimal design would be to test the video clip in isolation against the PIS to identify effects on recruitment. However, due to ethics requirements, participants had to receive the PIS prior to the baseline visit limiting our ability to formally test the video on recruitment. In a time of increased use of multimedia, it would be instructive to formally test whether there are complete alternatives to standard PIS that may be more acceptable and effective whilst being ethically robust. We were also unable to gain feedback from families that may have viewed the initial information but not enrolled in the study, which limits our understanding of participants’ reasons for not engaging with the study.
Implications for recruitment practice
Although somewhat contrary to our expectations, it was encouraging that the PIS appeared to be effective for this sample. Moreover, participants and researchers perceived an initial email contact, regardless of whether it included the PIS or video, as being helpful. This suggests that initial contacts of this kind may be helpful in facilitating initial conversations about research as well as decision making about taking part. It would be helpful to test this formally using a SWAT design in future studies.
Future research may be focused on testing the effectiveness of a video clip as an initial recruitment contact rather than following a screening phase. Moreover, it would add value to evaluate this recruitment method in different demographic populations (e.g. those with lower literacy, children and young people, those with intellectual disabilities) in order to better ascertain utility. The inclusion of data regarding participants’ engagement with a PIS would also be helpful to identify potential areas for improvement and general levels of engagement.
We found involvement with our PPI group extremely helpful in developing the video and SWAT study. We strongly recommend that other researchers involve PPI groups when developing patient facing materials and designing trial procedures. The video may have benefited from several rounds of user testing to optimise its potential; however, due to timeframes and potential burden on the PPI group, this was not possible on this occasion.
Future research could consider whether videos may be particularly helpful for specific groups, better utilised in shorter (e.g. 30 s) formats addressing specific topics (e.g. randomisation), or as a supplementary material after participants have read the PIS. It would also be of interest to examine the impact of either formats on participant retention.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.