Cost analysis and efficacy of recruitment strategies used in a large pragmatic community-based clinical trial targeting low-income seniors: a comparative descriptive analysis

Randomized controlled trials (RCTs) are widely recognized as the most robust study design for clinical research [1]. One of the greatest challenges in the conduct of clinical trials is participant recruitment [2]. A review of 253 terminated trials found that almost 40% were discontinued prematurely due to difficulties with recruitment [3]. Another study found that up to 50% of trials had to be extended to enroll a sufficient number of participants, and yet only about 30% of clinical trials meet their recruitment targets [4, 5]. As a result of under-recruitment, many studies suffer from a lack of statistical power [6]. This has potential risks: if researchers cannot make their study worthwhile, they waste participants’ time and expose them to unnecessary risks without the promise of improvements in clinical care, which is both unproductive and unethical [6, 7]. Recruitment challenges also result in considerable additional costs to extend study recruitment periods [8], which may deter funding institutions from supporting clinical trials [6].

Numerous authors have attempted to explain why recruitment for clinical trials is so difficult. Some have described problems related to study design [2, 9‐13] and recruitment methodology, such as undesirable trial arms (i.e., control groups) and ineffective recruitment strategies for the target audience [12, 14, 15]. Others have noted challenges specifically inherent to the potential participants [2, 6, 11‐13] and researchers [2, 12, 13].

Some recent clinical trials have included detailed descriptions about their recruitment process in the published literature. A recent Australian study of older men with diabetes and low testosterone showed successful recruitment through radio and TV advertisements as well as government mail-outs, where all three approaches were relatively cost-effective [16]. Other studies have successfully used health care referrals [16‐20], mail-outs [16, 21‐23], and community outreach [18, 24]. However, there are still gaps in our knowledge regarding the effectiveness of various recruitment strategies for different target populations [27], the costs associated with them [28], and their success in reaching specific subgroups, which is particularly lacking in the literature. These knowledge gaps are more prominent for researchers attempting to recruit older participants or socially vulnerable participants, as they are often excluded or under-represented in the majority of trials [29].

The objective of this paper was to use our experience with the Assessing Outcomes of Enhanced Chronic Disease Care Through Patient Education and a Value-based Formulary Study (ACCESS trial) to address this knowledge gap and to describe the most effective strategies for researchers seeking to enroll low-income seniors into pragmatic clinical trials. The specific objectives included (1) describing the effectiveness and types of patients enlisted by the various recruitment strategies used, (2) detailing the costs associated with each of these strategies, and (3) determining the cost per enrolled participant for each of these strategies.

Recruitment for ACCESS was time-consuming and costly, but ultimately successful. We used 14 substrategies to recruit the first 4013 participants into the study, at a cost of $354,330 CAD, which was approximately 20% of the overall study operating expenditures during this period. Despite eventual success, there was a lack of adequate planning and budgeting at the beginning of the study to successfully reach the target number of participants. Initial planning set out 12 months for recruitment, with only $20,000 CAD set aside as a dedicated recruitment budget to contact pharmacies, which was to be the sole recruitment strategy. As recruitment continued, the timeline had to be extended and considerably more resources had to be put towards recruitment—all of the other strategies developed once it became clear that the initial plans for recruitment were going to be inadequate. It is important for researchers to understand how much of the study budget may need to be spent on recruitment, and to plan their funding requests accordingly. No single strategy appeared to succeed in recruiting typically under-represented groups; rather, the strategies differed in their ability to recruit various types of people. This suggests that the use of multiple and varied recruitment strategies is important to increase the likelihood of achieving sociodemographic diversity [2, 12, 26]. The average cost per enrolled participant was $88 CAD; however, the costs involved with each strategy used in the ACCESS trial varied considerably. It is important to recognize how the availability of resources can restrict recruitment efforts, thus influencing what is seen as a successful recruitment strategy. As found in the ACCESS trial, each strategy differs in its cost, ability to recruit participants (including specific types), and timeliness.

Under-recruitment and the high costs of recruiting for clinical trials have been well documented in the literature. Many authors have provided insight on how to change study designs to help researchers reach recruitment goals within their desired budget and time frame [9]. Building upon this work, there were knowledge gaps our study was able to address, most importantly, how to recruit populations which are typically underrepresented in clinical research. Most studies that have analyzed the factors that impact recruitment (including the studies included in Treweek’s meta-analysis [9]) were enrolling younger and healthier participants and not actively targeting low-income seniors with chronic diseases (at high risk of hospitalization). Another important difference is that our study concentrated on understanding the effectiveness of specific recruitment strategies and their associated costs, whereas Treweek’s work had examined other aspects of the study design that can affect recruitment success (i.e., the consent process, the initial information given to potential participants, incentives, trial conduct, etc.) [9]. The study published by Bracken et al. [16] provides a cost analysis and detailed explanations of many commonly used and successful strategies such as radio and TV advertisements, health care referrals, mail-outs, and community outreach which were also used in the ACCESS trial; however, our study targeted a different participant base and also provides a demographic analysis of the subtypes of participants recruited by each strategy. The availability of both of these studies provides future researchers with greater insight on the types of participants to expect from various recruitment strategies, depending on their target demographic. By thoroughly analyzing each individual strategy, our paper provides researchers different perspectives on what could be considered a successful recruitment strategy, allowing them to determine which might be most suitable for their studies.

A common barrier to the recruitment of older participants is lack of trust in the researchers and recruiters [11‐13, 26]. This implies that recruitment may be more successful if participants hear about studies from trusted individuals, such as their physicians or pharmacists [2, 27], and it might explain why health care providers enrolled the most participants in our study. Similarly, personal referrals have been shown to be quite successful at persuading people to enroll [2, 11‐13, 26, 28, 32, 33]; which we also noted with the success of the ”word of mouth” strategy. Mail-based recruitment strategies can be challenging to implement. Suggestions for improving the success of mail-based recruitment strategies include personalizing the letters as much as possible using a pre-specified list that is more likely to include only individuals who are most likely to be eligible and interested [26]. These factors might explain why two of our mail strategies (coronary angiogram registry and contact after hospital discharge) were considerably more successful than the more generic Canada Post mail-out [2, 11, 26, 34]. However, it is important to note that even though the coronary angiogram registry had a high human resources cost, the active approach of calling participants directly made this strategy much more successful than the other mail strategies, which were more passive. Community-based approaches and cultural adaptation are known to enhance recruitment efforts [11]—which we saw in the moderate success of our outreach strategies. In terms of the ratio of men and women enrolled, each strategy had a relatively equal number of men and women recruited with the exception ”word of mouth” and the coronary angiogram registry. ”Word of mouth” and ”seniors outreach” recruited a higher number of females, potentially because females have been observed to be slightly more social than males [35]. The coronary angiogram registry recruited a higher proportion of males, most likely due to the fact that on average males have cardiovascular events earlier than females [36], making them more likely to be on the coronary angiogram registry.

As we observed, social media is typically thought to be more effective at reaching younger populations than low-income seniors who may not be as technologically inclined [13, 15, 27]. Previous studies have found the total costs of media recruitment to be relatively low [32], which was true for ACCESS as well. However, we had very few participants recruited by paid media, making its cost per enrolled participant quite high. In contrast to paid media, we found that unpaid media (such as coverage stemming from press releases) had the lowest cost per enrolled out of all strategies. This emphasizes the importance of using free advertising and building trust with local communities to help with the promotion of study recruitment [14]. Media coverage seemed to generate immediate interest in the study that was not sustained over time. It is quite clear that the first major press release in March 2016, which garnered much interest in unpaid print and online media, generated a clear spike in initial contacts.

Our analysis has some limitations that merit discussion. First, due to the contextual nature of this work, the findings of this study may only be generalizable to researchers studying a similar population (low-income seniors with chronic diseases) who are offering the possibility of a tangible benefit to study participants (i.e., free medications). Furthermore, the specific costs, and particularly those related to human resources, are highly contextual. Other researchers should anticipate that the exact costs of specific strategies will vary within their own locations, but the relative costs should be generalizable to all settings of similar studies. Second, we do not have detailed information on why individuals declined to participate, which could be useful in the future to modify recruitment approaches. Third, we were unable to account for the impact that media promotion may have had on other recruitment strategies. It is possible that a number of health care providers, senior outreach workers, and friends and family members who referred participants to our study initially heard about it from media sources. Similarly, participants might have heard about the study from multiple sources, but only called the study after the most recent contact. Therefore, the reach of some strategies may have been underestimated. Finally, this quantitative analysis does not offer any insights into why some strategies were more effective or how success was achieved. This would necessitate a qualitative inquiry, which may be informative to researchers - such an analysis has been conducted by our team separately [25].

In summary, we found that different recruitment strategies were better at targeting certain demographic groups and varied in their overall effectiveness. Overall, recruitment through personal referrals from health care providers was the most expensive but enrolled the most participants. However, strategies such as unpaid media and word of mouth—while enrolling fewer participants—were associated with the lowest cost per enrolled, highlighting the importance of community outreach methods in recruiting populations that are typically under represented in clinical trials. The experience of the ACCESS trial emphasizes the importance of understanding the target demographic of a clinical study and determining the most appropriate strategies to effectively recruit those desired individuals.