Retention strategies in longitudinal cohort studies: a systematic review and meta-analysis

Longitudinal cohort studies play a central role in advancing understanding of the onset and progression of physical and mental health problems. Cohort studies assess, and often compare, the incidence of a condition within a group of people who share common characteristics (e.g., being born in the same year) [1]. A key advantage of longitudinal cohort studies over other research designs is that repeated measures data temporally orders exposures and outcomes to facilitate causal inference [2]. However, significant and systematic attrition can reduce the generalisability of outcomes and the statistical power to detect effects of interest [3]. Systematic attrition in longitudinal research occurs most often in older, non-white male participants with limited education and/or multiple health problems [4]. Long duration and repeated assessments can also increase attrition due to the significant burden on participants [4]. Given the expense of longitudinal cohort studies, effective strategies that engage and retain cohort participants are critical to the integrity of research outcomes [5, 6].

In the last decade, longitudinal data collection methods and cohort retention strategies have evolved considerably. So too have participant expectations of organisations (research and otherwise) that seek information from individuals [7, 8]. Established retention strategies within longitudinal cohort studies include: cash or gift incentives, sending reminder letters to participants, re-sending surveys, and offering alternative methods of data collection (for a review, see [6]). Booker et al. [6] demonstrated that these strategies were effective in longitudinal cohort studies that used the traditional data collection methods of postal surveys, face-to-face visits (home or on-site), and telephone interviews or surveys. However, these cohort retention strategies may not be as well suited to contemporary methods of collecting longitudinal data, such as web and mobile surveys [9], wearable sensors (e.g., FitBits) [10], short message services (SMS) [11], and groupware systems (e.g., video conferencing) [12]. Novel methods of engaging participants such as web advertising [13], social media [14], and electronic reminders [15], are also now being employed in cohort studies using both traditional and modern longitudinal data collection methods.

A systematic review on the effectiveness of established and emerging cohort retention strategies in longitudinal cohort studies would provide guidance to researchers and funders on maximising cohort maintenance within these high investment programs of research. Previous reviews of retention strategies in health research include [4, 6, 16, 17]; only one of these reviews focused specifically on longitudinal cohort research designs [6]. Booker et al. [6] conducted a narrative review of retention strategies in longitudinal cohort studies, including incentives, reminders, repeat visits/questionnaires, and alternative methods of data collection, finding that incentives and reminder strategies improved cohort retention. However, this review was limited by the small number of studies identified for each retention strategy, which resulted in the identification of a restricted breadth of retention strategies and the inability to synthesise findings empirically.

Further, Booker et al. [6] did not include research completed after 2006 and thus were unable to investigate emerging cohort retention strategies. Brueton et al. [16] completed a more recent review of retention strategies that included both established and emerging digital data collection retention strategies. However, the authors specifically excluded longitudinal cohort studies and instead focused on participant retention in intervention trials. Differences between intervention and longitudinal cohort studies, such as research design factors (e.g., study duration) and the motivations of the participants in joining or withdrawing from studies, may impact the usefulness of retention strategies across both study designs [4, 6].

A review of retention strategies reported in modern longitudinal cohort studies is pertinent and timely, given the emergence of digital retention strategies alongside established retention methods. Maximising cohort retention in longitudinal research can reduce the administration costs of conducting research, improve the efficiency of research processes, and reduce outcome biases for studies by adopting an evidence-based cohort retention framework. In this review, we aimed to: (i) identify retention strategies used in recent longitudinal cohort studies; (ii) examine whether retention rate was moderated by different study or participant characteristics (i.e., number of waves, study duration, sample size, population type, gender, age, country); (iii) estimate the retention rate in studies that use specific retention strategies, and contrast this retention rate with studies that do not use specific retention strategies; (iv) examine whether retention rate is associated with the number of retention strategies used; (v) examine which retention strategies were the strongest independent predictors of retention rate; and (vi) contrast the retention rate based on whether studies utilised emerging or established strategies. Moreover, to ensure that recent innovations in retention strategies were identified, this review focused on literature published within the past 10 years.

This study aimed to identify retention strategies employed in longitudinal cohort studies during the past decade, and to examine their effectiveness. We identified 143 longitudinal cohort studies that described retention strategies and outcomes, resulting in 95 different retention strategies. We then investigated whether study or participant characteristics moderated retention, the relationship between retention rate and retention strategy type, and whether new cohort retention strategies have emerged since previous reviews. In so doing, this study is the first meta-analysis of retention strategies conducted in longitudinal cohort studies. This research particularly complements the previous narrative review that investigated cohort retention strategies in longitudinal research [6], and the wider literature investigating participant retention strategies across health research designs (e.g., 4,16,17). Such research has important implications for maximising cohort retention and reducing research administration costs, which will subsequently improve the efficacy and quality of health research.

We first investigated how study or participant characteristics may influence cohort retention. Study characteristics included sample size, study duration, number of waves, and country development level - none of which were associated with retention rate. Participant characteristics included mean age at baseline, cohort type (clinical or non-clinical), and gender. We found that cohort studies with a higher proportion of male participants had lower retention rates than studies with a higher proportion of female participants; no associations were found for participants’ age or cohort type. While difficulties in retaining male participants are well-documented in previous research (e.g., 4,20,21), our study noted that cohorts with a higher proportion of male participants were also more likely to be clinical samples than cohorts with a higher proportion of female participants. In addition, cohorts with a higher proportion of male participants were also disproportionately focused on high-risk groups, such as substance use and men who have sex with men (e.g., the Bangkok Men who have Sex with Men Cohort Study (BMCS) [25] and the International Multicenter ADHD Genetics (IMAGE) study [26]). Thus, the difficulties in retention reported in this study and the wider literature could potentially be attributed to the differential impact of these clinical issues that affect men more than women. Researchers working with hard-to-retain populations, such as men in particular clinical groupings, may benefit from investigating what retention strategies work within their specific populations and settings beyond the core retention strategies identified in this review.

Second, we investigated the relationship between retention rate and retention strategies. We identified 95 different retention strategies, grouped thematically into four classes: barrier-reduction, community-building, follow-up, and tracing. Specific strategies associated with improved retention rates included the barrier-reduction strategy of offering alternative methods of data collection to participants (e.g., completing an interview over the phone or in person); and the tracing strategy of collecting detailed contact information from participants at baseline via a locator document. Further, weak evidence was found for one community-building and two further barrier-reduction strategies: (i) sending participants thank you, birthday or holiday cards; (ii) having consistent research team members, and; (iii) offering site and home visits for data collection.

Overall, barrier-reduction strategies emerged as the strongest predictor of improved retention. Barrier-reduction strategies may be particularly useful in longitudinal research given participants are likely to experience significant changes in their capacity to remain involved across the study’s duration (typically years). Follow-up/reminder strategies, such as incentives and reminders, were associated with significantly poorer retention. This result was surprising, given that the previous review investigating retention strategies in longitudinal cohort studies found the opposite, that use of these follow-up/reminder strategies resulted in improved retention rates [6]. The lack of support for follow-up/reminder strategies found in the current review could be due to a number of extraneous variables including: (i) timing: studies may have implemented this strategy after other retention efforts proved ineffective; (ii) participant burden: the studies using follow-up/reminder strategies may have involved a high data collection burden (e.g., long surveys); (iii) sampling: studies using follow-up/reminder strategies may be over-represented in studies of difficult-to-retain populations, such as men. However, these explanations are unlikely, given that follow-up/reminder strategies were identified in most of the cohorts included in this review (111 out of the 143 cohorts), and the cohorts employing follow-up/reminder strategies did not differ by research design (sample size: t₍₁₄₁₎ = .67, p = .50; no. waves: t₍₁₄₁₎ = −.43, p = .67) or participant characteristics (age: t₍₁₄₁₎ = −.11, p = .91; gender: χ²_{(2, n = 143)} = .37, p = .85; HDI: χ²_{(2, n = 143)} = .01, p = .97). Differences were observed only for study duration (any M(SD) = 3.9(4.4); none (SD) = 5.8(6.4); t₍₁₄₁₎ = 2.00, p = .05). Alternatively, participants may perhaps view follow-up/reminder strategies as the research team “badgering” them to complete assessments, thereby damaging rapport. This negative perspective of follow-up/reminder strategies may be further exacerbated if the research team has not implemented sufficient barrier-reduction strategies to help make it easier for participants to remain involved in the study. Future research could consider investigating participants’ perspectives of retention strategies in longitudinal cohort studies, ensuring that both active and inactive participants are included, to better understand the costs and benefits of different approaches.

Interestingly, the current study found that simply adding more cohort retention strategies did not result in higher retention rates. These results contradict the findings of Robinson et al. [17] and Davis et al. [4], who both found that the use of more retention strategies across multiple classes was associated with improved retention rates. However, neither study specifically examined participant retention in longitudinal cohort studies, and both synthesised their retention results using a narrative rather than meta-analytic approach. Given that the implementation of retention strategies can be costly in terms of both time and money, the overall number of strategies employed is important to evaluate. The interaction of quantity of retention strategies used and provision of flexibility needs to be better understood, given research protocols that accommodate the changing lives of participants should remain a key focus of retention efforts.

Finally, we examined whether studies utilising new or emerging retention strategies had improved retention compared with studies using established strategies. Of the 95 retention strategies described in the included articles, 44 were identified as an emerging retention strategy that had not yet been described in extant systematic reviews examining participant retention [4, 6, 16, 17]. Emerging strategies included using social media and SMS to assist in tracing participants lost to follow-up, and the use of study websites and social media profiles for keeping participants up-to-date with study news and events. Emerging retention strategies were endorsed by only a handful of studies, and the use of a single emerging strategy was not significantly associated with retention rate. However, we found that studies that employed more emerging retention strategies were associated with improved retention rates. Importantly, emerging strategies were identified across all four retention strategy domains (barrier-reduction, community-building, follow-up/reminder, and tracing), demonstrating that the association between emerging strategies and improvements in retention are due to the use of modern technology to help achieve core cohort engagement goals. Thus, we recommend that researchers continue to innovate their retention efforts, particularly where such strategies may reduce participant burden.

Second, net retention rates were calculated by the difference between the first and last wave of data collection reported in the article. Where specified, ineligible participants (e.g., participants recruited after the first wave, or deceased participants) were excluded from the retention rate calculation. However, some articles did not provide detailed information on the eligibility of the sample at the final wave, and thus it is possible that the retention rates calculated for some studies may be slightly inaccurate. This limitation could be addressed by researchers providing details on the eligibility of their samples at each wave.

Third, high levels of heterogeneity were reported for most analyses in this study. This may best be explained by two factors. First, we expected to identify high heterogeneity given the diversity of research questions, methodologies, and cohorts reported across articles. Second, only a small number of studies were eligible for most meta-regressions in this paper, which reduces the precision of heterogeneity estimates [27]. This limitation could be addressed in future work, which could aim to investigate the effectiveness of different retention strategies within different subgroups.

Finally, by nature of synthesising retention results across different samples and settings, the current study is unable to disaggregate nuanced effects of various retention strategies across specific contexts and populations, given results are pooled across multiple studies. The current study did address this broadly by investigating the effects of study and sample characteristics on retention.

A final point to note is that available to researchers are a range of statistical or methodological approaches that can minimise potential biases introduced with attrition. Whilst beyond the scope of this paper, these approaches include formal statistical methods for addressing missingness due to attrition such as multiple imputation or full information maximum likelihood methods [28, 29]. Moreover, researchers may address attrition methodologically by using replacement sampling approaches that recruit new participants into a study to replace those who have dropped out, based on shared characteristics measured in the original sampling frame [30, 31]. All these methods provide useful avenues to address attrition once any employed retention strategies have been used to retain the largest proportion of the original sample as possible.

Overall, this study has important implications for the retention efforts of longitudinal cohort studies. Combined, these results suggest that researchers need to be strategic in choosing how to invest their resources to better target participant retention, rather than simply increasing the number of strategies applied. Projects should invest both time and funding into matching retention strategies to the sample prior to implementation, including careful consideration of unintended burden for participants. Finally, given the high number of emerging retention strategies identified, longitudinal research methods clearly continue to evolve. Longitudinal cohort studies may benefit from open and regular protocol revision to incorporate new strategies, particularly where these strategies may offer greater flexibility to participants.