Maximising follow-up participation rates in a large scale 45 and Up Study in Australia

The majority of population-based cohort studies rely on individuals’ willingness to give their time and effort to participate. In recent decades, many epidemiological cohort studies have experienced declining participation or response rates, raising concerns about the usefulness of cohort findings for research and practice [1, 2] and posing practical and methodological challenges. Chief among these is identifying strategies to maximising response rates for assuring external validity of the findings. Maximising participation rates at recruitment and during the follow-up period is crucial, given that the characteristics of non-responders may show a socio-economic gradient [3] or higher mortality than survey responders [4].

For many decades, researchers have attempted to increase response rates to postal questionnaires. An early systematic review [5] and a meta-analysis [6] identified factors such as increased contacts and reminders, pre-notification by mail, credible sources of the study and perceived relevance of the topic can influence study participation rates. A Cochrane review summarised strategies to increase postal questionnaire responses, and identified incentives, personalising follow up, and follow-up reminder questionnaires as among strategies that effectively increased response rates [7]. Despite the plethora of research, most studies were small-to-mid sized (less than 1000 people approached) [e.g. 9], and most assessed recruitment to enrol in a study or trial [8, 9]. A more recent report described recruitment to an epidemiological cohort [10] and another identified that different population subgroups responded differently to multiple mail-out reminders [11].

Retention is a further concern within cohort studies, with loss to follow-up as a recognised problem that may cause greater bias than initial recruitment. In recent years, researchers have focused effort on baseline recruitment, but few studies have tested different recruitment strategies to increase subsequent retention for follow-up in large population-based cohorts. Strategies to increase retention have been trialled in HIV prevention studies in younger adults [12] and in one cardiac trial, home visiting increased study retention [13]. Reminder letters, together with the provision of stamped or business-reply envelopes and delivery of a newsletter have also influenced follow-up response rates [14]. An older adult trial for neck injuries trialled a £5 incentive to increase four- and eight-month questionnaire returns, and achieved higher responses, but at a cost of £67 per additional questionnaire received [15]. A recent systematic review found incentives increased retention rates (increasing with higher values) in population-based cohort studies [16]. Overall, the research to date has used similar strategies for increasing recruitment and achieving continuing participation during the follow-up period. Existing evidence withstanding, little research has addressed retention in epidemiological cohort studies, using different postal pre-notification (advance notice) and reminder approaches to maximise follow-up questionnaire responses sent several years after the baseline. Additionally, it is important to consider that the factors influencing participation and retention may be different; that is, it is possible that different mechanisms may operate once a participant has consented and expect to be in a cohort study over an extended period.

This study focuses on a 3.5-year follow up of a large sub-sample of the 45 and Up Study Cohort [17]. The follow up sub-study assesses social, environmental and economic factors (SEEF) and subsequent health outcomes, as well as changes in several baseline attributes from the 45 and Up Study. The recruitment goal of the SEEF Study was to re-contact the first 100,000 participants from the baseline 45 and Up Study and invite them to take part in the follow-up sub study (i.e. the SEEF Study). Recruitment for the SEEF Study was challenging, as it involved a large sample, and therefore low-cost methods for optimal follow-up strategies were required. This paper evaluates the strategies used to maximise the response rate to the SEEF Study and presents cost analysis of the strategies, to inform subsequent 45 and Up Study follow-up studies and large cohort studies elsewhere.

The Study 1 pilot of 5000 mailed questionnaires had a 30-day response rate of 41.7 %. This response rate was similar by gender, and by age group (Table 1). In the three subsequent intensive mail-out conditions in Study 2, the 14-day response rates ranged from 28 to 29 %; Strategy A returned 282 completed questionnaires, Strategy B returned 284 questionnaires and Strategy C returned 290 questionnaires (Fig. 2). Overall response rates were incrementally greater with increasing intensity of follow-up procedures. A response of 50.5 % was achieved with the pre-questionnaire postcard only (Strategy A), 54.1 % responding to only the post questionnaire reminder (Strategy B), and 56.4 % responding to both the pre-postcard and post questionnaire reminder (Strategy C). Overall, the intensive conditions of Study 2 achieved a 53.7 % combined response rate (95 % CI 51.9–55.5), which was significantly higher than the 41.7 % (40.3–43.5 %) response rate in Study 1.

Compared to the pilot mail-out to 5000 participants in Study 1, the effect of sending a postcard when there is no reminder was 1.21 times as effective (95 % CI 1.13–1.30). Using a postcard prompt also resulted in increased likelihood of survey response (PR = 1.30, 95 % CI 1.22–1.39), as did if both a postcard and a reminder were sent out (PR 1.35, 95 % CI 1.27–1.44) (data not shown). The contrast analyses showed that using a reminder or a postcard were comparable, and adding a postcard to a reminder did not significantly improve the response rate compared to only sending a reminder. However, the combination of postcard and reminder was significantly better than postcard alone (PR 1.12, 95 % CI 1.03–1.21).

The costs of the recruitment per person were highest for Study 1, and similar for each strategy in Study 2. The most expensive enhanced recruitment approach was both reminder and postcard, which cost around $0.65 more than a postcard alone, per person recruited.

For the main SEEF Study mail-out (Phase 3 of this study), of the remaining 92,000 SEEF-eligible participants, a response rate of 61.6 % (95 % CI 61.3–61.9 %) was obtained, significantly higher than either Study 1 or Study 2. The recruitment approach for the main survey was an adaption of the Study 2 strategies and used a combination of mail-out, followed at 3 weeks by a reminder letter to non-responders, followed by a second reminder plus a questionnaire to the remaining non-responders.

Efficient methods for recruiting and following up existing participants are essential in large-scale prospective studies if they are to remain invaluable in epidemiological research. In this paper, we report the effectiveness of three enhanced recruitment strategies to improve follow up response rates in a social epidemiological study. The results of this study show that more intensive strategies can improve follow-up response rate. Among the three strategies trialled, the pre-postcard, mailed questionnaire and reminder mailed letter at 2 week intervals had the highest response rate. However, having any kind of prompt alone or in combination appears to be better than having no prompt, consistent with other studies using a pre-notification (advance notice) or reminders [3]. Another epidemiological study also found similar follow-up response rates using reminder strategies based on style of return envelope and delivery of a newsletter [14]. The cost analysis did not define a clear benefit, but all additional costs were cheaper per unit survey than in other research [15].

Overall, the SEEF study across Study 1, Study 2, and the subsequent adaptation that involved also sending a reminder questionnaire at follow-up resulted in 60,404 in the complete data set, indicating just over a 60 % response rate. This response rate represents a reasonable return rate at an acceptable cost, without compromising the scientific integrity of the study. Further, these results are within the ranges obtained by large epidemiological cohort studies [25]. However, age differentials were observed in this study, with lower response among those aged over 65 years. Such age-related declines in response rates during follow-up period are expected in older adult cohort studies as a result of attrition due to deaths and disability [2, 26].

Our study found some gender differences to response based on the strategies used. Women receiving a questionnaire plus a reminder were more likely than their counterparts receiving a postcard and a reminder, separately or in combination, to respond to the survey. For men, a combination of pre-notification and reminders generated a substantially higher response rate in this group compared to women, suggesting that male participants may respond better to frequent prompts and reminders. In other research, frequent reminder prompts to male participants also indicated a positive effect on retention rates [16]; although to our knowledge no study had specifically looked at gender differentials to repeat contacts. Further research will need to investigate the impact of frequency and recruitment methods to boost follow-up response rates among men, a group widely known to be less likely to take part in research studies.

A major strength of this study is its sample size, with 8000 population-based individuals approached in the two studies. The literature to date shows that studies of similar scope and size are rarely conducted (typically less than a total of 1000). The cost analysis is another strength, indicating that the additional costs associated with the most intensive strategy (Strategy C) is marginal when considering the higher response rate generated by the strategy. Our study focused on postal questionnaires and did not examine ethnic differential effects which may limit its generalisability to studies involving large non-English speaking participants. This study did not model other factors that could predict participation (e.g. age, gender, health profile, timing of joining the baseline 45 and Up Study). We did stratify the analysis by gender and age, and found response rate differentials across these factors. It is possible that the lower response among the oldest age group (65+ years) may have been the result of natural attrition due to the higher risk of morbidity or mortality among this older age group. Due to the sequential nature of the group allocation to the three recruitment strategies, it is also plausible that the time since last contact may affect the response rates. We acknowledge that our study is further limited by the lack of description of the sub-samples which were targeted by the different strategies, as the varying response rates may be related to the socio-demographic and health-related characteristics.

While this study shows that any kind of reminder alone or in combination is better than having no reminder (Study 1), we cannot state in absolute term which of the three enhanced recruitment strategies (Study 2) is better than another. Further, the pilot recruitment phase occurred during Easter and therefore some of the improvements noted in the following two phases may be attributed to recruiting outside a major public holiday period.

This study explored alternate methods for increasing follow-up response rates in a large epidemiological cohort study. The findings indicated that increasing the follow-up intensity resulted in improved response rates. Specifically, we found that the pre-postcard, mailed questionnaire and reminder mailed letter at 2 week intervals alone or in combination were more effective in eliciting a higher response rate than no prompts at all. This study has made an important contribution to maximising the follow-up response rate, and consequently the usefulness of the SEEF Study follow up among older Australian adults. Recruitment and retention, however, will remain a significant challenge for epidemiological cohort studies. Future research in this area needs to consider developing and testing low-cost, innovative recruitment and retention strategies, including investigating potential mechanisms that may influence participation and whether these would vary by gender.