Background
Underpowered studies and insufficient sample sizes often result, in part, from ineffective participant contact methods and associated poor recruitment and participation rates [
1,
2]. Unsuccessful participant contact leads to extended recruitment time, missing data, and increased cost and other resource utilization [
3]. Nearly 60% of randomized controlled trials had difficulties in meeting recruitment target or needed to extend recruitment period [
4]. Achieving a timely and efficient strategy to contact research participants is critical for conducting clinical research [
4].
Several studies have shown that participants who were female, Caucasian, younger, more educated, and employed tend to have faster response to surveys [
5‐
9]. However these findings are not universally confirmed, with conflicting results regarding factors, such as participant demographics and health status, influencing participant recruitment and contact [
10‐
18]. Some of these conflicting results may be due to different patient populations studied and different countries of research. Hence, we aimed to evaluate predictors of survey completion in a patient population and U.S. study setting similar to prior research by Chen et al. [
13]. Chen et al. [
13] studied 146 acute lung injury (ALI) survivors from a single-center, multi-site longitudinal observational study based in the U.S.
Building on the results of Chen et al. [
13], in the current study, we evaluated a similar population of ALI survivors, but with a larger sample size of 332 participants, recruited from 41 hospital sites at 12 centers across the U.S. Like Chen et al., this trial was conducted with participants who had previously been enrolled in an observational follow-up study evaluating physical and mental health outcomes. In our current study, the last research evaluation occurred a median of 34 months previously. As part of this study, we asked participants to complete a new one-page insurance survey that was not part of the evaluation in their prior research visits.
The current study was conducted via two sequential randomized controlled trials evaluating the effects of different mail- and then telephone-based strategies for contacting and recruiting participants, and evaluated participant-related variables associated with time to completing the survey. As commonly done in survey-based research, the study protocol was designed with initial participant contact via mail, with more resource-intensive telephone contact reserved for non-responders to the mailed survey [
13,
19‐
21]. Based on the results of prior research [
22‐
24] and the non-statistically significant signal favoring personalized form of communications observed in the smaller-sized study by Chen et al. [
13], we hypothesized that using personalized contact strategies, such as hand-written envelopes for mail and using a specific investigator’s name for telephone messages, would result in a faster time to completing the survey and a higher response rate. We also hypothesized that for participants with poorer physical and mental health status, at the time of last research contact, the time to survey completion would be longer.
Discussion
We conducted two sequential randomized controlled trials to evaluate separately the effects of personalized versus generic mail- and telephone-based contact strategies and also evaluated other potential predictors of time to participant survey completion for 332 ALI survivors recruited from 41 hospitals across the U.S. There was no significant difference in time to survey completion or response rate between a “personal” versus generic “business” style of envelope in the mail trial, or for a generic telephone message (that used the study name) versus a personalized telephone message (that used the principal investigator’s name) for non-responders to the mail trial. Among 18 demographic and physical and mental health status variables evaluated, only younger age (≤40 years old) and minority (vs. white) race were independently associated with a significantly longer time to survey completion within this sequential protocol of repeated mail then telephone contact strategies.
A multi-modal contact strategy, such as mail followed by telephone calls, and making repeated contact attempts, as done in this study, is typical of rigorous approaches for minimizing participant loss to follow-up and associated missing data [
19‐
21]. In our study, this approach involved active contact for up to 12 weeks (maximum of 4 bi-weekly mailings and 4 weekly telephone calls) with participants who were last contacted approximately 3 years earlier. Via this approach, we achieved an overall survey completion rate of 67%, with only 3% declining participation, and required a median (IQR) of 3 (2, 5) contact attempts among respondents. This combined 67% survey completion rate, with individual response rates of 49% in the mail trial and 51% in the telephone trial, compares favorably with other studies [
10,
22,
45]. Comparison of response rates between the mail versus telephone trials within our study was not conducted since participants in telephone trial represented non-responders from mail trial or participants without a valid mailing address, and thus a different population from the mailing trial. Contrary to our hypotheses developed based on prior studies [
13,
22‐
24], we did not observe a significantly shorter time to survey completion or a higher response rate with more personalized contact formats, such as a personal envelope format and a telephone message using a the principal investigator’s name (“Dr. Dale Needham”).
Relatively few studies have evaluated factors predicting time to participant survey completion [
10,
46]. Tennant and Badley [
46] evaluated age as a single predictor for non-response bias, while Chen et al. [
13] did not explicitly evaluate the effects of participants’ objective physical and mental health status. Although Chen’s study [
13] found no association of any participant demographics with time to survey completion, we found significant associations for age and race. Our study demonstrated that participants ≤40 years old had a longer time to survey completion. This finding is similar to the results of Tennant and Badley [
46] who demonstrated that participants ≤65 years old who were physically independent had a tendency to slower survey completion. We speculate that younger patients, especially after acute lung injury, may be less physically impaired and spent less time at home, making them less readily available to respond [
25,
26,
47,
48]. Alternatively, expanded use of mobile phones among younger adults may make them more responsive to telephone versus mail communication; thus, the slower time to survey completion for younger patients may be a result of initial contact attempts being made via mail. Another potential explanation is that younger participants may have changed addresses more frequently than older participants. In addition, our study demonstrated that although several predictors of physical and mental health status had a significant association with time to survey completion in bivariable Cox models, they were not significant in multivariable analyses. Since the last study contact, for purposes of evaluating health status, was approximately 3 years earlier, this timing issue may have contributed to a lack of effect; however, specifically in ALI survivors, physical and mental health impairments are long-lasting [
47,
49‐
51] which justified their evaluation in this study.
The median (IQR) number of contact attempts was 2 (1, 3) in the mail trial among those participants who completed the survey, which demonstrates that repeated attempts within a single contact modality are needed for survey completion. However, we observed a decreasing rate of survey completion over time in the mail and telephone trials (Figure
3), indicating that with repeated failed contact attempts, non-responders were less likely to respond. However, after changing from mail to telephone contact, additional participants were successfully contacted, with 51% of eligible participants completing the survey with a median (IQR) of 1 (1, 3) telephone calls among respondents. This finding suggests that the effects of single contact method (e.g. mail), despite some improvement with repetition, may be not adequate for timely and maximal participation. Utilizing at least two different types of contact strategies can increase participant response rate in clinical studies.
There are several potential limitations in our study. First, our study focuses on survivors of ALI, so our findings may or may not be generalizable to other specific populations. However, this research was a national study of participants initially recruited from 41 hospitals in United States, so our findings may be generalizable to similar populations across the U.S. Second, there was a lag between the participant’s last visit at the end of the ALTOS study and our attempt to contact them in this trial, so their employment and physical and mental health status may have changed over time, impairing our ability to identify the true association between these variables and survey completion. Third, our analyses may have omitted potentially relevant predictor variables that were not available for analysis in this study. For instance, Chen et al. [
13] evaluated factors related to prior research visits (e.g., incomplete data collection and missed visits), and revealed significant associations with participant contact. Moreover, other comorbidities and disease history, not evaluated in this study, may be related to timing of survey completion. Fourth, the difference between our contact strategies (generic versus personal) may have been too small to yield a significant difference in participant perceptions and response times. However, we felt that our interventions were not markedly different from prior studies [
22‐
24] that demonstrated a substantial difference in response rates between personal and generic contact strategies. Additionally, it’s possible that the personalized contact strategy would, in fact, shorten response time had these participants not already been enrolled in a lengthy study. However, their prior involvement may not have played a large role since this trial was conducted 3 years after their last follow-up visit. Lastly, the questionnaire mailed was a single-page, retrospective survey of insurance coverage status. Perhaps faster response time and higher responses rates would have been achieved, in the mailing portion of the trial, if the survey was more intriguing to the participant.
Acknowledgements
We thank all patients and their proxies who participated in the study. We acknowledge our dedicated research staff, including the following who assisted with this trial: Elizabeth Fuller and Kyle Schneck.
Funding/support: National Heart, Lung and Blood Institute funded this follow-up study (N01HR56170, R01HL091760 and 3R01HL091760-02S1), the EDEN trial (contracts HHSN268200536165C to HHSN268200536176C and HHSN268200536179C), and this analysis (1R24HL111895).
The National Heart, Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network: University of Washington, Harborview (*L. Hudson, S. Gundel, C. Hough, M. Neff, K. Sims, A. Ungar, T. Watkins); Baystate Medical Center (*J. Steingrub, M. Tidswell, E. Braden, L. DeSouza, J. Germain, C. Kardos, D. Kelley, L. Kozikowski, S. Ouellette); Baylor College of Medicine (K. Guntupalli, V. Bandi, C. Pope, C. Ross); Johns Hopkins University (*R. Brower, H. Fessler, D. Hager, P. Mendez-Tellez, D. Needham, K. Oakjones); Johns Hopkins Bayview Medical Center (J. Sevransky, A. Workneh); University of Maryland (C. Shanholtz, D. Herr, H. Howes, G. Netzer, P. Rock, A. Sampaio, J. Titus); Union Memorial Hospital (P. Sloane, T. Beck, D. Highfield, S. King); Washington Hospital Center (B. Lee, N. Bolouri); Cleveland Clinic Foundation (*H.P. Wiedemann, R.W. Ashton, D.A. Culver, T. Frederick, J.A. Guzman, J.J. Komara Jr, A.J. Reddy); University Hospitals of Cleveland (R. Hejal, M. Andrews, D. Haney); MetroHealth Medical Center (A.F. Connors, S. Lasalvia, J.D. Thornton, E.L. Warren); University of Colorado Hospital, Aurora (*M. Moss, E.L. Burnham, L. Gray, J. Maloney, M. Mealer); Denver Health Medical Center (I. Douglas, K. Overdier, K. Thompson, R. Wolken); Rose Medical Center (S. Frankel, J. McKeehan); Swedish Medical Center (M.L. Warner); Saint Anthony’s Hospital (T. Bost, C. Higgins, K. Hodgin); Duke University (*N. MacIntyre, L. Brown, C. Cox, M. Gentile, J. Govert, N. Knudsen); University of North Carolina (S. Carson, L. Chang, S. Choudhury, W. Hall, J. Lanier); Vanderbilt University (*A.P. Wheeler, G.R. Bernard, M. Hays, S. Mogan, T.W. Rice); Wake Forest University (*R.D. Hite, K. Bender, A. Harvey, P.E. Morris, Mary Ragusky); Moses Cone Memorial Hospital (P. Wright, S. Groce, J. McLean, A. Overton); University of Virginia (J. Truwit, K. Enfield, M. Marshall); LDS Hospital and Intermountain Medical Center (*A. Morris, A. Austin, S. Barney, S. Brown, J. Fergeson, H. Gallo, T. Graydon, C. Grissom, E. Hirshberg, A. Jephson, N. Kumar, R. Miller, D. Murphy, J. Orme, A. Stow, L. Struck, F. Thomas, D. Ward, L. Weaver); LDS Hospital (P. Bailey, W. Beninati, L. Bezdijan, T. Clemmer, S. Rimkus, R. Tanaka); McKay Dee Hospital (C. Lawton, D. Hanselman); Utah Valley Regional Medical Center (K. Sundar, W. Alward, C. Bishop, D. Eckley, T. Hill, B. Jensen, K. Ludwig, D. Nielsen, M. Pearce); University of California, San Francisco (*M.A. Matthay, C. Calfee, B. Daniel, M. Eisner, O. Garcia, K. Kordesch, K. Liu, N. Shum, H. Zhou); University of California, San Francisco, Fresno (M.W. Peterson, J. Blaauw, K. Van Gundy); University of California, Davis (T. Albertson, B. Morrissey, E. Vlastelin); Louisiana State University Health Sciences Center-New Orleans (*B. deBoisblanc, A. Antoine, D. Charbonnet, J. Hunt, P. Lauto, A. Marr, G. Meyaski, C. Romaine, R. Tejedor); Earl K. Long Medical Center, Baton Rouge General Medical Center Mid-City and Baton Rouge General Medical Center Bluebonnet (S. Brierre, J. Byrne, T. Jagneaux, C. LeBlanc, K. Moreau, C. Thomas); Ochsner Clinic Foundation (S. Jain, D. Taylor, L. Seoane); Our Lady of the Lake Medical Center (C. Hebert, J. Thompson); Tulane Medical Center (F. Simeone, J. Fearon). Clinical Coordinating Center: Massachusetts General Hospital and Harvard Medical School (*D. Schoenfeld, M. Guha, E. Hammond, N. Lavery, P. Lazar, R. Morse, C. Oldmixon, N. Ringwood, E. Smoot, B.T. Thompson, R. Wilson). National Heart, Lung and Blood Institute: A. Harabin, S. Bredow, M. Waclawiw, G. Weinmann. Data and Safety Monitoring Board: R. G. Spragg (chair), A. Slutsky, M. Levy, B. Markovitz, E. Petkova, C. Weijer. Protocol Review Committee: J. Sznajder (chair), M. Begg, E. Israel, J. Lewis, S. McClave, P. Parsons.
*Principal investigator.
Authors’ contributions
All authors have read and approved the final manuscript. VDD, KAS, and DMN developed the study concept and design. VDD, MH, KAS, MP, ROH, EC, and DMN analyzed and interpreted the data. VDD, MH, EC, and DMN drafted the manuscript and all authors have provided critical revisions for important intellectual content. The study was supervised by DMN.