Increasing response to a postal survey of sedentary patients – a randomised controlled trial [ISRCTN45665423]

Postal surveys are routinely used to obtain information from patients and groups within the general population, over a range of topics. Postal surveys are a cost-efficient method compared with intensive methods such as face-to-face interviews and capable of obtaining systematically, information on many thousands of people. A key quality component of postal surveys relates to the number of people sampled, and the proportion returning a completed useable questionnaire [1]. Lower response rates can reduce the statistical power of the study, and mask statistically significant relationships, which 'truly' exist within the population studied. Responders may also be different to non-responders. This can introduce bias into the survey findings, if the decision to respond (or not) relates to the outcome being analysed within the survey, thereby reducing generalisability to the initial reference population [2].

Many studies conclude that non-responders in surveys and other epidemiological studies can differ to responders with respect to a range of specific health, lifestyle and social variables. Non-responders have been found to differ with respect to their sex, age, race, social class, home circumstances, education, and healthy lifestyle behaviours [3‐7]. They can also differ in terms of existing health and healthcare utilisation [8‐11] with differences extending through to higher rates of mortality [12] compared with responders. However, it can be difficult to make clear conclusions about the characteristics of non-responders in surveys and other types of studies, as factors such as the purpose of the study and the way in which it was carried out, will no-doubt have some effect. Furthermore, differences have not always been found between responders and non-responders, at least in terms of what factors were assessed, and nonresponse will not always affect estimates of prevalence [13‐15]. Nevertheless, nonresponse bias should always be considered a possibility [16, 17]. Moreover, there is evidence that in general, response rates to postal questionnaires are falling, [18], making this topic worthy of continued investigation. There is however, no agreed level of acceptable response in postal surveys [19‐21].

A systematic review [22] found a number of factors associated with postal questionnaires can influence the likelihood of response. This included providing incentives; questionnaire length and appearance, method of delivery, method for return, if any pre-warning/contact was given, the content and layout of the questions, through to the origin/sponsor of the questionnaire and how it was communicated. The review found that monetary incentives, recorded delivery and using an 'interesting' questionnaire were the three strongest influences on response rates. Statistical heterogeneity was found for all of these factors, thus limiting the extent that pooling of results was viable. Moreover, only a third of studies were from medical/epidemiological/healthcare journals and with no distinction between studies of patients or of staff and subgroup analyses were absent. As such, it is difficult to generalise the findings from the review to particular groups of patients in a healthcare setting.

In the current study, we carried out a randomised controlled trial to examine the effect of a pre-warning letter on response rates to a postal questionnaire. The questionnaire was sent to patients who had previously been referred to a community based exercise referral scheme because of a sedentary lifestyle and sought information on the quality of the service offered. An earlier study [23] of a similar population suggested poor response rates could be a problem. With a limited budget, we were unable to offer financial incentives, as suggested in the review by Edwards et al [22], but wanted to explore the suggestion that pre-warning letters might increase final response.

The sample consisted of patients who had been referred to a community based exercise referral scheme during the past 12 months, identified from the service register. Patients were referred by a primary care practitioner because of concerns about their sedentary behaviour and its impact on their health. The questionnaire formed part of a project examining the relationship between patient service-expectations and service outcomes.

We examined the effect of a pre-warning letter, posted to patients seven-days before sending the main questionnaire, compared with no pre-warning letter. The pre-warning letter was printed on one-side of letter-headed paper, and informed the respondent that a survey would be sent to them within the next few days. It informed them about the purpose of the survey and the importance of it being completed and returned.

The main questionnaire was sent with a covering letter and a pre-paid business franked addressed envelope for its return [24]. A standard reminder letter was sent to all non-responders seven days after posting the questionnaire, and after a further seven days, persistent non-responders were sent a further copy of the questionnaire, with a standard letter and return envelope. Randomisation was done using computer generated random numbers, and stratifying by age and sex. Participants remained unaware as to group allocation.

The primary outcome was the final response rate to postal questionnaires after sending all reminder letters. This was calculated at least 6 weeks after sending the initial questionnaire, to allow for late responders. Differences in proportions between groups were examined using Pearson's chi-square test and logistic regression to adjust for age and sex with 95% confidence intervals (95% CI). To observe a difference of at least 10% between trial arms required 752 participants, based on a return rate of 60% in the control groups, with 80% power. Approval for the study was received in advance from the local research ethics committee and the research governance committee.

The number of patients referred to the exercise referral scheme in the past year with complete name and address information was 627. Their mean age was 48 years (SD 13, range 18 to 78) and 69.2% (434/627) were women. Randomisation allocated 49.0% (307/627) of patients to receive a pre-warning letter and 51.0% (320/627) no pre-warning letter (Figure 1). The two groups were balanced with respect to sex (66.8% female in the pre-warning group compared with 71.6% in the control group) and age (mean 48.7 years, SD 13.3, in the pre-warning group compared with mean 47.6 years, SD 13.9 in the control group).

The final response rate to the postal survey, after completing two stages of follow-up was 25.4% (159/627). In the pre-warning group, the response rate was 30.0% (92/307) compared with 20.9% (67/320) in the control group (χ² 6.75, p = 0.009) (Figure 2). Thus giving a difference between the two groups of 9.1% (95% CI for risk difference, 2.2% to 15.8%) (Figure 2). In a logistic regression model, the pre-warning letter increased the odds of returning the questionnaire by 1.61 (95% confidence interval 1.12, 2.32) and this was not altered after adjusting for age (in years) or sex (OR^{adj age sex} 1.60, 95% CI 1.11, 2.30).

Sending a pre-warning letter seven days in advance of mailing out a postal questionnaire had a modest impact on increasing final response rates by almost 10%, with a relative 43% increase compared with sending no pre-warning letter. Patients in our study were selected on the basis of a previous referral to a community based exercise referral scheme, and the questionnaire sent to them sought information about their perceptions of the service quality. An earlier trial examining the impact of this service on a similar group of patients, in terms of increasing physical activity, had achieved average response rates of 60% [23]. The much lower than expected response rate in the current study could have been influenced by the topic and purpose of the questionnaire along with the layout of the questionnaire [22].

Our findings are consistent with the evidence from a systematic review [22] that pre-contact can lead to an increased odds of response by as much as 50%. Our study confirms that this benefit extends to groups of sedentary patients known in advance, to be reluctant to reply. Pre-warning letters are simple to administer and relatively low cost compared with more labour intensive methods to increase response rates, such as telephone reminders or face-to-face visits. Moreover, this intervention method does not require any additional information or administrative systems over and above those required for sending the main questionnaire. Give that the response rate in the intervention group still remained low, at 30%, we may need to consider alternative approaches to postal questionnaires to obtain information from this group of patients, particularly if non-response could be associated with the outcomes examined within the survey – in this case, the patient experience of the exercise referral service.