Sample selection, recruitment and participation rates in health examination surveys in Europe – experience from seven national surveys

The main strength of this study is the inclusion of several European countries with significant experience of conducting nationally-representative health examination surveys. Differing definitions of response rates is a well-recognised issue [44, 45]. The limitation of these different definitions of non-response, in each individual survey, was overcome by use of agreed uniform definitions for both numerators and denominators. As with comparisons of the findings of disparate heath surveys across Europe, [46] the lack of uniformity of survey methods prevents assessment of the impact of cultural differences within countries, and aspects of sampling, recruitment, and data collection on participation rates. Analysis is therefore limited primarily to comparisons within each country.

In countries which have either national or local population registers, these were used as the sampling frames for individuals. Countries without population registers constructed their sample using address files. Population registers have good coverage of the population; updating procedures are usually regulated by law [47] and occur regularly. Population registers include information about the name of the person, their address and often their sex and age as well. Address files do not list individuals but buildings; they often include vacant buildings, offices and other non-domestic houses. The updating frequency of address files varies between countries. For example in Ireland, the GeoDirectory identified 2.7 % of residential addresses as vacant while the Census reported 15 % of residential addresses as vacant [41]. The large proportion of ineligible addresses in address files (>10 % on average) increases the cost of making contacts as interviewers have to check all addresses.

For practical reasons, people living in institutions are often excluded from health examination surveys, for example in the three surveys using household-based sampling. When people are asked to come to the fixed examination clinic, it may be impossible for institutionalized persons to participate. Also in cases where home visits are conducted routinely, or offered as an alternative for those not able to come to the examination site, inclusion of institutionalized persons may be difficult. Many institutionalized persons have illnesses which prevent their participation or ability to provide informed consent; in the case of prisoners, both the safety of the survey staff and the privacy of the examination may be difficult to guarantee. For the working aged population (18–64 years), the number of institutionalized people is a small proportion of the population and should not affect the representativeness of the survey results. In older age groups (65+ years), the proportion of the population living in institutions (such as care homes) is larger.3 %-6 % of people aged 64 and over in OECD countries, are living in long-term care institutions [48]: excluding them may affect the representativeness of the results.

The recent European recommendations (European Health Examination Survey) for conducting a national health examination survey determined that a sample size of at least 4000 persons is needed to survey the 25–64 year age group [49]. The EHES recommendations are similar to STEPS recommendations [50]. The sample size should be made up of equal numbers of men and women in each 10-year age group (500 in each group) in order to obtain nationally representative results. In each of the seven national health examination surveys, the sample size fulfilled this criterion even though in most of the surveys, a wider age group was covered.

Sending a short pre-notification letter for survey invitees before mailing out the invitation and questionnaire is technically easy to organize but for large population surveys it increases the cost of the survey. In some countries, sending a personalized pre-notification is not possible due to the sampling methods. In Finland, it was possible to send SMS messages for survey invitees as mobile phone numbers are widely available. In a recent, local HES within Finland, those receiving an SMS reminder were up to 25 pp more likely to participate [51], although telephone reminders have not been found to have a significant effect [25]. The rapid increase of mobile phone subscriptions and at the same time decline in land-line subscriptions is opening new possibilities for contacting survey invitees [52] in countries where listings of mobile phone numbers are available.

In health examination surveys, home visits have also been used to contact survey invitees who do not respond to the invitation [28]. In the Netherlands, this was an effective way to convince people to participate, although it was instituted at the same time as several other changes to the recruitment procedures, making formal evaluation difficult.

For health examination surveys, a Norwegian study reported that many participants felt that incentives would impose commercial features on the survey and could undermine the confidence in the survey. They also noted that there were clear differences between population groups regarding acceptance of incentives. People from lower socio-economic classes and younger age groups, the population groups which tend to have the lowest participation rates in health examination surveys, had more positive attitude towards incentives [27].

Although the larger the monetary incentive, the bigger the effect on participation rate [25], there are diminishing effects of greater amounts. In some countries, research ethics committees have prevented the use of monetary incentives in health examination surveys. Therefore, not all surveys have had an equal opportunity to use monetary incentives. In England, use of an unconditional £5 voucher in the advance letter to the household increased household participation rates by 3 % [53]. In large health examination surveys, providing monetary incentives can have a significant effect on the survey budget but it may also reduce number of contact attempts needed and therefore reduce survey staff costs.

Little has been reported about people’s preferences for day of the week and time of the day when it would be most convenient for them to participate in the survey. One would assume that availability depends on people’s daytime commitments and lifestage. It has been shown in the Health Survey for England that people attending the survey outside ‘office hours’ have different demographic and socio-economic profiles than those attending during the day on weekdays [29]. Therefore, it is important to organize health examination surveys so that everyone is able to be seen when convenient for the participant. This means offering a wide range of opening hours for examination clinics and/or home visits to maximize participation. On the other hand, extensive opening hours makes it difficult to organize collection of fasting blood samples (at least when 8–12 hours fasting is required), which are usually collected in mornings after overnight fasting. Longer data collection hours can also require more field work personnel, which increase the survey cost.

Health examination survey participation rates across seven European countries were generally higher in women and older people, whether conducted in participants’ own homes or an examination centre. Participation rates in the National Health and Nutrition Survey (NHANES) in the USA were also higher among women. Although participation rates in NHANES increased by age among women until recently, the declining participation with age among older men now also occurs in women [54].

Participation rates declined over time by 7–15 pp in the past decade in the four countries for which comparable data were available. In NHANES, reported (interview and) examination response rates fell from (99 % and) 74 % of adults aged 25-74y in 1971–74 to (70 % and) 67 % aged 30-69y² in 2011-2012 [54]. These figures are not directly comparable with rates in our study as the NHANES data may not match our standardised definitions, however, the NHANES response rates remain high. It is possible that financial barriers to routine healthcare in the USA mean that the opportunity for a free health check has been a greater incentive in those countries than in European countries with universal healthcare.

Participation rates for England and Scotland presented in the Tables are based on households in which at least one adult responded, as the age and number of household members is otherwise unknown. After multiplying individual rates by household response rates, interview rates in England and Scotland were similar to or better than in other countries. However, the two-stage method used in England and Scotland for data collection had a marked reduction in participation in the second stage, with blood pressure measured in substantially fewer individuals than weight. There was further reduction in participation rates when blood sample consent was compared with blood pressure measurements, which occurred in the same survey nurse visit. This may reflect an unwillingness to provide blood samples in a domestic, non-clinical environment, particularly when the samples are not required for clinical reasons.

Participation rates in the examination elements in the five countries where participants attend a central location were uniformly high as a proportion of those interviewed, and participation rates for the three measurements varied little. However, the proportion attending as a percentage of the whole sample varied widely by age, sex, and country. In the Netherlands, the first phase of NLdeMaat had much lower participation rates, although already using more intensive recruitment than in the 1993–1997 MORGEN study. Phase II of NLdeMaat demonstrated that it was feasible to reproduce 'historical' participation rates, but at a much higher cost than previously.

The lower participation rate in capital cities has been noted before within individual studies [55]; we have confirmed this as a problem in most primate cities and metropolitan areas. Lower contact rates (a necessary first step before participation can be requested in surveys based on an interviewer calling in person, such as the household-based surveys) occur in more highly urbanized areas [56]. Groves and Couper relate this in part to these areas have a higher preponderance of less accessible household types, with more multi-unit dwellings, single-person households (thus a reduced likelihood of anyone being home when an interviewer calls), renter-occupied homes, more time spent commuting, and more entertainment options [56]. Large metropolitan areas also have a younger and more mobile population (increasing non-contact rates), greater poverty (also increasing non-contact [57]), and larger immigrant populations with more language barriers to survey participation, both through comprehension problems [33, 58] and language-related barriers of perceived relevance [58]. In the German DEGS1 study reported here, participation rates were inversely proportion to conurbation size (53 % in localities with a population <5,000; 35 % in cities of 500,000+) [59]. Participation rates were 10 pp higher among German than other nationals in DEGS1 [59].

The main concern over lower participation rates is the extent to which the results are representative of the underlying population, with non-response bias if participation is not random. Groves summarised five theoretical models relating response propensity to the likelihood of non-response bias [60]. He concluded that while non-response bias does occur, it is not a simple function of the non-response rate: higher response rates reduce the risk of such bias but not the actual presence of such bias. Non-respondents to health examination surveys have higher mortality than participants [11, 15‐17]. Socio-economic position is associated with non-response [11, 23]; one study found that deprivation was associated with difficulty in making contact but individuals in less deprived areas had higher refusal rates [57]), Other studies found higher response rates among more educated individuals [13]. Some but not all studies have found both demographic and health differences between those responding early to a survey and those requiring greater effort to contact or to encourage into the survey [61, 62]. Representativity indicators have been proposed to replace response rates; they can be generated during fieldwork to target less represented potential participants (a responsive survey design) [63]. Field substitution to address differential non-response has little effect on the results [64] but post-survey adjustments (e.g. non-response; weighting) removes these differences, [61, 64, 65] with a suggestion to reduce fieldwork costs (or increase sample size) by reducing efforts to contact or convert non-responders and weighting data instead [61, 66].

Factors influencing decisions to participate in a survey include societal and individual level factors, survey design, and interviewer attributes [67]. In addition, the perceived relevance or sensitivity of the topic can affect participation. Thus health surveys may have health-related non-response bias both because those whose health is less good are more likely to be available [29] to contact, and because the survey may appear more salient to them. Gibson et al. have found reporting heterogeneity for similar health questions on limiting longterm illness, with lower prevalence among census respondents than health survey participants [68]. In contrast, alcohol consumption is often under-estimated by surveys; Gray et al. compared morbidity-linked survey and population data and proposed a method to develop more accurate (higher) alcohol consumption estimates using multiple imputation [69]. However, Hall et al. found that non-response weighting bias addressed health variables adequately, other than smoking, but not attitudinal variables [66].