The Oslo Health Study: The impact of self-selection in a large, population-based survey

Since the Black report was published in 1982 [1] a considerable number of papers have described social inequality in health and discussed possible reasons for the invert association between social position and mortality [1‐4]. Even in the egalitarian Scandinavian countries there has been reported a substantial difference in health between the least and the most privileged groups [5]. Ecological analysis in Oslo also showed large differences in mortality rates between residential areas characterised by social inequalities [6]. In order to identify variables at individual level to explain these health differences, the Oslo Health Study, an age stratified population-based health survey, was carried out in 2000–2001.

Valid estimates of health inequality, however, and especially prevalence figures depend heavily upon representative attendance. The low response rates observed of certain exposed and affected groups such as lower social classes, elderly, single-households, third-world country immigrants, receivers of social security benefits, urban city dwellers, people with poor self-reported health and unhealthy lifestyle, have been a major concern in many population-based health surveys [7‐18]. The Oslo Health Study (HUBRO), (HUBRO = eagle owl, acronym for the Norwegian title of the Oslo Health Study), provided the possibility to compare the prevalence and associations between already known background variables in the total study population with that of the attendees. We are not aware of any large population-based study using a complete set of data on both exposure and outcome variables concerning all invited subjects, analysed to assess whether the associations observed in health surveys are influenced by selection bias.

Some studies have compared own data with large-scale survey data for the total population [17, 19, 20], others have analysed fairly small samples [9, 21, 22] or compared a limited number of linked variables [8, 9]. Three Swedish population-based studies and one Dutch study have used administrative data linkage to analyse non-response bias. The first three did not, however, discuss possible bias in associations [10, 16, 23], which is of major interest, and the last paper studied only health care utilisation [15].

In the present analysis public register data from Statistics Norway (education, income, social- and disability benefits) covering all Norwegian citizens, were linked with data from the Oslo Health Study, using the individual's unique 11-digit personal identification number.

Thus the objectives of the present analysis were to:

The Oslo Health Study, a joint collaboration between the Oslo City Council, the University of Oslo and the Norwegian Institute of Public Health, was conducted in Oslo from May 2000 to September 2001. An invitation for participation in the health survey was sent to all residents born in the following years: 1924, 1925, 1940, 1941, 1955, 1960 and 1970. Additional groups, not included in the present study, were also invited to HUBRO [24].

The non-attendees in our study were characterised by being young, unmarried, males, inner city dwellers, and belong to the lower income and educational echelons. They also received, to a larger extent, disability benefit than did the attendees. This corresponds to results from other studies where the same sub-groups tend to be under-represented [8, 10‐18, 22, 23, 29]. Also in accordance with some studies [7, 10, 12, 13, 15, 16, 22] we found those persons born in non-western countries to be under-represented.

The participation rate in HUBRO was 42.4% in men and 49.3% in women, resulting in a total number of responders of 18 770. The attendance rate was lower than in previously comparable surveys in Norway [30], but only slightly lower than reported in the population-based Oslo Study of men in 1972–73 where 42.8% attended among 20–39 years old and 62.6% in those 40–49 years old [31]. Since the 1970's the Oslo population has become more heterogeneous with regard to ethnic origin and socioeconomic situation. During the last years the response rates have declined in Norway as well as in other countries [18, 32‐36].

It is not evident that a higher response rate in HUBRO would have prevented selection bias. Several other studies have demonstrated only moderate changes in prevalence estimates and sociodemographic distribution when comparing results by increasing the response rates in the range from around 30% to 70% [18, 37‐41].

The register linkage to data in Statistics Norway made it possible to estimate prevalence values of selected health variables based on information regarding certain socioeconomic and demographic variables for all invited. The observed values of self-rated health, smoking, BMI and mental health (HCSL) in the attendees differed only slightly from the estimated prevalence values in the target population when weighted by the inverse of probability of attendance. Thus, self-selection by sociodemographic variables did not influence prevalence estimates. This is reassuring and in accordance with other studies weighted in similar ways [18]. The analyses were, however, based on the assumption of similar prevalence among non-attendees and attendees within each sociodemographic group.

Assuming that the prevalence in non-attendees differs by no more than 50% from the prevalence in the attendees, the calculated prevalence for the target population do not differ much from the estimated prevalence values. Most studies report differences of less than 25–30%. In the Rancho Bernardo study [42] diabetes was about 30% higher in non-responders (5.5%) compared to responders (4.2%) whereas the difference in smoking was below 20% (29.5 % in non-responders and 24.7% in responders). The utilisation of various types of health care was 3–32% lower in a Dutch study [15]. The corresponding difference in non-fatal stroke and myocardial infarction between non-responders and responders in a Swedish follow-up study with linkage to public registers, were less than 15% [16].

For specific diseases or conditions the difference between attendees and non-attendees could, however, be higher than 50%, which means that the true prevalence in the population will diverge from the prevalence estimated from the attendees only. In our study the ratio between non-attendees and attendees with respect to disability benefits reached a maximum of 2 in some sub-groups of men born in Norway. Previous analyses have shown that the prevalence in early responders differed insignificantly from the prevalence in late responders in age groups less than 70 years [43]. In the oldest age group, however, the prevalence of diabetes, daily smoking and symptoms of mental distress was significantly higher in late responders, with the highest prevalence ratio (2.2) for mental distress in men. This suggests that a prevalence ratio of 2 could occur in the elderly leading to an underestimation of the prevalence in the population when based on the attendees. For the younger age groups we conclude that, except for special conditions obstructing people from attending health screenings physically or mentally, the prevalence estimates in the present survey, are valid for public health and administrative purposes.

The associations between education, residential region and marital status – and disability benefit, measured by odds ratios, were similar among the invited population and the attendees. This demonstrates the robustness of the effect measurements. The only divergent finding was an overestimation of the odds ratios for disability benefit in non-western born compared to Norwegian born, when calculated from attendees only. Kleinbaum et al. [44] have described how selection can be characterised conceptually by comparison of the fourfold table describing the actual sample (i.e. education low/high × disability yes/no of those who attended the screening) with the corresponding table describing the total invited population. There is no bias in odds ratio if the cross product of the selection probabilities (attendance rates) for each cell is one. Based on the numbers presented in tables 5 and 6 we have calculated these cross products. They were close to one for education and disability, whereas for country of birth and disability, the cross products were generally larger than one. This was due to a collectively over-representation of persons born in Norway without disability benefit and persons born in non-western countries with disability benefit. The overestimation of the odds ratio was, however, moderate (33% in men aged 59–60 years and 57% in men aged 40+45 years). In general, we may assume little bias in odds ratio estimates of variables of the same nature as disability benefit when the observation is limited to the attendees.

Several other studies [19, 45‐47], including the few with information on all or almost all individuals [15, 48, 49], show a rather small impact of non-response on risk estimates of health or disease with regard to various background characteristics. In the Rancho Bernardo study the mean odds ratio error was about 15% and most error terms were reasonably close to 1 [48]. Of the 54 OR of health care utilisation that were estimated for various background characteristics in a Dutch study, only 11 differed by, at the most, 10% between the respondents only and the entire sample [15]. In a previous Norwegian community cohort study [49] no overt differences were observed in associations between gender, age and smoking – and respiratory disorders when the analyses were based on initial (65% attendance) compared to "all" respondents (89% attendance).

The main emphasis of the Oslo Health Study was to provide survey data to identify and monitor social, ethnic and geographical differences in health and associated risk factors for disease, to assess the need for health services, and to initiate etiological research. The demand for representative study groups is valid when the inferential goal with the data is a description of the target population. However, a highly representative sample of participants is no longer considered essential for generalisability in etiological studies that report risk estimates rather than prevalence estimates [15, 19, 46]. Generalisability depends on the ability to abstract universal scientific hypotheses or theories from a set of observations and not only from the statistical framework of these observations [50]. In biological science, in which we include etiologic medical research, we select subjects with certain characteristics enabling us to make valid comparisons, but not necessarily being representative for the population from which they have been recruited. Rothman and Greenland [50] argue against the notion that generalisation from a study group depends on the study group being a representative sub-group of the target population, in the sense of survey sampling.

There are a number of striking examples of "unrepresentative" studies in the epidemiological literature, studies which have contributed significantly to the domain of medical knowledge [51‐55]. All these studies are characterised by being highly unrepresentative of the general population, but the etiological issues being studied are likely to represent valid contributions to the current scientific domains.

It is, of course, important to obtain as large sample size and response rate as possible to avoid selection bias. However, because the presence of selection bias is determined from the selection probabilities within the fourfold table, selection bias may occur even with a relatively large overall response rate, according to Kleinbaum et al. [44]. Conversely, there may be no selection bias despite small response rates.

The attendance in the Oslo Health Study varied according to sociodemographic variables, which is in agreement with other population-based studies in the Western world. It is therefore likely that the results from the present study have a general validity corresponding to similar studies, with the same social distribution of the attendees and the same attendance rate. Furthermore, the weighted prevalence estimates, based on background variables from the total population, were close to un-weighted estimates. This indicates that self-selection by sociodemographic background does not influence prevalence estimates to any great degree, assuming the same prevalence between attendees and non-attendees within each stratum of background variables. Assuming dissimilarities, sensitivity analyses have shown a high robustness of the prevalence estimates for prevalence ratios of < 1.5 in non-attendees versus attendees. Unhealthy and sick persons may participate to a less extent than healthy persons as illustrated by those receiving disability benefit. But this selection was independent of education, marital status and residential region leaving the associations between disability benefit and these sociodemographic variables unbiased. If any bias were present, we would expect an overestimation of the association between ethnicity (non-western born) and chronic disease. But sensitivity analysis established as probable that the association of selected variables could not be totally explained by selection bias.

The main strength of this study is that we have complete information from both attendees and non-attendees for selected variables. The main weakness is that it is difficult to generalise to other populations, although we may assume little bias in odds ratio estimates of variables of the same nature as disability benefit when the observation is limited to the attendees.