Challenging the role of social norms regarding body weight as an explanation for weight, height, and BMI misreporting biases: Development and application of a new approach to examining misreporting and misclassification bias in surveys

The Survey of Lifestyle, Attitudes and Nutrition (SLÁN) 2007 is a nationally representative cross-sectional study of the adult (18+ years) population residing in the Republic of Ireland. Detailed methodology is found in the SLÁN 2007 Main Report [23]. In brief, trained interviewers conducted face-to-face interviews with 10,364 subjects selected using a multistage area probability sampling procedure. In addition, SLÁN 2007 incorporated two sub-studies for (1) the measurement of anthropomorphic characteristics (N = 967, age 18-44 years) and (2) physical examination with clinical laboratory tests (N = 1207, age 45+ years). In the main survey, self-reported height and weight were obtained and used to calculate self-report-based BMI (BMI_SR), and in both sub-studies trained interviewers measured body height and weight for the calculation of measured BMI (BMI_M).

Subjects who completed the SLÁN 2007 main survey and one of its sub-studies were selected for the present analyses, resulting in an initial sample size of 2174 (age 18+ years). Subjects were excluded if the interviewer deemed the height or weight measurements to be unreliable or slightly unreliable (N = 18) or if data was missing for BMI_M (N = 3), self-reported height (N = 75), self-reported weight (N = 48), or any of the analysis variables (N = 68). The differences between self-reported and measured height or weight were determined, and the five most extreme cases in either direction were excluded because these cases were considered to have a high probability of data recording or entry error, as opposed to reporting error (N = 20). The total number of cases excluded was 232, resulting in a final sample size of 1942.

Self-reported weight or height was deemed acceptably accurate if within ±2.0 kg or ±2.0 cm of measured weight or height, respectively. These cut-off points were determined to allow for various factors that might explain differences between self-reported and measured height and weight that can occur even in the absence of any intentional or unintentional misreporting. One factor is the technical error of measurement of weight (negligible) and height (0.5 cm), and another is temporal variation in weight and height depending on food/liquid consumption, water balance, and postural differences. In addition, height and weight in an individual can vary with age, and subjects may have meticulously reported their weight and height based on a measurement that was taken months or years ago. Finally, most subjects (97%) reported their height in terms of feet and inches, which entails additional errors. The combined effects of errors in self-reported height and weight on BMI bias were simulated and led us to the conclusion that self-reported BMI should be considered accurate if within ±1.40 kg/m², under-estimated if < -1.40 kg/m², or over-estimated if > +1.40 kg/m² (see additional file 1: Method of BMI error simulation to establish a BMI accuracy definition). This simulation method can be readily applied in samples of other populations, as it uses the mean measured weight and height of the sample.

Measured weight (kg) and height (cm) were subtracted from their corresponding self-reported values to determine errors in self-reported weight and height. Degrees of error were categorized in 1.0 cm or 1.0 kg intervals, and the frequency of error within each interval was determined and plotted in a frequency map to demonstrate the distribution of errors in the study population (Figure 1). Using the definitions of accurately reported, under-reported, and over-reported weight and height, we assigned subjects to one of nine possible types of error (visually demarcated in Figure 2, listed in Table 1). The number and proportion of subjects for each error type was determined. The difference between self-reported and measured BMI (BMI_D), an estimate of the degree and direction of bias, was calculated for the total population undergoing analysis and for each error type. To express the relative contribution of each error type to BMI_D, the following calculation was used:

where b = -1 when BMI_D is negative or b = +1 when BMI_D is positive for a given error type. The factor b accounts for the error type's direction of bias. These analyses were carried out in the total sample and the normal, overweight, and obese BMI ranges. Underweight subjects were not analyzed in this manner because the number of observations (n = 15) was not large enough for stratification by error type but were included in the total population analysis. Interval estimates and statistical comparisons were not performed because the error types are not fully stochastic.

Pearson χ² tests were used to compare the proportions of subjects in covariate sub-categories according to the definitions for under-estimated, accurately estimated, or over-estimated BMI (Table 2). Similar comparisons were performed for subjects whose self-report-based and measured BMI categories were negatively discordant (BMI_SR category < BMI_M category), concordant (BMI_SR = BMI_M), or positively discordant (BMI_SR > BMI_M) (not shown). Univariate and multivariate logistic regression were used to determine the associations (odds ratios [OR] and 99% confidence intervals [99%CI]) of self-described weight status or attempted weight management with the following binary outcomes: (1) under-estimated vs accurately estimated BMI, (2) over-estimated vs accurately estimated BMI, (3) negative discordance vs concordance, or (4) positive discordance vs concordance. Associations were also determined for under- or over-reported weight and height.

To determine self-described weight, subjects were asked: "Given your age and height, would you say that you are about the right weight, too heavy, too light, or not sure?" Attempted weight management was determined by asking subjects: "Are you actively trying to manage your weight?" If a respondent answered affirmatively, they were asked: "Is it to lose, gain, or maintain weight?" In a first analysis (Table 3, Model 1), associations were adjusted for age, sex, social class, ethnicity, marital status, highest level of education, physical activity level, current smoking status, and alcohol consumption. A subsequent analysis (Table 3, Model 2) included the same covariates as in Model 1 with BMI category as an additional covariate. A final model (Table 4) was constructed with all covariates, including both self-described weight and attempted weight management. All analyses were conducted using PASW Statistics v18.0 (Macintosh). Significance was set at P ≤ 0.01.

SLÁN 2007 was approved by the Research Ethics Committee of the Royal College of Surgeons Ireland and was funded by the Department of Health and Children in Ireland.

As BMI_M category increased, the prevalence of under-estimated BMI_SR and of negative discordance between BMI_SR and BMI_M categories increased (Figure 1A and 1B, respectively, both P < 0.001). Opposite relationships were observed for the prevalence of over-estimated BMI_SR and positive discordance, where in both cases the prevalence decreased with increasing BMI_M category (Figure 1A and 1B). Underweight subjects had the highest prevalence of accurately estimated BMI_SR (87%), with none of them having under-estimated BMI_SR. Normal range subjects often had under-estimated BMI (22%), but they had the highest prevalence of concordance (93%), and only 2% had negatively discordant BMI categories. Obese subjects were the least likely to have accurate BMI_SR estimates and to have concordant BMI categories.

To determine whether the trend for negative bias in BMI_SR was due mainly to misreporting of weight or height, we assigned subjects to one of nine error types that correspond to the nine regions depicted in Figure 2 and listed in Table 1. Figure 2 visualizes the distribution of errors in self-reported weight and height and suggests a trend towards under-reported weight and over-reported height. Table 1 indicates that the degree of under-estimation in BMI_SR increases as weight category increases. The overall BMI bias was -1.34 kg/m² and was -0.49, -1.33, and -2.66 kg/m² in the normal, overweight, and obese BMI ranges, respectively. Underweight subjects (n = 15) were excluded from this analysis because numbers in each group were too small for meaningful analysis. Although subjects in the normal BMI range exhibited a weak negative bias (-0.49 kg/m²), this degree of bias remained well within the acceptably accurate range of -1.40 to +1.40 kg/m². The error types that contributed the most to bias in the total study population and in each of the BMI categories analyzed were consistently, in decreasing order of influence: (1) under-reported weight combined with over-reported height, (2) under-reported weight with accurately reported height, and (3) accurately reported weight with over-reported height.

The proportions of subjects for each covariate category used in logistic regression analyses are shown in Table 2. Multivariate logistic regression without adjustment for measured BMI category (Table 3, Model 1) suggested that describing oneself as too heavy and actively attempting to lose weight were significantly associated with a greater likelihood of under-estimating BMI_SR and exhibiting negative discordance. These covariates were also associated with an increased likelihood of under-reporting weight (OR 1.4, 99%CI: 1.1-1.8 for both covariates, P < 0.01) but not of over-reporting height (data not otherwise shown).

However, when Model 1 was further adjusted for measured BMI category, the associations described above were reversed (Table 3, Model 2). Specifically, in Model 2, describing oneself as too heavy and attempting to lose weight were, within an given BMI category, significantly associated with a lower likelihood of under-estimating BMI_SR and exhibiting negative discordance within each BMI category. Describing oneself as too heavy was also associated with a lower likelihood of under-reporting weight (OR 0.5, 99%CI: 0.3-0.7, P < 0.001) and over-reporting height (OR 0.7, 99%CI: 0.6-1.0, P = 0.007). Similarly, attempting to lose weight was associated with a lower likelihood of under-reporting weight (OR 0.7, 99%CI: 0.5-1.0, P = 0.011) and over-reporting height (OR 0.7, 99%CI: 0.5-1.0, P = 0.008) (data not otherwise shown).

The associations described above in Model 2 for describing oneself as too heavy held when both covariates -- self-described body weight and attempting weight loss -- were entered simultaneously (Table 4). Attempting to lose weight was partially confounded by self-described weight. Four additional factors emerged as being significantly associated with BMI mis-estimation: sex, age, measured BMI category, and marital status. The strongest factor associated with all outcomes shown in Table 4 was BMI category.