Measuring obesity in the absence of a gold standard

Highlights

• The proportion of obese people correctly classified using BMI ranges from .55 to .87.

• The proportion of the non-obese correctly classified using BIA ranges from .47 to .86.

• The misclassification rates for BMI and BIA are highest for women.

• Waist circumference outperforms both BMI and BIA in all demographic groups.

• Estimated latent obesity is highest for Hispanic women and lowest for Black men.

Abstract

Reliable measures of body composition are essential to develop effective policies to tackle obesity. The lack of an acceptable gold-standard for measuring fatness has made it difficult to evaluate alternative measures of obesity. We use latent class analysis to characterise existing diagnostics. Using data on US adults we show that measures based on body mass index and bioelectrical impedance analysis misclassify large numbers of individuals. For example, 45% of obese White women are misclassified as non-obese using body mass index, while over 50% of non-obese White women are misclassified as being obese using bioelectrical impedance analysis. In contrast the misclassification rates are low when waist circumference is used to measure obesity. These results have important implications for our understanding of differences in obesity rates across time and groups, as well as posing challenges for the econometric analysis of obesity.

Introduction

Obesity is defined as an excessive accumulation and storage of fat in the body and is a leading cause of morbidity, disability and premature death and increases the risk for a wide range of chronic diseases (WHO, 2009, Antonanzas and Rodriguez, 2010, Konnopka et al., 2011). Cawley and Meyerhoefer (2012) estimate that total medical costs of obesity for the full non-institutionalised population of adults aged 18 and older in the U.S. was $190.2 billion in 2005. In 2012 the American Medical Association put a resolution to its delegates asking that obesity be recognised as a disease in the hopes that doing so would change the way the medical community tackles this complex health issue. In the ensuing debate the Council on Science and Public Health (2012) published a report outlining the advantages and disadvantages of such a move. In particular they expressed concerns with existing diagnostic tests of obesity and noted that “if obesity is to be considered a disease, [then] a better measure of obesity than BMI is needed to diagnose individuals in clinical practice.”¹ In this paper we draw on work from other areas of biostatistics to propose a new way of evaluating alternative tests for obesity and use our results to make recommendations for the diagnosis and management of obesity.

The traditional and most popular measure of obesity is based on an individual's body mass index (BMI). Despite its widespread use there is a body of research that argues that BMI is, at best, a noisy measure of fatness since it does not distinguish fat from muscle, bone and other lean body mass (Johansson et al., 2009, Burkhauser and Cawley, 2008, McCarthy et al., 2006, Smalley et al., 1990). Because of these shortcomings in BMI, a World Health Organisation expert consultation on Obesity drew attention to the need for other indicators to complement the measurement of BMI (WHO, 2000). Consequently, a number of alternative measures have been proposed. These include percent body fat estimated using bioelectrical impedance analysis (BIA), measures based on waist circumference (WC), Waist to Hip ratio (WHR) and the ABSI index of body shape². Different approaches to measuring fatness not only yield different rates of obesity (Burkhauser and Cawley, 2008), have different impacts on outcomes (Johansson et al., 2009, Krakauer and Krakauer, 2012, WHO, 2011, Song et al., 2013), but also give rise to different trends in obesity over time (Elobeid et al., 2007, Burkhauser et al., 2009, Ford et al., 2014).

When evaluating alternative measures of fatness the tendency to date has been to settle on a specific, preferred measure as a gold-standard and use this measure to benchmark other diagnostic tests (Smalley et al., 1990, Mei et al., 2002, Burkhauser and Cawley, 2008). For example, using BIA measures as the gold standard Burkhauser and Cawley (2008) find that 61.25% of women classified as non-obese by BMI are false negatives, with no false positives, while for men 14.20% of those classified as obese by BMI are false positives and 33.5% classified as non-obese are false negatives.

In this paper we take a different approach to comparing the accuracy of alternative measures of obesity, motivated by the fact that, a-priori, there is no strong basis for choosing any single measure of obesity as a gold standard. In their survey of alternative measures of obesity Freedman and Perry (2000) note that “The lack of an acceptable gold-standard limits the assessment of the validity of field methods that can be used to estimate body fat.” Hu (2008) provides a detailed discussion of the strengths and weaknesses of alternative approaches to the measurement of body composition. Recently developed high-tech imaging options, such as computed tomography and magnetic resonance imaging, offer excellent accuracy and allow researchers to distinguish between visceral and subcutaneous fat, a distinction that is important in helping understand the consequence of obesity. However, Hu (2008) notes that their cost, technical complexity and lack of portability prohibit their routine use in large scale studies. To date the use of these advanced approaches have been limited to small-scale studies³.

Rather than specifying a gold-standard ex-ante we allow all measures to be potentially imperfect indicators of fatness. When one test is specified as a gold standard evaluating all other possible tests is straightforward. However, in the case where all of the tests are potentially imperfect the task of evaluating diagnostic tests is more difficult because the true underlying disease status of each individual in the study is unknown. However, by treating the true unknown disease status as a latent variable, it is possible to use latent class analysis (LCA) to estimate the true underlying prevalence of the disease along with the characteristics of each of the tests (Walter and Irwig, 1988, Biemer and Wiesen, 2002, Biemer, 2011)⁴. This approach has been used elsewhere in biostatistics, for example when comparing alternative skin tests for the presence of tuberculosis (Hiu and Walter, 1980), comparing diagnosis of myocardial infarction (Rindskopf and Rindskopf, 1986), evaluating diagnostic tests of autism (Szatmari et al., 1995) and malaria (Gonçalves et al., 2012). However, to my knowledge, LCA has not been used before to evaluate alternative measures of obesity.

Using data from a representative sample of US adults I show that that while obesity rates based on BMI and BIA misclassify large numbers of individuals, this is not the case for measures based on WC. The error rates for WC measures of obesity are of the order of 3% compared to error rates as high as 45–70% with BMI and BIA. In particular we show that BMI suffers badly from a high rate of false negative diagnoses while BIA suffers from a high rate of false positives. These results have important implications for differences in obesity rates across time and groups, as well as for traditional econometrics analysis of obesity.

In Section 2 of the paper we discuss latent class modelling in diagnostic testing, while Section 3 discusses the NHANES data used throughout the analysis. Section 4 presents and discusses my key results and Section 5 examines the robustness of these findings. Section 6 concludes.