Urban health indicators and indices—current status

Soon after the Millennium Declaration, a Health Metrics Network, [4] funded by the Gates Foundation, was established to assist member nations in developing and interpreting health data. This partnership has recognized the key relationship between an indicator of health and an indicator of health disparity, and has provided leadership in deriving the latter from the former. An important contribution of those involved in the Network is a concise summary of the methodologies available to examine indicators and assess inequalities (first published in Spanish [5] and subsequently in English; [6‐8] this discussion was based on Mackenbach and Kunst; [9] see also Houweling et al. [10]).

The authors point out that the vast majority of Indicators are based on data aggregated by geopolitical unit. A subset are cumulative markers (Gross National Product (GNP), percent of literacy, unemployment rate) that lack meaning at the individual level [6]. It is readily inferred that most analysis of Indicators is ecological, and thus constrained by the statistical limitations of correlational analysis. For example, in constructing an Index, indicators that co-vary do not necessarily increase the amount of information in the Index (though in some instances, such as a latent variable construct, they may). But conversely, indicators that are not correlated render interpretation of the Index problematic, since the level and trajectory of the Index then results from the complex interaction of disparate measures.

Perhaps more important, the authors [6] point out that indicator selection should be predicated on the ability to demonstrate disparity. By using information on the total population, and being sensitive to the size and distribution of the population along socioeconomic groupings, an overall indicator (say, GNP) can be transformed into a measure of inequality. In these articles, the authors then describe the major ways in which disparities are expressed (see Table 1). In fact, all of these are some form of ratio or difference, manipulated to highlight certain aspects of the contrast. For example, the measures that are based on the slope of a regression line (the ratio of a change in a variable compared to a unit change in another) simply provide a model-based contrast as opposed to the simple empirical observation of say, the ratio of highest percentile to lowest. The Lorenz Curve and the Concentration Curve are more complicated ratio measures, and have been shown to be specific examples of the class known as Relative Distribution Measures. Such measures raise a fundamental question for all methods of representing disparities. A simple ratio is dimensionless, and can thus be used for the comparison of many populations. A simple difference is in units of the underlying measures (money, frequency, incidence, area, etc.); some of these units permit direct comparability and others do not. Other measures of disparity compare observed data to an absolute standard (such as one of no disparity, as with the Lorenz curve and Gini coefficient). Still others use a standard embedded in the total data (say, highest or mean value). Relative distribution measures, [11, 12] on the other hand, compare two distributions directly so that, for example, the level of disparity in two urban areas can be directly described. Other measures permit this, but may require extra steps. In addition, the common practice of rank ordering, if performed without reporting the actual disparities, would not be sufficient to provide the actual difference between two areas since the space between ranks is not uniform.

More recently, Talih [13] introduced the symmetrized Renyi Index, based on prior work using entropy measures to assess disparities. An important advantage of this measure is its invariance with respect to a reference group (say, the population average or the least well off group). In addition, population-weighted and equal-weighted versions can be calculated, and an “aversion” parameter can be included that reflects the investigator’s judgment as to values that society attributes to inequality [14].

The choice of indicators, from the myriad available, should be predicated on some agreed upon set of criteria. Flowers et al. [15] provide a checklist of 20 facets of a proposed indicator: several are descriptive (title, origin, rationale, routine or special collection, frequency); others deal with general characteristics (strengths, weaknesses, perverse incentives, influence on practice or behavior). A simpler, and perhaps more forceful summary of the ideal characteristics of indicators is provided by Etches et al. [16] whose keywords bear repeating: consensual, conceptual, valid, sensitive, specific, feasible, reliable, sustainable, understandable, timely, comparable, and flexible. These authors stress as well the need for a conceptual framework [16], Fig. 1, p.34 from which the appropriate indicators can be drawn and indices can be constructed. Such a framework can be the basis for multilevel modeling [17] and for causal analysis [18]. Though neither of these approaches is necessarily involved in the formation of indices, they are part of the intellectual basis for prior and subsequent analysis.

Several statistical problems bedevil indicators. The Will Rogers phenomenon, for example, is the paradox observed “when moving an item from one set to another moves the average values of both sets in the same direction,” [15], p.243 and refers to migration of an item to a group vastly different from its own. Such a situation obtains when the highest value in one population is less than the lowest value in another, so that movement of the highest item lowers the mean of both groups. Indicators are also subject to regression to the mean, a phenomenon that reflects the random distribution of measurement error. A more extreme value will likely be followed by one less extreme because, based on typical distributions, the error in measurement of the second value is likely to be less extreme than that of the first value. As noted earlier, indicators or indices are often presented as ranks, which are ordinal rather than interval or ratio quantities despite the use of integers. Ranks convey a sense of better or worse that may not be merited by the underlying data. In addition, entities are not equally separated, and some may be bunched so that ties are resolved by resorting to a non-meaningful number of significant digits. Most indicator assessments and ranking procedures do not contain an appropriate estimate of uncertainty, and an assumed difference may be spurious. Flowers et al. [15] suggest the use of such devices as funnel plots (a standard part of the meta-analysis armamentarium) to detect aberrations in the distribution of values that may point to real differences.

As described by Saltelli et al. [19] indices are composite statistics that have generated polarized views of their value: either a mashed together collection of unrelated numbers or a usable distillation of reality. But these authors go on to point out that such statistics are really mathematical models developed through a social process: the community of scientists, policy makers, and practitioners must largely agree on their makeup and utility. The European Commission Joint Research Centre group on Composite Indicators [20] has explored the mathematical, political, social, and economic aspects of composite indicators in detail [19, 21‐26]. This complex analysis provides a rigorous basis for combining criteria and for legitimate ranking schemes. As these and other investigators point out, linear aggregation, with either equal weighting or some other weighting scheme, is simplest, most commonly used, and often least reproducible, in that it does not derive from pre-established criteria, but rather from experience and negotiation. Geometric aggregation, usually by multiplying the nth root of n items, has been used successfully by the UNDP Human Development Index [27] but does require higher technical capacity. The most complicated of the approaches—multi-criteria analysis [28, 29] —is less adaptable for use on the local level, but a toolbox of techniques has been developed, [21, 22] and the use of this approach is a good example of the potential value of an academic and public health partnership [30].

In its simplest form, an index is constructed from a set of indicators that have been transformed (standardized, normalized, scaled) so that they are directly comparable, and then added together. Simple arithmetic combination, often mistakenly called “unweighted,” implies that each indicator is given the same unit weight. The resulting Index may be bounded (such as a proportion or percent) or unbounded at one or both ends. A simple example is the Social Health of the States, [42] a long-running Index from the Institute for Innovation in Social Policy. It combines 16 indicators that have been scaled and averaged so that the worst possible score is 50 (smaller is better). The difference between a state’s actual average and 50 is then expressed as a percentage of 50. The states are rank-ordered and grouped in quintiles (1–10 are excellent; 41–50 are poor). Similarly, the Michigan Index of Urban Prosperity [45]—one of the specifically urban indices—combines nine indicators from multiple sources (crime rate; property value change; median household income; employment rate; employment change; graduation rate; Michigan Education Assessment Program passing rate; young adults; population change). It uses the ratio of each site-specific indicator to the overall state indicator (actually, to the overall mean) and averages them, deriving a number in the vicinity of 1.0. A somewhat more complicated urban metric is the Index of Resident Economic Well Being, [46] which combines indicators from five Domains (unemployment rate; poverty rate; labor force participation; median household income; per capita income) by using a linear combination of N-scores (deviations from the median, as opposed to z-scores, which are standard deviations from the mean).

Perhaps the most important of these is the Human Development Index (HDI) [27] now in its 25^th year, published by the United Nations Development Program (UNDP). The measure is constructed from life expectancy at birth, measures of schooling and expected years of schooling, and gross national income per capita. Each of these is standardized by taking the country value as a percent of the range of the most extreme values for any participating country over the past 20 years compared to subsistence value: ([country value – subsistence value]/[maximum value – subsistence value]). The resulting value is a proportion between 0.0 and 1.0. The two education values are combined by taking their arithmetic mean and combining them with the other two measures using their geometric mean ([life expectany^1/3 x schooling^1/3 x income^1/3]). A country’s HDI is then rank-ordered among all the others, and its place over time can provide the size and direction of relative progress (“relative” because a country’s change in rank may not reflect its change in absolute values). Since the standards for “best” and the “worst” are fixed, and each nation’s values are placed on a scale with that range, the concept of disparity is an integral part of the measure. Though urbanicity is not the focus of the HDI, its approach and methodology are suited to the development of a measure of urban health and disparities. In addition, the UNDP introduced an inequality-adjusted HDI, [47] a measure that accounts for inequality by adjusting each indicator’s value by its level of inequality, based on work by Atkinson [48] wherein he used the analogy between ranking inequality distributions and ranking probability distributions based on utility. Each of the three indicators is adjusted by the ratio of the geometric mean of the distribution to its arithmetic mean. Using a similar statistical approach, they have also introduced a Gender Inequality Index that captures the difference in reproductive health, empowerment and the labor market for men and women. A third measure—the Multidimensional Poverty Index—diverges from the Human Development Index by using microdata from household surveys. Each person is classified as poor or non-poor based on his or her family deprivation and the data are aggregated to form a national index. The actual computation bears considerable resemblance to previously discussed aggregations of indicators (weighted linear combinations), though the mechanism for combining information on the 10 indicators used is complex.

Another example of a more complex measure, the Bertelsmann Transformation Index (BTI) [49] takes a wholly different approach. In their process, 17 criteria (“Domains”) are represented by 52 questions (“Indicators”) that are answered in a report completed by 128 participating nations. The answers go through two levels of review and calibration (not further defined) by experts in the responding country and by the BTI board. Scores are combined by linear aggregation (not further defined), and an overall score and sub-domain scores are calculated. The approach may be described as a modified, interactive, Delphi technique that is heavily dependent on expert opinion, and may or may not be reproducible. Such an approach, however, recognizes, and in fact embraces, the political process that is an important part of index development.

A third approach is typified by the Corruption Perception Index [50] produced by Transparency International that ranks countries by the perception of corruption in their public sector. They collect information from a variety of sources (of which the BTI is one) and use at least three different sources for each country. This approach represents a substantial divergence from most of the others in that a uniform data set is not used for each country. Rather, they subject available information to substantial mathematical manipulation: data are standardized by using matching percentiles (reminiscent of the relative distribution methods), then undergo beta transformation, and a linear average of the transformed values is taken. The final index and ranking are substantially removed from the raw information. This approach acknowledges presumed exchangeability of indicators after mathematical manipulation.

Still another approach might be termed the “organic” Index, one that grows, shape-shifts, and is tested for its credibility and consistency. An example is the Deprivation Index, first proposed by Townsend [51] in 1987 and Carstairs [52] in 1989. These were constructed as the sum of four standardized variables. The Townsend Deprivation Index used percentage of unemployed people in the active population, percentage of not-owner-occupied households, percentage of households without a car, and percentage of overcrowded households. The Carstairs Index replaced no-owner-occupied households with the percent of low social class persons (a measure available in England based largely on occupation). In a subsequent review of Deprivation Indices [53], Carstairs describes other variations, such as the Jarman Underprivileged Score [54], constructed from rankings by general practitioners and subsequently used as part of a reimbursement scheme. Carstairs demonstrates that the Deprivation Index, as she developed it, was strongly correlated with measures such as overall mortality and cancer registrations.

In more recent years, the Deprivation concept has been retained, but the details altered. Sivakuman [55] proposed a Human Deprivation Index based on percent below the poverty line, infant mortality, and illiteracy rate. One-third of each is added together to form the Index. Messer et al. [56] constructed a Deprivation Index based on five sociodemographic domains: income/poverty, education, employment, housing, and occupation. They used principal components analysis, taking the first principal component as representative of neighborhood deprivation, an assertion supported by the consistency of component loading across study areas. Rey et al. [57] explored the properties of their previously developed Index, FDep99, which had been constructed from: median household income; the percentage high school graduates in the population aged 15 years and older; the percentage blue collar workers in the active population; and the unemployment rate. This measure was also constructed using principal components analysis, and the first principal component accounted for 68 % of total variation in mortality. The authors provided an empirical analysis that purported to show that FDep99 was superior to their slightly altered versions of the Townsend and Carstairs Indices.

The aforementioned Indices are instructive in providing a typology, but only touch on the extant composites that have been developed. In a systematic review, Kaltenthaler and colleagues [58] described 18 health indices culled from the literature from 1966 to 2000, and summarized information on their origin (US, UK, Canada, and Europe), characteristics, purpose, types of indicators, methods of aggregation, data sources, and validation. Several major points emerged. First, only four of the indicators had been validated, two by professional judgment, two by inference. Second, the user groups were not clearly defined, so that the target geopolitical level was not always clear. Reasons for choice of indicator were opaque. Weights appeared to be arbitrary, or at least not justified by standard criteria. The data upon which many of these indices were based were not always publically or universally available. The authors concluded that this set of indices would not be suitable for health policy makers in the United Kingdom (the place of origin of the study). Nonetheless, the authors reaffirm “the need for a population-based health index at either national or local level.” [58], p. 254.

Unfortunately, the literature on Indices that reflect urban health specifically is sparse. Those that include both urbanicity and health tend to focus on the former. As an example, Shane and Graedel [59] propose a set of indicators that includes a measure each for air, water, soils, transportation, energy, resource use, population, urban ecology, livability, and general environmental management. They do not include health measures per se, but do use the Human Development Index as an environmental measure. Instead of a composite index, they propose a novel graphic: a triangle made of four layers (planning, waste, resource, human factors). Each of the 10 metrics is represented in the triangle by a grey scale corresponding to its adequacy (high, middle or low rating). The resultant “picture” can be compared to triangles from other areas, and can be used as a marker for evaluation over time.

An exercise in index construction was conducted by Stephens et al. [60] who used a workshop environment to build an Index of Deprivation that compared Accra, Ghana with Sao Paolo, Brazil. Interestingly, groups working on the two areas devolved on the same Domains (income, education of head of household, number of persons per room, sanitation, and safe water access), but had to use different Indicators within those Domains. The collected data produced an overall picture that concealed substantial differences between the two areas. Those differences were demonstrated, however, by a simple choropleth map comparing the two cities by using four levels of socio-environmental conditions. Nonetheless, the authors felt that the data and resulting indices did not fully capture the political and social complexity of the cities. They do, however, cite several positive policy changes that resulted from the exercise. An important message from the study is the need for greater flexibility in the choice of indicators that make up an Index, since their true function may be as a catalyst for local change.

We have recently published [61] an Urban Health Index (UHI) that focused more on method than content. Adopting approaches used for the Human Development Index, [27, 47] the UHI permits construction of a variety of composite indices related to urban health, urban health disparities, and health determinants, and is coupled to a technique for mapping that provides visual display of disparities for contiguous small areas. Indices are standardized by transforming the values for each small area into a proportion of the range for the overall location, and are then combined by taking their geometric mean. The method, still under empirical investigation, may be of use in demonstrating health disparities and the geographic distribution of inequalities. It is an example of the reorientation of composite indices from methods for ranking to flexible tools for use by local public health workers to assess health status, needs, and disparities. In addition, it highlights the need to collect data as an integral part of the construction of indices. Small area data—differing only in scale from the more routinely collected large area data—are critical for understanding the urban microenvironment.

A complementary approach to the assessment of health disparities is the burgeoning field of geovisualization. The growing armamentarium of data and geographic tools has given rise to alternate methods for measuring disparities, some of which can be married to the just described indicators and indices. For example, measures of urban design (enclosure, scale, transparency, complexity) can be obtained directly from digital sources and used to define urban space that may house the disadvantaged [68]. Remote sensing has been coupled with GIS methods (in Bangladesh, for example [69]) to demonstrate concentrations of poverty and the heterogeneity within impoverished areas. Techniques for assessing access to parks and other environmental landmarks have been used to provide measures of the availability of health activities within an urban space [70]. Google StreetView makes possible measures of local food availability with easy connection to population density and other factors that may affect disparities [71].

A series of studies from Australia demonstrate the potential melding of health and environmental indicators and geovisualization. Badland and colleagues identified 11 domains for “liveability” that included 61 usable indicators, and developed a framework that connected these indicators to social determinants of health [72]. They applied this concept to demonstrate the connection between Public Open Spaces (POS) and the mechanisms by which they influence health [73]. Similarly, a set of public transport indicators were developed, and their pathways of connection to population health explored [74]. This work-in-progress promises to bring environmental factors (open spaces, transport) together with health determinants in real space, and to serve as a complex metric for identifying health disparities.