Using a positive deviance framework to identify Local Health Departments in Communities with exceptional maternal and child health outcomes: a cross sectional study

Since the articulation of the ten essential public health services, researchers have used a variety of strategies to assess public health system performance including grant reporting requirements, program evaluations, and community health assessments [15]. These assessment tools, however, tend to rely on self-reported, often incomplete data that may not be externally validated. In our study, we used a systematic means of rigorously identifying LHDs in communities with better than expected MCH outcomes compared to peers in their state – positive deviants – in order to determine what factors support positive unexpected population-level achievements.

Positive deviance is a framework that identifies and learns from units that perform beyond expectations. In public health, positive deviance originated from the concept that interventions could be designed around uncommon beneficial health behaviors that some community members already practiced, [16] and more recently, to identify system-level approaches to improve population health outcomes by reducing unwarranted variability between or within institutions [17].

According to Pawson [18] a primary function of evaluation research should be to capture a clear understanding of why some programs achieve better outcomes than others. Traditional evaluation approaches such as meta-analysis and narrative review are often either so general as to make no effort to understand how programs work contextually, or are so specific to times and places that lessons cannot be transferred to other contexts [18]. To overcome these limitations, evaluation research should seek to understand program mechanisms, put findings in context, and generalize findings to external situations (Context + Mechanism = Outcome). We utilized this framework to organize the methods for identifying positive deviant LHD jurisdictions in order to explain not only who was performing well, but why.

Numerous factors influence population health outcomes, over which LHD administrators have varying levels of immediate control. Factors over which LHD leaders have limited or no control - the context in which they find themselves [19] - may include state and local budgets, population size, and geography. Many of these factors have been found to have a significant impact on LHD performance [8, 20]. LHD staff, however, cannot generally change these circumstances.

Factors over which LHD leaders have some control include the activities they choose to undertake - mechanisms [19] used to improve health outcomes - including establishing partnerships, applying for external funding, and choosing which services and how much of them to provide in the community. Many of these variables have been shown to impact community health outcomes and are much easier to change than context factors.

Outcomes result from a combination of context and mechanisms (context + mechanism = outcome) [19]. We identified LHDs in communities that experienced better outcomes than their in-state peers to better understand the interaction between the environments in which LHDs exist (context) and how specific activities are triggered (mechanisms) that lead to exceptional health outcomes. The positive deviance approach can help improve the performance of public health systems by building a shared evidence-base of best practices to improve outcomes across institutions and geographic locations, while taking into consideration local context and resources.

We used multivariate regression models to identify positive deviant LHDs jurisdictions, in relation to MCH outcomes. Our goal was to exploit the linear relationship between contributing factors and outcomes, [21] rather than test hypotheses or make inferences. The model we used sought to explain MCH outcomes (Y) as a function of contextual factors (Z) (over which LHDs have little or no control) and internal mechanisms (X) (over which LHDs have some control [18].

We included the following contextual variables in our analysis (Appendix): total LHD expenditures, county level population size, number of Medicaid-funded births, Core-Based Statistical Area (CBSA) (metropolitan, micropolitan, rural), [25] Robert and Reither’s Social Disadvantage Index, [26] percent of children under 18 living in poverty, percent of persons 25+ with high school or more education, percent of African American residents, percent of Hispanic residents, and per capita health care providers (nurses, midwives, and doctors). Similar variables have been used as controls in other PHAST MCH studies [21, 24]. As controllable mechanisms (X), that have been shown to be related to LHD service provision and/or health outcomes, [27] we included whether or not certain public health services are being provided instead of - or also by - an “alternative provider” [21, 28] and if the LHD’s lead executive is a clinician [13, 29]. We also classified LHDs based on a latent measure developed from activities described as provided by individual LHDs in the 2008 National Association of County and City Health Officials (NACCHO) National Profile of Local Health Departments (Profile) survey [30]. The latent measure developed from the Profile capture an LHD’s general approach to service delivery as in a range of overall service delivery: limited to comprehensive [31]. For outcomes (Y) we included rates of teenage pregnancy/births, rates of late or no prenatal care, infant mortality rate, and percent of low weight births. These are proximal MCH outcomes where relationships between investments and outcomes have been found [21].

Identifying positive deviants required classifying all cases exceeding or not exceeding a threshold for each outcome [32]. We defined positive deviants using three steps:

Examining each outcome where lower was better; we identified potential positive deviants as those with studentized residuals less than −1, as they performed better in the outcome than the model predicted (better than their peers). Using a cutoff of −2 was too restrictive and did not allow for variation in LHD context. We identified influential leverage points (n = 4, 1 %) to ensure we were obtaining the same positive deviants with and without them. The analysis was robust to the removal of leverage points except for two LHDs each in 2009 and 2010 which were removed from the Florida models for low birth weight, infant mortality rate and late or no prenatal care. The positive deviant observations we identified were largely robust to sensitivity analyses with and without these influential leverage points. The few that were not robust¹ had relationships between the outcome and inputs that were consistent with the overall trend. We ran additional sensitivity analyses to ensure that LHDs with missing data were similar to those included in the models and that selection of positive deviants was robust to removal of highly correlated covariates and to the use of more parsimonious models.

We conducted these analyses within states for each outcome. This allowed us to control for state-level differences in expenditure and outcome measures and created a consistent scale for identifying state-specific outliers. To ensure reliability in our identified positive deviant observations, we created a matrix and included only LHDs that were identified as outliers over multiple years and/or outcomes. An LHD, for example, that had exceptional outcomes in low birth weight in 2009, and low birth weight and low teenage birth rate in 2010 would be included, but an LHD with only exceptional rates of low birth rate in 2009 would not be included.