Benchmarking health system performance across regions in Uganda: a systematic analysis of levels and trends in key maternal and child health interventions, 1990–2011

We identified data sources for MCH indicators via in-country meetings with collaborators and relevant stakeholders. For this analysis, we used 17 household surveys and two population censuses (Table 1). While we identified administrative data sources including drug distribution from National Medical Stores, ITN distribution campaigns, and HMIS data, we excluded them from our analysis due to concerns about completeness and lack of reliable denominator information.

We included 20 MCH indicators that have previously been shown to be related to child survival [22] (Table 1): antenatal care (ANC, 1 and 4 visits); skilled birth attendance (SBA); immunization with the Bacillus Calmette-Guérin (BCG), measles, oral polio (three doses, OPV3), and pentavalent vaccines; exclusive breastfeeding (EBF); the receipt of at least two tetanus toxoid injections during pregnancy; reported use of oral rehydration salts for diarrhea; reported seeking of care at a health facility for children under 5 in response to suspected pneumonia; prevalence of childhood underweight and stunting; intermittent preventive therapy against malaria during pregnancy (IPTp, one and two doses); household ownership of ITNs; ITN use by children under 5; household receipt of IRS; the proportion of households that either owned at least one ITN or received IRS; the proportion of children under 5 who either slept under an ITN the night before the survey or lived in a household that received IRS; reported receipt of ACTs for children under 5 who had a fever in the last 2 weeks; and the proportion of ACT antimalarials prescribed for febrile children under 5. Additionally, we included seven socioeconomic covariates previously shown to be related to child health outcomes [23‐26]: household size, female headship, electricity availability, improved wall type, improved sanitation, improved water source, and years of education for women aged 15–44. Due to data availability, we restricted our analysis to between 1990 and 2011.

We focus on the results For a subset of indicators in this paper, but trends for all indicators can be found in Additional file 1. We also produced national estimates by pooling microdata within and across surveys. All regional and national results can be downloaded from the Global Health Data Exchange (GHDx), a publicly available health data catalogue [27].

We report estimates for 10 sub-regions (Central 1, Central 2, Kampala, East Central, Eastern, North, Karamoja, West Nile, Western, and Southwest) delineated in the 2011 Demographic and Health Survey (DHS; Additional file 2) [28]. We sought to conduct our analyses at the district level, but experienced barriers to doing so appropriately. Uganda has frequently redistricted its administrative boundaries, creating new district boundaries and reformulating existing ones several times since 1990 (i.e. there were 44 districts in 1997, 79 in 2006, and 112 in 2010 [29, 30]). GPS coordinates were not available for every survey in this analysis, and since there was insufficient documentation of restricting activities, we could not consistently extract and reassign estimates to their corresponding district in 2010. As a result, we opted to use the 10 regions delineated in the 2011 DHS.

We calculated an annual series of under-5 mortality estimates for each region from surveys and censuses that included birth history modules (Table 1). We applied synthetic life table methods to pooled complete birth history data from sources where complete birth histories were collected (i.e. DHS) to generate direct estimates of under-5 mortality [15]. For sources where summary birth histories only were collected (i.e. AIDS Indicator Survey and both censuses), we applied the combined version of the maternal age cohort and maternal age period methods to generate indirect estimates of under-5 mortality [31].

We calculated regional intervention coverage estimates for each survey-year of available data. Aside from information extracted from Netmark survey reports (Table 1), all estimates were produced using microdata and accounting for each survey’s complex multistage design. For most child-level indicators, we included children aged between 0 and 59 months; however, we excluded children under 12 months of age for immunizations, except for BCG, due to their recommended dosing schedule [32]. For BCG immunization, however, we computed coverage estimates for two age groups – under 12 months and under 59 months – in order to both reflect timely BCG vaccination and overall protection among children under 5. We report on BCG immunization rates for the latter age group below, but additional information on BCG coverage estimation and corresponding results can be found in Additional files 1 and 3, respectively.

Most survey questions pertaining to ANC, SBA, and IPTp only covered the respondent’s most recent birth. As this restriction disproportionately excludes older children from high parity mothers, we restricted the age range for children aged 0–12 months to avoid bias. Additionally, the Uganda National Panel Survey (UNPS) only ascertained pentavalent and measles immunization status for children aged 0–24 months, and weight measurements were taken for children aged 6–59 months [33]. In order to maximize data inclusion while keeping age groups consistent across surveys, we used the UNPS age groups for all survey data extraction pertaining to these indicators. Immunization coverage estimates were obtained using vaccine cards whenever they were available; in cases where cards were not present at the time of survey, maternal recall was used to supplement immunization data.

We adapted a previously described modeling and validation framework for modeling regional trends in under-5 mortality in Uganda [15]. Specifically, we applied the following model: where 5q0 _i,t,s is under-5 mortality in region (i), year (t), as measured by source (s). The β terms are fixed effects; β ₀ and β ₁ are the global intercept and slope, respectively, and β ₂ is an adjustment coefficient included for non-DHS sources to account for an observed discrepancy between under-5 mortality estimates derived from DHS surveys and those derived from other surveys. All other terms are random effects. Specifically, u _i and v _i are a regional-level random intercept and slope, respectively, and are both assigned conditional autoregressive priors [34]; these terms allow for each region to deviate from the global level and linear trend in under-5 mortality. w _t is a year-level random intercept assigned a first-order random walk prior [35]; this term allows for non-linearity in the global time trend. Similarly, δ _i,t is a region-year level random intercept with the prior given by the interaction between a conditional autoregressive prior for spatial trends and a first order random walk prior for temporal trends [36]. This random effect allows for non-linearity in the region-specific time trends. Finally, γ _i,s is a source-year level random effect assigned an independent and identically distributed normal prior and is included to account for autocorrelation in estimates of under-5 mortality derived from the same source in the same region. Weakly informative normal priors were assigned to all fixed effects and weakly informative gamma priors were applied to the log precision of all random effects. To generate predictions from this model, we approximated the posterior distribution of θ _i,t by setting I _{s ∉ DHS} and γ _i,s to 0. The median, 2.5th, and 97.5th percentiles of this distribution were inverse-logit transformed to generate the point estimates and confidence intervals (CIs) for 5q0 _i,t in each region and year.

We used a two-step modeling approach to generate regional trends from 1990 to 2011 for each indicator. In the first stage, we fit the following linear mixed-effects model with random intercepts and slopes for each region.

Observations are indexed to region i and year t. For modeling coverage estimates, which are bounded between 0 and 1, logit transformation was applied. On the other hand, for variables such as years of maternal education log transformation was used. We used a one-knot natural cubic spline with two basis functions (h ₁ and h ₂ ) to act as a smoother. The random effects (γ _0i , γ _1i , and γ _2i ) allow the levels and trends to vary between regions.

In the second step, the predicted trend from this linear model acts as the mean prior for Gaussian process regression (GPR), which is implemented with a Matern covariance function [37, 38]. GPR is a nonparametric technique for interpolating non-linear trends that has been used extensively in the estimation of time series data [39‐43]. Briefly, it takes into account the model variance as well as the relative sampling uncertainty of the observed data to estimate a posterior mean function. We generated trends with uncertainty for each indicator by drawing 1,000 times from the posterior distribution and back-transforming to the original scale. The point estimate was based on the median of the draws, and 95 % CIs were obtained by taking the 2.5th and 97.5th percentiles of the samples.

We created two overall intervention coverage metrics to summarize regional intervention levels. First, we estimated an overall intervention coverage metric that included 11 indicators spanning the spectrum of interventions included in this analysis: the proportion of households with IRS, ITN ownership or both; IPTp2; self-reported receipt of ACTs after fever; EBF; BCG, measles, OPV3, and pentavalent immunization coverage; ANC4; SBA; and the proportion of children who were not underweight. When constructing overall coverage metrics, prior theory-based judgments may be incorporated to reflect the relative value of interventions [44]. For simplicity and ease of interpretation, we applied equal weights for each intervention included in the overall coverage metric, therefore defining overall intervention coverage as the average of the 11 interventions. We compared the overall intervention coverage trend to under-5 mortality and socioeconomic variables using Pearson correlation coefficients.

All analyses were conducted using Stata 13.1 (StataCorp, College Station, TX) and R version 3.0.1 (R Foundation for Statistical Computing, Vienna, Austria). The model used for mortality estimation was fit using the INLA package [45].

Permission to implement this research project was obtained from the Ministry of Health of Uganda. Ethical approval for this study was obtained from the institutional review board of the University of Washington, Uganda National Council of Science and Technology, and School of Medicine Research Ethics Committee. The study was conducted in compliance with national regulatory and ethics guidelines.