Background
The Global Polio Eradication Initiative (GPEI) is closer than ever to a polio-free world. This success can be attributed in part to the use of supplementary immunization activities (SIAs) or campaigns, at the largest scale ever seen in public health [
1,
2]. In order to achieve poliovirus (PV) eradication, it is crucial for policymakers to know which areas are most vulnerable to PV and what impact SIAs are having in addressing this vulnerability.
During a SIA, health workers attempt to provide oral polio vaccine (OPV) to all children under 5 years old in a given area, typically through house-to-house vaccination campaigns. Reaching all children is difficult for a variety of reasons, including out-of-house children, vaccine refusals, and incomplete household maps. Currently, two types of post-campaign surveys are used to assess campaign coverage: independent monitoring [
3], and cluster lot quality assurance sampling (LQAS) [
4]. However, coverage estimates from independent monitoring are often unrealistically high and inconsistent with observed epidemiology [
5]. On the other hand, coverage estimates from LQAS are considered more accurate, but are less precise and not as widely available. Further, both LQAS and independent monitoring are household-based surveys, and may miss children in informal settlements, hard-to-reach areas, and mobile populations [
4].
In order to gauge the need for SIAs, policymakers use indicators of vulnerability based primarily on dose-histories of non-polio Acute Flaccid Paralysis (NP-AFP) cases collected through the global polio surveillance network [
6]. While reported dose histories may suffer from recall bias, they have been shown to be robust indicators of individual immunity [
7,
8], and the collection of dose-histories from AFP cases in a particular area serve as useful indicators of population immunity [
9]. However, sparse data limits accuracy of vulnerability estimates at small spatial scales [
9], and also the responsiveness of the estimates to immunization activities. Perhaps more importantly, reported doses are the result of past SIAs, but the impact of SIAs on reported doses is not directly measured or used in SIA planning.
In this manuscript, we introduce a novel method for estimating campaign quality, which we call campaign effectiveness, from dose histories of non-polio AFP data in northern Nigeria, and demonstrate how these quality estimates can be used to estimate population immunity. We use the Bayesian hierarchical model to account for sparse data and improve estimates at the district level. We then show how these quality estimates and immunity can be used prospectively in SIA planning. We validate our method by comparing estimated population immunity to the occurrence of confirmed wild poliovirus (WPV) and circulating vaccine-derived poliovirus (cVDPV) cases, and by comparing campaign effectiveness estimates to LQAS data.
Discussion
The Nigeria polio program underwent a restructuring process in October 2012 that led to the creation of the Polio Emergency Operation Center, an assembly of polio public health experts from international development agencies, such as WHO, UNICEF, Red Cross, and CDC, working closely together under the leadership of the Federal Government of Nigeria.
The recent success against polio has been attributed to the widespread use of data for action, innovative methods such as GIS/GPS tracking [
18], use of dashboards, and statistical modeling to project population immunity. These population immunity projections and maps provided valuable guidance to the program in determining areas of low population immunity where the PV was likely to take seed and cause an outbreak. By studying these immunity maps, the program was able to take pre-emptive actions to ensure that the best vaccination teams and supervisors were deployed to these weak or vulnerable areas, to ensure that the immunization campaigns were of the highest quality, improve population immunity and, therefore, stave off any PV outbreaks.
In 2012–2013, when PVs were being frequently isolated despite improving campaign quality, population immunity maps
1 were used as advocacy tools to encourage political office holders not to relent on their political and financial support to the program despite setbacks.
Because population immunity maps closely matched the other indices used to assess the quality of supplemental immunization activities, they provided real proof to the GPEI that Nigeria was making progress towards stopping polio transmission despite skepticism on the part of many about the quality of data generated from immunization activities.
Furthermore, over the course of several months, population immunity maps were used as an additional layer of evidence to determine the performance of LGA teams in the implementation of the accountability framework. Thus, LGA programs were more likely to be acknowledged and rewarded if population immunity projections also aligned with other criteria used for performance management.
Our calculations of campaign effectiveness showed steadily increasing quality from 2004–2014, in line with increased focus and resources devoted to the country’s polio program [
19,
20]. In Kano, the large improvement in 2014 was correlated with the widespread use of GPS vaccinator tracking [
18] and GIS-based micro planning [
19] to ensure all settlements are visited by vaccinators. In addition, mobile health camps providing additional routine immunizations, oral rehydration salts, and other medications delivered during vaccination campaigns began to be used at scale in 2014 – especially in Kano.
In contrast, campaign effectiveness showed mixed improvement in northeast states, particularly Borno, comparing 2010 and 2014 results. The northeast has been the center of civil conflict between the extremist group Boko Haram and civil authorities, and accessibility of areas to polio vaccinators has been affected [
21]. Our results suggest that efforts to improve campaign effectiveness in the northeast have been impeded, most likely due to regional insecurity.
When comparing our campaign effectiveness estimates with LQAS data, we found a good correlation but a systematic downward bias of our estimates [
5], suggesting that the number of doses reported by AFP cases could be systematically under-reported, that the populations captured by AFP are relative underserved compared to those captured by the LQAS surveys, or that the LQAS estimates are biased upwards, e.g. when difficult to access populations are not surveyed.
Similarly, immunity reconstructions show overall improvement in northern Nigeria, particularly for type 1. This is mirrored in case incidence: the last case of type 1 WPV was in July 2014 [
22]. However, the explicit link between vaccine usage and our immunity model shows that type 2 results are far more mixed: long periods of no or limited tOPV use result in declining immunity. Efforts to rapidly increase type 2 immunity before cessation of type 2 OPV in April 2016 with increased campaigns are reflected in our immunity results, which also illustrate what may be expected by applying different campaign calendars. These results emphasize lingering concerns over type 2 immunity in Borno.
Our estimates of campaign effectiveness and population immunity rely on the accuracy of AFP data. The varying causes and detection rates of AFP make bias in surveillance data both likely and difficult to quantify [
23]. It is possible that AFP data is more effective than traditional cluster surveys in measuring under-served populations, as those people who become paralyzed may seek treatment even if they are not represented within a sampling frame or present at the time of the survey. Conversely, remote populations may not have access to health services and thus not report their cases of paralysis.
Assessing campaign quality from surveillance data is also complicated by the difficulty to recall vaccination dose history, for example in areas which receive many campaigns, or when the child is older. Poor recall in older age groups may be captured by the age-structure model to some extent. In our model, we saw that the estimated campaign effectiveness for older children was lower than for younger children, which could be evidence of inaccurate dose recall.
Our method also relies on the quality and granularity of historical campaign data. In certain special cases, the official campaign calendar may not reflect circumstances when campaigns were cancelled due to accessibility issues (e.g. in Borno state), or when special interventions are conducted to deliver additional doses (e.g. health camps and permanent vaccination teams). In these rare cases, it is possible for our estimates of campaign effectiveness to be biased downward (when campaigns are cancelled) or upward (when vaccination is conducted outside of campaigns).
Explicitly incorporating information about routine immunization (RI) coverage into our methods would improve the accuracy of our estimates of immunity and campaign coverage. Currently, we estimate the vaccine mix received by an AFP case using only the campaign history. This is a good approximation in areas which use tOPV exclusively or where RI coverage is low. In areas where multiple types of vaccines are used (tOPV, mOPV1, mOPV3, bOPV), the accuracy of immunity estimates would be improved by considering RI: increasing the type 2 and decreasing the types 1 and 3. Our estimates of campaign effectiveness assume all received OPV doses are due to vaccination campaigns, such that the ratio of received doses to the number of campaigns exposed to is a measure of campaign effectiveness. In areas where RI coverage is high, multiple OPV doses can be attributed to RI instead of campaigns; incorporating this effect, our estimates of campaign effectiveness would be reduced, and the accuracy of our estimates would be improved. In northern Nigeria, RI coverage is low and the number of vaccination campaigns is high, such that the method presented produces good results.
Going forward, we could also incorporate regional differences and the uncertainty in per-dose vaccine efficacy, and we could attempt to explicitly model within-district heterogeneity in campaign coverage to better capture the effect of chronically missed children.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
AUB, AV, GCC, and HL contributed to design, methods development, and analysis. AUB, AV, GCC, HL, and FS contributed to the interpretation, writing, and critical review of the article. All authors read and approved the final manuscript.