Characterization of immunization secondary analyses using demographic and health surveys (DHS) and multiple indicator cluster surveys (MICS), 2006–2018

In past decades, great progress has been made in expanding the reach of immunization programme and introducing new vaccine s[1‐3]. However, vaccine-preventable diseases (VPD) remain a significant cause of morbidity and mortalit y[4, 5].

Several initiatives have allied countries and immunization stakeholders towards reaching more people and preventing more vaccine-preventable diseases (VPDs). The Global Immunization Vision and Strategy 2006–2015 (GIVS )[6], then the Decade of Vaccines’ Global Vaccine Action Plan (GVAP )[4], and soon the Immunization Agenda 2030 (IA2030 )[7], have guided efforts of achieving the vision of delivering universal access to immunization. Yet, coverage levels worldwide have plateaued at about 85% for 3 doses of diphtheria-tetanus-pertussis vaccine (DTP3) and coverage gaps persist between countries, as well as within countrie s[8]. Thus, better understanding immunization program performance can help immunization programmes better tailor strategies to reach un- and under-immunized children [4].

Vaccine coverage (percentage of target population vaccinated) is a major indicator for immunization programme performance at the local, national, regional and global levels. It is used to target resources and identify areas requiring special attention, and is a core indicator for access to basic health service s[9]. Estimates of immunization coverage generally come from several sources, including administrative systems and registries and from different household survey s[6]. Household surveys are frequently used to complement administrative data, and in some cases as the primary measurement of vaccination coverage. Oftentimes, it is thought that household surveys can give the more accurate and precise estimates, although they have the disadvantage that they cannot provide timely enough information for program planning, and can be subject to several biase s[10‐13]. The Expanded Programme on Immunization (EPI) cluster surve y[14], the United Nations International Children’s Emergency Fund (UNICEF) Multiple Indicator Cluster Survey (MICS) [15] and the Demographic and Health Survey (DHS )[16] are the three main household surveys used to obtain vaccination coverage.

DHS and MICS almost always include immunization-related questions and report coverage indicators. These coverage estimates are used at the global level to inform the World Health Organization (WHO) and UNICEF National Estimates of Immunization Coverage (WUENIC )[3] and to assess progress by other immunization partners like Gavi, the Vaccine Alliance [17]. Nevertheless, it is unclear how much the national immunization programs of countries where these surveys are implemented use DHS and MICS results, and especially whether secondary analyses on immunization are done, and if so, whether they are used to inform immunization program planning. To answer the first question, we conducted a review and characterization of immunization-related secondary analyses published in the literature between 2006, the year that GIVS was launched [6], and 2018.

As of August 2018, 308 standard DHS had been conducted (with 22 DHS ongoing at the time of the first writing in December 2018) in 91 low- and middle-income countries (LMICs) since 1985. Similarly, 352 MICS had also been done (with 60 not yet available) in 113 LMICs since 1993 (supplements 1,2) [19]. Regarding data publicly available, 94.1% (269/286) of DHS and 67.1% (196/292) of MICS datasets were available.

Searches in 15 database identified a total of 1411 papers, of which 229 passed title, abstract and key words content screening; 115 were eligible for our study [8, 20‐133]. One additional article [134] was supplemented by the Pan American Health Organization (PAHO), serving as the WHO regional office for the Americas (Fig. 1). Thus, a total of 116 articles were included (supplement 3).

Of the 116 articles included, 90 (77.6%) were published on or after 2012 and 84 (72.4%) analyses used data from a survey conducted before 2011; 66 (56.9%) analyses published within 5 years of the survey, with an average delay of 5.4 years (SD = 2.6) (Table 2). Publication of secondary immunization analyses increased overtime (Fig. 2). Regarding author affiliation, most, 80 (68.9%), of the first/corresponding authors were from high-income countries, 41 had a first or corresponding authors from the survey country, and 53 analyses had at least one of authors from the survey country. Most, 79 (68.1%), analyses used survey data from single country, and of those, only 38 (48.1%) had an author with an affiliation in that country. Nearly half of analyses conducted by high-income country used data from multiple countries.

Most articles, 100 (86.2%), only used DHS data; 55 (41.7%) focused on a specific vaccine and 53 (40.2%) focused on complete or full immunization; 77 (55.4%) and 21 (15.1%) focused on determinants of vaccination and immunization coverage inequalities, respectively. Of the 21 studies that focused on immunization inequalities, 16 focused on socioeconomic inequalities, and 3 studies on gender-related inequalities; one study focused on geographical inequality, and one focused on both urban-rural residency and gender inequality. As per acknowledging survey or study limitations, 80 (69.0%) did not mention the percentage of card/health records seen; 17 (14.6%) did not mention how persons without documented vaccination were handled; 45 (38.8%) did not mention whether the analyses was weighted; 62 (53.4%) did not mention any potential biases; and 27(23.3%) did not mention limitations of the analyses (Table 2). There was a significant difference on specifying how persons without documented vaccination were handled between main authors from LMICs (66.7%) compared to those from high-income countries (93.8%).

To understand any trends of immunization secondary analyses, we categorized the articles by publication year (before or after 2012, the year GVAP was endorsed by the World Health Assembly). The percentage of secondary analyses mentioning potential bias and limitations increased from 23.1 to 53.3% and 50.0 to 84.4%, respectively. Other characteristic of immunization secondary analyses did not change significantly before and after 2012.

Figure 3 shows the distribution of immunization outcomes used of the secondary analyses included. Among all studies, 55 (41.7%) focused on specific vaccines, with measles-containing vaccines (MCV) (32.7%), diphtheria, tetanus and pertussis containing vaccines (DTP) (21.2%), and polio vaccines (20.2%) being the most frequent. 53 (40.2%) focused on complete or full immunization, mostly defined as having received one dose of BCG and three doses of DTP and one MCV dose (38 of the 53, 71.7%).

Figure 4 shows the distribution of affiliation countries, the top three countries contributing most of the published studies, with a first or corresponding author from that country, included the United States (US), India and the United Kingdom (UK), with 35, 15, and 12 publications, respectively. Compared with high-income countries, analyses whose main authors had a LMIC affiliation were more likely be for a single country (P < 0.001), to have a longer time lag between survey and publication > 5 years (P < 0.001), focus more on complete immunization (P = 0.003), and were less likely to include information on how persons without documented evidence of vaccination were handled (P < 0.001). No significant differences were found for other characteristic like publication year, survey type, type of analysis, mention of potential biases and the other proxies for quality (Table 3).

In our literature review, we analysed the immunization secondary analyses using DHS or MICS data since 2006 in order to better understand to what extent the DHS and MICS immunization data has been utilized beyond the coverage indicators included in the standard DHS and MICS reports. We also explored what vaccination issues were most looked at and whether some metrics related to “quality” were being included in these articles. We identified 116 published immunization secondary analyses using data from DHS and MICS, with over 3 out of 4 being published after 2012. The endorsement of GVAP by the World Health Assembly in 2012 may have been factor for the increase in secondary immunization analyses from multipurpose household surveys.

Interestingly, many more secondary analyses have used DHS data, as opposed to MICS, suggesting that the longer history of DHS database standardization and sharing may be a factor, but other factors may exist. Given that now more MICS surveys are being conducted, including sub-national surveys in many countries, it is possible that this difference will disappear going forward.

Most of the secondary immunization analyses focused on complete or full immunization, or in coverage with specific vaccines, and explored factors associated with these outcomes. Ideally, to make results more useful to immunization managers, these analyses need to be timely and be available to them. Even though DHS and MICS are only conducted in LMICs, most analyses are led by organization outside these countries; the underrepresentation of health researchers from LMICs in the scientific literature has been recently highlighted [135] and the PREPSS initiative has been born to support researchers from LMICs publish in the peer-reviewed international journals [136]. The median time delay between survey and secondary analysis publication was over 5 years. Though we did not explore the delay between survey implementation and report and dataset release by DHS and MICS, this is likely an important factor explaining this lag. Nevertheless, the time delay was worse when an organization in a LMICs contributed to the secondary analysis. Time delays and foreign authors likely reduce the usefulness of the results for immunization programmes and this should make us reflect and strategize on how to facilitate the conduction of secondary analyses immediately after DHS and MICS databases are released, and with the engagement of the country where the survey took place.

Other concerning findings are the fact that many articles did not provide information to assess the limitations of the analyses, for example how persons without documented evidence of vaccination were handled (the validity of recall has been increasingly questioned [137‐139]); whether the analysis was adequately weighted; and potential bias. At least, some of these elements seem to be increasingly reported. Finally, only a small proportion of analyses focused on more immunization-specific performance analyses such as timeliness, coverage trends and missed opportunities for vaccination (MOVs), which, in general, cannot be obtained from routine immunization information systems. To support timely, quality and in-country survey immunization analyses, WHO has released a suite of programs, along with detailed documentation, to calculate vaccination coverage survey indicators that go beyond only coverage, in a documented, standardized and replicable manner; these programs are known as the Vaccination Coverage Quality Indicators (VCQI) [137, 140]. Also, several capacity-building activities related to survey analysis are ongoin g[141].

Factors related to diverse immunization outcomes and immunization inequalities were commonly assessed. Inequities are a top priority for the global immunization community and in the human health development agenda, as highlighted by the Immunization Equity Reference Group (ERG) [142], and surveys can help diagnose these issues [143].

Studies exploring inequalities looked into socioeconomic factors and analyzed individual factors such as parental education, religion, urban/rural residence, etc. Overall, the most common factors associated to lower immunization coverage include place of residence, socioeconomic status, maternal education and birth order, but other factors that have been found to affect immunization uptake, such distance to immunization centres, quality of immunization services, behaviour and attitude of health personnel cannot be assessed from DHS and MICS, as they do not collect those variables. Now, research about interventions that will actually decrease inequities and identify and reduce immunization gaps between the privileged and vulnerable populations are still lackin g[144]. Regarding timeliness of vaccination, studies considered a dose timely if the child had received it within a certain timeframe of the recommended age, but definitions varied between studies; this issue has also been noted by other author s[145].

An important limitation in household surveys like DHS and MICS is the unknown magnitude of information bias. DHS and MICS first determine immunization history by documented vaccination, usually by transcribing information from the home-based record (HBR) or vaccination card; occasionally they seek vaccination information in health facilities. If documented evidence of vaccination is missing, caregiver’s recall is used. Thus, the percentage of immunization records seen and how people without vaccination documentation are handled is important to reflect about the accuracy of the results [138, 139, 146]. In our review, though almost three quarters of the analyses (74.1%) included recall information of caregiver, 14.6% did not mention how the results from recall were handled and less than a third (31.0%) reported the specific percentage of documented vaccination seen, even as this percentage is available in all DHS and MICS reports. Thinking about other limitations of the surveys and the analyses, there is also room for improvement. Only half of analyses were explicit about the potential biases of the survey and close to 40% of the studies failed to indicate whether the analysis was weighted (as it should be to generalize the results, given the complex sampling design used by DHS and MICS). Not all is bleak; a higher percentage of articles specify bias and limitations after 2012, compared to the earlier period.

Our systematic review has several limitations. We included only DHS and MICS surveys given their standard methods and more availability of their datasets for public use. Thus, nutrition and other multipurpose surveys that also include immunization were not included. Datasets from EPI surveys are rarely made publicly available, limiting the possibility of conducting secondary analyses. Given the limited time to conduct this study, the initial article screening and data extraction were performed only by the first author (YH) with support from the senior one (MCD-H), which may have resulted in excluding relevant articles, and also non-English papers may have been missed. We did not search the EMBASE database because of inaccessibility at WHO, but we searched all 15 related available databases in WHO, especially including databases (LILACS, Popline, African Journals online, IMSEAR, WPRIM) from developing regions of Latin America, Africa, South-East Asia Region, and Western Pacific Region, which benefit the review by increasing the chance of including LMIC publications. We also contacted experts from WHO, Gavi and UNICEF for them to supplement any article we might have missed. Thus, we believe that the thoroughness of the search and review would have minimized this possibility and the main conclusions of this review stand. Further analyses of the types of journals where articles are published was not done. Also, we did not explore whether the metrics analysed have changed following the release of the revised WHO Vaccination Coverage Cluster Survey Manual in 2015 and the suite of supporting materials to conduct analyses beyond coverage thereafter. Finally, maybe our main limitation is that we did not search the grey literature [147], perhaps analyses done as thesis for university students or country immunization staff, that can potentially be more useful to immunization managers than those reported in the peer-reviewed literature.

We identified over a hundred publications of secondary immunization analyses and characterised them. Most secondary analyses used DHS, rather than MICS, data and more such analyses have been done in recent years. The fact that most authors of these analyses are affiliated with institutions in high-income countries (as opposed to LMICs where these surveys are conducted), the delays between survey implementation and the publication of the analyses, in addition to some incompleteness in reporting some proxies related to the quality of the analyses illustrates that more needs to be done to build capacity to do, and publish, these analyses in LMICs and to make them useful to inform immunization programs. It is now time to better understand country awareness about the conduction of immunization secondary analyses and their use, to promote taking further advantage of the immunization component of DHS and MICS.