Strengthening vaccination delivery system resilience in the context of protracted humanitarian crisis: a realist-informed systematic review

There is no broadly agreed definition of the term “protracted crisis” in the literature. We adopted the approach used in the Global Humanitarian Assistance Reports, including any country subject to at least five, consecutive years of United Nations (UN)-coordinated humanitarian action at any point between 2001 and 2021 [29]. A shortlist of eligible countries was generated using this approach to determine which contexts to include in searches (see Additional file 2: Appendix 2).

We gathered evidence on system strengthening interventions aimed at one or more of the World Health Organisation (WHO) health system building blocks at meso- or macro-level, with explicit or potential effects on either or both of the primary and secondary outcome domains set out below. For inclusion, [i] there needed to be sufficient programmatic detail in the article to form a clear view of intervention design and how it had been implemented, and [ii] interventions had to operate at meso- or macro-level. Meso-level interventions included supply-side measures such as area-based interventions (i.e. district, governorate or equivalent level and above), health sector interventions addressing system resilience directly, or alternatively specifying activities under one or more of the WHO’s six health system building blocks; or interventions targeting specific tranches of the vaccination delivery pathway e.g. cold-chain maintenance; or demand-side interventions focused on refugee or otherwise displaced populations in crises with demonstrated effects on population health outcomes (e.g. cash transfer programmes), with the overall objective of increasing population demand for, and uptake of, vaccination. Macro-level interventions, on the other hand, addressed system resilience at national level directly, or concurrently addressed a number of the WHO’s six health system building blocks at national level; or described interventions targeting specific tranches of the vaccination delivery pathway at national level. Micro-level interventions such as the use of tailored text-messaging or other forms of individualised outreach were excluded from the review. Shorter-term activities such as vaccination campaigns were included provided there was demonstrable evidence that these contributed in some way to longer-term resilience-promotion (e.g. through training of workforce cadres, leveraging of existing structures in new ways).

A full outline of inclusion and exclusion criteria is given in Table 1. We included studies published between 01/01/2001–09/11/2021, in Arabic, English and French. We focused on publications in these languages given the expertise mix within the team but also given the balance of countries identified as eligible for inclusion (Additional file 2: Appendix 2).

The primary outcomes of interest in this study were population level vaccination coverage for the vaccines outlined in Table 1. We also considered secondary outcomes in two categories: [i] vaccination delivery outcomes as defined by included studies but including rates and reported caseloads for VPDs; access to routine immunisation (RI – typically defined as the proportion of eligible children within a given time period in receipt of a particular antigen dose); drop-out rates (for multiple dose regimens); and [ii] system resilience indicators where these were given e.g. the presence of systems for protecting financing for vaccination, composite measures such as facility readiness metrics, available vaccine stock, workforce numbers and reserve and so on (with variations from study to study according to design and setting).

Keyword-structured searches were performed in MEDLINE, EMBASE and Global Health (all via Ovid), CINAHL (via EBSCOHost), the Cochrane Collaboration and WHOLIS. These were accompanied by targeted searches for grey literature through the United Nations Children’s Fund (UNICEF), GPEI and Technical Network for Strengthening Immunization Services (Technet – https://​www.​technet-21.​org/​en/​) websites, because of the role these organisations and initiatives have played in producing and collating evidence and technical guidance to support RI delivery including in humanitarian contexts. All literature searches were performed between 3rd and 9th September 2021. A sample search strategy is provided in Additional file 3: Appendix 3

Studies were independently screened for inclusion on title and abstract, and then full texts reviewed by two members of the research team working independently, using the criteria outlined in Table 1. The first stage of screening (title and abstract) for articles identified through established search engines was performed using Rayyan QCRI, a free web application to support the conduct of systematic reviews by researchers working remotely [30]. For results obtained from the UNICEF, GPEI and TechNet websites, initial screening was performed in MS Excel because it was not possible to download full reference details in an appropriate format for use in referencing software. Full text screening for all sources was conducted exclusively using MS Excel, and with reference to article PDFs. At each step, disagreements were resolved based on discussion between the two members of the screening pair.

Data were extracted in duplicate from each included study using a pre-developed extraction template in MS Excel. The extraction template was structured around the context-mechanism-outcome triumvirate emphasised in guidance on realist reviews [25, 31]. The template gathered basic study characteristics, general features of the intervention context (including the type of humanitarian crisis, and whether the intervention was geared towards prevention or outbreak response), the target disease; data on intervention structure using an approach informed by the WHO’s health system building blocks but including additional components such as service-user-focused aspects (demand management approaches such as community mobilisation and communications initiatives to bolster vaccination uptake); and finally measured outcomes. Importantly, the tool focused specifically on how inputs contributed to absorption, adaptation or even delivery system transformation (i.e. the intervention mechanism). Findings were synthesised using a thematic approach informed by the framework given in Fig. 1.

Study quality was assessed independently using the Mixed Methods Appraisal Tool (MMAT), a validated tool developed which has been developed to help facilitate appraisal of public health research studies with multiple study designs and interventions within a single framework [32]. Results from the duplicate extractions were inspected and areas of disagreement resolved by discussion between the two authors engaged in the extraction.

Although – as the summary table in Additional file 4: Appendix 4 makes clear – the vast majority of studies referenced community engagement as a trust-building measure to a greater or lesser degree, four studies explicitly focusing on this aspect met the inclusion criteria for the review, all of which concerned multi-dimensional community mobilisation activities [71‐73], and one of which combined community mobilisation activities with direct observation of polio vaccination by officials to ensure proper administration of vaccination [74]. All four were implemented in Nigeria, and most were concerned with improving polio vaccination uptake. Community mobilisation activities in Nigeria all took place in districts in the north and east of the country, against a backdrop of the Boko Haram insurgency, and all involved a strong element of cooperation with local community and religious leaders to build trust with affected communities. These complex interventions involved multiple outreach strategies including the use of roadshows, in-kind incentives (e.g. provision of soap or detergent, foodstuffs), health camps, and communication via multiple channels [72], agreement of mutually convenient vaccination point locations with community leaders [73] among other activities; a number incorporated recruitment and training of volunteer community mobilisers as part of the intervention [71]. These strategies were used to encourage service use through demand-generation activities, reducing access barriers by positioning access points in areas where service users were more likely to take them up, and by enhancing trust in service providers.

Study outcomes focused predominantly on measures of administrative coverage and missed opportunities for vaccination. Three of the studies relied on narrative assessments of impact without formal statistical testing [71‐73]; the final study presented a rudimentary time series analysis [74]. Two of the four studies found declines in the proportion of children who had not received any OPV doses during the study period [71, 74]; another showed concurrent, large increases in pentavalent vaccination coverage from an integrated intervention designed to improve uptake in hard-to-reach communities in Northern Nigeria [73], albeit with marked variations across geographies.

Eight studies addressed sustained, multi-dimensional interventions. These interventions also tended to act at multiple levels, spanning macro-level governance and coordination changes, through to health worker training, and community outreach activities. Three of these studies looked at integrated, strategic interventions in response to poliomyelitis outbreaks, all in conflict-affected settings [75‐77] and all featuring vaccination campaigns as part of the overall package. All emphasised the importance of strengthened coordination between key actors. A study from Somalia documented the introduction of a national polio control and coordination room to bring together key partners, improve communication and provide a framework for information/intelligence sharing; national coordination mechanisms were important in Ukraine and Syria [76, 77]; and regional mechanisms were used in Middle East and North African (MENA) countries for the 2017–18 polio outbreak response [78]. Multi-modal social mobilisation activities were also central to achievement of improved outcomes.

Of these three studies, two considered regional activities in response to polio outbreaks in conflict-affected countries in the MENA, both involving multi-phase response plans not just to interrupt initial transmission, but then to focus on high-risk areas for importation. These evaluation reports indicate that through intensive community engagement it is possible to raise vaccine coverage even in areas of profound insecurity and constant population movement, but also that focused surveillance is really important in allocating resources appropriately and that multiple data points were triangulated for this [77]. The first of these studies emphasised importance of targeting in identifying polio hotspots, and the contribution of electronic syndromic surveillance systems in picking up emergent VPD case clusters and improving situational awareness [77]. Results from the second study indicate that intermittent SIAs can be supportive of long-term system capacity by refreshing training and other essential functions and bolstering support for community outreach and surveillance initially built during an acute outbreak response, although these effects tail off over time [78]. This study noted that gradual degradation of systems since the first SIA in 2013–14 may have contributed to the risk of re-emergence of polio cases in Syria in 2017–18 [78].

A further four studies looked at integrated, preventive responses, three focused on SIAs [79‐81] and one on a district-level, NGO-led intervention to increase cholera vaccination uptake in urban slums in Haiti following the 2010 earthquake [82]. SIAs achieved their measured effects through a combination of community mobilisation activities, capacity building of local health staff, mobilisation of funds from multiple sources (including international donor support) and supply chain strengthening activities. In Somalia, improvements linked to the SIAs contributed to long-term cold chain strengthening for all RI delivery [79]. The district-level intervention in Haiti comprised multiple components including the introduction of new governance mechanisms (a coordinating committee), a communication plan, and re-allocation of the majority of NGO staff to supporting vaccination delivery for the duration of the intervention. This adaptive response to low coverage in their target communities benefited from an ability to surge staff from other areas – a disruptive approach for the NGO’s broader activities which is unlikely to have been sustainable beyond the term of the intervention [82].

This review identifies a series of interventions that may support vaccination system resilience capacities for improved routine childhood vaccination coverage (some of which also addressed catch-up vaccination delivery for older children) in settings of protracted humanitarian crisis. Most of these interventions reinforced adaptive resilience capacities but some (especially governance and workforce interventions) had transformative potential even if study results did not necessarily indicate radical change had been achieved within the term of the study. Although the diversity of settings described in this review precludes easy generalisation regarding important contextual factors, more successful interventions relied on leadership from domestic ministries of health, funding and – importantly – flexibility from agency and donor partners, and an ability to negotiate safe access for vaccinators and outreach workers. However, findings drew heavily on evidence from polio programming, and to a lesser extent cholera and measles vaccination delivery, potentially limiting generalisability to delivery of other antigens in these settings.

Considering adaptive capacity, vaccination campaigns were the most extensively evidenced interventions and while their effects may appear to be short-lived, many of the examples considered here both built on existing capacities and contributed to development of new ones in a range of areas, including through introduction of novel governance structures and workforce capacities developed during previous activities. The most successful campaigns were multi-dimensional interventions that incorporated a mix of service delivery approaches (fixed-site, mobile team and mass-vaccination sites), intensive community mobilisation efforts, health worker training, and supply chain strengthening work. They were also often multi-phased to help both break chains of transmission (where the primary strategic objective was outbreak control) and prevent future outbreaks. The long-term sustainability of capacity strengthening through campaigns necessarily depends, however, on mobilisation of funding and other resources beyond the intervention lifecycle – some of which can be addressed by sustaining intervention through SIAs. None of the studies exclusively focused on campaigns offered detail on sustainability planning post-intervention.

Other adaptively-focused interventions included service delivery changes such as service integration and the use of mobile health teams. All included studies on service integration demonstrated improvements in vaccination coverage and course completion rates albeit with varying effect sizes and in studies of generally low quality. Mobile health teams seemed to improve vaccination coverage by enabling outreach especially into poorer and more marginalised communities, by improving trust through regular interactions and the supporting activities community mobilisers drawn from the communities they served.

Some common mechanisms emerged across the higher-impact interventions identified in the review. For example, strengthening trust and increasing the range of access points were key themes especially among communities living in security-compromised zones, and many interventions directly addressed this through outreach models including the use of community volunteers, messaging through community (including religious) leaders, but also more controversially through cooperation with security personnel. Political and ethical challenges revolving around the deployment of security personnel in support of routine vaccination are considerable and this route is unlikely to be feasible or desirable in all settings. On the other hand, community volunteers were frequently identified as critical in tackling scepticism towards government and officials, but also contributed to bolstering situational awareness through improved case ascertainment for VPDs (as a linked, situational awareness-improving function).

Finally, a large proportion of the papers related directly or indirectly to lesson learning from polio eradication, and the resilience-building contributions GPEI infrastructure could make with respect to routine immunization more generally. Particular areas of learning included the use of multi-modal approaches to vaccination and surveillance in security-compromised areas, the value and use of existing outreach networks into communities, and messaging strategies. Objectives for polio control (specifically eradication) are very different to those for many other VPDs where primary aims are more likely to be interruption chains of transmission in the context of outbreaks or to keep the burden of mortality and morbidity low in the face of endemicity (for diseases such as measles for example). This should not, however, preclude the use of infrastructure developed for polio control to support wider RI objectives, especially given calls elsewhere to consider how GPEI activities can better support routine immunisation delivery overall and lessons learnt through these regarding effective community engagement strategies in security-compromised areas [83, 84].

Evidence in some areas was notable by its scarcity. For example, material on governance and financing was both limited and showed conflicting evidence on vaccination-related outcomes, although some promising interventions were identified in these domains, including the use of direct disbursement mechanisms for health worker payment, and a model of cross-border cooperation in the Horn of Africa to support preparedness and planning to reduce poliomyelitis risk. In addition, while some studies touched on leadership and oversight capacity development, we identified no studies that explicitly focused on leadership models contributing to resilience capacities for RI. Similarly, evidence on governance reforms such as decentralisation, which have been implemented in some eligible countries (e.g. Kenya and Tanzania) was not forthcoming, although one study briefly considered the importance of decentralised decision-making as a contextual contributor to the success of a nutrition-immunisation integration intervention [60]. Finally, although we did not specifically screen to include studies on cost and cost-effectiveness, data on costs associated with intervention implementation were conspicuously absent from included studies. This is a notable shortcoming given the fundamental importance of sustainable financing for health system resilience especially in crisis-affected settings.

A number of systematic reviews have been published in recent years addressing the effectiveness of interventions in humanitarian settings, some including data on vaccination delivery [17, 85], and there is now a large body of evidence considering the effectiveness of interventions to improve vaccination coverage from community level upwards in low- and middle-income settings more generally (e.g. [16]). This review is, however, distinctive in its macro- and meso-level interventional focus, in employing a realist perspective to understand how, where and why interventions may be effective, and its emphasis on gathering data from protracted crisis settings. Many existing studies and guidance documents focus on acute-phase responses without consideration to ways in which these may support, or undermine, long-term capacity within the system to respond to changing circumstances on the ground, or the mechanisms by which they may do so [20, 86, 87].

Nevertheless, many of the central messages from this review – including the value of recruiting local staff to improve trust and vaccination uptake, flexibility in service delivery modes (including the use of mobile services), and use of electronic systems to strengthen supply chain management and health intelligence – support findings from reviews and guidance elsewhere [85, 86, 88, 89], including recently published evidence on improving integration of refugees into routine immunisation planning and services in contexts of displacement [90]. Findings also suggest that SIAs, as integrative interventions, can have important and wide-ranging effects not just on vaccination uptake, but also on wider system capacity through e.g. workforce training, introduction of new governance and coordination mechanisms, and support to outreach activities. These effects should be balanced against potential undermining effects on RI through fixed centres that have been observed in some settings [91], but SIAs are likely to remain an important part of the adaptive armoury to expand coverage in humanitarian settings given their effectiveness at picking up children missed through routine delivery [92]. In addition, a number of promising practices are identified, including the use of flexible contracting for healthcare workers and novel financial disbursement mechanisms, for which evidence remains limited but which nevertheless could address delivery problems widely acknowledged in these settings.

Limitations to the findings reported here relate to both the nature of the underlying evidence base, and the way in which the review was conducted. As the quality appraisal results show (see Additional file 5: Appendix 5), most included studies were observational works with significant methodological limitations. This particularly affected descriptive, quantitative analyses included in the review (most of them program evaluations). Outcome reporting was in general poor: measures were incompletely described and frequently related weakly to study objectives or included no clear baseline data against which to measure effects. The quality and detail of intervention description was highly variable, making it difficult to tell which particular intervention components were driving reported results.

Although we were careful to use explicit definitions to guide the review, clarity in the wider literature on some of the key terms is lacking and may have contributed to relevant results being missed. For example, we used a response plan-driven definition of protracted crises that assumes that the introduction of a response plan (RP) coincided fairly precisely with the duration of the crises of interest. This is often not the case: there may be delays of up to several years before RPs are formulated and agreed – as in the case of the Syria Crisis response, for example. Secondly, an important goal of this review was to capture data from a wider range of sources than the peer-reviewed literature alone, to better capture emergent best practice. Structured searching of grey literature sources remains challenging and it is likely that some relevant material was missed.

Some of the criteria applied to searches in this study may also have excluded relevant material. The restriction to three focus languages (Arabic, English and French) may have preventing us from accessing material published in other languages although preliminary searches did not suggest sizeable, relevant literatures elsewhere. Secondly, the focus on material published since 2001 will have excluded earlier material on vaccination delivery in other settings in the 1980s and 1990s that may have informed our findings.

As with all systematic reviews, our findings also cannot account for unpublished or negative results. This may explain the lack of data relating to absorption as a resilience mechanism – that they were not reported simply because they involved services performing in much the same way but more intensively (e.g. through changes to service opening hours, workforce redeployment etc). Because of the diversity of intervention types, study designs and study contexts included, it was neither possible nor desirable to produce summary statistics of intervention effects beyond those reported in the results section.

Finally, we noted imbalances in the geographical representativeness of studies included. On one hand, the balance of included evidence was skewed strongly towards unstable, low income settings. This is unsurprising given that most countries in protracted crisis today fall into this category, but it reduces the potential transferability to middle income countries hosting large displaced populations today. Evidence from countries such as Jordan, Iran and Turkey, for example – all of which host large populations displaced by conflict – was notably absent in this review. On the other, a large proportion of included studies were conducted in Nigeria – reflecting historical challenges with vaccine delivery and particularly polio control in the north of the country.

Findings from this review suggest that no “silver bullet” solution exists to promoting resilience in vaccination delivery systems in protracted humanitarian crises, and adaptations are likely to be needed across a range of fronts to address significant access barriers (many of which stem directly from population displacement and chronic insecurity), low trust in service providers, and limitations to effective VPD prevention and control imposed by national borders. Periodic intensification of vaccination delivery via campaigns and SIAs is likely to be a mainstay of adaptive responses to crises whatever the context, to account for shortfalls in routine delivery. However, there is a strong steer from this work for recruitment of non-traditional workforce cadres from within affected communities – including community mobilisers – to help enhance uptake over the long-term, especially in security compromised areas where trust in government and agency representatives may be low.

Resilience in vaccination coverage is also likely to be enhanced through the concurrent use of multiple service channels to reach affected populations, including mobile health teams and integration with in-demand services such as nutritional support. However, there will inevitably be trade-offs in cost terms to expanded service availability in this way given resource constraints in many humanitarian settings, and the success of any of these interventions will ultimately depend on the willingness of domestic and international actors (including donors) to ensure stability in funding flows to crisis-affected countries.