Background
Vaccination is one of the most effective interventions against infectious diseases and is estimated to prevent 2-3 million deaths annually, with an additional 1.5 million deaths that could be averted with improvements in global vaccination coverage [
1]. Vaccines can provide cost-effective, long-term protection and have resulted in the eradication of two major pathogens, rinderpest and smallpox, as well as the local elimination of others, such as measles [
2,
3]. Most vaccines have been shown to be safer than therapeutic medicines and are deemed second only to safe drinking water in terms of public health benefit [
4‐
6].
Of the burden that does occur due to vaccine preventable diseases, the majority is in low- and middle- income countries (LMICs) [
7]. In many countries, the lack of reliable surveillance data and the inability to observe the burden that would have occurred without vaccination makes it difficult to directly calculate the mortality and morbidity prevented by vaccination. Hence, mathematical models are crucial for providing estimates of the burden averted by immunisation as they can project a no-vaccination (counterfactual) scenario and scenarios with vaccination [
8]. The Vaccine Impact Modelling Consortium (VIMC) [
9] was established in 2016, formally bringing together several modelling groups and a secretariat with a history of working together to estimate the impact of vaccines against ten pathogens, namely, Haemophilus influenzae type b (Hib), hepatitis B (HepB), human papillomavirus (HPV), Japanese encephalitis, measles, Neisseria meningitidis serogroup A (Meningitis A), rotavirus, rubella, Streptococcus pneumoniae and yellow fever (YF). Estimating the impact of vaccination is important as this reveals the effectiveness of current global vaccination strategies and whether any changes are needed.
Calculating vaccine impact is complex as for some pathogens, such as HepB and HPV, the consequences due to disease are not seen until long after infection. Thus, any vaccine impact is not immediately evident, but is still substantial leading to questions around how to appropriately attribute the impact of vaccination activities in order to inform the public and policy makers of their value. Vaccination activities also face a paradox as “vaccines are victims of their own success” [
10]. For example, when the transmission of a fatal disease from an outbreak is prevented by vaccination, political and public support is assured. However, when vaccination is successful and the formerly-feared disease starts disappearing, the benefits of vaccination become less clear whilst the costs remain visible. As a consequence, public and political support may begin to decline. The benefits of the success of vaccination and control of diseases become less obvious with time, no longer being viewed with the same sense of urgency.
Through mathematical modelling of vaccination scenarios, we can explore the impact of vaccination activities stratified by characteristics such as birth cohort. Mathematical modelling, such as that carried out by the VIMC, is able to provide estimates of disease burden in terms of the number of cases, deaths and/or disability adjusted life years (DALYs). DALYs measure the years of healthy life lost due to disability from the disease and premature death. Comparing burden estimates in scenarios with and without vaccination can then quantify the burden averted by vaccination. However, impact may be attributed in different ways, some of which may be more appropriate in some settings than others. For example, one may be interested in the effect of vaccination on a particular birth cohort, so the impact of vaccination activities over the lifespan of that cohort needs to be aggregated. In contrast, for advocacy, planning and financing, it is valuable to ascertain the impact attributable to a particular year’s vaccination activities.
As the burden in each vaccination scenario is calculated using mathematical models of pathogen transmission dynamics and implementation of vaccination activities, the impact of vaccination will vary depending on the underlying model being used. The models within the VIMC vary from being static, whereby the direct effects of vaccination on FVPs are modelled assuming that pathogen transmission intensity is not modified by vaccination coverage, to dynamic, whereby infectious disease transmission dynamics, direct effects of vaccination on FVPs and herd effects are modelled. Herd effects account for any indirect effects of immunity (both natural and/or from vaccination activities) due to a reduction in transmission. For example, an increase in FVPs offers indirect protection as it reduces the risk of a non-vaccinated individual coming into contact with the disease. Herd effects will not arise for all vaccine preventable diseases, for example, Japanese encephalitis is vector-borne (i.e. not transmitted person-to-person) and the majority of cases are due to spillover events [
11]. Hence in this case, FVPs will not provide protection to others.
The VIMC uses vaccine coverage estimates from the World Health Organization (WHO), United Nations Children’s Fund (UNICEF) and Gavi, the Vaccine Alliance [
12‐
14]. Due to changes and uncertainties in data on vaccination coverage, estimates of coverage are constantly refined and updated [
13], requiring estimates of vaccine impact to be recalculated accordingly. Whilst mathematical modelling of vaccine impact is vital to the process of the VIMC and ascertaining the effect of vaccination, it can be incredibly computationally expensive and time-consuming. This limitation sometimes precludes it from being used for quick updates as new coverage becomes available; an event which can happen multiple times a year. As such, the VIMC secretariat has developed an impact extrapolation (IE) method whereby the impact ratio (impact per fully vaccinated person) from one modelling exercise can be applied to a new coverage scenario in order to extrapolate the impact calculation to the latest coverage estimates.
In this paper, we describe the methods that have been implemented by the VIMC to calculate the vaccination impact by calendar year, birth year and year of vaccination (YoV). To show how these methods perform, we use examples from HepB, measles and YF. Further, we describe an IE method which facilitates the estimation of impact from new vaccination coverage scenarios without full model runs. This is an approximation of the model projections which, whilst informative, is not a replacement for full model estimation. The IE does, however, allow for new approximate estimates to be produced very quickly and we detail a number of approaches each with their strengths and weaknesses.
Methods
We detail a range of methods for calculating the impact of vaccination (
D) defined as the burden averted for a given disease, country, year and, in some cases, birth cohort. Each method is used to address a specific point of interest, such as the number of lives saved by vaccination in a particular year (calendar year method), the number of individuals born in a particular year that will be saved due to vaccination (birth year method), and the number of individuals that will be saved due to a particular year’s vaccination activities (YoV methods). These usually involve the comparison of a focal scenario, the vaccination scenario we wish to assess, to a counterfactual or baseline scenario, which is the vaccination scenario we wish to compare against. We also detail the IE process and the scenarios carried out. All parameters and variables are defined in Table
1. Impact ratios are shown in Table
2 and a summary of the impact methods is shown in Table
3.
Table 1
Parameter and variable definitions
y | Year | Bb | Burden in baseline/counterfactual scenario |
Yv | Years of vaccination | Bf | Burden in focal scenario |
Ym | Years evaluated | Db−f | Difference or impact between baseline and focal burden |
a | Age | | baseline and focal scenarios (Bb−Bf) |
Av | Age at routine vaccination | ρ | Impact ratio (impact per FVP) |
Am | Ages modelled | D | Vaccine impact |
c | Country | FVP | The number of fully vaccinated persons |
v | Vaccine | | |
k | Birth cohort | | |
Table 2
Stratifications of the impact ratios (ρ)
No stratification: Vaccine impact does not vary between vaccination activities or birth cohorts in a population. | Stratification by activity type: Vaccine impact varies between RV and CV but does not vary across birth cohorts. |
\(\rho = \frac {D_{R} + D_{C}}{\text {FVP}_{R} + \text {FVP}_{C}}\) | RV: \(\rho _{R} = \frac {D_{R}}{\text {FVP}_{R}}\) and CV: \(\rho _{C} = \frac {D_{C}}{\text {FVP}_{C}}\) |
Stratification by birth cohort: Vaccine impact varies across birth cohorts in a population but does not vary between vaccination activities. | Stratification by activity type and birth cohort: Vaccine impact varies between vaccination activities and birth cohorts. |
\(\rho (k) = \frac {D(k)}{\text {FVP}(k)}\) | RV: \(\rho _{R}(k) = \frac { D_{R}(k)}{\text {FVP}_{R}(k)}\) and CV: \(\rho _{C}(k) = \frac { D_{C}(k)}{\text {FVP}_{C}(k)}\) |
Table 3
Summary of methods used by the VIMC for calculating the impact of vaccination (refer to Tables
1–
2 for parameter values and for stratifications of the year of vaccination method, respectively)
Calendar year | Impact for a particular year | Intuitive. | Does not account for the long-term impact of vaccination on individual disease risk and cannot be linked to specific vaccination activities. |
Birth year | Impact for a particular birth cohort | Captures the long-term effects of vaccination in protecting those vaccinated. | Duration of modelling needs to be pathogen-appropriate, e.g. the effects of HepB require a longer modelling time frame. Does not account for inter-cohort effects (e.g. herd protection). |
Year of vaccination (YoV) | Impact attributed to year in which vaccination occurred | Possible to assess the effects of a specific year’s activities and thus provide a direct comparison of the number of doses provided and their effect. This is useful for planning and economic purposes. | An impact ratio is required which can be affected by the years/activities or birth cohorts included, see Table 2 for details. Does not account for inter-cohort effects (e.g. herd protection). |
Impact by calendar year (cross-sectional impact)
The calendar year method calculates the impact accrued over all ages for a specific year. It is generally the most intuitive and frequently used method to calculate the impact of vaccination. In this case, the vaccine impact, Db−f(a,c,y), for age a, country c in year y is defined as the difference in disease burden between baseline and focal scenarios for a given year, Db−f(a,c,y)=Bb(a,c,y)−Bf(a,c,y). Here, the baseline scenario can have no vaccination or different coverage to the focal scenario. Aggregating the impact over all ages modelled (Am) means the impact for a country c in year y is given by \(D(c,y) =\sum _{a \in A_{m}} D_{b-f}(a, c, y)\). This does not account for the future disease burden averted through current vaccine activities.
Impact by birth year (lifetime impact)
The birth year method accounts for the long-term impact accrued over the lifetime of a particular birth cohort k. For country c, this is given by \(D(c,k) =\sum _{y-a=k} D_{b-f}(a, c, y)\), where y∈Ym and a∈Am. The duration of modelling needs to be appropriate to the pathogen of interest as in some cases, such as HepB, disease occurs later in life. For example, if we model vaccination for birth cohorts born from 2000 to 2030 and model disease burden until 2100, we do not account for the vaccine impact for those born in 2030 once they are over 70 years old. The method also does not specifically account for the impact of vaccinating a cohort outside the cohort vaccinated (e.g. because of herd protection).
Impact by year of vaccination
The YoV methods are vital for determining the long-term impact of vaccination due to activities carried out in a particular year. We obtain the impact ratio,
ρ, as the impact attributable per fully vaccinated person (FVP) calculated as the coverage × cohort size. This ratio can be stratified by different characteristics, such as birth cohort in order to catch temporal changes in transmission or healthcare or by activity type to capture the differing effects of routine and campaign vaccination (RV and CV, respectively); see Table
2 for details. The impact ratio allows effects due to a particular year’s worth of vaccination to be attributed to that year. Additionally, the impact ratio can then be projected with the IE method by updating the impact with a new estimate of FVPs; see impact extrapolation section for further details.
Impact by year of vaccination: unstratified impact ratio
The simplest approach to calculate the impact by YoV is with an unstratified impact ratio which assumes that the effect of a vaccine is invariant over time and activity type. As such, the impact ratio,
ρ, for a country,
c, is the attributable vaccine impact divided by the number of FVPs:
$$ \begin{aligned} \rho(c) = \frac {\sum_{y-a \in Y_{v} - A_{v}} D_{R}(a,c,y)+ \sum_{y \in Y_{m}} \sum_{a \in A_{m}} D_{S}(a, c,y)} {\sum_{y \in Y_{v}}\text{FVP}_{R}(c,y) + \sum_{y \in Y_{v}}\text{FVP}_{S}(c,y)}, \end{aligned} $$
(1)
where
Yv−
Av refers to cohorts vaccinated in
Yv through RV. Thus, the impact attributed to a year of vaccination is given by:
$$ D(c,y) = \rho(c) \times \text{FVP}(c,y), $$
(2)
where FVP(c,y)=FVPR(c,y)+FVPS(c,y) and the impact ratio is the same across age groups. This method does not take into account any changes in treatment or transmission over time, nor the differing effects of vaccination through RV or CV. However, it is less complex to calculate and could be considered a weighted average of impact ratios for each vaccination activity.
Impact by year of vaccination: impact ratio stratified by activity type
To calculate the impact by YoV using impact ratios stratified by activity type, we assume that RV and CV, which target multiple age groups, have different effects; for example due to dosage clustering. Hence, this method produces multiple activity-specific impact ratios which can then be multiplied by the number of FVPs to calculate impact.
For RV, the impact ratio is defined as the impact for all cohorts who are vaccinated over time period
Yv per the additional FVPs between the baseline and focal scenarios. The impact for RV,
DR, is given by
\(D_{b_{R}-f_{R}}\), where
bR and
fR are the baseline and focal RI scenarios, respectively. Here, the impact ratio for RV is:
$$ \rho_{R}(c) = \frac{\sum_{y-a \in Y_{v}-A_{v}} D_{R}(a, c, y) }{\sum_{y \in Y_{v}} \text{FVP}_{R}(c, y) }, $$
(3)
where
Yv−
Av are cohorts receiving vaccinations in years
Yv. The impact by year of vaccination is then:
$$ D_{R}(c,y) = \rho_{R}(c) \times \text{FVP}_{R}(c, y), $$
(4)
where FVP R are FVPs vaccinated through RV and the impact ratio is the same across age groups. This does not account for the impact generated by indirect effects from and to cohorts not eligible for routine vaccination in Yv, nor does it account for interactions between RV and CV that may occur during or before years Y−v.
For CV, the impact ratio is averaged evenly over all ages across the entire time period (
Ym). This is because we do not attempt to predict future CV coverage after the final year of credible campaign schedules. Therefore, the only impact due to CV comes from CV years
Yv and all campaign impact for birth cohorts born after this period can be attributed back to these vaccination years. The impact of CV,
DC, is given by
\(D_{b_{C}-f_{C}}\), where
bC and
fC are the baseline and focal CV scenarios, respectively. The impact ratio is given by the following:
$$ \rho_{C}(c) = \frac{\sum_{y \in Y_{m}}\sum_{a \in A_{m}}D_{C}(a, c, y)}{\sum_{y \in Y_{v}}\text{FVP}_{C}(c, y)}. $$
(5)
The impact by year of vaccination is then given by the following:
$$ D_{C}(c, y) = \rho_{C}(c) \times \text{FVP}_{C}(c, y), $$
(6)
where FVP C are FVPs vaccinated through CV. Again, this does not account for interactions with the level of routine coverage across or before the same years.
The aggregated impact by YoV for both activities is the sum of the impact from RV and CV, i.e. sum of Eqs.
4 and
6.
Impact by year of vaccination: impact ratio stratified by birth cohort
In this method the impact ratio is invariant to vaccination activity type. However, vaccine effect is assumed to vary over time through birth cohorts. This means that rather than averaging the effect of vaccination over time, we account for the variation in transmission and health of the population. This influences how one year’s vaccination may work compared to another. For example, if therapeutic treatments for a disease improve over time, we may expect the impact of vaccination in 2050 to be less than that now as the population is generally healthier. In this case, the impact ratio is cohort specific, given by the following equations:
$$ \rho(c, k) = \frac{D_{b-f}(c, k)}{\text{FVP}(c, k)}, $$
(7)
where
k is the cohort defined by
k=
y−
a and
a∈
Am. In order to find the impact attributed to one year of vaccination, all cohort-specific impacts for those cohorts vaccinated in the year of interest must be aggregated. The impact by YoV is then given as follows:
$$ D(c, y) = \sum_{k} \rho(c,k) \times \text{FVP}(c,k,y), $$
(8)
Impact by year of vaccination: impact ratio stratified by both activity type and birth cohort
In the above methods, we illustrate the potential ways the impact ratio could change between birth cohorts or by vaccination activity. However, it is likely that in reality the impact varies with respect to both of these aspects. In the following method, we use the approach of the activity stratification over birth cohort to arrive at an impact ratio per birth cohort
and activity. The impact ratios by activity (R for RV; C for CV) are thus the following:
$$\begin{array}{*{20}l} \rho_{R}(c,k) &= \frac{D_{R}(c,k) }{\text{FVP}_{R}(c,k)}, \end{array} $$
(9)
$$\begin{array}{*{20}l} \rho_{C}(c,k) &= \frac{D_{C}(c,k) }{\text{FVP}_{C}(c,k)}. \end{array} $$
(10)
Therefore the impact by YoV is provided by the sum of the activity and cohort-specific components:
$$ \begin{aligned} D(c,y) &= D_{R}(c,y) + D_{C}(c,y) \\ &= \sum_{k}{ \rho_{R}(c,k) \times FVP_{R}(c,k,y)} + \sum_{k}{ \rho_{C}(c,k) \times FVP_{C}(c,k, y)}. \end{aligned} $$
(11)
Note that in the above method the numerator may also be affected by vaccination activities experienced by cohorts around them, so not captured in the denominator. Hence the impact ratio could potentially be infinite.
Summary of impact methods
We summarise the key features of each vaccine impact method in Table
3.
As coverage estimates are frequently updated within the WHO/UNICEF Estimates of National Immunization Coverage (WUENIC) and Gavi’s Operational Forecast (OP) [
12‐
14], one of the vital tasks of the VIMC is to provide updated impact estimates to inform future RV and CV. Translating these coverage estimates into tangible estimates of vaccine impact in terms of mortality and morbidity averted is crucial. However, given the regular updates of vaccination coverage and the effort and time required for full model updates across the VIMC, a simplified process, the IE, has been developed to update past rounds of impact estimates with new coverage data.
The IE is primarily a linear interpolation of current vaccine impact estimates with new coverage estimates assuming constant country- and delivery strategy-specific rates of mortality and morbidity averted per dose of vaccine. It can be applied to the YoV methods as these provide us with an impact ratio (
ρ in Table
2). Hence, when coverage is updated, the updated number of FVPs can be calculated by multiplying the new coverage with the target population. The updated FVPs (FVP
∗) are then multiplied by the previously calculated
ρ to calculate the updated impact (
D∗):
$$ D^{*}(c) = \rho(c) \times \text{FVP}^{*}(c). $$
(12)
Models and vaccination scenarios
In this analysis, we focus on calculating impact estimates for HepB (dynamic model with long-term outcomes and herd effects), measles (dynamic model with short-term outcomes and herd effects) and YF (static model without herd effects) [
15‐
18]. Model details are provided in the supplementary material. We use anonymised countries for each of the pathogens, denoted by Country A for HepB, Country B for measles and Country C for YF, with a focus on the years 2000–2017. The countries were chosen as they included examples of relevant vaccination activities over the time period. Standardised demographic data (live births per year, death rates) are based on the 2017 United Nations World Population Prospects (UNWPP) [
19]. Immunisation coverage from 1980 to 2016 corresponds to the WHO/UNICEF Estimates of National Immunization Coverage (WUENIC) as published in July 2017 [
13], and 2017–2030 coverage corresponds to Gavi’s October 2017 Operational Forecast (OP) [
14,
15]. Using each of the vaccine impact methods, we calculate the number of deaths averted due to vaccination over the years 2000–2017. Though we only show this for 2000–2017, we are taking into account the impact of vaccination activities from 2000 to 2030 as we follow individuals through their lives to capture later deaths averted.
We also investigate the performance of the IE under the YoV methods with no stratification, stratification by activity and cohort separately and in combination by applying it to HepB, measles and YF in Countries A, B and C with updated coverage data. More specifically, for each of these pathogens, we update RI coverage from 1980 to 2017 based on WUENIC published in July 2018 [
13], and 2018–2030 routine and campaign coverage is updated to Gavi’s October 2018 OP [
14]. Changing coverage of activities alters the number of FVPs (population demography and models remain the same). We then re-run the models with this updated coverage scenario and compare to the IE projections in order to determine the reliability of the IE.
Discussion
We have presented the main methods used by VIMC to estimate vaccine impact (summarised in Table
3). We also have shown how these methods perform when providing an IE. Each of the methods differ and the appropriate method will depend on the perspective of interest.
The vaccine impact methods primarily differ by how they attribute impact. The calendar year method attributes impact to the year examined irrespective of which age groups are targeted and when the vaccination occurred. In contrast, impact by birth year attributes impact to the birth cohorts of interest, irrespective of when the vaccination occurred over their lifespan. When only RV occurs in the first year of life, the impact by YoV and impact by birth year will be similar. However, when vaccination activities vary in who is targeted and when, the estimation of impact becomes complex.
The two impact by YoV methods differ not in
when they attribute impact but in the assumptions around how the effect of vaccination may change over time or by delivery method. The birth cohort stratification accounts for changes in transmission and healthcare over time, allowing the impact of vaccination to be adjusted given improvements in other interventions such as treatment or the effects of climate change on vector-borne diseases as seen for YF [
21,
22]. The issue with this method is that it may be sensitive to the introduction of RV or CV as these dictate the vaccination experience of cohorts to come, for example, a birth cohort born the year after a mass vaccination campaign should be well protected by the population vaccinated in that campaign. This characteristic is a strength in that it accounts for herd effects. However, the denominator for this method does not include vaccinations given to birth cohorts vaccinated outside the year of interest that might have an indirect (herd) effect on the birth cohort of interest. Furthermore, as the vaccination coverage, implementation and, arguably, demography are all subject to high degrees of uncertainty, this characteristic will also exacerbate any inaccuracies.
The most accurate way to capture herd effects would be to repeatedly run the underlying mathematical models incrementally, i.e. starting with the baseline and adding vaccination of a single birth cohort, year and activity type with each iteration. This has the advantage of correctly attributing the full indirect impact of each additional set of vaccinations to the whole population. The disadvantage of this method is that it is computationally expensive.
Overall, in our examples, we have shown that the IE works well and is an effective tool to update vaccine impact. Notably, the activity stratification is accurate for static models with dose dependency but may overestimate the impact of coverage improvements when dynamic herd protection is included. The birth cohort stratification may underestimate impact if coverage declines before improving in later years.
Caution needs to be taken when deciding the appropriateness of implementing an IE. In cases where coverage is reduced drastically or an activity is delayed, if the pathogen poses a risk of outbreaks and the model is dynamic, the IE may not be accurate as it may miss capturing the impact of any outbreaks that may occur during this period. With larger delays to activities, the risks of the IE not capturing such outbreaks would rise. Additionally, if a follow-up activity cannot reach the individuals missed, using the IE may not be accurate. In such cases, new model runs would be required to provide updated estimates of vaccine impact.
For computational and time efficiency, the IE is used by the VIMC to approximate the impact of vaccination given a new scenario around FVPs relative to the one modelled. This means that uncertainties in the modelling process are propagated into the IE. The update was developed in order to provide estimates on a short timescale given the latest available demographic and vaccination data. We show that impact by YoV methods perform well in many situations and given updates in coverage, each stratification would be appropriate for their related insights. However, there are limitations to the IE, particularly for diseases that pose a risk of outbreaks and that are modelled dynamically, if there are drastic changes to activities (e.g. coverage drops or delays to activities) as the IE may not account for rises in case numbers during such periods. A careful choice of the impact ratio informed by the natural history of the disease, individual vaccination strategy and modelling approach is needed to provide an effective extrapolation.
The impact we show is measured primarily through mortality; however, there are far reaching benefits of vaccination including reduction in morbidity as well is reducing pressure on health care services shown. The VIMC uses the methods shown here in a similar manner for cases and DALYs (disability adjust life-years); deaths are presented as there is a more widely accepted, and comparable, definition of this outcome. Whilst the methods of modelling and impact calculation provide an indication of the benefits of vaccination, they are limited to very specific research questions and outcomes. As such, they cannot replace wider surveys and alternative modelling efforts to examine the broader benefits of vaccination such as economic, equity and healthcare improvements.
The methods shown have their strengths but are also limited in certain ways, and as further research questions develop, new methods for calculating vaccine impact will need to be created. Though the YoV methods capture changes over time due to healthcare or transmission variation and the different effects of vaccination activity, they do not capture issues around vaccination and healthcare clustering which has been shown to be influential [
3,
23]. An extension would be to account for the potential of re-vaccinating the same individuals each time, a method that could highlight and identify groups of zero-dose children and health equity [
24]. Similarly, the current activity stratification allows us to further explore questions around vaccine delivery and health access. Finally, the results shown are presented nationally, whereas RV and CV may be delivered sub-nationally and the disease transmission and health access may be spatially heterogeneous especially for large countries. As such, the next steps will be to examine the sensitivity to these national assumptions and account for the spatial heterogeneity in vaccine impact.
Whilst vaccination is one of the most effective interventions against infectious diseases, the specific implementation of vaccination requires substantial planning, support and financing. Calculating the public health impact of vaccination and understanding the different methods for doing so is vital. We have presented multiple methods to express the complex effects of vaccination which will continue to be developed for ever increasing questions around optimal vaccine use.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.