Countries that introduce vaccines post-licensure should plan to continue surveillance of the targeted disease to allow evaluation of vaccine impact. Vaccine efficacy from pre-licensure studies is often erroneously used as a surrogate for predicting reductions in disease burden, but inaccurately predicts impact for the reasons previously discussed. Higher than expected post-licensure reductions will occur where a vaccine has a high indirect effect. An additional issue can arise when burden reduction estimates are based on a surrogate. For example, for pneumococcal conjugate vaccine (PCV), vaccine efficacy against non-bacteremic pneumonia was based on vaccine efficacy for invasive pneumococcal disease. If the former is substantially less than the latter, then disease burden estimates would have been underestimated. Since PCV has large indirect effects through reduction of carriage, vaccine serotype pneumococcal pneumonia would eventually be eliminated even with a lower vaccine efficacy.
Lower than expected post-licensure reductions in disease burden may indicate problems with vaccine delivery, low immunisation coverage (due to programmatic problems or vaccine hesitancy), cold chain limitations or reflect different vaccine schedules compared to those used in pre-licensure trials. In theory, lower than expected reductions may occur if lower efficacy or effectiveness is present in some epidemiological settings, such as the hypothesis that observed lower rotavirus vaccine efficacy in some settings occurred due to differences in gut flora. Practically, however, we are not aware of circumstances in which licensing trials using an iRCT design overestimated disease burden reductions. For the rotavirus vaccine trials, settings in which vaccine efficacy was lower had a higher vaccine-preventable disease incidence (VPDI) due to much larger background incidence rates. Regardless, documenting and publicising vaccine impact under real-life settings can provide important public and political support for routine immunisation programs. Post-licensure surveillance, including surveillance for adverse events following immunisations, is also important to identify rare adverse events or unexpected effects. Robust vaccine impact data can help to counter anti-vaccine messaging.
Methods
Post-licensure vaccine effectiveness trials are population-specific trials that focus on estimating the public health impact of the vaccine in a particular setting under real world conditions [
4]. Most post-licensure studies are observational given the ethical issues of withholding a licensed vaccine from a control group. Under some circumstances, however, randomised studies may be conducted. For example, if disease burden is unknown (and thus a vaccine would not be introduced), a vaccine can be used in a probe study approach [
8], as occurred during a
Haemophilus influenzae type b (Hib) vaccine trial in Indonesia [
9]. Vaccine introduction may also need to be implemented in stages due to programmatic, product or financial limitations that would allow for a stepped wedge design, although, for a variety of reasons, this design has almost never been used in a vaccine trial [
10,
11].
A last justification for a randomised trial using a cluster design in the post-licensure phase may be to demonstrate vaccine performance (including vaccine efficacy and impact) in a resource-poor setting where many health priorities compete for scarce health sector funding. In this case, cluster randomisation is particularly important since the goal is to provide as accurate an estimate as possible of vaccine-associated reductions in adverse health outcomes. One example of this was a phase IV double-blind, placebo-controlled trial of a two-dose regimen of bivalent killed whole-cell oral cholera vaccine over 5 years in a slum area of Kolkata, India [
12]. A second example is a phase IV trial for a single dose of the Vi polysaccharide typhoid vaccine in slum-dwelling residents of Kolkata, India [
13].
Mathematical models can be used to extrapolate from the results of clinical trials to estimate the impact of vaccination programmes. The models that are used for this purpose can be classified as dynamic or static. Dynamic transmission models (often shortened to ‘dynamic models’) are able to capture the direct and indirect vaccine effects by assuming that the probability of a susceptible individual becoming infected at any one point in time (the force of infection) is related to the number of infectious individuals in the population. If this changes (for instance, vaccination would be expected to reduce the number of infectious individuals), then the model recalculates the force of infection. Thus, the remaining susceptible people experience a reduced risk of infection through indirect protection [
14,
15]. These assumptions closely mirror the real-world epidemiology of most vaccine-preventable diseases. Static models, by contrast, do not recalculate the force of infection – it remains at a fixed level (usually the pre-vaccination level) – and therefore the remaining susceptible individuals in the modelled population do not experience any indirect protection as a result of the vaccination programme. By omitting indirect effects, static models underestimate immunisation programme impact. Despite this, static models remain widespread. Indeed, most economic analyses of vaccination programmes employ these methods [
16] and therefore underestimate the cost-effectiveness of vaccination programmes. There are other, potentially important, effects of vaccination programmes that are not captured by static models, and that can be predicted by appropriately parameterised dynamic models. These include increases in the average age at infection following infant immunisation (which may have important public health consequences if the risk of serious outcomes of infection increases or decreases with age as is the case with rubella and malaria, respectively) [
15], increasing gaps between epidemics, and replacement of vaccine-targeted serotypes with non-vaccine types (as has been demonstrated with PCVs that target a relatively limited repertoire of the more than 90 pneumococcal serotypes). These different effects, which are often vaccine-specific, require the development of specific dynamic models.
Measures
We propose that the measures of VPDI, the number needed to vaccinate (NNV) and total cases prevented should be used in a more systematic manner for all vaccines [
17,
18]. VPDI has several synonyms, including vaccine-attributable risk and vaccine-attributable rate reduction. It is the incidence of a given disease syndrome preventable by vaccine in a given context [
19], and is defined as “
outcome incidence in an unvaccinated population X vaccine efficacy” [
8], and thus incorporates both vaccine efficacy and the underlying burden of disease. This is mathematically equivalent to the incidence in the control group minus the incidence in the intervention group. VPDI derived from a clinical trial is reported as cases per 100,000 vaccinated persons per year for the duration of the trial. In principle, and as indicated above, VPDI is best calculated from cRCTs as this allows incorporation of the vaccine’s ability to prevent disease through both direct and indirect mechanisms and, in this case, it is the overall incidence reduction in the vaccinated population that is achievable with vaccine. By contrast, VPDI calculated from an iRCT gives only the reduction in incidence achievable from direct immunisation of individuals. Community randomised trials, however, are rare. For example, studies of PCV [
9] and rotavirus [
17,
18] vaccines used iRCT designs to assess VPDI against clinical outcomes; these studies likely underestimated VPDI since rotavirus vaccine and PCV provide indirect protection. Studies of dengue [
20] and the RTS,S malaria [
10] vaccines similarly used an iRCT design, but the consequence of this is unknown since vaccination against vector-borne diseases affecting the entire population may provide minimal transmission reductions when the vaccine target age range is highly constricted. An additional issue is that iRCTs and cRCTs both usually target a limited age range even if indirect benefits may accrue to other persons. For example, PCVs may provide most of their benefit via indirect protection of unvaccinated older persons.
The NNV is often used as a metric of the value of vaccination programmes, and can also be used for cost effectiveness studies. NNV is a measure to quantify the number of people that need to be vaccinated, or the number of vaccine doses that need to be used, to prevent one occurrence of a target health outcome [
21]. Unlike VPDI, NNV is not a rate but instead the overall number of cases prevented for a given number of persons vaccinated, and thus incorporates the length of the trial or, outside of a trial, the duration of immunity. Consequently, if VPDI is reported as cases per 100,000 vaccinated persons per year, NNV is calculated as 100,000 divided by VPDI divided by length of study/immune duration.
While VPDI and NNV can be calculated for etiologically confirmed outcomes, as public health measures, these metrics have more utility when calculated for clinical outcomes as this adjusts for the inevitable failure to confirm all prevented outcomes. Although less specific outcomes lead to lower vaccine efficacy, the baseline incidence for less specific outcomes is often much higher, leading to higher and more accurate VPDI estimates. For example, during PCV trials in The Gambia [
9] and South Africa [
22], the vaccine prevented approximately 4- to 5-fold more clinical pneumonia than vaccine serotype invasive pneumococcal disease. A trial of Hib conjugate vaccine in Indonesia used a community randomised design to assess VPDI for clinical pneumonia and suspected meningitis, and found a VPDI for all clinical meningitis 10-fold higher than that for confirmed Hib meningitis [
23]. Similar effects are seen in developed countries and emphasise the difficulty in accurately confirming etiology for all or even most cases in which an organism forms part of the causal chain. For example, in Finland, rotavirus VPDI was over twice as high for all-cause compared to confirmed rotavirus acute gastroenteritis [
24].
Another advantage of focusing VPDI and NNV on clinical outcomes is that clinical outcomes are usually of greater public health importance and allow for more accurate comparisons between vaccines. For example, public health officials have a greater interest in preventing hospitalisation for pneumonia than in preventing pneumonia through a serotype invasive pneumococcal disease vaccine. When valuing a new vaccine such as for RTS,S malaria or dengue, it may be more sensible, from a public health perspective, to compare VPDI or NNV against severe fever (or severe fever hospitalisations) rather than VPDI or NNV for PCV or Hib vaccine impact against severe pneumonia or rotavirus vaccine against severe gastroenteritis [
7].
Nevertheless, VPDI and NNV have some limitations. Because these metrics incorporate baseline disease incidence, they depend on the local epidemiological context and thus require an appreciation of local epidemiological nuances when extrapolating from one setting to another. However, this may also represent an advantage since it emphasises that decision-making around vaccines should reflect not just the degree to which the vaccine works against the target etiology but also how much disease can potentially be prevented.