Background
In an effort to decrease the rate of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections and cases of severe coronavirus disease 2019 (COVID-19), eight billion doses of COVID-19 vaccines have been deployed worldwide as of November 21st, 2021 [
1]. In the United States (US), SARS-CoV-2 vaccines include two messenger RNA vaccines (mRNA) BNT162b2 [
2] and mRNA-1273 [
3] and one adenovirus vector vaccine (Ad26.COV2.S [
4]). As of November 21st, 2021, 59% of the United States and 42% of the world’s population were fully vaccinated against SARS-CoV-2 [
1]. In randomized placebo-controlled Phase III trials, the BNT162b2, mRNA-1273, and Ad26.COV2.S vaccines showed 95%, 94%, and 67% efficacy against symptomatic disease due to SARS-CoV-2. However, the temporal evolution of vaccine protection against future SARS-CoV-2 infection, symptomatic, and severe COVID-19 remains poorly understood. As countries around the world face surges of COVID-19 cases, the question of waning immunity and its contribution to new outbreaks must be urgently addressed. In this meta-analysis, our objective was to evaluate the overall, age- and vaccine-specific efficacy/effectiveness (VE) of BNT162b2, mRNA-1273, and Ad26.COV2.S vaccines against SARS-CoV-2 infection, symptomatic, and severe COVID-19 disease over time.
Methods
Results were reported following Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) 2020 [
5]. This study was deemed exempt by the Penn State Institutional Review Board. The study protocol is provided in Additional file
1: Text S1.
Data sources and searches
MEDLINE, Scopus, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, the World Health Organization Global Literature on Coronavirus Disease, and CoronaCentral databases were searched from December 2019 to November 2021 for peer-reviewed studies reporting COVID-19 Vaccine effectiveness or efficacy without language restriction. Clinical trial registries through The World Health Organization (WHO) International Clinical Trials Registry Platform search portal (
https://trialsearch.who.int/) and conference proceedings were also searched. Reference mining of primary studies was conducted to identify additional literature. The following Medical Subject Headings and keyword search terms were used; “vaccine effectiveness” OR “vaccine efficacy” AND “SARS-CoV-2” OR “COVID-19” OR “severe acute respiratory syndrome coronavirus-2” OR “coronavirus disease 2019”. Full search terms are provided in Additional file
1: Table S1.
Study selection
Studies were selected according to Participant (P), Intervention (I), Comparator [C], Outcome (O) and Study type (S) [PICOS] criteria [
6]:
Participants Persons of all ages and sex included in studies that investigated COVID-19 vaccines efficacy or effectiveness.
Intervention COVID-19 vaccines (BNT162b2, mRNA-1273, Ad26.COV2.S). We did not include studies that evaluated booster doses because very few studies on efficacy of boosters were available at the time of our meta-analysis.
Comparison Unvaccinated cohorts.
Outcome of interest Vaccine effectiveness or efficacy (VE) calculated as 100 × (1 − IRR), where IRR (incidence rate ratio) is the ratio of the rate of COVID-19 in the vaccinated group to the corresponding rate in the unvaccinated group. A vaccine’s efficacy is a measure of how well vaccines work in clinical trials. In contrast, vaccine effectiveness is a measure of how well vaccines work in real-world settings outside of a clinical trial [
7]. Outcome measures and the definitions of infection and severity of COVID-19 are provided in Additional file
1: Text S2.
Study type Randomized clinical trials (RCT) for efficacy and observational studies for effectiveness. Pairs of independent investigators (PS and AES) screened the titles and abstracts of all citations. Studies included by either reviewer were retrieved and independently screened by two investigators (PS and AES).
Data extraction and quality assessment
A standardized data extraction form was developed and two investigators (PS and AES) worked independently to extract study details. The following information was extracted: year of study publication, country and time frame, type of vaccine; mRNA-1273 (Moderna); BNT162b2 (Pfizer-BioNTech); Ad26.COV2.S (Janssen), inferential statistical test estimates (vaccine effectiveness or efficacy and its 95% confidence intervals), follow-up time after full vaccination (2-doses for mRNA-127 and BNT162b2 and 1 dose for Ad26.COV2.S), study-level descriptive statistics (mean (SD)/ median (IQR) age in years, proportion (%) female, male and obese), follow-up time (days), and definitions of symptomatic, and severe COVID-19. Authors were contacted for missing or incomplete information. The risk of bias of the included RCTs was evaluated with the Cochrane Collaboration’s Risk of Bias 2 tool (Additional file
1: Table S3) [
8]. Methodological quality for nonrandomized observational studies was assessed with the Newcastle–Ottawa Scale (NOS) [
9]. Based on the NOS criteria, we assigned a maximum of 4 stars for selection, 2 stars for comparability, and 3 stars for exposure and outcome assessment. Studies with fewer than 5 stars were considered low quality; 5 to 7 stars, moderate quality; and more than 7 stars, high quality.
Grading the quality of evidence
We assessed the quality of evidence using the GRADE (Grading of Recommendations, Assessment, Development and Evaluations) framework using four levels of quality of evidence: very low, low, moderate and high [
10].
Data synthesis and analysis
Statistical analyses were performed with R software version 3.6.2 (R Project for Statistical Computing). R package
ggplot2 was used to display the scatter plots.
Meta and
Metafor R packages were used to conduct formal meta-analyses and create forest plots. Descriptive statistics were used to summarize study-level demographics. Meta-analyses were stratified by length of follow-up after full vaccination where continuous time was discretized in months. The DerSimonian-Laird random-effects model with Hartung- Knapp-Sidik-Jonkman variance correction was used to combine VE estimates if the number of studies included in the meta-analysis was greater than three [
11‐
13]. In a situation where VE was reported at more than one time point in a single month of study, a fixed-effects model was utilized to pool the estimate within the study before conducting the random-effect meta-analysis. Results from fixed-effects model were reported for subgroups of 3 studies or fewer.
Heterogeneity between studies was evaluated with the
\({I}^{2}\) indicator expressed as percent low (< 25%), moderate (50%), and high (≥ 75%) [
14]. Prespecified subgroup analyses were conducted according to age, vaccine type, WHO regions, and study design. Publication bias was quantitatively evaluated with Egger’s linear regression and Begg’s rank test [
15,
16] and qualitatively with funnel plots. Two-sided p < 0.05 was deemed statistically significant.
Discussion
In this systematic review and meta-analysis of 18 peer-reviewed studies, which included nearly 7 million individuals, we found evidence of waning immunity against SARS-CoV-2 infection from a high of 83% at one month to 22% at five months or longer after being fully vaccinated. Similar trends were observed for symptomatic COVID-19. VE against SARS-CoV-2 infection declined more rapidly in individuals ≥ age 65 years but was less than 50% in all age groups by month five. VE varied by vaccine type with highest pooled VE among mRNA-1273 recipients. Reassuringly, VE against severe disease remained robust; 90% at five months following vaccination, although protection was lower in older individuals (≥ 65 years) and in those who received Ad26.COV2.S.
The impact of temporal waning of vaccine effectiveness against SARS-CoV-2 infection raises concern that initially effective vaccination strategies will not be sufficient to mitigate the individual and population level effect of COVID-19 long-term. Historically, waning immunity for other infectious diseases has been addressed by administering subsequent doses of vaccine, e.g., booster doses [
34]. In line with other highly vaccinated countries, the US Food and Drug Administration (FDA) amended its Emergency Use Authorizations (EUA) for COVID-19 vaccines on November 19, 2021 to allow a single booster dose for all adults age ≥ 18 years and the Centers for Disease control and Prevention has endorsed booster doses for all adults. This approach in well-resourced countries has led to international debate about the implementation of booster vaccine rollout when much of the world’s population has yet to receive a single vaccine dose. Understanding the public health goal of booster vaccinations–prevention of infection versus prevention of symptomatic disease versus prevention of severe COVID-19 outcomes– is crucial to implementing a global strategy moving forward. At the population level, the majority of SARS-CoV-2 infections and arguably more importantly, severe cases, continue to be identified in the unvaccinated or in those unlikely to mount a robust vaccine response [
35]. The drivers of population level transmission in regions of variable vaccine uptake have yet to be determined.
Based on our data, VE against SARS-CoV-2 infection and symptomatic COVID-19 are clearly waning and have fallen below the WHO’s minimal criteria of 50% when considering the outcomes of infection and symptomatic disease [
7]. If impact on less severe COVID-19 is chosen as a global goal, then booster doses will certainly be needed to attempt and restore higher effectiveness targets. Early data from BNT162b2 vaccination campaigns in Israel and the United Kingdom suggest that a booster dose of COVID-19 vaccine will in the short-term increase vaccine effectiveness against confirmed infection and symptomatic disease [
36‐
38]. The longevity of this protection, beyond a few weeks after vaccination, has yet to be determined and the impact on transmission and utilization of healthcare resources remains unclear.
In terms of severe COVID-19, our data support robust protection that persists over time. A pooled VE of 90% at five months post-vaccination remains well above the WHO’s preferred estimate of 70% and minimal estimate of 50% when considering effectiveness against the outcome of severe disease [
7]. We did note an increased risk of severe disease in individuals ≥ 65 years of age and those initially vaccinated with Ad26.COV2.S that warrants further consideration. In this regard, many countries that have implemented booster policies have targeted older individuals first. The impact of this approach is largely unknown, although observational data from Israel suggests that a third dose of BNT162b2 decreased severe COVID-19 in individuals 40 years of age or greater in the short term [
39]. However, severe outcomes in immunocompetent individuals who have received BNT162b2 or mRNA-1273 continue to be rare [
33] and the side effect profile of a subsequent dose, particularly in fully vaccinated younger individuals at very low risk of COVID-19 complications, has not been fully delineated.
Individuals who received Ad26.COV2.S have lower VE against all outcomes, including pooled VE of 74% against severe COVID-19. Unpublished data presented to the Advisory Committee for Immunization Practices on October 21, 2021 suggested that a booster dose of Ad26.COV2.S given two months after the initial immunization increased VE in the short term to 100% against severe disease [
40].
Several important limitations of this meta-analysis should be considered when utilizing the data to influence public health policies. There is high between-study variation in the included estimates. Several factors may have contributed to this variation. First, study designs ranged from high-quality randomized controlled clinical trials to lesser quality cohort and test-negative case–control studies. When considering only the highest quality studies, VE was 83%, 93% and 91% for infection, symptomatic, and severe COVID-19, respectively. However, while the RCTs had higher estimates and controlled trials are generally considered to have a higher level of evidence, some of the RCTs included had short follow-up periods, very different participant groups (teenagers), and potential bias (baseline comparison groups not similar) as shown from Additional file
1: Table S2. Second, study populations varied geographically according to the WHO regions and included the Americas (United States, South America), Europe (United Kingdom, Israel), Africa (South Africa), and Eastern Mediterranean Region (Qatar). Random-effects models were adopted to control for the possible difference in effect estimates by region. However, potential differences in study demographics, rates of post-infection immunity, and political and social interventions exercised by these distinct geographic regions to control the pandemic could have introduced variations in VE. Third, we chose to focus our analysis on only mRNA-1273, BNT162b2, and Ad26.COV2.S. Therefore, our findings are not generalizable to other globally-available vaccines. Fourth, we could not conduct time-varying estimates stratified by the vaccine type due to small number of studies. Fifth, our analysis was temporally limited to 5–6 months after full vaccination as dictated by data availability. Impact of waning VE on all outcomes beyond the time frame of this study remains unknown. Lastly, our analysis was conducted before the emergence of the Omicron variant, which is associated with lower VE than the Delta variant [
41], suggesting potential variability in VE as future variants emerge.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit
http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (
http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.