Risk factors affecting COVID-19 vaccine effectiveness identified from 290 cross-country observational studies until February 2022: a meta-analysis and meta-regression

A computerized search of the relevant literature was undertaken using the MEDLINE and Excerpta Medica dataBASE (EMBASE) databases and MedRxiv, a free online archive of complete preprints. Eligible studies, with no language restriction, had to be published between 1 December 2020 and 28 February 2022. Moreover, weekly searches of MEDLINE via PubMed were conducted over the same period of time and a recursive search of references in published reviews or meta-analyses was done manually to identify additional studies beyond the computerized search.

Relevant publications were selected using “immunization,” “COVID-19,” and “effectiveness” as keywords and their synonyms. The full search formulas are shown in Additional file 2: Table S4.

Eligible studies met the following inclusion criteria: (1) observational studies with a control group represented by unvaccinated participants, (2) exposure defined by type-specific or commercial vaccines including the number of doses, (3) outcome assessed by disease of any symptomatology and severity, and (4) stated VE including the confidence interval (CI). The primary outcome of our meta-analysis was to identify the risk factors of VE at least 7 days after full immunization (i.e., usually after two vaccine doses as defined by the manufacturer). The secondary outcomes of partial vaccination (with one vaccine dose) or booster immunization (after three doses) were supportive to the primary outcome. The first year of the worldwide vaccination campaign was dominated by four FDA- or EMA-approved vaccines, i.e., two mRNA vaccines: BNT162b2 (Comirnaty; Pfizer-BioNTech) and mRNA-1274 (Spikevax; Moderna) and two adenoviral vector (AdV) vaccines: ChAdOx1 (Vaxzevria, AstraZeneca) and Ad26.COV2.S (Jcovden, Janssen). Therefore, only studies reporting homologous immunization with these vaccines or with these vaccine types were eligible for our quantitative synthesis.

Extracted data included characteristics of the study, participants, interventions, comparison, and outcome. Studies scoring ≥7 stars using the Newcastle–Ottawa Quality Assessment Scale (NOS) were defined as low risk of bias (RoB) (Additional file 3) [25‐29]. To assess the quality of evidence resulting from this quantitative synthesis, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) [30] criteria were considered: number of VE records, study limitations, heterogeneity, and indirectness based on the difference between vaccinated and unvaccinated participants, imprecision, and publication bias of studies.

The meta-analysis was conducted to obtain pooled VE data stratified according to risk factors. These risk factor categories included sex; age-stratified sex; age stratified as <18 years, 18–65 years, and >65 years; patients with specified comorbidities (i.e., type 2 diabetes mellitus, hypertension, obesity, cardiovascular, renal, respiratory, neurological, and liver diseases, cancer, immunosuppression or transplantation, etc.); and the specific population of healthcare workers (HCWs) or high-risk groups (HRGs) including residents of long-term care or residential homes. The key predictors were the post-vaccination interval and the SARS-CoV-2 variant. The very often missing post-vaccination interval was compensated for by the month of study termination stratified into 2-month intervals from January 2021 to February 2022. Variant-dependent VE records were assessed as untyped, with the α, β, and γ variants merged into one group and the δ or ο variants assessed separately.

Given the high heterogeneity of extracted VE records resulting from an inconsistency index >50%, quantitative synthesis was conducted using the random-effects model (DerSimonian–Laird method; D–L). The fixed-effect model (inverse variance method, I-V) was used to reveal a potential publication bias. The primary analysis was performed as a meta-regression where the dependence of log-transformed effect sizes derived from VE as 1–VE/100 on the categorical independent variables representing the risk factors was investigated. To obtain adjusted VE, the most robust reference subgroup of each factor was selected. Additional analyses were performed to assess (1) the effect of small studies determined using a meta-regression model with Egger’s test [31], (2) the summary effect of asymmetry with the identification of any unpublished or unfound studies estimated by the trim-and-fill method [28], and (3) the prediction interval estimated from the standard errors of studies heterogeneity and pooled VE [28]. Statistical analyses were performed using STATA version 17.0 (StataCorp. 2021. Stata Statistical Software. College Station, TX, USA) with two main modules of “meta summarize” and “meta regress” at a significance level of α = 0.05 with a two-tailed 95% CI.

The level of protection against any COVID-19 within 14 months was documented by the pooled 81% VE (95% CI, 81–82%) for mRNA vaccines and 64% VE (95% CI, 63–65%) for AdV vaccines. Both pooled VE values against severe COVID-19 rose to 92% (95% CI, 91–92%) and 85% (95% CI, 84–86%) after mRNA and AdV vaccination, respectively.

However, adjusted VE against any COVID-19 depended on study termination regardless of vaccine type. The early 96% VE (95% CI, 95–96%) observed after mRNA vaccination in January–February of 2021 decreased significantly to 67% (95% CI, 65–69%) by November–December of 2021 (Fig. 2). The waning protection against severe COVID-19 was also demonstrated by the significantly decreasing effectiveness specific only for mRNA vaccines, i.e., from 98% (95% CI, 97–98%) to 86% (95% CI, 82–88%) a year later (Fig. 3). A similar decline occurred after AdV vaccination, i.e., the 86% VE (95% CI, 83–89%) in March–April of 2021 dropped to 67% (95% CI, 62–71%) in November–December of 2021. The waning protection against severe COVID-19 was also demonstrated by the significantly decreasing effectiveness specific only for mRNA vaccines, i.e., from 98% (95% CI, 97–98%) to 86% (95% CI, 82–88%) a year later. However, this decline was slower compared with that for any COVID-19.

Protection against any COVID-19 was not negatively affected by the pre-Omicron variants as demonstrated by the non-inferior VE value associated with specific variants versus variant-untyped VE.

Nevertheless, the Omicron variant significantly decreased the VE against any COVID-19 to 40% (95% CI, 33–47%) and that against severe COVID-19 to 87% (95% CI, 82–90%) after mRNA vaccination. A decrease in protection against any COVID-19 associated with Omicron, i.e., 28% VE (95% CI, 10–43%), was also apparent in participants immunized with AdV vaccines.

Other risk factors decreasing protection against any COVID-19 included age >65 years, comorbidities, and living in long-term care or residential homes. It was just these specific populations, which exhibited decreases by 14–17% and 5–16% in VE against any and severe COVID-19, respectively, independently of vaccine type. Higher levels of effectiveness irrespective of disease severity were observed in participants vaccinated with either the mRNA-1273 or ChAdOx1 vaccine versus those vaccinated with the same type of vaccine, i.e., BNT162b2 or Ad26.COV2.S. Full immunization with mRNA vaccines conferred better protection against disease than against infection whereas protection against infection and disease was just the opposite after AdV vaccination. The VE against death was not inferior to that against hospitalization regardless of vaccine type.

The overall pooled VE values of booster immunization with mRNA vaccines were 83% (95% CI, 80–85%) against any COVID-19 and 91% (95% CI, 90–93%) against severe COVID-19. The level of protection depended on the month of study termination as demonstrated by the significant decline in January or February of 2022 for any or severe COVID-19 (Fig. 4).

The booster VE against any COVID-19 caused by the Omicron variant decreased to 59% (95% CI, 48–67%) while remaining unchanged against severe COVID-19.

An increased risk of any COVID-19 was found in patients with a comorbidity or those living in long-term care or residential homes showing lower protection than the general population after booster vaccination.

Higher levels of protection were achieved against disease compared with infection and death versus hospitalization. Although no different adjusted VE against any COVID-19 was found between both commercial mRNA vaccines, a 2% higher level of protection against severe COVID-19 was observed with the BNT162b2 versus the mRNA-1273 vaccine.

The effectiveness of partial vaccination was influenced by risk factors identical to those of full immunization (Additional file 6: Fig. S1; Additional file 7: Fig. S2). Incomplete immunization with any mRNA vaccine provided pooled VE values of 58% (95 CI% 55–60%) and 73% (95% CI, 71–75%) against any and severe COVID-19, respectively, which were significantly decreased in the last two 2-month intervals of 2021 regardless of COVID-19 severity. The pooled VE values against any and severe COVID-19 were 47% (95% CI, 45–49%) and 70% (95% CI, 68–73%), respectively, after one-dose immunization with the ChAdOx1 vaccine. Despite the waning levels of protection against any COVID-19 observed between May and December of 2021, these levels against severe disease did not change significantly within 12 months.

Protection by partial immunization with an mRNA or AdV vaccine either against any or severe COVID-19 was significantly decreased by the Omicron variant and in specific risk populations. Among the mRNA vaccines, better protection was shown with one mRNA-1273 dose regardless of COVID-19 severity.

A sufficient number of VE records was obtained for the predefined predictors except for sex and age <18 years. Over 70% of records assessing partial or full vaccination against COVID-19 of any symptomatology came from studies at low RoB; hence, no serious limitation was anticipated. Conversely, the quality of evidence was lower for outcomes of severe COVID-19 and booster immunization since the analyses were more often performed using records from studies with an unclear limitation, i.e., 46–69% of records came from low RoB studies. The random-effect model had to be applied because of the high heterogeneity of studies.

Partial and full immunization regardless of vaccine type and disease severity were investigated using ≥80% records of studies with no differences among participants; therefore, no serious indirectness of evidence was assumed. The indirectness of evidence assessing booster immunization was unclear since 54–64% records achieved acceptable comparability of vaccinated and unvaccinated participants. No serious imprecision of outcomes was found as demonstrated by a pooled standard error <2%.

Publication bias established by the difference of outcomes of both random-effect and fixed-effect models was confirmed regardless of the number of doses, COVID-19 severity, or vaccine type. However, as the difference between results of both models was <0.1, publication bias was considered acceptable. Quantitative analyses were burdened by the effect of small studies increasing pooled VE with missing studies decreasing pooled VE. Moreover, the prediction intervals of full vaccination confirmed that the VE against any COVID-19 ranged between 44 and 94% for mRNA and between 24 and 83% for AdV vaccines and against severe COVID-19 between 72 and 97% for mRNA and between 50 and 96% for AdV vaccines. The same intervals for the booster dose of an mRNA vaccine ranged between 10 and 97% for any COVID-19 and between 43 and 99% for severe COVID-19. A summary of acceptable/no serious and unclear strengths of evidence for studies grouped according to investigated predictors is available in Additional file 8: Table S7.