Comparative effectiveness of BNT162b2 and ChAdOx1 nCoV-19 vaccines against COVID-19

We used data from the IQVIA Medical Research Database (IMRD) (previously called The Health Improvement Network (THIN)), an electronic health records database of general practitioner (GP) based healthcare covering 839 practices and 19 million individuals in the UK. The IMRD was established in 2003 as a collaboration between the company owning Vision (In Practice Systems) and the CSD Medical Research Group (now Quintiles IMS). The database contains computerised information on socio-demographics, anthropometric characteristics, lifestyle factors, and details from visits to GPs (i.e., prescriptions, diagnosis, diagnoses and interventions from specialist referrals, hospital admissions, and results of laboratory tests). The READ classification system is used to code specific diagnoses [21], whereas a dictionary based on the Multilex classification system is used to code drugs [22]. The validity of the IMRD for use in clinical and epidemiological research studies has been demonstrated in a previous study [23]. The scientific review committee for the IMRD database and the institutional review board at Xiangya Hospital approved this study, with a waiver of informed consent. This study followed the recommendations of the STROBE initiative for reporting observational studies in epidemiology [24].

We emulated a target trial to compare the risk of SARS-CoV-2 infection and its sequelae, i.e., hospitalisation and death in individuals who received the BNT162b2 vaccine with those who received the ChAdOx1 vaccine. We included individuals between 18 to 89 years of age, who received their first covid vaccination with either of these two vaccines from January 1, 2021, to August 31, 2021, and had at least two years of continuous enrolment with a general practice prior to entering the study. The details of vaccination records were based on the READ code in IMRD (Additional file 1: Table S1) [22]. The date of the first dose of either the BNT162b2 vaccine or the ChAdOx1 vaccine was assigned as the index date. Individuals were excluded if they had received a SARS-CoV-2 infection diagnosis or a different COVID-19 vaccine (i.e., mRNA-1273 [Moderna] and Ad26.CoV2.S [Janssen/Johnson &Johnson]) prior to the index date.

We divided the baseline study period into weekly time blocks. Eligible individuals were allocated into these blocks according to their index dates. In each weekly time block we calculated the propensity score (PS) and adopted an overlap weighting approach to balance baseline characteristics (Fig. 1) [25, 26]. Specifically, the PS for the BNT162b2 vaccine was calculated in each weekly time block using a logistic regression model that included potential confounders. Individuals receiving the BNT162b2 vaccine were weighted by the probability of not receiving the BNT162b2 vaccine, i.e., 1-PS, and individuals receiving the ChAdOx1 vaccine were weighted by the probability of receiving the BNT162b2 vaccine, i.e., PS. Overlap weights were bounded and smoothly reduced the influence of individuals at the tails of the PS distribution without making any exclusions.

The primary outcome was a confirmed diagnosis of SARS-CoV-2 infection, and the secondary outcomes were hospitalisation for COVID-19 and death from COVID-19. A confirmed diagnosis of SARS-CoV-2 infection was based on READ codes recommended in national guidelines (Additional file 1: Table S1) [27, 28]. According to National Health Service Guidance and Standard Operating Procedures for Primary Care, and the UK Faculty of Clinical Informatics guidelines, confirmed SARS-CoV-2 infection codes represent a positive RT-PCR test [29]. Hospitalisation for COVID-19 was defined as a hospitalisation record in the IMRD [30] within 30 days after documentation of SARS-CoV-2 infection, and death from COVID-19 was defined as a death within 30 days after documentation of SARS-CoV-2 infection.

We used two negative control outcomes (i.e., SARS-CoV-2 infection occurred over the 14 days after receiving the first dose of the COVID-19 vaccine and death from cardiovascular disease [CVD] defined as a death within 30 days before or after documentation of myocardial infarction, stroke or heart failure) to evaluate for potential residual confounding [31]. No difference in the risk of the outcomes between two comparison groups should be observed unless there were residual confounders (e.g., health status or healthcare-seeking behaviour).

The variables in the PS model consisted of sociodemographic factors (i.e., age, sex, Townsend Deprivation Index), geographic location, race, body mass index (BMI), lifestyle factors (i.e., alcohol use and smoking status), influenza vaccination history during the past one year before the index date, comorbidities prior to the index date (i.e., hypertension, diabetes, chronic kidney disease, pneumonia or infection, chronic obstructive pulmonary disease, influenza, cancer, venous thrombosis, atrial fibrillation, ischemic heart disease, congestive heart failure, stroke, trauma, fracture, liver disease, fall, dementia, and depression), medication use (antidiabetic, antihypertensive, statin, diuretics, glucocorticoids, non-steroidal anti-inflammatory drugs, opioids, proton-pump inhibitors, biologic disease modifying antirheumatic drugs) and healthcare utilization during the past one year immediately before the index date. A directed acyclic graph of the comparative effectiveness of BNT162b2 and ChAdOx1 nCoV-19 vaccines against COVID-19 and potential confounders was shown in Additional file 1: Figure S1.

The baseline characteristics of individuals who received the BNT162b2 vaccine were compared with individuals who received the ChAdOx1 vaccine using standardized difference. For the primary outcome, person-months of follow-up for each individual were calculated as the amount of time from the index date to the first of the following events: confirmed diagnoses of SARS-CoV-2 infection, death, disenrollment from a GP practice participating in IMRD, 10 months follow-up, receiving a different COVID-19 vaccine of the second dose, or the end of the study (October 31, 2021). We calculated the rate of SARS-CoV-2 infection and plotted cumulative incidence curves of SARS-CoV-2 infection for individuals who received the BNT162b2 vaccine and individuals who received the ChAdOx1 vaccine, respectively. We estimated the absolute rate difference (RD) of SARS-CoV-2 infection between the two comparison groups. We obtained the hazard ratio (HR) of SARS-CoV-2 infection for the individuals who received the BNT162b2 vaccine vs. those who received the ChAdOx1 vaccine using Cox proportional hazard model accounting for competing event (i.e., death). We tested the proportional hazard assumption by plotting the cumulative incidence curve of each outcome. We then conducted a weighted cox regression to obtain a non-proportional HR if the proportional hazard assumption was violated [32]. We repeated the analysis to evaluate the comparative effectiveness of the two vaccines for secondary outcomes.

We performed four sensitivity analyses to assess the robustness of the study findings. First, we conducted subgroup analyses to assess the comparative effectiveness of these two vaccines according to sex (male vs. female), age (≥65 vs. <65 years), and study period (i.e., SARS-CoV-2 alpha-variant predominance period from January 1 to May 16, 2021 vs. delta-variant predominance period from May 17 to October 31, 2021 [13]). Second, to evaluate the potential residual confounding effect, we examined the relation of the BNT162b2 vaccine vs. the ChAdOx1 vaccine to the risk of SARS-CoV-2 infection in the first 14 days after the first dose of vaccination and death from CVD, respectively. Third, we performed the analysis by including the participants who had SARS-COV-2 infection prior to the index date. In this analysis, we added the history of previous SARS-COV-2 infection when calculating the PS. Fourth, we assessed the risk of SARS-CoV-2 infection among participants who received two doses of vaccines and one dose of vaccine separately.

To date, results from their respective phase III clinical trials demonstrated that the BNT162b2 vaccine had a 95% efficacy in protecting against COVID-19, while the ChAdOx1 vaccine had a 70% efficacy against COVID-19. Consistent with the findings of clinical trials, results from several observational studies also indirectly showed a slightly higher effectiveness of the BNT162b2 vaccine than the ChAdOx1 vaccine when compared with unvaccinated individuals [12‐16]. Meanwhile, other observational studies suggested similar effectiveness of the BNT162b2 and ChAdOx1 vaccines against SARS-CoV-2 infection, symptoms, hospital admissions, and death from SARS-CoV-2 infection when compared with unvaccinated individuals [4, 17‐20] Our study demonstrated greater effectiveness of the BNT162b2 vaccine, indicated by both absolute (i.e., RD) and relative (i.e., HR) effects on the risk of SARS-CoV-2 infection and hospitalisation for COVID-19 than the ChAdOx1 vaccine over ten months follow-up. Although the infection rate was different between the SARS-CoV-2 alpha-variant predominance period and the delta-variant predominance period, which may be due to a differential vaccine waning, the comparative effectiveness of the BNT162b2 and ChAdOx1 vaccines against SARS-CoV-2 infection was consistent across these two study periods. This real-world empirical data provided critical evidence of the relative effectiveness of these two vaccines in mitigating the COVID-19 burden in the general population [33].

Both the mRNA and AdV vector vaccines encode the production of the SARS-CoV-2 spike (S) protein, which is the major target for neutralizing antibodies generated from natural infection and for therapeutic monoclonal antibodies [34‐36]. However, the two mechanisms leverage different aspects of the immune response, particularly the innate immune response, which may lead to differences in immunogenicity [34]. Previous studies have reported that the BNT162b2 vaccine might induce higher antibody titres than the ChAdOx1 vaccine [7‐9], which may partially explain the differences in the effectiveness against COVID-19 between these two vaccines.

There are several strengths to our study. First, we emulated a hypothetic trial using a population-based electronic database to evaluate the comparative effectiveness of two different vaccines against COVID-19. Second, the findings are consistent with the indirect comparison of the effectiveness of these two vaccines based on the results of their respective clinical trials. Third, results from the sensitivity analyses using negative control outcomes confirmed that the effect of residual confounding, if present, is likely to be minimal, supporting the validity of our main findings. Fourth, considering that the time-varying confounders (e.g., COVID-19 vaccines where rollout in the period specified started in the oldest age groups) [37] may impact the results, we divided the participants’ enrolment period into weekly time blocks when they received either the BNT162b2 or ChAdOx1 nCoV-19 vaccines. In each time block we calculated propensity score (PS) and adopted an overlap weighting approach in each weekly time block to balance the baseline characteristics.

However, our study also has some limitations. First, although we used rigorous approaches to control for confounding (i.e., demographic characteristics, medical history, comorbidities, and health care utilization), unmeasured residual confounding, such as the severity of comorbidities, cannot be ruled out. Second, owing to a lack of detailed information in IMRD, we are unable to assess the comparative effectiveness of two vaccines on several other sequelae of COVID-19, such as intensive care unit admission, mechanical ventilation, or length of hospitalisation. Future studies are needed to evaluate these additional outcomes. Third, we cannot access the data held in the hospital and were not reported back to GPs (e.g., tests were performed at the hospital and were not reported back to GPs). Thus, misclassification of the COVID-19 diagnosis could occur and bias the study findings. However, such bias, if occurred, is likely to be small and non-differential. As a result, it would bias the observed associations towards the null. Moreover, although the original PCR test results were not available and the sensitivity for capturing COVID-19 cases through the Read Code has not been validated in the IMRD, a recent study of COVID-19 codes recorded in the primary care setting suggested that clinical diagnosis of COVID-19 by physicians followed a similar trend to test positive cases confirmed by the UK national testing service [38]. Fourth, we could only access data within the GP system. Thus, we were unable to evaluate the comparative effectiveness of BNT162b2 and ChAdOx1 nCoV-19 vaccines among individuals who were not registered with a GP or whose vaccination record was not captured by their GP. Future studies are needed to assess whether the effectiveness of BNT162b2 and ChAdOx1 nCoV-19 vaccine varies among this population.

This emulation of a target trial of a head-to-head comparison of two vaccines, provides evidence that the effectiveness of the BNT162b2 against COVID-19 appears better than the ChAdOx1 vaccine for both SARS-CoV-2 infection and hospitalisation for COVID-19 but not death from COVID-19. The percentage of fully vaccinated population remains low (<20%) in most countries of Africa and several countries of West Asia [6]. Although our findings suggest BNT162b2 vaccine may be preferred to reduce the SARS-CoV-2 infection rate and hospitalisation for COVID-19, the choice of vaccine type should not only be based on the potential differences in effectiveness. The decision regarding which COVID-19 vaccine to receive should consider other factors including availability, cost, and individual characteristics [39].