From trial to practice: incidence and severity of COVID-19 vaccine side effects in a medically at-risk and vaccine-hesitant community

Med Center Health is a six-hospital system serving over 280,000 individuals across ten counties in southcentral Kentucky. Within three days of COVID-19 vaccine authorization, Med Center Health established a vaccine clinic at its main campus in Bowling Green, Kentucky. As vaccine became more plentiful, clinics were established at four other hospitals within the system. At the peak of vaccine rollout, these facilities collectively administered approximately 900 doses per day (M. Joyce, pers. comm.). Over the study period, more than 41,380 first doses of COVID-19 vaccine were administered.

Med Center Health followed the four-phase vaccine rollout plan developed and stipulated by Kentucky Governor Andy Beshear, in accordance with national and state public health officials [32]. Because of patterns of vaccine availability and distribution, most individuals received the BNT162b2 vaccine, while somewhat fewer received mRNA-1273; a small number received the Janssen (JNJ-78436735) vaccine.

We deployed an online survey to individuals who received at least one vaccination through Med Center Health between December 14, 2020 and May 1, 2021. Med Center Health staff compiled a cell phone contact list of 18,711 such individuals, who were each sent a survey invitation via text message on May 5, 2021. The invitation, which included a link to the online survey, was sent successfully to 17,760 devices. Text reminders were sent on May 12 and June 12. On June 12, we texted the survey invitation to an additional 1830 individuals vaccinated after May 5, bringing the total pool to 19,590 potential respondents. One final reminder was sent to the entire list on June 18 and the survey closed on June 21, 2021.

Data were collected anonymously following Dillman’s Tailored Design Method [33] for internet surveys. The questionnaire was optimized for mobile viewing [34] and consisted of several pathways. The various pathways were pilot tested among 24 college students, office staff, pharmacists, medical interns and physicians for validity, clarity, timing and mobile layout.

The survey questionnaire was developed by the research team in Qualtrics and approved by the Medical Center IRB #1. All respondents provided informed consent in order to enter the questionnaire and all collected response data were kept on a secure device.

Respondents were first asked a series of demographic questions, followed by general information on their vaccine status and which vaccine type they had received. The majority of remaining question items were identical to those used in clinical vaccine trials, for the BNT162b2 [2] and mRNA-1273 [3] vaccines. These large question blocks asked respondents to rate the severity of 15 potential side effects. Severity rating categories were Likert-based and included: none (1; no symptoms), mild (2; did not interfere with activities), moderate (3; interfered with some activities), severe (4; prevented regular daily activity), and medical care required (5; ER, doctor, or hospital).

Additionally, respondents were asked to rate the severity of any unsolicited adverse events they experienced, if they required medications to treat their side effects, and the number of days missed from work or other activities as a result of side effects. They were also asked to rate their overall reactivity to other common vaccines (e.g., flu, pneumonia, tetanus and shingles), if they had been diagnosed with COVID-19 prior to vaccination and if so, how long ago in reference to their vaccine date.

All statistical analyses were carried out using SYSTAT, version 13. Because of sample size limitations, statistical comparisons were restricted to respondents who received two doses of the BNT162b2 or mRNA-1273 vaccine.

Individual items were tested for significant differences in level of side effects reported between vaccine brands using chi-square tests of association, separately for first and second doses. Overall side effect severity scores were calculated for each respondent as the mean response level across all listed side effects; separate severity scores were computed for first-dose and second-dose reactions.

Side effects incidence rates were compared to those reported from clinical trials data via calculation of incidence rate ratios (IRR). Statistical significance of IRRs was determined based on 95% confidence intervals, and clinical significance was qualitatively assessed using Cohen’s h [35].

Differences in severity scores by age and gender were tested for significance using a Kruskal-Wallis test, and relationships between age and severity scores were estimated using least-squares regression. Differences between BNT162b2 and mRNA-1273 vaccines were tested for significance using Kruskal-Wallis one-way nonparametric ANOVA. Differences between first- and second-dose severity scores were examined using a Wilcoxon signed-ranks test applied separately between vaccine types. Differences between respondents who reported previously tested positive for COVID-19 vs. being COVID-19 naïve were tested using a Kruskal-Wallis test. Correlations between overall severity scores for both first and second doses of each COVID-19 vaccine type were compared to respondents’ self-reported severity of reactions to vaccines for influenza, pneumonia, tetanus, and shingles using Pearson correlation coefficients. Significance of all comparisons was based on Bonferroni-adjusted criteria applied within families of tests.

Respondents were predominantly female (73.1%) and identified as Caucasian (91.8%). The median age group was 56–60 years. The largest number of respondents were 61–65 years of age (13.7%), and greater than 50% of respondents were aged 51–75 (55.2%). Healthcare workers made up the largest single group of respondents (28.9%), followed by individuals aged 70 and older (18.5%), 60–69 years of age (15.2%), K-12 school personnel (10.4%), adults aged 15–59 (7.8%), and essential workers (7.0%). These demographic characteristics mirror the ethnic makeup of the community from which the sample was drawn [36], combined with Kentucky’s phased rollout of the vaccine during the timeframe of the study [32]. Complete respondent characteristics are given in Table 1.

The majority of respondents (85.3%) received the BNT162b2 vaccine. The percentages (Table 1) are reflective of the availability and distribution of vaccine types at the Med Center Health Vaccine Clinic from which the sample of respondents was drawn (M. Joyce, pers. comm.).

The majority of respondents (61.8%) cited the “ability to protect oneself from COVID-19” as the most important factor in taking the vaccine, while a substantial number (24.7%) cited “a desire to protect others” as their primary motivator (Table 1). A modest percentage (10.9%) reported having contracted COVID-19 previously. Of these, 60.6% were exposed > 90 days prior to receiving the vaccine (Table 1). The majority of respondents reported having previously received an influenza (90.2%) or tetanus (88.2%) vaccine, with smaller percentages having received vaccinations against pneumonia (52.6%) or shingles (39.4%) (Table 1).

The most commonly-reported side effects following the first dose of either the vaccine were arm soreness near the injection site, fatigue, muscle ache near the injection site, and headache; these were reported by greater than 25% of respondents (Table 2). Injection site redness, chills, and fever were reported in 10–25% of respondents. Other symptoms were reported by fewer than 10% (Table 2). Allergic reactions were the least-reported side effect, cited by only 2% of respondents (Table 2). No rare severe adverse effects – such as myocarditis, thrombosis, or hemorrhage – were reported.

In all cases, side effects were generally quite mild. Arm soreness resulting from the first dose of mRNA-1273 had the highest mean severity (2.14 + 0.03), which was still in the mild category; all other side effects across both brands had mean severity values less than 2.00 (Table 2). The majority (64.2%) reported taking medications to alleviate their symptoms following the vaccine. The vast majority (89.8%) reported that the side effects did not cause them to miss work or other activities; of those who reported having missed activities, the majority indicated the number of days during which their activities were impacted was one (Table 2).

Reported side effects were more pronounced among respondents who received mRNA-1273. For nine of 17 side effects – arm soreness, fatigue, muscle ache, injection site redness, chills, fever, injection site itching, rash, and medications required – this difference was statistically-significant (Table 2).

Second dose side effects were similar to those reported for first doses. Arm soreness, fatigue, muscle ache, and headache remained the most commonly-reported, followed by chills, fever, and injection site redness (Table 3). Allergic reactions were again quite rare, reported by only 2% of respondents. In most cases, second dose side effects were stronger than those reported for the first dose (Table 2, Table 3); the only exceptions were arm soreness (BNT162b2), injection site itching (both brands), and general itching (mRNA-1273). The magnitude of the differences was less than 5% in all cases. No rare severe adverse events were reported.

Arm soreness had the highest mean severity following the second dose, though still in the mild category (Table 3). Only injection site itching resulting from mRNA-1273, and medications required following BNT162b2 showed less severe reactions following the second dose (Table 2, Table 3). The mean number of days in which activities were impacted following vaccination was zero to one (Table 3).

Second-dose side effects to mRNA-1273 were generally stronger than those to BNT162b2. Thirteen side effects showed significant differences in severity between brands. These included all 11 side effects that showed significant differences after the first dose, as well as additional side effects of headache, nausea, lymph node swelling, and missed activities (Table 3).

Most side effects (12 of 17) tracked in our study were common to BNT162b2 and/or mRNA-1273 clinical trials. Incidence rate ratios (IRRs) indicated general consistency between our data and clinical trials. While the majority of IRRs comparisons were significantly different from zero, the magnitude of differences was of limited clinical significance in most cases. Only five comparisons were characterized by Cohen’s h values in the moderate (0.50 < h < 0.80) or large (0.80 < h < 1.0) range [21] (Table 4). Among BNT162b2 recipients, the use of pain medication was clinically greater than reported in the clinical trial (IRR = 2.69 + 0.12, p < 0.05, h = 0.85). Among mRNA-1273 recipients, injection site redness was clinically greater following the first dose compared to the clinical trial (IRR = 10.48 + 0.84, P < 0.05, h = 0.82), while fever was of moderate clinical significance following both first (IRR = 20.31 + 1.62, p < 0.05, h = 0.63) and second (IRR = 2.57 + 0.20, p < 0.05, h = 0.56) doses. Injection site redness was also of moderate clinical significance following the second dose of mRNA-1273 compared to the clinical trial (IRR = 3.14 + 0.26, p < 0.05, h = 0.50). Nevertheless, the severity in all cases was consistently in the mild to moderate range; fever following the second dose of mRNA-1273 was the only side effect in which the incidence of either a severe reaction or one requiring medical attention exceeded 5% (Tables 2, 3).

Respondents listed 531 “other” side effects across their first (n = 234, 4.8%) and second (n = 297, 6.2%) vaccine doses; however, more than 20% of these were identical in language to previously listed side effects. Most others listed included a general sense of feeling “unwell,” “bad,” or “flu-like.” Very few respondents (0.4%) listed potentially severe side effects such as seizures (n = 2), Bell’s palsy (n = 1), anaphylactic shock (n = 1), and heart arrhythmias (n = 2). Only two types of unsolicited adverse events were commonly reported: dizziness, including vertigo, brain fog and lightheadedness (n = 71); and changes in menstruation (n = 17).

There was a significant difference in severity score as a function of gender, for both first (Kruskal-Wallis H = 95.33, df = 1, p < 0.001) and second (Kruskal-Wallis H = 208.08, df = 1, p < 0.001) doses; overall, females reported more severe reactions, though still in the generally mild category. There was a significant negative relationship between age and severity score for both first (b = − 0.019 + 0.001, p < 0.001) and second (b = − 0.037 + 0.002, p < 0.001) doses, though the relationships were relatively weak (r² = 0.055 and r² = 0.095, respectively).

Overall side effect severity scores were significantly higher among respondents who received the mRNA-1273 vaccine. This pattern was seen following both first (Kruskal-Wallis H = 54.22, df = 1, p < 0.001) and second (Kruskal-Wallis H = 220.66, df = 1, p < 0.001) doses (Fig. 1). Second-dose severity scores were significantly greater than first-dose severity scores for both BNT162b2 (Wilcoxon z = 4.42, p < 0.001) and mRNA-1273 (Wilcoxon z = 11.69, p < 0.001) (Fig. 1). Most interesting, the difference between first- and second-dose reactions was three times greater among mRNA-1273 as compared to BNT162b2 recipients (18% vs. 6%; Fig. 1).

Respondents who reported previously tested positive for COVID-19 showed significantly higher side effect severity scores following their first vaccine dose than did COVID-19 naïve respondents (Kruskal-Wallis H = 79.77, df = 1, P < 0.001; Fig. 2); first-dose reactions of COVID-19 positive respondents was on-par or slightly higher than the second-dose reactions seen among COVID-19 naïve respondents (Fig. 2). However, severity scores in COVID-19 positive respondents were only slightly higher than first-dose reactions in this same group (Kruskal-Wallis H = 6.31, df = 1, p < 0.01; Fig. 2).

There was a positive correlation between severity scores calculated for first and second doses of both vaccines and respondents’ self-reported severity of reaction to previous influenza, pneumonia, tetanus, and/or shingles vaccines (Table 5). While the magnitude of these correlations was rather low (r < 0.23 in all cases), 11 of 16 comparisons were statistically significant (Table 5). The strongest correlations were seen between the first dose of BNT162b2 and other vaccines; in fact, first-dose correlations were higher than correlations observed from second-dose reactions (Table 5). By contrast, second-dose correlations of the mRNA-1273 vaccine with other vaccines were higher than first-dose correlations (Table 5).

Our findings derived from rollout of BNT162b2 and mRNA-1273 vaccines within a diverse, at-risk community align well with data from clinical trials. The most common side effects reported by our 4825 respondents – arm soreness, fatigue, muscle ache at the injection site, and headache – were the same as reported in both clinical trials [2, 3]. Chills, nausea, and fever were also commonly reported by both our respondents as well as those participating in clinical trials. The duration of side effects impacting regular activities among our group of respondents was generally zero to 2 days, also consistent with both trials [2, 3].

BNT162b2 [2] and mRNA-1273 [3] trials reported a higher incidence of both local and systemic side effects in younger participants compared to those in the older age categories. This is consistent with our finding of a significant negative relationship between side effect severity and age. Neither clinical trial compared incidence or severity of reactions between males and females, while our data indicated that females reported more severe impacts than did males, though generally in the mild range.

Our results are also consistent with previously published studies of real-world populations citing injection site pain or arm soreness as the highest reported side effect [4, 21, 37‐39]. Fatigue and headache were also commonly reported, as were a range of other symptoms including swelling and tenderness at the injection site, chills and/or general malaise. Incidence rates of common symptoms were broadly consistent with what we observed, though there existed a fair amount of variation among studies; in particular, Menni et al. [21] reported lower incidence rates for all of these common side effects. Several studies mirrored our results and reported a higher incidence and/or severity among females as compared to males [21, 38] and younger vs. older recipients [21, 38]. Ripabelli et al. [38] attributed these trends to well-established differences in pharmacological responses between groups, including differences in cellular immune response between genders and the relative robustness of the immune system in younger versus older individuals.

As in previous studies [4, 21, 37‐39], we found evidence for increased side effects following the second dose of the vaccine. Our data also support prior data indicating stronger reactions to mRNA-1273 than to BNT162b2 [2, 3, 39, 40], particularly following the second dose. This difference may reflect the higher dose amount of mRNA-1273 (100 vs. 30 micrograms), and longer inter-dose interval (28 vs. 21 days). While these differences may account for the consistently higher side effect severity associated with mRNA-1273, neither fully explains the fourfold greater difference in severity between first and second doses of mRNA-1273. In fact, the one-week greater interval between doses in the mRNA-1273 series – assuming a linear relationship between inter-dose interval and side effect severity – would be expected to result in only an 8% greater difference, vs. the 18% observed.

Other adverse side effects, beyond those explored in trails, have been noted in response to the vaccine, and our incidence rates of such effects closely mirror other similar investigations [38]. Serious side effects of note include neurological symptoms [41], anaphylaxis [42, 43], and myocarditis [44, 45]. Our respondents listed very few if any of these types of events. However, the prevalence of dizziness and changes in menstruation may warrant further exploration. Dizziness, light-headiness and brain fog are similar to reported symptoms of COVID-19 infection [46]. The basis for our respondents’ symptoms are unclear, but other studies have found dizziness to be a common neurological side effect of the vaccine [47]. Ripabelli et al. [38] suggested this might reflect anxiety-related symptoms associated with the vaccination process. Menstruation-related symptoms have also been reported elsewhere [48], and a National Institutes of Health funding initiative was established to investigate these effects [49].

These findings support the conclusion that side effects experienced by individuals in a diverse population, characterized by a range of comorbidities, are likely to be similar in pattern and severity to what has been described from clinical trials of healthy individuals. Practitioners, health departments, and other entities can thus have confidence in promoting both BNT162b2 and mRNA-1273 as safe and minimally-disruptive of normal activities.

Individuals in our sample with a previously-documented COVID-19 infection reported a higher level of side effect severity than did COVID-19 naïve respondents, across both vaccine types. However, this increased severity was only significant following the first dose. Moreover, first-dose reactions in respondents previously infected with COVID-19 mirrored second-dose reactions in COVID-19 naïve respondents. Mathioudakis et al. [4] were among the first the show the link between prior COVID-19 diagnosis an increased risk of incidence or severity of vaccine side effects. A similar finding was reported by Ebinger et al. [50], who compared 35 patients with prior COVID-19 infection to an additional 528 COVID-19 naïve individuals. While smaller in scope than our study, these authors used antibodies to confirm prior COVID-19 infection.

Menni et al. [21] studied more than 2000 individuals previously diagnosed with COVID-19 who received the BNT162b2 vaccine. They found local side effects were one- to two-times more common, and systematic side effects two- to nine-times more common in such individuals [21].

Applying a multivariable logistic regression approach to a same of over 19,000 individuals, Beatty et al. [39] found that prior COVID-19 infection led to a significantly higher odds ratio of adverse effects following COVD-19 vaccination, second only to vaccine dose in terms of magnitude. Jeskowiak et al., [37] found previously diagnosed individuals had stronger adverse effects after the first dose of the vaccine, which they postulated could be due to a weakened antibody-dependent enhancement, whereas COVID-19 naïve respondents experienced stronger side effects after the second dose. Taken together, these findings suggest that prior COVID-19 infection may effectively serve as a priming dose for the immune response (analogous to the first vaccine dose in COVID-19 naïve individuals). The similarity of second-dose responses in both COVID naïve respondents and those with prior COVID-19 infection in our study further suggests responses to booster doses might be expected to be on par with individuals’ second doses. Such a pattern, if confirmed by subsequent studies, could encourage individuals to take booster doses when recommended.

There was a positive correlation between severity of side effects to either COVID-19 vaccine and individuals’ reported responses to other vaccines. Individuals who reported a higher incidence or severity of side effects to the BNT162b2 vaccine also tended to report a higher level of overall severity of responses to influenza, pneumonia, tetanus, and shingles vaccines; these correlations were significant across both first and second doses. With mRNA-1273, we only observed significant correlations between the influenza vaccine and either dose of the COVID-19 vaccine, and between the pneumonia vaccine and the second dose of the mRNA-1273 series. In all cases, however, the correlations were low. A similar positive association between adverse events resulting from the second dose of the COVID-19 vaccine and prior vaccinations was reported by Ripabelli et al. [38]. These data suggest that individuals might expect to respond to COVID-19 vaccination similarly to their response to other vaccines. Conversely, Beatty et al. [39] found that receiving an influenza shot during the prior year was associated with lower odds of severe a response to COVID-19 vaccination. While the relationship between COVID-19 and other vaccines remains somewhat unresolved, such data could be useful to individuals in making the decision as to whether and how to schedule their COVID-19 vaccination.

Measuring or affecting vaccine hesitancy was not a primary objective of this research. However, our results have been used in local community-led vaccination efforts, and it is prudent to discuss the potential of this study, and those like it, in persuading the vaccine hesitant. This is particularly important in regions like Kentucky which are characterized by a high level of comorbidities that increase susceptibility to severe COVID-19 [22, 24] and political and social-economic factors that increase the rate of vaccine resistance [51, 52] and continue to this day.

Research suggests that vaccine holdouts may be reachable with the right information provided by trusted sources [9, 10, 53, 54], (Vlasceanu M, Coman A: The impact of information sources on Covid-19 knowledge accumulation and vaccination intention, forthcoming). Use of local messaging [55, 56], and social norms [57] are thought to increase trust and serve as effective motivators for behavioral change. Message source is vitally important in persuasion [55, 58] as shared values between the messenger and the audience affect message acceptance [59], especially among conservatives and moderates [60]. This “local source” phenomenon is related to the use of normative behavior, or social norms [61], which are included in various models and theories of health-related behavior change [57, 61, 62]. Research has also demonstrated that behavior change is most likely when information comes of an individual’s own social network [63‐65], and that social norms and circles can influence health behaviors and attitudes toward vaccination uptake [66, 67] especially via social media [18]. Additionally, interventions targeted at individual healthcare teams and their patients can optimize efforts to address vaccine hesitancy [68].

Given these considerations, local research projects such as ours have increased credibility and potential to persuade at the community level. Our results were reported by local news outlets and on social media. Study participants were our own community members, which increases the normative value of getting vaccinated. Further, the large number of participants means individuals from many different social circles in our community were likely engaged.

The current vaccine-hesitant moment in which we find ourselves will continue to impact the public health sector in the context of COVID-19 and beyond. The return of measles to the world-stage is a cautionary tale. Eradicated from the U.S. in 2000, there were 1200 reported cases in 2019, while Europe saw more the 90,000 in the first half of the year. Increases across the African continent have also been reported, and in most cases, these increases are tied to precipitous declines in vaccination rates [31]. Similar trends have also been noted for the seasonal flu and HPV [17]. Additionally, COVID-19 is certain not be the last outbreak to require a rapid vaccine rollout, the cited cause for increased misinformation [17]. In fact, the current outbreak of monkeypox recently prompted the U.S. FDA to authorize the same “emergency use authorization” for the JYNNEOS vaccine (Modified Vaccinia Ankara, MVA) against monkeypox [68].

Given these disturbing anti-vaccine trends and the inevitability of novel and resurgent viruses, it is prudent to identify strategies targeting vaccine hesitancy and combatting misinformation [69] at the community level. This study was quick and inexpensive; it could be easily recreated to measure the side-effects of recurring (e.g., seasonal flu), childhood (e.g., MMR and HPV) and novel vaccine programs (e.g., monkeypox). Use of such locally sourced data in vaccine campaigns, even outside of the COVID-19 pandemic, and especially through social media [18], may be a strategy worth pursuing.

We acknowledge several limitations with our study. As with any survey, results are dependent upon self-reporting of both side effects and severity level. This limitation was most evident with regards to unsolicited side effects, as respondents commonly listed side effects that were included in Likert scale questions. Since many respondents were healthcare workers, there may have been a bias towards over-reporting of side effects, as this population was attuned to side effects that had been reported in clinical trials. Additionally, it had been several months since some respondents received their COVID-19 vaccine, and they may not have accurately remembered their side effects and/or severity, potentially leading to overstating or understating symptoms. This same recall bias may have influenced respondents’ assessment of their responses to other vaccinations [70] and has been documented in other analysis of cognitive bias and vaccine-induced adverse events [71]. Some respondents may not have known they were previously COVID-19 positive, especially if they had been asymptomatic or had very mild symptoms; however, this represents a conservative error with respect to our finding of significant differences in response of prior COVID-19 positive vs. COVID-19 naïve individuals. We also did not ask respondents about any comorbid conditions, though regional demographics suggest that people with comorbid conditions were included in the sample.