COVID-19 underreporting and its impact on vaccination strategies

Surveillance and notification systems in Public Health are subject to uncertainties that cause difficulties to estimate the morbidity and mortality rates affecting populations. Among the diverse causes of uncertainty two distinct levels of surveillance in Public Health should deserve special attention, under-ascertainment, when not all cases seek healthcare; and underreporting, a failure to adequately report symptomatic cases that have sought medical advice [1]. In the context of mortality, it is possible to identify the concepts of under-ascertainment and underreporting since both events are expected to happen in real systems of Public Health. Thus, in what follows, we unify under-ascertainment and underreporting as “underreporting”.

Underreporting cases of infectious diseases poses a major challenge in the analysis of their epidemiological characteristics and dynamical aspects. Without accurate numerical estimates it is difficult to precisely quantify the proportions of severe and critical cases, as well as the mortality rate [1]. Such estimates can be provided, e.g., by testing the presence of the virus. However, during an ongoing epidemic, such testing implementation is a daunting task.

Different strategies were proposed to estimate the true amount of COVID-19 cases. Some of these strategies are based on seroprevalence studies [2‐6] that found seroprevalence proportions much larger than the reported accumulated cases in different periods of 2020 in Chicago and NYC, as well as across Denmark, Mexico, and the United States, respectively. Other works estimate underreported infections and deaths from the official reports in combination with different techniques. For example, in [7, 8] the authors consider the excess of deaths caused by respiratory infections in 2020 and found significant underreporting proportions in Brazil. The number of excess deaths is also estimated for England and Wales in [9]. Based on data provided by the World Heath Organization (WHO), the article [10] compares case-fatality risk measures for different countries to estimate underreporting. By using the data from South Korea as a benchmark, the authors in [11] built an underreporting estimation technique based on the predictions of a susceptible-infected-removed-type (SIR-type) model, that are adjusted using demographic data from different places. The work [12] proposes a Bayesian framework based on an SIR-type model to estimate the true case fatality ratio (CFR) and the corresponding underreporting using official reports from the Brazilian health authority. Similarly, in [13], the authors use a Susceptible-Exposed-Infected-Removed-type (SEIR-type) model to estimate the CFR and the underreported cases in Iran, based on data from WHO and the Iranian Health authority. Another application of an SIR-type model to estimate underreporting was performed in [14], using data from California and Florida. In [15], the authors estimate underreported deaths in Italy by comparing mortality data and making use of regression techniques, as well as demographic information. The work [16] proposes a machine learning algorithm to predict underreported infections for all the 50 states in the US and other countries, using official reports and the infection-fatality-rate estimated in [17] as the training dataset.

COVID-19 control is dependent, in complex non-linear ways, on several variables that include the incidence of infection, on non-pharmaceutical interventions like the use of masks and social distancing, the speed with which the vaccination can be implemented, and the efficacy of the available vaccines. The uncertainties and interactions between these variables make the use of mathematical models to quantify and optimise the effects of vaccination on the COVID-19 pandemic urgently needed [18]. Mathematical models, therefore, have played a key role in helping the understanding of COVID-19 dynamics as well as in determining the best decisions of mitigation strategies [19]. In this sense, models remain essential tools for evidence synthesis, planning and forecasting, decision analysis for COVID-9 control, as well as policymaking [20].

This work presents a methodology to estimate underreported infections based on approximations of the stable rates of hospitalization and death found using daily reports of infections, hospitalizations, and deaths, as well as testing data. As an important byproduct, we evaluate the impact of underreporting in the designing of vaccination strategies because the larger the number of unaccounted infections, the larger the chances of vaccinating an already immune individual. This can restrict the capability of vaccination in reducing hospitalizations and deaths, as simulated scenarios using an SEIR-like model [21] show. It is worth mentioning that, understanding such limitations is particularly important to help scientists and authorities addressing the politicization of the vaccination, the polemic around safety and efficacy of the vaccines, and the anti-vaccination campaigns that contribute to vaccination hesitancy and vaccination delay [22, 23].

This section starts by presenting how the stable rates of hospitalization and death are obtained. Then, the technique to estimate the potential underreporting of COVID-19 infections is introduced. Finally, a Susceptible-Exposed-Infected-Removed-like (SEIR-like) epidemiological model is proposed to quantify how underreporting may affect vaccination strategies. The schematic description for this methodology is shown in Fig. 1.

In order to find stable rates of hospitalization and death, we seek specific time periods when the daily rate of testing is sufficiently large with respect to the population size, and the number of positive tests is small enough. During such periods we evaluate daily empirical rates of hospitalization and death, looking for those whose rates fluctuate around some mean value. This is performed by means of an accurate data analysis producing different statistical indicators leading to the necessary correction. A schematic representation that summarizes the proposed methodology can be found in Fig. 1. We use time series of seven-day moving averaged reports from Chicago and New York City (NYC), in the US, the province of Buenos Aires (BA), in Argentina, and Mexico City (MC), in Mexico. Since COVID-19 severity strongly depends on age and gender [17, 24‐27], we evaluate the above-mentioned rates accounting for demography to improve the estimation accuracy of the number of infections. The latter will be called corrections. These corrections are evaluated using the empirical rates of hospitalization and death as follows: For an observed rate of hospitalization or death, and a given day in the time series, we evaluate the corresponding infection number. For example, if for this day the reported hospitalization rate is one and the projected rate is one half, then, the correction is twice the reported infections.

Underreporting Estimation In order to estimate underreported infections, the formulas in Eq. (2) are used, considering the daily cases of COVID-19. The graphical comparison between the observed and corrected numbers of infections for Chicago can be found in Fig. 6. Table 2 presents the corrected and observed accumulated numbers of COVID-19 infections in Chicago, during the period 01-Mar-2020 to 23-Dec-2020. In order to observe the effect of corrections, we divided the period 01-Mar-2020 to 23-Dec-2020 into three periods, namely, 01-Mar-2020 to 31-July-2020, 01Aug-2020 to 05-Oct-2020, and 06-Oct-2020 to 23-Dec-2020. Additional results considering the data from other places can be found in the Additional file 1.

Corrections using hospitalization rates present smaller values than the ones obtained with death rates. This can be explained by the considerably larger values of the death rate in hospital observed during the outbreaks of March to May and of October to November. The estimated numbers for 01-Mar-2020 to 31-July-2020 are larger than the ones estimated for other periods, indicating that underreport can be more likely in the beginning of the pandemic. Corrections suggest that, for 01-Mar-2020 to 31-July-2020, the number of infections can be 32–632% larger. For 01-Mar-2020 to 23-Dec-2020, COVID-19 infections can be 10–238% larger. Thus, from 8% to 25% of the population of Chicago could have being infected in the study period, instead of the observed proportion of 7.3%. Such figures are in remarkable agreement with with the seroprevalence study [2], carried out between June and December 2020 in Chicago, which pointed out a seroprevalence of 17.9%.

During larger outbreaks we expect that the stabilization of the daily rates does not occur, as we observed in the case of the daily hospitalization rate during the second wave in Chicago. So, we did not use values from this period in our analysis. On the other hand, the daily death rate remained stable during the second wave in Chicago, suggesting its robustness.

The datasets from NYC do not have daily reports by age range or gender. We considered two different periods to estimate the stable rates of hospitalization and deaths and corrected infections can be found in Additional file 1: Table S.2, representing 7.5–30% of the NYC population, instead of the observed proportion of 4.41%. A seroprevalence study [3] estimated about 1.7 million accumulated infections in NYC by the end of May, which is very similar to our results for the same period, i.e., 1.47 million (1.25 million–2.18 million).

For BA, unfortunately, during the period of study the percentage of positive tests was mostly above 10%, making difficult the empirical analysis. However, we consider the period when the positive rate was below 20%. Additional file 1: Table S.4 presents the estimated rates of hospitalization and death. Death rates for individuals younger than 60 years old are like the corresponding rates observed in Chicago. However, for older individuals in BA, the death rates are considerably larger. Corrections from Additional file 1: Table S.5 suggest infection numbers varying from 3.4% to 303% larger than the notified cases, representing 4.7–18% of the estimated BA population for 2020, instead of the reported 4.53%. Unfortunately, we could not find a seroprevalence study for BA that could be used for comparison.

For MC, we could not identify a period when the rates of death or hospitalization stabilized around mean values. Thus, we used the rates estimated for Chicago to provide corrections. Using the death rates by age-range from Chicago seems to be the more accurate way to estimate underreported cases in other places, since the data from Chicago satisfied the hypotheses made to find stable rates. Corrections are 44–681% larger than the observed cases, representing 5.39–29.1% of the estimated population of MC for 2020. In spite of the issues of the MC data, such estimates are pretty much similar to the seroprevalence of 30.7% (95% CI: 28.3–33.1%) found in the study [5] during December 2020, in the Region Central in Mexico, that includes Mexico City.

In Denmark, from 01-Sep-2020 to 31-Oct-2020, more than 4% of the countrywide population was tested weekly, with positiveness proportions of less than 2%. We used this period to estimate the rates of death and hospitalization in Additional file 1: Table S.7. The corresponding estimations of accumulated cases in 2020 can be found in Additional file 1: Table S.8. Corrections are 23.6–295% larger than the reports, representing 3.35–10.7% of the estimated Danish population in 2020. Such numbers closely agree with the estimated seroprevalence of 4.0% (95% CI: 3.4–4.7%) found by the study [4].

Underreport Impact on Vaccination Scenarios Let us now turn to the impact of underreporting on the capacity of vaccination strategies in reducing hospitalizations and deaths. We consider three different scenarios. The first two consider random-mass vaccination under contained and uncontained spread, whereas in the third an age-range-dependent vaccination is performed under contained spread. The parameters used in these examples are estimated using reports from Chicago and NYC [34].

In all three cases we assume that the proportion of the population in the recovered, exposed or in some infective compartment in the model in Eqs. (345678910)–(11), ranges from 5% to 30%. Moreover, only the amount of 5% is observed in all cases. This means that the probability of vaccinating someone that has already had contact with the virus is proportional to the percentage of the population distributed in the exposed, non-hospitalized and infective, and recovered compartments that were not included in the reports. Thus, in our simulations if 5% of the population was infected, then 100% of the vaccinated individuals were susceptible, whereas, if 30% of the population was infected, then only 73.4% of the vaccinated individuals were susceptible. We also assume that the vaccine is 90% effective, and 0.5% of the population is vaccinated every day, for 150 days. The hospitalization rate also decreased proportionally to the number of underreports.

Under contained spread, the transmission parameter amongst mildly infective individuals is set to \(\beta _M = 0.23\). Under uncontained transmission, the parameter \(\beta _M\) is set to 0.44. The resulting accumulated numbers during the vaccination strategy, in both situations, can be found in Table 3.

The assumed size of this hypothetical population is of 2,693,976 individuals. In Table 3, the numbers in the row Total Vaccinated correspond to the vaccinated individuals that were in the susceptible compartment. As the underreported infections increase, the number of effectively vaccinated individuals decreases. The recovered individuals are considered permanently immune. The capacity of vaccination in reducing hospitalizations and deaths is hampered due to underreporting, both under contained and uncontained disease spread. However, if the disease transmission is not under control, then, as underreport increases, the number of hospitalizations and deaths can decrease, indicating the achievement of herd immunity. Therefore, estimating underreporting helps to quantify and explain possible limitations of vaccination strategies.

In the age-range-dependent vaccination case, we use the same vaccination efficacy, and vaccination starts with those aged 80 years or older, then, 10 days after, those individuals aged 70 years or older are included, and so on. Individuals younger than 18 years are not vaccinated. The experiment runs during 150 days, and at each day, 0.5% of the population in each age range included in the strategy for such day is vaccinated. The resulting accumulated numbers can be found in Table 4. The model used to simulate this example is the generalization of the present one as in [23, 34].

The accumulated numbers in Table 4 present a similar pattern to those in the previous examples, as expected, illustrating that the underreporting issue can also limit the effect of age-range-dependent vaccination strategies.

This work proposes possible ways to estimate underreported COVID-19 infections, based on daily reported of cases, hospitalizations, and deaths, considering demography. The proposed methodology of correction is then applied to data from Chicago, NYC, BA, MC, and Denmark. Moreover, it estimates the potential impact of underreporting in vaccination strategies by using an SEIR-like model with parameters estimated from real data.

Estimating underreporting in an ongoing epidemic is a hard task, and only a seroprevalence study can address this task appropriately. However, if we can estimate the stable rates of hospitalization and death related to the disease, then we can use reports to estimate the correct number of infections. The major difficulty of this approach is to identify the period when these rates can be observed or approximated. Firstly, we assume that the number of tests performed daily must be sufficiently large, then the number of positive tests must be sufficiently small. Setting up this is subtle, and we must compare the data from different places. For Chicago and NYC, we set that the rate of positive tests must be below 10%, for BA, it was 20%, and for Denmark it was 2%, since we identified, in the corresponding periods, a stabilization of the rates around mean values. For MC, we could not find such period.

For Chicago, NYC, MC, and Denmark during the period of study, corrections suggest that the number of infected individuals could reach 30% of the population of these places, which represents, in some cases, more than six times the reported numbers. These estimated numbers are in remarkable agreement with the estimates from seroprevalence studies carried out in Chicago, NYC, MC, and Denmark during 2020 [2‐5]. Moreover, the death rate corresponding to 0.97% estimated in [3] for NYC also agrees with the estimated death rates from Additional file 1: Table S.1, i.e., 1.22% (90% CI: 0.82–1.42%). Such estimates must be considered when evaluating the aftermath of vaccination strategies, since underreporting, as illustrated by numerical examples, can reduce the impact of vaccination in reducing mortality and hospitalization rates. Estimating underreports can be useful, for example, to adjust the daily numbers of given vaccines in order to reach the target of reducing the numbers of infections, hospitalizations, and deaths.

Using age-dependent death rates seems to be a reliable way of estimating underreporting, since such rates can be used even if the age pattern of the infected population changes during the epidemic. Thus, we expect that the more demographic information we incorporate into the death rates, the more reliable are the corrections.

We tested the proposed methodology with data from BA and MC where the positive test proportion was considerably higher than 5% to “stress test” the model, verifying if our premises were still valid when the small positiveness proportion was violated. For MC, we compared our results with the estimates from the seroprevalence study [5] finding again a close agreement between them, in spite of the issues in MC data. This illustrate the possibilities of this approach, since in the developing world seroprevalence studies are generally scarce, and our methodology can shed light on the underreporting issue, providing at least a rough picture of the real number of infections. We believe that our approach represents an accurate alternative to seroprevalence studies that allows anyone who has access to daily reports of infections, deaths and hospitalizations, as well as testing data, to keep track on the underreporting issue. Moreover, for disease surveillance purposes, it can be used as the main underreporting estimation technique or as an independent source of results to validate results from seroprevalence studies.

By considering different vaccination strategies under different disease spread trends, we observe that underreporting can also limit the impact of vaccination in the reduction of hospitalizations and deaths, based on the results obtained with the SEIR-type model in Eqs (345678910)–(11).

In summary, using the proposed methodology described in Fig. 1 and employing a judiciously chosen data analysis implementation, we estimate COVID-19 underreporting from publicly available data. This leads to a powerful way of quantifying underreporting impact on the efficacy of vaccination strategies. Furthermore, based on the insights given by the observed rates of hospitalization and death in Chicago, as well as the number of tests performed, we may infer that, during the outbreak of March to May 2020, the number of COVID-19 infections was considerably underestimated. Another byproduct of our analysis is that during the outbreak, only people with more severe symptoms were looking for hospital care thus decreasing the hospitalization rate for all age ranges except for the 0–17 years old cohort. Finally, the studies performed for the Chicago case were also conducted for Mexico City, the Province of Buenos Aires, and Denmark resulting in similar conclusions. A natural follow up would be to extend these studies to other metropolitan areas. In the cases of Chicago, NYC, MC, and Denmark, estimated underreported infections closely agreed with seroprevalence studies.

Moreover, by considering vaccination strategies under different disease spread scenarios, using an SEIR-type model, we found that underreporting can also limit the observed reduction in the numbers of deaths and hospitalizations caused by vaccination.