Stringent thresholds in SARS-CoV-2 IgG assays lead to under-detection of mild infections

Serological tests for antibodies to SARS-CoV-2, the virus that causes Covid-19, have been used to estimate the extent of Covid-19 exposure in national [1, 2] and regional populations, as well as in subgroups of interest including healthcare workers (HCWs) [3, 4]. Additionally, antibody results may be informative about risk of future infection, at least in the short term [5].

One key element of deploying antibody testing is determining assay thresholds for confirming the detection of antibodies. Typically, this has been done based on collections of pre-pandemic sera (i.e. known negative samples) and samples from patients with PCR-confirmed Covid-19 (deemed highly likely to be antibody positive) [6‐8]. Often, these ‘known positive’ sera have been derived from individuals who accessed PCR diagnostic testing based on the presence of clinical symptoms, with a bias towards those with severe enough symptoms to present to hospital where diagnostic testing was concentrated early in the pandemic. As a result, it is not clear how applicable antibody assay thresholds are to those with prior asymptomatic or mild infection which may not have met syndromic criteria for testing, such as presence of fever. Furthermore, even in symptomatic individuals, those with milder disease may have lower viral loads [9, 10] and therefore be more likely to be falsely PCR-negative and omitted from assay calibration cohorts. Limited numbers of studies have looked specifically at those with asymptomatic or mild infection [6], however lower IgG titres have been reported in patients with milder infection compared to those admitted to ICU [11] and one study of 26 PCR-positive asymptomatic individuals showed detectable IgG in only 4 (15%) a median of 29 days post PCR [12], in contrast another showed neutralising antibodies in 47/48 (98%) HCWs with mild infection by 28–41 days [13].

We have recently undertaken SARS-CoV-2 serological testing in a large cohort of UK HCWs [3]. In keeping with other researchers [14‐16], we reported that loss of smell (anosmia) or taste (ageusia) were highly predictive of Covid-19 [3].Additionally, several HCWs tested were antibody-negative despite clinical syndromes consistent with Covid-19, including household contacts of PCR-confirmed cases. Therefore, here we use the presence of anosmia/ageusia along with other risk factors for Covid-19 as probes for Covid-19 infection in our HCW cohort. We use these to search for Covid-19 that was potentially undiagnosed by immunoassays by investigating their relationship with antibody readings either side of the assay threshold.

Eleven thousand four hundred seventy-seven hospital staff provided a first serum sample between 23rd April and 20th August 2020, of whom 11,475 provided complete associated symptom and risk factor survey data. The median (IQR) age of staff was 39 (30–50) years, 8463 (74%) were female. Detailed breakdowns of the occupational roles and specialities of staff have been provided previously [3], the largest groups included nurses and healthcare assistants (4383, 38%), doctors (1746, 15%), administrative staff (1384, 12%) and therapists and other allied health professionals (1054, 9%). Within the 11,475 serum samples obtained, 9324 were analysed using an anti-spike ELISA developed in Oxford and 906 (9.7%) had IgG detected. Eleven thousand three hundred forty-two samples were analysed using the Abbott Architect CMIA and 1093 (9.6%) had anti-nucleocapsid IgG detected. Within 9191 samples tested by both platforms, 788 (8.6%) had IgG detected by both Abbott CMIA and Oxford ELISA, 114 (1.2%) by only Abbott CMIA and 106 (1.2%) by the Oxford ELISA only.

More individuals have had Covid-19 than are identified using immunoassays calibrated on PCR-positive cases enriched for hospitalised patients. We show, using two different immunoassays that target different SARS-CoV-2 antigens, that intermediate assay results, below diagnostic thresholds for positivity, are associated with increased rates of anosmia, ageusia and other symptoms, as well as being more frequent in individuals with a higher risk of Covid-19. Additionally, we show that around 1 in 10 previously symptomatic and PCR-positive HCWs had negative antibody assays when tested a median (IQR) 37 (30–52) days following a first positive PCR test.

We used self-reported anosmia and ageusia rates across antibody readings to estimate how many mild/asymptomatic Covid-19 cases might be missed by current assay thresholds. Although these were subjective and retrospectively reported symptoms, we observed a clear relationship with antibody readings (Fig. 2). Adjusting for rates of anosmia/ageusia in those with high negative (equivocal) antibody readings produces an estimated test sensitivity for the Oxford ELISA of 89.8% (95%CI 86.6–92.8%) rather than the previously reported 99.1% (97.8–99.7%). Similarly, the adjusted Abbott CMIA sensitivity is 79.3% (75.9–82.7%) compared to 92.7% (90.2–94.8%) [8]. However, we observed several individuals reporting anosmia/ageusia with a positive PCR result but negative antibody results, including low negative readings (Fig. 1). Therefore, as a sensitivity analysis we calculated the adjusted sensitivity assuming an example background rate of anosmia/ageusia of 3%, which lead to estimated sensitivities for the Oxford ELISA and Abbott CMIA following mild/asymptomatic infection of 75.6% (95%CI 68.3–82.0%) and 70.5% (67.6–81.1%) respectively. This example background rate is plausible as self-reported rates of anosmia in previous studies vary, e.g. from ~ 1 to 5%, with rates of 3% in the ages typical of the HCWs studied in one series [19]. However, these are studies of established objective anosmia, subjective rates may differ, and the question used for our study asked only about new onset loss of smell or taste since 01 February 2020, i.e. excluded established anosmia. It should also be noted that we analyse antibody readings from assays designed for qualitative use, and so these readings may not change linearly with changes in underlying antibody responses.

Studies of closed communities or households can also be used to put a lower bound on test sensitivity if universal exposure is considered likely. For example, seroprevalence on the US Navy ship the USS Theodore Roosevelt was 60% following a large outbreak, demonstrating immunoassay sensitivity must be at least this amount (a further 6% were PCR-positive, but antibody negative, possibly reflecting sampling shortly after Covid-19 onset before antibody levels could rise) [14].

Our analysis aims to estimate assay performance in mild infections at current diagnostic thresholds, rather than to propose new thresholds. As such, the specificity of each assay is that previously reported, i.e., ≥99% for both assays [8]. However, if assay thresholds were lowered to improve sensitivity in mild infection, this would also result in reduced specificity. For example, in our dataset 449/9324 (4.8%) of Oxford ELISA results fell within the high-negative (equivocal) range we defined, as did 686/11342 (6.0%) of Abbott results. If the majority of these individuals are uninfected, then using this lower threshold specificity might be expected to fall to < 95%, i.e. unacceptably low in low prevalence settings. As such it may not be possible to detect all previous mild infections serologically while maintaining adequate specificity. Future studies of PCR-confirmed mild infection and pre-pandemic samples are needed to provide sufficient data to propose any adjustment to assay thresholds. In populations with a high pre-test probability of infection, the underlying prevalence may be sufficiently high that lower specificity can be tolerated, and in this setting considering reporting equivocal results, which may or may not prompt testing on second assay may be helpful.

The main limitation of our study is the lack of a PCR test in all HCWs reporting anosmia. We cannot say what proportion of those with equivocal antibody responses and anosmia/ageusia would have had a positive PCR test if tested shortly after infection. Furthermore, given the anosmia/ageusia were self-reported and may have had other causes, our study is limited by the lack of a pre-pandemic control group asked the same question regarding new onset loss of taste or smell between February and May of a previous year. There is therefore uncertainty about the background rate of anosmia/ageusia, and the attributability of this to other respiratory viral causes in this context [20, 21]. Reported anosmia/ageusia may also vary between settings and populations, for example less anosmia is reported in East Asian patients with Covid-19 [22], however we found similar rates of anosmia/ageusia in staff from Asian ethnic groups (predominantly south and south-east Asian) with Covid-19 to staff identifying as white. Additionally, some antibody responses may have been missed, although 99% of all those reporting anosmia/ageusia and providing a date of symptom onset were tested ≥14 days after symptom onset and before antibody responses began to fall substantially. Follow up studies are needed to evaluate the extent to which individuals with symptoms or exposures strongly suggestive of Covid-19 such as anosmia or a PCR-confirmed household contact, but negative antibody results, have other evidence of infection, for example from T cell assays and also to assess the extent of neutralising antibody activity across a range of antibody readings. Specific Covid-19 T cell responses have been reported in seronegative individuals who have been exposed to SARS-CoV-2 [23], including in HCWs from our own setting [24].

Antibody results can be used for multiple applications, including epidemiological and modelling studies. The sensitivity of SARS-CoV-2 serology in those with mild symptoms, i.e. the majority of the infected population, is likely to be lower than previously reported, 90% or less, which is likely to have implications for epidemiological modelling and forecasting. Antibody results may also inform assessments of the risk of an individual being re-infected. However, for individuals with a high pre-test probability of Covid-19, negative serology does not exclude previous infection and possible protective immunity.

Following mild SARS-CoV-2 infection 10–30% of individuals may have negative immunoassay results. While negative SARS-CoV-2 immunoassays following mild infection may not be unexpected, here we are able to quantify that this may be more common than previously appreciated. Our data also highlight, that in contrast to the approach used in evaluations to date [6‐8], samples from individuals with mild and asymptomatic infection should be included in SARS-CoV-2 immunoassay evaluations in sufficient numbers to assess sensitivity in different populations. Our findings have important implications for epidemiological studies and individuals interpreting their antibody results.