Introduction
Non-PCR-based point-of-care testing (POCT) has been widely introduced into national test strategies and independently evaluated using different settings and approaches [
1‐
4]. While the detection and quantification of SARS-CoV-2 genomes by nucleic acid amplification testing represent the gold standard in diagnostic laboratories, the reagent supply chain can be limiting and turnaround times for PCR testing prolonged, making an effective clinical and outbreak management difficult at times, especially when incidences are high.
To add to the repertoire of quality-controlled, laboratory-based SARS-CoV-2 testing from respiratory material, several companies have recently introduced automated, quantitative SARS-CoV-2 Ag assays. First field studies indicate that these medium- to high-throughput assays’ sensitivities ranged from 40 to 93% and specificities between 91 and 100% [
5‐
16]. While assay specificities were frequently found to be relatively high [
5‐
16], assay sensitivities according to international and national guidelines for rapid Ag tests in general, requiring positive rate percentages ≥ 80, have frequently not been met [
17,
18]. Further, the studies published so far did not compare different automated SARS-CoV-2 Ag assays among each other, which will be critical for diagnostic laboratories seeking to implement such assays.
Since the beginning of 2021, VOCs have started to dominate the pandemic in different parts of the world. For Germany, the Robert-Koch Institute estimated that in the middle of April 2021 ca. 90% of newly diagnosed SARS-CoV-2 infections were caused by VOC Alpha (pangolin lineage B.1.1.7), in line with surveillance reports from many other countries [
19,
20]. In South Africa, the VOC Beta (pangolin lineage B1.351) has been driving the second wave in late 2020/early 2021 [
21] and has been detected around the globe, including many European countries [
19,
22]. It is largely unclear whether SARS-CoV-2 Ag assays are able to sensitively detect VOCs, which besides their defining mutations in spike also carry up to four amino acid mutations in the respective nucleocapsid proteins, i.e., Alpha: D3L, R203K, G204R, S235F; Beta: T205I; Gamma: P80R, R203K, G204R; Delta: D63G, R203M, G215C, D377Y [
23].
The aim of our study was to compare four different automated SARS-CoV-2 Ag assays, which all detect the nucleoprotein of SARS-CoV-2 applying different technologies, for their analytical performance. In parallel, a widely used POCT rapid Ag test was used with an identical set of samples. We evaluated respiratory samples collected from patients at the University Hospital of Munich (LMU Klinikum) at the end of the second pandemic wave in Germany (February–March 2021) as well as patient-derived isolates, including EU1 (pangolin lineage B.1.177), VOC Alpha (B.1.1.7) and VOC Beta (B.1.351), which had been expanded in cell culture in a biosafety level 3 laboratory.
Discussion
In the current study, we evaluated the performance of four commercial, automated SARS-CoV-2 Ag tests that were launched on the European market in early 2021 for the quantitative detection of SARS-CoV-2 in respiratory samples from COVID-19 patients hospitalized at the LMU Klinikum and three LMU München teaching hospitals as well as on cell culture-expanded clinical isolates, including VOCs. We assessed the sensitivity and specificity as well as method-based test characteristics of these four laboratory-based Ag assays and also compared them to a widely used POCT.
The minimum requirements of performance recommended by international organizations for rapid SARS-CoV-2 Ag tests (sensitivity ≥ 80%, specificity > 97%) are certainly not simply transferable to automated Ag assays. Since this study was conducted at a time during the pandemic when the number of newly infected individuals in Germany was declining at the end of the second pandemic wave, it was difficult to include primarily fresh respiratory samples from patients at the time of initial diagnosis and presumably high viral load. Therefore, the low clinical sensitivities, which only have relevance in the specific cohort investigated, are not discussed further. The analytical sensitivity (95% detection limit) of the four automated SARS-CoV-2 Ag tests examined ranged from 420,000 to 25,000,000 Geq/ml. In comparison, the detection limit of the Roche Rapid Ag Test was 9,300,000 Geq/ml, which is also in good agreement with a recent report [
1].
This indicates that automated Ag testing does not necessarily go along with an increased sensitivity. However, it is evident that test indication and overall usefulness of automated Ag tests cannot be generalized. Rather, cutoffs must be individually adapted to and evaluated for the specific diagnostic focus. The ROC analyses in this study demonstrate impressively that lowering the cutoff for either specificity or sensitivity is not suitable for every test. In the case of CLIA, ELISA and ECLIA, it is evident that cutoffs have already been set by the manufacturers just above the lower asymptote of negative results. Only CLEIA shows a wide dynamic range that allows marked variations. For this assay, the manufacturer also explicitly recommends adjusting the cutoff to the requirements of the specific laboratory performing the assay. Recently, Häuser et al
. performed similar analyses on the automated CLIA [
12]. In their study cohort of more than 130 COVID-19 patients hospitalized in the period December 9, 2020 to January 29, 2021, the overall analytical sensitivity was 40.2% at a specificity of 100%, while lowering the assay cutoff from 200 to 100 arbitrary units (AU) per mL increased the sensitivity to 49.7% while decreasing the specificity to 98.3%.Possible areas of application of automated Ag tests are the screening of asymptomatic patient groups with regard to possible infectivity at the time of testing during a phase of increased incidence or testing for de-isolation of known PCR-positive patients in the hospital setting in addition to local symptoms-dependent hygiene concepts.
The specificity of an Ag test determines the number of false-positive results depending on the pre-test probability. Independent studies and meta-analyses [
2] now show that high specificity can be achieved with SARS-CoV-2 rapid Ag tests [
2,
3]. In our current analysis, the RAT used confirms this observation. Independent testing and summaries of automated Ag test specificities are also important, as substantial variability can occur due to individually definable cutoff values for tests that have a quantitative or semi-quantitative output measure. In our current study, all Ag tests investigated were able to stay above the limit required by the WHO for the specificity of SARS-CoV-2 rapid Ag tests of more than 97%. To our knowledge, there is no corresponding recommendation for automated laboratory Ag tests from any official institution. Laboratory operators should responsibly and proactively plan the use of automated Ag tests depending on local incidence and circulating variants, the latter potentially also affecting assay performance.
Undoubtedly, any positive Ag test, POCT or laboratory based, has to be confirmed by PCR. Specificities of around 97%, as found in our current study for the CLEIA and ECLIA, can lead to significant rates of false-positive results and corresponding burden on laboratory workflows due to a need for PCR retesting with associated delays in the time to result.
Recent analyses showed that individuals infected with VOC Alpha had a viral load at primary diagnosis of one order of magnitude higher than individuals infected with a non-VOC [
36‐
41]. We were able to observe this trend between these two patient groups, despite the small number of COVID-19 cases. As a consequence, Alpha VOC-infected patients may be more likely to be recognized by an Ag test early during the course of infection. Unfortunately, we were unable to demonstrate such an effect, likely due to the small number of primary diagnoses. In this context, however, it is more important to investigate whether different virus variants—especially VOCs—can be detected by automated Ag tests with comparable sensitivity. Changes in the virus genome can lead to non-synonymous amino acid exchanges. In the spike protein, for example, the polymorphisms K417N and E484K in VOC Beta can reduce the efficacy of neutralizing antibodies. The nucleocapsid gene harbors a mutational hotspot in the region around amino acid positions 202–205, which is probably an important phosphorylation site for packaging. While these SNPs are not VOC specific, almost all current SARS-CoV-2 lineages carry mutations in this area of the genome compared to the Wuhan reference sequence. In addition to these four amino acid positions, there are five other amino acids in nucleocapsid that can differ between VOCs and the early pandemic isolates (D3, D63, P80, G215, S235 and D377). Mutations in the nucleocapsid of the major VOCs and the wild-type virus used in this study are shown in Table
4 [
23].
Table 4
Summary of known amino acid mutations with > 75% prevalence in VOCs Alpha, Beta, Gamma and Delta and EU1/B.1.177 from March 2020 used in this study with non-synonymous amino acid substitutions at the indicated amino acid positions in the nucleocapsid of SARS-CoV-2
EU1 | | | | | | | | V | | |
Alpha | L | | | K | R | | | | F | |
Beta | | | | | | I | | | | |
Gamma | | | R | K | R | | | | | |
Delta | | G | | M | | | C | | | Y |
Four of the five assays reported in this study are based on monoclonal antibodies that bind to the viral nucleocapsid protein in the immunoassay. Only CLIA uses polyclonal antibodies for Ag detection and Fujirebio states for the CLEIA that they use a mixture of different monoclonal antibodies. Although the stage of infection and pre-analytical conditions may have the greatest impact on a false-negative Ag test result, it is essential to also investigate failures in the binding ability of the antibodies used in these assays due to VOC-specific mutations. An immunoassay that includes polyclonal antibodies certainly has a conceptual advantage of reliably detecting multiple epitopes on the nucleocapsid protein even if mutations occur at a single site, whereas the use of a monoclonal antibody in the assay design is more vulnerable to drastic loss of binding and assay failure [
42]. Since different manufacturers are likely to have the diagnostic antibodies bind in different domains of the nucleocapsid protein, this survey must be repeated for each test on the market. It can be assumed that Ag test antibodies react to more conserved domains of the nucleocapsid, but so far this information has not been disclosed by the manufacturers. In our experiments with supernatants from cell culture of primary patient clinical isolates of Alpha and Beta, we did not observe a marked drop in test sensitivity. This result is in line with the findings of a recent publication in which no difference was found between cell culture supernatants of Alpha and Beta VOC isolates and non-VOC isolates for detection rates in RATs [
43].
In summary, automated assays for the detection of SARS-CoV-2 nucleocapsid protein from respiratory samples significantly differ in their assay dynamics, with marked differences in analytical sensitivity. However, sensitivity is not consistently higher in automated, laboratory-based assays compared to a widely used RAT and no marked difference in detecting a SARS-CoV-2 isolate from early 2020 relative to VOC Alpha and Beta could be established. Nucleic acid-based testing remains the gold standard for high-performance SARS-CoV-2 detection and quantification in a laboratory setting.
Acknowledgements
We are grateful to members of the Diagnostic Laboratory of Virology at the Max von Pettenkofer Institute for support. We like to thank Volker Fingerle, Nikolaus Ackermann, Alexandra Dangel, Ute Eberle, Mona Hoyos, Lena Mautner, Andreas Sing and the Bavarian SARS-CoV-Public Health Laboratory Epidemiology Team at the LGL for support. We thank Dr. Xaver Sewald for critical discussions.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.