Analytical performance of the point-of-care BIOSYNEX COVID-19 Ag BSS for the detection of SARS‐CoV‐2 nucleocapsid protein in nasopharyngeal swabs: a prospective field evaluation during the COVID-19 third wave in France

The 2019 coronavirus pandemic (COVID-19) continues to spread worldwide. The effective isolation and early treatment of patients infected by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) require rapid, accurate, and straightforward diagnostic tools.

While currently recommended nucleic acid amplification tests (NAAT), such as real-time reverse transcription-polymerase chain reaction (rtRT-PCR) assays, remain the gold standard cornerstone for the diagnosis of SARS‐CoV‐2 infection [1, 2], viral antigens can be detected using immunological methods [2‐4]. Indeed, conducting rtRT-PCR is expensive, time-consuming, and requires special equipment and qualified operators. Point-of-care antigen-detecting rapid diagnostic tests (Ag-RDT) constitute simple and less expensive alternative tests [3]. Ag-RDT relies on direct detection of SARS-CoV-2 viral proteins in nasal swabs and other respiratory secretions. The N-nucleocapsid protein is frequently targeted because of its relative abundance and conserved structure, or other viral proteins such as the spike protein [4]. Most Ag-RDTs rely on sandwich catching using anti-SARS-CoV-2 monoclonal antibodies to detect viral antigens in the simple-to-use lateral flow immunoassay format allowing results in < 30 min. However, significant variability has been reported about their diagnostic performance and a lack of external validation for many available tests, which still require clinical validation [5‐9].

Our study aimed to evaluate the qualitative membrane-based immunochromatographic BIOSYNEX COVID-19 Ag BSS Ag-RDT (Biosynex Swiss SA, Freiburg, Switzerland; reference SW40006; abbreviated by BIOSYNEX Ag-RDT) using monoclonal antibodies detecting SARS-CoV-2 N-nucleocapsid protein to diagnose COVID‐19 from prospectively collected nasopharyngeal secretion samples in adults living in the Paris region throughout the third wave of the COVID-19 epidemic in France.

Paired swab samples were obtained from 967 participants, including 741 from site A and 226 from site B (Table 1). Participants ranged in age from 18 to 95 (median = 34 years). The main reasons for testing were air travel (35.6%), contact-case exposure of an individual infected with SARS-CoV-2 (35.1%), suspected COVID-19 (n = 212, 21.9%), preoperative assessment (4.4%), and control of SARS-CoV-2 infection in the previous 30 days (3.0%). The majority (722/967, 74.7%) of included persons were asymptomatic, while a minority (245/967, 25.3%) reported at least one COVID-19-related symptom [including 212 suspected COVID-19 cases, 29 (8.5%) contact cases, 3 (0.9%) travelers, and 1 (3.0%) patient with a recent history of COVID-19]. The median symptom duration before sampling was four days (range, 0–20 days). All comparisons between positive and negative Ag-RDT and rtRT-PCR testing results for both sites and all other variables did not achieve statistical significance (not shown).

Among the 148 positive samples using the gold standard rtRT-PCR, 146 were positive for the three gene targets, and two were positive for only E and N genes. The mean ± SD of the C_t values were 26.1 ± 4.4 arbitrary units (a.u.) for the E gene, 26.5 ± 5.0 a.u. for the RdRP gene, and 26.9 ± 5.1 a.u. for the N gene.

The vast majority (114/123, 92.7%) of positive results were visible in the window of the cassette of the Ag-RDT within the first 5 min. Table 2 shows the test results and primary performance characteristics of the BIOSYNEX Ag-RDT compared with the reference rtRT‐PCR in the study population according to COVID-19-compatible symptoms. Using rtRT-PCR as the standard, three false-positive BIOSYNEX Ag-RDT results occurred among specimens from asymptomatic individuals (n = 2) or symptomatic patients (n = 1). Of the 148 rtRT-PCR positive results, 27 (18.2%) were false-negative BIOSYNEX Ag-RDT (23 specimens from asymptomatic persons and 4 specimens from symptomatic patients). Overall, the BIOSYNEX Ag-RDT showed high sensitivity (81.8%), specificity (99.6%), PPV (96.6%), and NPV (97.5%). Among symptomatic patients, sensitivity was 95.0%, specificity was 99.4%, PPV was 95.6%, and NPV was 96.3% (Table 2). Within 7 days from symptom beginning, the BIOSYNEX Ag-RDT showed a sensitivity of 96.6%, a specificity of 99.4%, whereas the PPV and NPV were 95.7% and 99.4%, respectively.

The Table 3 shows the analytical results based on the level of viral excretion assessed by the N gene C_t values using the reference rtRT‐PCR. Overall, the BIOSYNEX Ag-RDT showed high agreement (97.0%), reliability using Cohen’s κ coefficient (0.87), and accuracy using Youden’s J index (81.6%) to detect SARS-CoV-2.

In case of high or very high viral loads (C_t ≤ 33), the BIOSYNEX Ag-RDT had a good analytical performance (sensitivities between 83.3% and 100.0%, specificities of 99.8%, PPV between 98.3% and 98.6%, and NPV between 97.7% and 100.0%). In case of low or very low viral loads (C_t > 33), the sensitivity of the BIOSYNEX Ag-RDT had reduced analytical performance (sensitivity of only 55.2%), while its specificity remained high (98.8%). Similar observations were made when the C_t values of the E or ORF1ab gene targets were chosen for stratification of viral load in clinical samples (data not shown).

We evaluated the analytical performance of the novel point-of-care BIOSYNEX Ag-RDT compared to multiplex rtRT-PCR as gold standard for detecting SARS-CoV-2 RNA in a real-life setting. In this study, the sensitivity of the BIOSYNEX Ag-RDT was lower among specimens from asymptomatic persons (79.4%) than among specimens from symptomatic patients (95.0%). It was high in patients with suspected COVID-19 (96.8%). Specificity (> 99.0%) was high in specimens from both asymptomatic individuals and symptomatic patients. The prevalence of SARS-CoV-2 RNA-positive rt-RT-PCR results in this population was 15.3% overall, 9.4% for asymptomatic individuals, and 32.6% for symptomatic patients. The estimated PPVs and NPVs of the BIOSYNEX Ag-RDT were elevated in all groups of participants. However, administering the Ag-RDT in low prevalence settings will likely result in lower predictive values. In the event of significant viral excretion (i.e., N gene C_t values below 33 based on reference rtRT-PCR), the BIOSYNEX Ag-RDT showed high sensitivity (from 83.3% to 100.0%) and specificity (> 99.0%) for SARS-CoV-2 RNA detection. Concordance, reliability, as well as accuracy were great with the reference assay and PPVs and NPVs above 97.0%. However, the sensitivity of the study Ag-RDT dropped to 55.2% with low or very low viral shedding (C_t > 33). Together, these observations demonstrated the high analytical performance of the BIOSYNEX Ag-RDT. This performance made it suitable for use as point-of-care Ag-RDT in various hospital and non-hospital settings where a rapid diagnosis of SARS-CoV-2 is necessary. Although less sensitive than RT-PCR, the BIOSYNEX Ag-RDT could be beneficial by obtaining quick results, ease of use, and independence from existing laboratory structures. Testing criteria focusing on patients during the early onset of symptoms could further increase its diagnostic value.

The sensitivity of the BIOSYNEX Ag-RDT was 81.8% overall, and the positive detection rate was comparable to the rtRT-PCR in the majority (88.2%) of subjects with C_t ≤ 33. False-negative test results of 12/14 (85.7%) subjects with significant viral excretion (C_t ≤ 33) were asymptomatic, although conflicting evidence exists about the relationship between symptom severity and viral shedding [12]. False-positive test results were rarely observed, providing 99.6%-specificity, exceeding the performance recommended by the World Health Organization (WHO) [13]. False-positive results have been reported as well in other antigen tests [14‐16]. False positivity could be associated with high viscosity of tested specimen samples as well as interference with mucosal antibodies [17].

Finally, the BIOSYNEX Ag-RDT meets the current WHO criteria which stipulate that Ag-RDTs for SARS-CoV-2 antigen detection must have a sensitivity greater than 80% and a specificity greater than 97% (97%–100%) [13]. Furthermore, analytical performances comparable to those in our study Ag-RDT were previously reported for some Ag-RDTs in lateral flow immunoassay format [7, 9, 14, 18‐28], while several studies have reported much lower sensitivity levels contrasting with consistently high specificity [3, 29‐34]. In addition, the BIOSYNEX Ag-RDT also fulfilled the current recommendations of the French High Authority of Health (Haute Autorité de santé, Saint-Denis, France) for a screening Ag-RTD stating that, at minimum, Ag-RDTs would need to correctly identify significant proportions of symptomatic patients (sensitivity ≥ 80%) as well as asymptomatic individuals (sensitivity ≥ 50%) and have high specificity (≥ 90%) [35].

We analyzed our results based on the estimated viral load in SARS-CoV-2 in the samples. There is an ongoing debate about the C_t value corresponding to the threshold of infectivity (i.e., patient considered as contagious) [7, 36, 37]. La Scola et al. found that patients with C_t values > 33 are not infectious because of the low number of positive cultures [38]. The Centers for Disease Control and Prevention (CDC), Atlanta, USA, propose a C_t cut-off value of 33 as a marker for contagiousness [39], and stress that C_t values ≤ 20 correspond to very high viral excretion [7, 36, 40, 41]. Our results confirm that the analytical performances of the BIOSYNEX Ag-RDT were much better in specimens with a high viral load. These observations demonstrate the capability of the BIOSYNEX Ag-RDT as a rapid rule-in test for COVID-19 with samples at high viral load in symptomatic patients, for example, and raise caution about its use as a singular rule-out test, especially in samples with lower viral loads.

Our study has several strengths. All samples were collected from one nasopharynx with flocked swabs, optimal for evaluating Ag-RDT clinical performances in our study. The Ag-RDT and reference rtRT-PCR were carried out in parallel. The study population included various situations outside the hospital setting, with mostly young adults without comorbidities who had typical and mild COVID-19 symptoms when being symptomatic.

The study presents also some limitations. Participants may have inadvertently reported general, non-specific symptoms as COVID-19 compatible symptoms. This investigation evaluated the BIOSYNEX Ag-RDT; the results presented here cannot be generalized to other agencies-authorized SARS-CoV-2 antigen tests. Otherwise, the CDC clarified that C_t values using the rtRT-PCR platform is not a quantitative measure of viral burden in clinical samples and cannot be used to assess whether a person is infectious [42]. Consequently, our stratification of samples according to C_t values of the N gene does not necessarily reflect the actual infectivity of the participants. Finally, higher rate of asymptomatic persons in the study (with lower virus level) could have resulted in decreased sensitivity of the BIOSYNEX Ag-RDT.