Evaluation of self-administered antigen testing in a college setting

The investigation was conducted between February 22nd–April 20th, 2021 at a primarily residential college in Georgia, USA. The level of COVID-19 transmission in the surrounding county was moderate at the beginning of the investigation and low at the end [7]. At the beginning of the investigation period when vaccine eligibility was limited 14.6% of the total county population had received at least one dose of COVID-19 vaccine and 9.1% were considered fully vaccinated [7]. On March 25th vaccine eligibility was expanded to everyone ≥ 16 years. By the end of the investigation, 28.7% of the county population had received at least one dose of the COVID-19 vaccine, and 22.2% were considered fully vaccinated [7]. The vaccination rate at the college at the beginning of the investigation was 19.8% of staff and 5.4% of students. Mass vaccination events were held on campus the weeks of March 22nd and April 19th; by the end of the investigation, vaccination rates among staff and students were 75.6% and 62.2%, respectively.

Most students (90%) attending the college live in residence halls on the controlled-access campus. The school instituted several COVID-19 prevention strategies, including mask mandates, physical distancing in classrooms, enhanced facility cleaning, limiting campus access, and encouraging students to form small, mutually exclusive social “bubbles” of ≤ 5 students. Students attending the college (N = 1982), staff/faculty (N = 643), and affiliates (e.g., spouses of staff) associated with the college were eligible for the investigation. Any person who was in quarantine due to SARS-CoV-2 exposure or who had COVID-19 symptoms was not eligible for testing. In the two weeks before the investigation, seven COVID-19 cases were reported on campus (prevalence: 133.3/100,000).

The Quidel QuickVue At-Home COVID-19 Test (hereafter self-administered antigen test) is a lateral flow assay that relies on qualitative detection of the SARS-CoV-2 nucleocapsid protein. Results are read visually on a test strip after ten minutes. At the time this investigation began this test was under FDA review for Emergency Use Authorization (EUA) for self-administration and received an EUA in March 2021 [8].

Participants were provided with self-administered antigen tests and manufacturer instructions and instructed to test twice weekly for seven consecutive weeks. During week three, nasal swabs for paired rRT-PCR testing were collected on the same day as antigen testing (Fig. 1).

Participants were asked to submit their self-administered antigen test results and a photo of their test strip through the college’s symptom screening online reporting system. Participants who tested positive by self-administered antigen test were counseled to obtain same-day confirmatory testing by RT-PCR and to self-isolate.

To evaluate the number of SARS-CoV-2 infections the serial twice-weekly self-administered antigen testing missed, we examined SARS-CoV-2 specific IgG seroconversion, with paired baseline (week 1) and endline (week nine) serology testing (Fig. 1). The college provided participant COVID-19 vaccination records and positive SARS-CoV-2 test results within 90 days prior to and during the investigation.

To delineate infection versus vaccine induced antibody responses among vaccinated participants, plasma specimens were analyzed with V-plex SARS-CoV-2 panel 2 IgG kit (Meso Scale Diagnostics [MSD]) as directed by the manufacturer. This multiplex assay detects antibodies against SARS-CoV-2 nucleocapsid (N), spike (S), and the spike receptor binding domain (RBD). Specimens that were positive for nucleocapsid antibodies, in addition to S and/or RBD, were classified as having infection induced antibody response; endline specimens positive only for S and/or RBD were classified as having vaccination induced antibody response only (i.e., no evidence of infection).

During the third week of the investigation, participants were asked to submit a nasal swab specimen for rRT-PCR testing and to take a self-administered antigen test on the same day. Participants were directed to self-collect a bi-lateral anterior nasal swab for rRT-PCR testing. These swabs were stored in tubes with viral transport media in coolers with cold packs and transported daily to the Georgia Public Health Lab and stored at 4 °C until testing. Within 48 h of collection, specimen nucleic acid was isolated using the Perkin Elmer Chemagic Viral DNA/RNA assay on the Perkin Elmer Chemagic 360 instrument (Perkin-Elmer) and analyzed using the CDC Influenza SARS-2 (Flu SC2) Multiplex assay according to the manufacturer’s instructions for use [9]. Residual frozen specimens positive for SARS-CoV-2 by either test underwent viral culture. Specimens were cultured by limiting dilution in Vero E6/TMPRSS2 cells and were observed daily for cytopathic effects in 96-well plates, described previously [10]. Supernatant from cells that exhibited cytopathic effects was harvested and tested by rRT-PCR using the Flu SC2 Multiplex assay to confirm the presence of SARS CoV-2. A specimen was culture-positive if the first viral passage had a cycle threshold (Ct) value at least two Ct values lower than the clinical specimen.

Participants were administered surveys during weeks two and eight of the investigation to assess characteristics and acceptability of twice-weekly self-administered antigen testing via an online survey platform (Qualtrics). Participation in the 2nd survey was low and results are not reported. Survey questions are included in Additional file 1.

Analyses were conducted using SAS (version 9.4; SAS Institute) and verified by an independent analyst. Frequencies of descriptive variables were calculated among groups of participants. Sensitivity, specificity, positive predictive value, and negative predictive value were calculated for paired self-administered antigen testing compared to rRT-PCR testing and seroconversion. Percentages were calculated for testing circumstances and acceptability of testing methods from the surveys.

Baseline serology results were available for 1081 participants, of which 34.3% (N = 371) tested positive for antibodies to the SARS-CoV-2 spike protein (Fig. 2). Previous infections since August 2020 were reported for 204 participants with baseline serology available; 181 (88.7%) tested positive and 23 (11.3%) tested negative. In addition, 103 participants received at least one vaccine dose prior to baseline serum specimen collection; 98 had detectable anti-S antibodies (95.2%; including 16 who also had a history of past infection), and the remaining five vaccinated participants who tested antibody negative received their first dose less than a week prior. There were 108 participants who were baseline antibody positive who did not have a record of previous infection or vaccination.

Results from ≥ 1 self-administered antigen test were reported by 1347 participants (Table 1); 983 students (73.0%), 363 staff (27.0%), and one affiliate (0.1%). A total of 9971 self-administered antigen test results were reported. Although participants were asked to test twice weekly for seven weeks, only 217 reported results from ≥ 14 tests (16.1%). The distribution of the total number of tests varied by baseline antibody status (Fig. 3a). Among 155 participants who were baseline antibody positive, most (~ 86%) reported taking ≤ 4 tests, particularly after results of baseline antibody testing were returned (March 8–14; Fig. 3b). Among participants who were either baseline antibody negative (n = 503) or who did not participate in baseline testing (n = 689), the median number of tests taken was eight. Participation attrition was observed among these groups as well, with the number of self-administered antigen test results reported peaking during the first two weeks of March 2021 and then steadily decreasing (Fig. 3b).

A total of 11 participants reported positive self-administered antigen test results, all of whom received RT-PCR testing within 24 h; three infections were confirmed through this testing (27%). Baseline antibody results were available for 4/11; one was seropositive and three were seronegative.

There were 324 individuals seronegative at baseline who participated in self-administered antigen testing and endline antibody testing (Table 1, Fig. 1). Among these individuals, there were three positive self-administered antigen tests, only one of which was confirmed by RT-PCR (positive predictive value [PPV] = 33%). Of the two not confirmed by RT-PCR, 1 person tested antibody negative at endline and the other had antibodies consistent with vaccination. Three additional individuals who never reported a positive self-administered antigen test or had any record with the school of COVID-19 vaccination were antibody positive at endline; these participants reported one, two, and seven total self-administered antigen test results. However, two of these individuals tested positive only for spike protein antibodies, and not nucleocapsid antibodies, suggesting they may have been vaccinated and this information was not provided to the school. The third individual had negative results for the spike and nucleocapsid protein antibodies using the multiplex assay, but they were positive by the qualitative VITROS anti-SARS-CoV-2 total antibody in vitro diagnostic test; the reason behind these inconsistent results is not clear. An additional four individuals who never reported a positive self-administered antigen test and who received at least one dose of COVID-19 vaccination were positive for spike and nucleocapsid protein antibodies, consistent with a SARS CoV-2 infection. Based on the five COVID-19 infections with complete and consistent data, the sensitivity of the implemented serial self-administered antigen testing was 20% (1/5). When we considered the subgroup of 186 participants who reported results from ≥ 10 self-administered antigen tests, there were no positive antigen tests and there were three missed infections based on seroconversion.

There were paired self-administered antigen and rRT-PCR specimens for 665 participants, 449 (67.5%) students and 216 (32.5%) faculty or staff (Table 1). Four rRT-PCR specimens were positive for SARS-CoV-2 (prevalence = 0.60%) (Table 2). Paired self-administered antigen tests were positive for two of these individuals (sensitivity = 50%). No specimens were positive by self-administered antigen test and negative by rRT-PCR (specificity = 100%). The PPV of a single self-administered antigen test compared to rRT-PCR was 100% and the negative predictive value was 99.7%. No specimens were positive for Influenza A or B.

The Ct values for the two concordant specimens were 18.5 and 19.4, while for discordant specimens they were 33.5 and 35.6. Culturable virus was detected in 1/4 (25%) rRT-PCR positive specimens. This specimen had a Ct value of 18.5 and was also positive by the self-administered antigen test (sensitivity = 100% among culture positive specimens). Both participants with concordant specimens developed symptoms; the two participants with discordant specimens were asymptomatic.

Among participants who submitted antigen test results, 753 (641 students, 112 staff) completed the survey (Table 3). Most students reported taking the self-administered tests alone in their dorm rooms, and few reported problems taking the test (0.5%). Almost half of students reported RT-PCR testing with a nasal swab specimens as their preferred COVID-19 test type (46.3%), but approximately 1/3 of students reported preferring self-administered antigen testing (35.7%). Results were similar for staff, who were also likely to take the test alone and reported few problems taking the test (Table 3).

The acceptability of the self-administered test was high among students and staff (Table 3). More than 2/3 reported agreeing or completely agreeing with the statement “I like using home antigen testing every week to reduce the spread of COVID-19”. More than 85% of respondents agreed or completely agreed that the amount of time spent completing self-administered antigen testing was manageable. The percentage who agreed or completely agreed that serial self-administered testing would have an impact on reducing the spread of COVID-19 at their college was 66.4% for students and 65.8% for staff. Similar results were observed for the impact on reducing spread of COVID-19 in the community. Staff were less likely than students to report that self-administered antigen testing would make them less likely to catch or spread COVID-19, but more than half of students and staff agreed or completely agreed that weekly self-administered antigen testing makes them feel more comfortable attending classes or working on campus in person.