Covid-19: Two antibody tests are “highly specific” but vary in sensitivity, evaluations find

Two tests for covid-19 antibodies developed by the drug companies Roche and Abbott are “highly specific” but one was found to have lower sensitivity than was previously reported by the company, evaluation by Public Health England has shown.

Abbott had reported that its assays had a sensitivity of 100% (the true positive rate) 14 days after symptom onset. However, PHE found that it had a sensitivity of 93.4% at 14 days, respectively.

PHE published the reports after experts called for transparency in relation to the evaluation results, so that any claims could be fully scrutinised.1

Now that the evaluations have been made public, experts have said it is not clear whether these tests are any better than other available antibody tests and emphasised that the evaluations did not meet the standard needed for accreditation.

Roche’s Anti-SARS-CoV-2 serology assay was tested between 5 and 7 May by the PHE diagnostic support group,2 which used 93 serum samples from convalescent patients and 472 negative samples. The team reported that all negative samples tested negative, giving a specificity of 100%, while the overall sensitivity was 83.87%, rising to 87.0% at 14 days after onset of symptoms, 87.7% 21 days after, and 100% more than 40 days after.

Abbott’s SARS-CoV-2 IgG kit was tested between 4 and 7 May,3 with 96 samples from convalescent patients and 759 negative samples. The assay was found to be 99.63% specific, with a sensitivity of 93.90% at 14 days after symptom onset, dropping to 93.40% at 21 days and 87.5% at 40 days.

Sheila Bird, formerly programme leader at the MRC Biostatistics Unit at the University of Cambridge and current member of the Royal Statistical Society’s Covid-19 Taskforce, told The BMJ, “PHE’s evaluation does not meet the standards that would be expected for confirmation that these tests meet the criteria set by UK’s Medicines and Healthcare Products Regulatory Authority for accreditation as point-of-care tests.”

She pointed to several limitations, including that an equally powered evaluation of the two tests was not provided; that there was a lack of transparency about the use of repeat serum samples per patient, meaning that readers could not discern how many positive patients contributed to the evaluation of sensitivity; and that the age and sex of the patients whose serum samples were being analysed was not taken into account.

“Demography matters for whether patients develop severe covid-19 disease. Also, since immune responses differ intrinsically with age, the proportion of patients who develop IgG antibodies may differ by age, as may their persistence,” Bird said.

She added, “A level playing field between the Roche and Abbott evaluations was not apparent in terms of whether the tested sera came from patients who had been hospitalised for covid-19 disease or had been only mildly symptomatic. Both matter, especially if IgG antibody tests are to be used for population surveillance, and test performance may be different by symptomatology.”

Meanwhile, Jon Deeks, professor of biostatistics at the University of Birmingham, told The BMJ that “the tests are not game changers.”

He said, “While both tests rarely give false positive results, they can give false negative results. So whilst positive test results clearly indicate that an individual had been infected with covid-19, a negative test result does not rule out possible infection with covid-19 . . . It is not clear that their performance appears superior to that of other antibody tests which are available.”

He also expressed concern over it being widely reported that PHE had approved the tests and that these claims were being used by various companies now offering home sampling kits.

Deputy chief medical officer Jonathan Van Tam previously said in a public briefing that the Roche test had been “validated by PHE” and would be “rapidly rolled out in the days and weeks to come.”

PHE has pointed out that it evaluates diagnostic kits but does not accredit or endorse them.

A spokesperson for Abbott said that the PHE evaluation “again suggests that the Abbott test has excellent performance, accurately identifying those individuals with and without antibodies, and hence provides essential information for epidemiology and clinical use.”

Roche was approached for comment but had not responded at the time of publication.