Were SARS-CoV-2 self-tests used for their intended purpose? The experience in Belgium

To our knowledge, this is the first evaluation to examine the extent to which and why people are using SARS-CoV-2 self-tests on a national scale. We used the number of reported STs sold at pharmacies as a proxy for the total number of used STs. This is most certainly an underestimation because the reporting was not mandatory and STs were also sold in supermarkets from July 1st, 2022. No figures are collected on the number of STs sold at supermarkets and we can assume that they represent an important number because of the relatively lower prices and barriers compared to tests sold at pharmacies. We also cannot rule out that the ratio of tests sold in supermarkets/pharmacies changed over time. However, we believe that the ratio remained fairly constant. At the time when STs increased in use, they had been available in supermarkets for a long time and people were familiar with both options for buying STs. Our proxy therefore allows detecting trends.

The share of STs in overall testing increased sharply in the last months of 2021. The proportion of all tests that were reported sold STs since that period was 37%. We can therefore conclude that they represented a high share of all testing, especially considering it being an underestimate. The increase coincided with the beginning of the Delta wave and stronger government awareness. We observed a peak in the use of STs during the Christmas period, probably reflecting the government’s strong recommendation to take a self-test before participating in the festivities around Christmas and New Year.

We calculated the percentage of all positive tests that were confirmatory tests of positive STs to estimate the share of self-testing in detecting COVID-19 cases. However, not all people register or confirm their positive self-test. Indeed, in the COVID-19 health survey, 18% reported that they had not confirmed their positive test result. Also, people presenting COVID-like symptoms can access testing free of charge without mentioning a previous positive self-test. Anecdotal evidence appears to indicate that this proportion might have further increased, and that confirmation is mostly asked when a sick leave certificate is needed. Thus, the number of reported positive STs is also likely an underestimate. Also, the ratio of reported positive STs over the number of sold tests is lower than the positivity rate reported by survey respondents, supporting the underreporting hypothesis. Still, in the period 17 January – 3 July 2022, 18% of all confirmed cases originated from a positive self-test. Hence, STs appear to have had an important share in detecting COVID-19 cases, at least from January 2022 onwards. The most plausible reason is that starting then, high-risk contacts and travelers who came from high-risk areas were no longer systematically tested. This significantly decreased the number of provider-administered tests and likely increased the use of self-testing after a high-risk contact.

We found that both the number of STs sold, and the number of positive STs followed largely the same trend as provider testing, with increases during periods when the incidence of COVID-19 was highest. We do not know why people were more likely to self-test during those periods, but the data from the two surveys of a representative sample of the adult population both point toward frequent use when they were symptomatic or had a high-risk contact. This was consistent with anecdotal information from health professionals and the findings of another online study of Belgian citizens conducted in early December 2021 [9]. In that study, more than 56% of the participants had taken a self-test or had a family member who had used one. Of the participants who had not yet used a self-test, 61% reported that they had not yet experienced symptoms that prompted them to use one. Thus, it is hypothesized that people are more likely to use a self-test during waves of high incidence because more people experience symptoms and have high-risk contacts. This is further corroborated by the equal trend in the number of sold STs and the trends in number of provider-administered tests in symptomatic people and high-risk contacts, and not for other indications. While it is possible that people self-test more during the waves for the formal indications, we consider it unlikely that this would cause such an increase. The relatively high percentage of positive STs reported by survey participants (17% and 7%) also seem to indicate use for indications with a higher probability of infection than the formal self-test indications. The ratio of reported positive STs over the number of sold tests (on average 0.04) is much lower than the positivity rates in provider-tested symptomatic people and high-risk contacts (on average 26% and 14%, respectively). However, this could be a result of the underreporting of positive STs.

We can therefore conclude that the Belgian government’s guidelines, which explicitly emphasized that self-testing should not be used when having symptoms, and initially also not after a high-risk contact, were not followed. Why people used their own judgement rather than the guidelines we do not know. We can only assume it is a combination of easier access/time savings and unfamiliarity with the guidelines. Which of these factors is predominant requires further investigation.

If and how this affected the isolation and contact tracing strategy, and thereby the control of the epidemic, remains unknown. On the one hand, self-testing comes at the cost of loss of accuracy. Several studies have shown that provider-administered RATs are less sensitive than RT-PCR, particularly when the viral load is low [10‐13]. Moreover, studies with point-of-care tests for other infectious diseases already highlighted the importance of trained staff [14‐16]. The few studies that have assessed the accuracy of self-administered RATs have indeed shown further decreases in sensitivity [17‐20]. Furthermore, while the specificity of RATs is high, it is less than 100%, resulting in a low positive predictive value when the positivity rate is very low [21‐25]. For example, with a specificity of 99.5% and a sensitivity of 85%, the positive predictive value is only 77.6% at a prevalence of 2.0%. Another disadvantage is the difficulty in reliable reporting of the results of STs.

The rationale for the restricted self-test indications was indeed a concern for false-negative results due to the lower sensitivity of STs, and false positive results, especially in a context of low virus circulation. If a false-negative self-test result would prevent people with symptoms or high-risk contacts from seeking formal testing, positive cases would go undetected. This would then negatively impact isolation and contact tracing strategies and interfere with epidemic monitoring. These concerns were also shared by international agencies [26]. In people without COVID-symptoms, a false-negative result could lead to abandoning of preventive measures, such as physical distancing and mask-wearing, and thus infection of others. Indeed, in the APB study, a significant proportion of respondents were found not to know how to behave after a negative test result. This is consistent with other international studies that showed that at-home COVID-19 self-test kit users may not follow the recommendations when they test negative. In a randomized trial in the US, for example, 33% of people with a high pre-test probability (having symptoms or after a high-risk contact) said they would not quarantine if the self-test were negative, while this was recommended by the authorities [27].

It can also be speculated that people who tested positive with a self-test and did not report it, were less compliant with the isolation requirement than those who were provider-tested. However, the results of the APB survey seem to indicate that people are well aware that they should isolate if the self-test is positive, as other research has also shown [27].

On the other hand, self-testing has also some major benefits and the use of STs when having symptoms or after a high-risk contact might have had a positive effect as well [28‐30]. First, self-testing lowers the threshold for getting tested, a threshold that has shown to be often high. For example, an analysis of the data of the first COVID-19 wave in France estimated that only 31% of individuals with COVID-19-like symptoms consulted a doctor [31]. This is confirmed by seroprevalence studies that show that a high percentage of people with antibodies against SARS-CoV-2 (including when having had COVID-like symptoms) report not having consulted a health care provider or having been tested [32, 33]. Many of the people who self-tested might not have been tested at all if STs had not been available.

Second, STs provide an immediate result and, if testing positive, people will thus isolate sooner. Furthermore, the reduced sensitivity is less a problem in people with a recent onset of symptoms when viral load is high. Several countries expanded therefore their indications for self-testing in late 2021/early 2022 to include when one has (mild) symptoms [34]. Reduced sensitivity is important for high-risk contacts who do not have symptoms and in whom the false negative rate is higher. The period when self-testing was widely used and high-risk contacts had to be tested with PCR was, however, relatively short (November 2021-March 2022).

It is thus possible that the benefits of reduced barriers to testing and a faster result outweigh the risks of reduced sensitivity. In addition, self-testing has also important societal benefits, such as a low cost and relieving overburdened health providers from collecting specimens.

Our evaluation has several limitations. Online surveys have a substantial risk of selection and reporting bias. The survey by DayOne used an existing database of a representative sample of Belgian adults and we believe this minimizes potential selection bias. The Sciensano COVID-19 health surveys were non-probability web surveys because the data were needed quickly. The survey samples were therefore prone to biased estimates as they relied on self-selection and excluded people without internet access or skills. However, Sciensano minimized this bias by setting up partnerships with trustworthy organizations, using a diverse recruitment strategy including multiple platforms to reach different subsets of the population, and assessing the results after every survey and making extra efforts during the next survey when realizing that some population groups were not enough represented. In addition, post-stratification weights were applied in the analysis to partly correct for the sampling bias. Both surveys used closed-ended questions with a fixed order of response options which might have biased the results through primacy effects. However, we do not believe that these potential biases can fully explain the high proportion of using STs when symptomatic or after a high-risk contact. The similarity in the responses between the two surveys further strengthens our conclusion that these were indeed the main reason why people use STs. We did not have data on the number of STs used and the number of negative results and had to use proxies. Nevertheless, we do believe that the bias introduced by these proxies was consistent over time and that it therefore did not have an effect on the trends.