Quantifying the potential value of antigen-detection rapid diagnostic tests for COVID-19: a modelling analysis

We denote an ‘Ag-RDT-led strategy’ as any testing strategy in which an Ag-RDT is the first diagnostic test performed (with the potential for follow-up NAT confirmation). As an illustrative example, we focused on an Ag-RDT with sensitivity and specificity of 80% and 98% respectively, relative to NAT, and costing 5 US$ per test, consistent with recent WHO interim guidance and antigen-detection tests authorised for emergency use by the United States Food and Drug Administration (FDA) [14, 15]. We compared the impact of using an Ag-RDT-led strategy to that of a ‘NAT-based strategy’ in which NAT was the only virological test performed, with reliance on clinical judgement where sufficiently rapid NAT results were not available (see Fig. 1 for a summary of the diagnostic strategies modelled). To inform relevant use case scenarios, we consulted experts from India, South Africa, Nigeria, and Brazil to elicit expert opinion on the ways in which Ag-RDTs could offer value in their own country settings (Additional file 1: Text S1). Based on this input, we selected two use case scenarios, as listed in Table 1: (i) a ‘hospital setting’, where the test is used to support infection control and treatment decisions amongst patients being admitted to hospital with respiratory symptoms and (ii) a ‘community’ setting, where the test is used in decentralised community clinics to identify cases of COVID-19 who should self-isolate. Although Ag-RDTs could also be considered for use in identifying asymptomatic infections, both of these focal scenarios involved testing of only symptomatic individuals.

For both use cases we constructed decision trees (Fig. 1; Additional file 2: Text S2) that represent the diagnostic use of the Ag-RDT, actions taken in response to the test results (or lack of results), and resulting outcomes. For simplicity and transparency, this model does not incorporate transmission dynamics but approximates epidemiological benefits based on the incremental change in the number of days that infectious individuals spend out of isolation; the magnitude of downstream impact would depend on factors, such as the rate of epidemic growth and the contact patterns of symptomatic versus pre- or asymptomatic cases, that are not specified in our model. Our focus is therefore on the direct benefits that would accrue to patients receiving the test and, by extension, their immediate contacts (see right-hand column of Table 1).

Model parameters, listed in Table 2, represent the contextual factors to be examined (including plausible ranges for each), with the aim of identifying those factors that are most influential for the value of an Ag-RDT-led testing strategy relative to NAT-based testing. Our expert consultation highlighted that no standard guidance for whether or how Ag-RDTs should be used in conjunction with NAT existed at the time (e.g. whether NAT should be used to confirm an Ag-RDT negative result). Thus, we also defined and modelled three different options for the adjunctive use of NAT in an Ag-RDT-led algorithm: (i) no confirmation of Ag-RDT results, (ii) NAT confirmation of Ag-RDT negative results, or (iii) NAT confirmation of Ag-RDT positive results). Due to the lower sensitivity of Ag-RDT compared to NAT, confirmation of Ag-RDT negative results with NAT reduces the probability of false negatives, whereas confirmation of Ag-RDT positive results reduces the probability of false positives. The latter is especially important in settings with a low prevalence of COVID-19, where even small shortfalls in specificity can lead to substantial numbers of false positive diagnoses [32].

Although NAT specificity is near 100% for current or recent infection [17], not all NAT-positive cases are necessarily infectious, given the potential to detect unviable viral genetic material after the infection has resolved [17‐19] and for severe symptoms to develop near the end of the infectious period [33]. By contrast, Ag-RDTs may detect only acute, but not recently cleared, infection [34, 35]. These distinctions have significance for the intended purpose of the test: where the purpose is to guide clinical decisions for treatment, knowing the aetiology of severe symptoms is important, regardless of viral antigenic load. On the other hand, where the purpose is early identification of infectious cases, detecting recently cleared infection can detract from the utility of a test. We captured these elements of both NATs and Ag-RDTs by distinguishing ‘acute’ from ‘recent’ infection and assuming that (i) only acute infection is infectious, (ii) NAT is able to detect both acute and recent infection with equal sensitivity and (iii) an Ag-RDT is able to detect only acute infection [34]. We accommodated wide uncertainty in the proportion of patients with acute infection in both the hospital and community settings; considering that viral load is highest just before symptom onset, and that the average COVID-19 patient is hospitalised 3 to 10 days after symptom onset [36], it is possible that a certain proportion of patients are no longer infectious by the time they are hospitalised. As discussed below, although these are useful simplifications for the purpose of the current analysis, these categorisations conceal potentially important complexities relating to temporal and between-individual variation in viral load, infectivity, and detectability by a given test. In the present analysis, we incorporated a parameter for the proportion of those with COVID-19, amongst the population being tested, that are still in the acute phase at the point of testing, allowing this parameter to occupy a wide range of values between 50% and 100%, acknowledging the existing uncertainty (See Table 2).

In the hospital setting, we assumed that the test would guide decisions about whom to isolate and whom to treat with dexamethasone [37] and moreover that all patients (regardless of test result) would receive supportive care such as oxygen support. We assumed a baseline of no COVID-19-specific intervention (i.e. supportive care, but with no NAT or Ag-RDT testing strategy, nor treatment with dexamethasone). We assumed that hospitalised COVID-19 patients have a case fatality rate between 20 and 30% [7] without treatment, and that dexamethasone reduces this by 7–25% (Table 2), consistent with recent study results for corticosteroid treatment of COVID-19 [7]. We assume a 10-day treatment course of dexamethasone [7]. We thus denoted ‘deaths averted’ as the reduction in deaths that would be achieved by a given testing strategy, relative to no intervention.

Similarly, as a simple proxy for the impact of a test on transmission in both hospital and community settings, we first assumed a uniform distribution for the number of infectious days remaining per patient amongst patients presenting with acute infection (Table 2). We then recorded the number of patient days of acute infection that were not spent in isolation, whether because of missed diagnosis or (in the case of NAT) delayed diagnosis without isolation, while awaiting a test result. We denoted ‘infectious person-days averted’ as the reduction that would be achieved by a given testing strategy, relative to no-intervention baseline. For the community setting, we estimated the impact of a test only in terms of infectious person-days averted, as it is likely that most individuals receiving a test in a community setting suffer from mild COVID-19.

We also estimated the cost to the health system of the different interventions. For the hospital setting, we estimated the cost of testing, treatment and isolation. For the community setting, we estimated only the cost of testing.

Using the model illustrated in Fig. 1, we estimated the impact (deaths or infectious person-days averted) and cost of each testing strategy. We stratified Ag-RDT-led strategies by the adjunctive role of NAT in confirmation of a test result (i.e. whether to confirm Ag-RDT-negatives, Ag-RDT-positives, or not at all). For NAT-based strategies, we assumed that only a proportion of eligible individuals receive a NAT result (assuming a broad range of 10–100%), with the remainder managed through clinical judgement alone. For each use case, we sampled all parameters from the uncertainty ranges in Table 2 using Latin Hypercube Sampling. For each sampled set of parameters, we calculated both incremental costs and the incremental primary outcome (deaths averted or infectious person-days that were isolated) under an Ag-RDT-led strategy or a NAT-based strategy, relative to no intervention (that is, a scenario of no testing, nor clinical management of COVID-19). To quantify uncertainty, we calculated uncertainty intervals (UIs) as 2.5th and 97.5th percentiles over 10,000 samples and reported median values as point estimates.

To compare testing strategies, we first estimated the cost per death averted (in the hospital setting), and the cost per infectious person-day isolated (in both hospital and community settings) under NAT-based and Ag-RDT-led strategies. However, we did not aim to determine whether or not an Ag-RDT would be cost-effective, given the uncertainties surrounding appropriate willingness-to-pay thresholds for emergency outbreak response [38]. Instead, we compared the two strategies (Ag-RDT vs NAT) using a simple approach of plotting their relative impact against their relative cost for each sampled set of parameters (see Additional file 3: Fig.S1 for a schematic illustration of the approach). It is important to note that this approach is distinct from a conventional cost-effectiveness plane, as the axes are shown on a relative, rather than a nominal, scale. In the example of deaths, we denoted A_RDT as the deaths averted by Ag-RDT-led testing, relative to no intervention, and likewise for A_NAT. Similarly, we calculated the incremental cost C_RDT of an Ag-RDT-led strategy relative to no intervention, and likewise for C_NAT. We then plotted the relative impact (A_RDT/A_NAT) against the relative incremental cost (C_RDT/C_NAT).

We defined an Ag-RDT as being ‘favourable’ relative to NAT, wherever its use resulted simultaneously in more deaths averted than NAT (i.e. A_RDT > A_NAT), and a lower incurred cost per death averted than NAT (i.e. C_RDT/A_RDT < C_NAT/A_NAT). We defined an Ag-RDT as being ‘non-favourable’ otherwise. We performed corresponding calculations for the outcome of infectious person-days successfully isolated. Our focus in the following analysis is on identifying which circumstances would lead to an Ag-RDT being ‘favourable’ relative to NAT.

Where simulation outputs were equivocal on the favourability of Ag-RDTs, i.e. straddling favourable and non-favourable regions, we evaluated the correlation between each parameter and relevant model outputs using partial rank correlation coefficients (PRCC), to identify those parameters that were most influential on the proportion of a simulation falling in a favourable region. In brief, PRCC is an efficient approach to global sensitivity analysis that quantifies the strength of association between any given parameter and model output, while simultaneously taking account of variation in all other parameters (for more details on its implementation, see refs [39, 40]). In particular, where simulation outputs straddled the vertical dashed line shown in Additional file 3: Fig.S1, we evaluated correlations against the relative impact of Ag-RDT-led vs NAT-based testing strategies. Where simulations straddled the diagonal line in the upper-right quadrant, we evaluated correlations against the relative cost-per-unit impact (i.e. per death averted or per infectious person-day isolated). Overall, in this way, we sought to identify the contextual conditions under which an Ag-RDT-led strategy would, and would not, be favoured over NAT.

Finally, we analysed sensitivity to model assumptions not covered by the analyses above. As a focal model output for this sensitivity analysis, we chose the proportion of simulations that were favourable, under a given use case and a given scenario for the adjunctive use of Ag-RDT and NAT. First, while the main analysis adopted fixed values for sensitivity and specificity of an Ag-RDT, sensitivity analyses examined how the proportion favourable would vary if Ag-RDT sensitivity ranged from 75 to 95% and if Ag-RDT specificity ranged from 98 to 100%. Second, we examined how the proportion favourable would vary if prevalence of COVID-19 ranged from 10 to 30% in the hospital setting and 1–10% in the community setting. Finally, for simplicity in the main analysis of the community setting, we assumed perfect adherence to isolation after a positive test result and no self-isolation amongst those testing negative. We relaxed these assumptions in the sensitivity analysis and assumed that non-compliance with self-isolation reduced the infectious days averted by a proportion p, while those not required to isolate (i.e., those with false-negative test results, and those awaiting NAT confirmation of an initial Ag-RDT result in the community) nevertheless self-isolated to a degree that reduced transmission by a factor q. We examined how the proportion favourable varied as either p or q ranged from 0 to 50%.

This work was funded by the Foundation for Innovative New Diagnostics (FIND), through a grant from WHO. Authors JS and SS are employees of FIND. Otherwise, neither FIND nor WHO had no role in the study design, analysis, or interpretation.