Performance of plague rapid diagnostic test compared to bacteriology: a retrospective analysis of the data collected in Madagascar

Bacteriological culture is the gold standard for the identification of Y. pestis. A lateral flow plague rapid diagnostic test (RDT) based on Y. pestis F1 antigen detection has been developed, produced, and successfully evaluated to support healthcare agents [1]. In 2006, the WHO revised recommendations for standard plague case definition based on laboratory results, including those of plague RDTs, and the epidemiological context (Additional file 1: Table S1) [2]. RDTs have since been produced by the CLP and distributed with their sampling kits and secured packaging to health care centers (HCCs) every year.

Since 2002, plague RDTs have been routinely used by primary HCCs and hospitals in plague endemic districts of Madagascar and at the CLP for biological samples arriving with a notification form. From 2008 on, they have been used as a screening test: only samples with a positive RDT result have been tested by bacteriological culture. This protocol has been followed until the early 2017–2018 plague season. Indeed, in 2017, Madagascar faced a pneumonic plague (PP) epidemic in two urban areas. A new molecular biology-based diagnostic test was implemented at the CLP and adapted to the epidemic context of PP [3, 4].

After this recent, mainly pneumonic, plague epidemic and because of the changes in laboratory protocols for plague confirmation, we decided to retrospectively analyze the plague RDT performance of both pneumonic and bubonic plague cases. Bubonic plague (BP) cases accounts for more than 80% of notified cases during endemic periods.

For Madagascar, all clinically-suspected plague cases, with compatible clinical presentation (fever, sepsis syndrome, lymphadenopathy, and/or acute pneumonitis) and epidemiological features (such as exposure to infected animals or humans and/or evidence of flea bites and/or residence in or travel to a known endemic focus within the previous 10 days), must be reported to the national surveillance system. Clinical and epidemiological data are collected for each patient on a standardized notification form sent to the CLP, accompanied by biological samples (bubo aspirate, sputum, or postmortem organ puncture, as appropriate) collected on sterile swabs and conveyed in Cary-Blair transport medium for RDT and bacteriological diagnosis [5, 6]. Briefly, the bacteriology consists in Y. pestis isolation from biological samples either by direct culture on Yersinia selective agar medium (Oxoid Ltd., United Kingdom) or by mouse inoculation followed by culture of spleen-sample from a dead mouse. Culture incubation was done at 26 °C–28 °C for 48 h or more as Y. pestis cultures grow slower than other bacteria. Suspected colony was identified by phage lysis test and biochemical identification on API20^E strip (bioMérieux, France).

Information from the notification form and diagnostic test results have been recorded in a Microsoft Access® file and were transferred to REDCap (Version 7.6.5) in 2018. A total of 16,221 cases were recorded (January 1, 1998 to April 3, 2019), 76% of which were of BP, 19% PP, and other cases which were categorized as neither PP nor BP (missing information from the clinical form = NP). Table 1 summarizes the available data by period between 1998 and 2019, according to the diagnostic protocol used by the CLP, with the period 1998–2001 being defined as Period 0, as no RTDs were routinely used.

We assessed RDT performance over time using the bacteriology as a gold standard and RDTs performed at the CLP to avoid bias due to handling effects in HCCs, despite training. We used plague cases for each identified period during which the two tests were performed independently and systematically, i.e. when RDTs were not used as a screening test to direct bacteriological testing. Indeed, only three time periods were used for our analysis: (1) from 2002 to 2007 for all clinical forms (Period 1), (2) between September 11 and October 3, 2017 for PP notified cases from non-endemic zones (Period 3, Protocol 2) and, from January 2018, all notified cases (Period 3, Protocol 3), and (3) 2018–2019 (until April 3rd) for all notified cases (Period 4).

Agreement between the RDT and bacteriological culture was assessed by examining the percentage of concordant results and the Kappa coefficient with 95% confidence intervals (95% CI) [7]. RDT sensitivity and specificity were also estimated such as the positive predictive values (PPV) and the negative predictive values (NPV) [8]. SPSS Statistics (version PASW® Statistics 18) and R (version 3.5.3) were used for statistical analysis.

In accordance with the Plague National Control Program, notification of suspected plague cases by HCC remains mandatory and thus no ethics approval was required for the use of the human data (de-identified) for this study.

We compared the plague RDT and bacteriological culture results, according to the clinical form of plague, for all notified cases during the three time periods for which inclusion and exclusion criteria were satisfied (Table 2).

In the early 2000’s, the use of RDTs for BP greatly contributed to the improvement of case management of patients living in remote areas of Madagascar. After more than 15 years of their use for plague diagnosis, we evaluated their performance using bacteriological culture as a gold standard in addition to our recent work [9]. Although positive results obtained using bacteriological techniques were undeniable, other factors can lead to false-negative results: sample quality (saliva rather than sputum), contaminated field samples, preliminary use of antibiotics (usually not reported on the form), and long transport delays to the CLP. Conversely, F1 antigen detection on RDT remained stable within samples contaminated and/or collected after antibiotic use. Nonetheless, bacteriological culture is still the gold standard for Y. pestis detection, according to the 2006 WHO recommendations.

The strength of our analysis, in which we included only cases for which the bacteriological culture was performed independently of RDTs, is also a pitfall. Indeed, we ended up with a small number of cases, leading to estimates with large confidence intervals for the 2017–2018 and 2018–2019 plague seasons and cannot exclude false-negative bacteriology sample due to contamination.

Nevertheless, our results are consistent over study time period, except the period 3, and considering bacteriological culture as the gold standard, the sensitivity of the RDT was 100%, confirming the relevance of its use until the availability of a new point-of-care test for remote areas. However, with a specificity of approximately 70 to 80%, RDT cannot be used as a screening test. The sensitivity and specificity of the RDT appear to be similar for both cases of PP and BP, although the confidence intervals were wider for PP. The 2017–2018 season was an exception, RDT was negative for 75% of cases that were positive by bacteriological culture and only a limited number of cases were available for analysis. Such poor performance of the RDT during the response to the PP outbreak stresses the potential variability of RDT performance under extreme circumstances, when HCCs and laboratories are overloaded and work under pressure. In particular, for PP, good-quality sputum samples are much more difficult to obtain than bubo aspirates, emphasizing the need for sputum collection standard process establishment and regular training of healthcare staff in sample collection and the use of RDTs during the inter-plague season. Indeed, during the 2018–2019 season, reinforcement of training and improvement of laboratory condition have positively impacted the specificity results.

Since the urban PP epidemic, the CLP- Plague Unit has systematically carried out a newly-implemented real time PCR (qPCR), targeting pla and caf1 genes. If results from this test are inconclusive, a conventional PCR targeting pla, caf1, and inv genes is performed. The new decision tree has been reported along with all details concerning findings during the epidemic [3]. Comparisons between RDT and bacteriological culture may be skewed by factors which could affect the isolation of Y. pestis. Thus, we are currently using a Bayesian approach on the epidemic data to model the conditional dependence between the multiple tests (RDT, bacteriological culture, and molecular biology testing). Given the recent improvement of molecular biology techniques for Y. pestis diagnosis, the WHO recommendations may be updated in the near future. Improving all tools for plague diagnosis, including those suitable for point-of-care screening in remote areas remains a research priority to control human plague [10].

RDT performance appeared to be similar for the diagnosis of BP and PP except during the 2017 PP epidemic in Madagascar where RDT performance was low for PP. This RDT, with its good sensitivity on both plague clinical form during a normal plague season, remained a potential test for alert in plague endemic areas. Particularly for BP, it may be of great value in the decision process for the initiation of therapy. However, for PP, RDT may deliver false negative results due to inconsistent sample quality (prone to errors). A combination with other rapid tests, microscopy or a new generation of RDT, could address the issue of its moderate specificity. Plague diagnosis could be improved through the development and implementation of next generation of RDTs.