Multicenter evaluation of TB-SPRINT 59-Plex Beamedex®: accuracy and cost analysis

Tuberculosis (TB) remains a major health problem worldwide. According to the World Health Organization (WHO), it is estimated that 10 million cases of TB occurred in 2017, causing 1.6 million deaths [1]. Multidrug Resistant tuberculosis (MDR-TB) is characterized by resistance to at least isoniazid (INH) and rifampicin (RIF), the two most important anti-TB drugs. Around 558,000 new cases of MDR-TB occurred worldwide in 2017 [1, 2].

In Brazil, about 82,676 TB cases were noted in 2016, out of which 1900 were estimated to be MDR-TB [1]. Brazil is one of the 30 high TB burden countries with cure rates that differ among the states. The median value of these cure rates is 71%, still far from the 85% goal of the WHO END TB Strategy [1].

The major challenge facing the success of TB treatment is patient acceptance of the treatment drug regimen along with a correct and early diagnosis of drug resistant strains. In most laboratories in Brazil and other countries with limited resources, smear microscopy examination is routinely used for TB diagnosis, while culture and drug susceptibility testing (DST) are mostly performed in reference laboratories as they require a biosafety level 3 laboratory [3, 4].

Molecular tests can provide rapid detection of TB and MDR-TB. The WHO has already endorsed and recommended some techniques for the rapid detection of TB and MDR-TB, such as the Genotype® MTBDRplus (MTBDRplus, HAIN Life Sciences, Nehren, Germany) and the Xpert® MTB/RIF (Cepheid, Sunnyvale, CA, USA) [1, 3, 5]. Recently, a consensus was reached regarding the importance to consider the test accuracy, time to produce a result, and costs incurred by a new diagnostic method, before its incorporation into the healthcare system [6‐8].

In this study, a molecular technique called TB-SPRINT 59-Plex Beamedex® (TB-SPRINT, Beamedex, Orsay, France) was used to identify TB and MDR-TB. This technique is a microbead-based assay developed to run on Luminex® devices (Luminex Corp., Austin, TX, USA) for the simultaneous spoligotyping and detection of rpoB, katG, and inhA mutations associated with resistance to RIF and INH. It has also been successfully used in previous studies and performed well when compared to the MTBDRplus [9‐11]. However, no data on its accuracy and costs have been gathered from a multicenter laboratory study considering all components of the cost chain. This includes the cost of the test (which is currently very low given its current marketing as a research use only (RUO) as well as an in vitro diagnostic (IVD) [9].

The aim of this study was to evaluate the performance of TB-SPRINT for MDR-TB detection at three laboratory research centers in Brazil and to evaluate its mean cost (MC) and activity-based costing (ABC).

Overall, TB-SPRINT had results comparable to those of DST and MTBDRplus, presenting high agreement in values for INH and RIF resistance detection at all sites. Regarding accuracy, TB-SPRINT was able to detect resistance mutations in the RRDR region of rpoB gene, regarding RIF resistance, and in codon 315 of katG and in the position − 15 of the promoter region of inhA gene, regarding INH resistance. These results are close to the MTBDRplus accuracy values found in this study and previously described [19, 20]. When comparing the results between the sites, there was an important variation, which shows a reduced reproducibility of this molecular test.

A large number of indeterminate results in TB SPRINT were observed, mainly for the analysis of mutations in the rpoB gene. These results were similar to other studies observed in MTBDRplus, [21‐23]. In previous studies, regarding evaluation of the frequency of indeterminate results of TB-SPRINT, only accuracy was evaluated [9, 11]. Most mutations conferring RIF resistance were identified in the 81-base pairs RRDR region of the rpoB gene, more frequently at codon 531, followed by codon 526, and codon 516. For INH resistance, most of the mutations were identified in the katG gene codon 315, followed by the inhA gene. These data are in agreement with the data described in the literature [11, 24]. Although most of the mutations found in the rpoB gene were the classic and most commonly observed ones (516 GAC-GTC, 526 CAC-GAC/TAC, and 531 TCG-TTG/TGG), other mutations were identified by sequencing in the evaluated MDR samples, which are not covered in the TB-SPRINT (Table 5). Such mutations can make the process of hybridization results analysis difficult to interpret, requiring more attention of the operator. Given the high number of indetermined TB-SPRINT outcomes observed in this study, and that the operator was blinded from the sequencing results at the time of the execution and data analysis, possible absence of signal in WT probes in resistant strains with other mutations were not detected, and the indetermined results, although described, were not evaluated in the statistic analysis. This is important to point out as every molecular result must be evaluated in conjunction with the patient’s clinical data, which could explain and avoid the false-negative results found [24].

The technique using Luminex devices allows the analysis of 96 samples at the same time, generating rapid results as recommended by the WHO [2]. Although excellent results were described previously, this was the first multicenter study that evaluated this test under routine conditions, with the same sample panel, changing only the operator and instruments [9, 11]. In this study, reduced reproducibility results have been observed, particularly for RIF, due to a high number of indeterminate results.

Differences in the laboratory structure, such as the fact that each site used its own reagents for PCR and hybridization, as TB-SPRINT did not provide PCR reagents in the kit at that time, may have contributed to the differences in reproducibility between sites and impacted the outcome of this test. Also, DNA extraction method is crucial to ensure good results, and as a limitation of this study, DNA extraction was performed only at UFMG and this material was distributed to the three sites, so the outcome regarding variability due to DNA extraction could not be accessed. Standardization of all laboratory flow and continuous personal training must be considered to achieve uniformity in the results. Despite the technical issue, the MC and ABC of TB-SPRINT (USD 127.78 and USD 109.94, respectively) are considerably higher than the average costs and ABC of MTBDRplus (USD 84.21 and USD 48.38, respectively) [25]. If the high number of repetitions of TB-SPRINT that would have been necessary was taken into account, these values would become even higher, causing this assay not suitable to be implemented in low and middle-income countries. Recently, after this study was performed, TB-SPRINT was improved, now it not only provides coupled beads but also dNTP, primers, and Taq Polymerase (www.​beamedex.​com) without significant extra costs.

The main components that increase the costs of TB-SPRINT are the equipment necessary supplies, where we highlight the high values of the calibration and validation kits indispensable for the use and maintenance of Luminex/Bioplex 200®, according to the manufacturer’s recommendation [26].

Other than the costs associated with TB-SPRINT, this kit also requires several steps to execute the test and does not provide all the required supplies. The supplies that are not provided by the kit and need to be acquired by the local laboratory include PCR plates, reading plates, adhesives, microtubes, and reagents for DNA extraction, PCR amplification, hybridization, and washing buffers [15]. Notwithstanding the cost increase, we observed a major variability between the laboratories that performed the assay, when evaluated under field conditions. This suggested that the protocol can be sensitive to variations in the instrument and reagents. Also, a protocol with many steps increases the amount of time dedicated by human resources and the need for equipment and materials. Although these are usually used in a molecular biology laboratory, they increase the values of both the MC and ABC. These data on TB-SPRINT MC and ABC, as well as the cost component analysis of this method, indicate that it may not be economically sustainable to incorporate TB-SPRINT into the drug resistance diagnostic routine of Brazilian public laboratories. The LMR/UFMG used as a model for the economic study, is a public laboratory that presents a reality very close to the other sites of this study and others public laboratories in Brazil, so is important highlight the necessity of laboratories perform the economic evaluations in loco in parallel to the performance evaluations, when incorporating new technologies.

Despite the fragile results for the identification of drug resistance in a routine laboratory, TB-SPRINT is an excellent method for performing spoligotyping as previously described by research laboratories as it gives great quality results in a short period of time [10, 11, 14].

The TB-SPRINT results were compared to DST by using MGIT, which was used as the standard method according to other studies evaluating the accuracy of this test for routine clinical detection of MDR-TB [9, 11]. Some strains present well-known resistance mutations, but are sensitive in DST. This event could occur due to the presence of heteroresistant strains, but it needs to be confirmed in the future studies, for instance retesting the strain for the DST together with minimum inhibitory concentration (MIC) determination, and whole sequencing genome [27].

This study presents some limitations: the relation of the clinical data of the patients with the results of the molecular tests was not performed, the cost-effectiveness was not evaluated, and the DNA extraction and the MTBDRplus assay were performed at one site. Given the WHO recommendations for rapid and reliable MDR strain detection, it is crucial to perform other studies to carry out laboratory validation and cost analysis, under field conditions, for any new molecular tests before their incorporation into the health system.

In conclusion, the current version of TB-SPRINT 59-Plex Beamedex® may not be applicable yet in routine laboratories especially in locations of low resources, given the cost of maintenance and materials, however TB-SPRINT is also an unique assay, besides whole genome sequencing, that may provide interesting clues on MDR-TB transmission rates in a given setting. As TB-SPRINT not only describes the major drug-resistance mutations but also provides the genotyping (through spoligotyping). In this sense, TB-SPRINT could be well suited for large retrospective population-based multidrug resistance national survey.

TB-SPRINT 59-Plex Beamedex® showed good results for INH and RIF resistance identification, but still needs improvements to achieve IVD standards. The low cost of TB-SPRINT is also hampered by the high cost of purchasing a Luminex device and the costs associated with routine calibration-controls. The spread of the MagPix®, a 50-Plex LED based fluorescence imager (Luminex Corp, Austin, TX, USA), could significantly lower the routine cost of TB-SPRINT. Improved protocol and new cost analysis should be pursued for TB-SPRINT to be made suitable for routine MDR-TB molecular diagnostics.