Background
With the adoption of artemisinin-based combination therapy (ACT) as the first-line treatment for falciparum malaria in the early 2000s, the World Health Organization (WHO) recommended parasite-based diagnosis first for older children and adults, then for all suspected cases of malaria regardless of patient age [
1]. This recommendation led to greatly expanded use of antigen-detecting malaria rapid diagnostic tests (RDTs) worldwide. Total annual RDT sales grew from 46 million in 2008 to 314 million in 2014 [
2]. To guide selection of quality RDTs in the face of multiple products on the market, variable reports of test performance, and weak national regulatory systems, WHO, the Foundation for Innovative New Diagnostics (FIND), and other collaborators established the WHO Malaria RDT Product Testing Programme in 2008 at the US Centers for Disease Control and Prevention (CDC) [
3]. By December 2015, the programme had completed six rounds of assessment and evaluated 171 products. The programme constitutes the laboratory evaluation component of WHO’s prequalification process for in vitro diagnostics (IVDs) and forms the basis for WHO’s malaria RDT procurement recommendations. Results inform procurement decisions by multilateral and donor agencies, non-governmental organizations (NGOs), ministries of health, and others across the malaria-endemic world [
2].
Procuring good quality RDTs, however, does not necessarily guarantee good field performance. Shipping, handling, and storage can affect RDT accuracy as can the training and supervision of health workers in both test preparation and interpretation [
4‐
9]. All have received attention in the literature.
One area that has received less attention is the quality of the buffer and accessories required to conduct RDT testing. The term “accessories” here refers to alcohol swabs, lancets, and blood transfer devices (BTDs). Buffer is referred to as a component. “Accessories” are so described because one brand or style can be substituted for another without compromising test performance. For instance, if tests come packaged with dry alcohol swabs, users can substitute alcohol and cotton or a different alcohol swab. Buffer, in contrast, is manufactured for a specific RDT brand and lot. Substituting buffer from a different brand, lot, or type of test, using water, or using any solution other than the one specifically designed for that lot and test, is contraindicated and could produce incorrect results. For this reason, WHO classifies buffer as a “component”: an integral part of the test. Manufacturers package alcohol swabs, lancets, BTDs, and buffer solution into boxes of 25, 30 or 50 test cassettes, commonly known as ‘hospital packs.’ Alternatively, RDT ‘single packs,’ include a test cassette, accessories and a single-use buffer vial in one envelope. In either case, defective accessories can contribute to anomalous test results. For example, some substandard BTDs can result in collection of too much blood; this can obscure results if the test strip background remains red and test lines illegible. Others can result in collection of too little blood; this can generate false negative results if the blood volume contains insufficient antigen to generate a visible test line. With RDT single packs, single-use vials that contain too little buffer or allow leakage or evaporation can also produce invalid or false negative results. In hospital packs, bottles with insufficient buffer volume may run out before all tests are used. This can lead to tests being wasted or providers substituting water or some other inappropriate solution for buffer. Until recently, however, the WHO-FIND Lot Testing Programme included only limited assessment of buffer and accessories [
10].
This paper reports on problems with leakage and stability of single-use buffer vials and quality concerns with accessories encountered in a project promoting RDT use among private sector health care providers in Kenya, Madagascar, Nigeria, Tanzania and Uganda. The paper also describes resulting actions taken by WHO and project partners. The project, “Creating a Private Sector Market for Quality-Assured RDTs in Malaria Endemic Countries,” took place between April 2013 and April 2016. Project details and outcomes will be described elsewhere. Briefly, private providers in the five countries were recruited, trained in malaria diagnosis using RDTs, supplied with an initial stock of test kits, linked with local supply chain partners, and supported by the project’s two implementing partners: Population Services International (PSI) in Kenya, Madagascar, and Tanzania, and Malaria Consortium (MC) in Nigeria and Uganda. WHO, FIND, and the Johns Hopkins Bloomberg School of Public Health (JHSPH) provided support in health worker training, product quality assurance activities, public-sector engagement, and development of relevant guidelines and standard operating procedures (SOPs).
The project’s first year involved working with national regulatory authorities, ministries of health, national malaria control programmes and others to develop or revise guidelines and policies on private sector RDT use. Project partners developed procurement protocols, carried out lot testing on samples drawn from among 500,000 RDTs, recruited and trained an initial cadre of private providers, and engaged with manufacturers, importers, and distributors [
11].
Discussion
This paper’s results reveal several quality issues related to malaria RDT buffer and accessories heretofore not described in the literature. Previous studies have focused primarily on clarity of instructions, package labelling, or desiccant status [
16‐
19]. Some of the issues described here raise more serious concerns than others. Dry alcohol swabs, for instance, may represent only a minor additional cost and inconvenience to providers who must purchase their own alcohol and cotton. Without access to these supplies, however, a provider might wipe the patient’s finger with the dry swab or water or leave it uncleaned, augmenting the risk of infection. Bent, broken or detached lancet tips could pose a risk to both health worker and patient. Some users might procure their own lancets—another additional cost—others might be tempted to substitute a non-sterile or non-disposable device. Poorly functioning BTDs represent an even greater concern. A stiff, difficult to use plastic pipette, manufacturing defects such as pinholes that interfere with suction, or absence of a mark indicating correct volume could result in transferring the wrong blood quantity to the RDT. Excess blood could leave a dark red background in the results window, obscuring test lines. This could cause a health worker to misread a positive result as negative, especially with a low parasite density infection where test lines tend to be faint. Too little blood could also produce false negative results if antigen quantity is insufficient to generate a visible test line. In either case, a malaria-infected patient could go untreated.
Assessment during lot testing showed blood volume variations using the inverted cup, a BTD previously reported as easier to use and more accurate than others [
20,
21]. Variations exceeded those in past reports, perhaps because different BTD manufacturers use different designs or plastics. (Note that the Hopkins et al. study was based on 3 transfers from each of 227 health workers; data here are based on 10 transfers with a single BTD by a single technician [
21]). The fact that some BTDs show up to 46% volume variation suggests that not all RDT manufacturers perform an adequate quality control of BTDs. Most RDT manufacturers source accessories such as BTDs from other suppliers; under ISO 13485, manufacturers are responsible for verifying the quality of any materials they include in their test kits.
Problems with single-use buffer vials represent another significant concern. An empty vial might tempt an RDT user to substitute water or some other liquid. At least one study has shown that substitution of tap water, distilled water, or saline solution for buffer can all produce false positive results in both
P. falciparum-only and multi-species RDTs [
22]. For vials in which evaporation has left insufficient buffer volume, reagent concentration poses a similar risk. Beyond lots procured for this project, the FIND-WHO RDT lot-testing programme documented buffer evaporation in 145 lots of single-use RDT kits representing 11 different products from three manufacturers between 2014 and 2015 [
23]. Anecdotal reports from various countries were also discussed during a meeting of the malaria RDT procurement task force in October 2014. As a result of these various findings, the WHO IVD Prequalification Programme (WHO PQ) issued a notice of concern recommending a halt to procurement and delisting of WHO-prequalified affected products [
24]. In parallel, WHO PQ also asked relevant manufacturers to notify those who procured the affected products, investigate the problem, and develop or source alternative, stable, single-use buffer vials. Two affected manufacturers accomplished this using higher density plastic, but sourcing the material and conducting the required stability studies resulted in the product being unavailable for 20 months.
Deficiencies with buffer and accessories could undermine both provider and client confidence in RDTs. It has been widely reported that RDT users doubt negative test results and prescribe antimalarials to RDT-negative patients for fear of leaving an infected but incorrectly diagnosed client untreated [
25‐
28]. Research also shows that both health care providers and patients are reluctant to end a clinical visit without prescribing or receiving some sort of treatment [
25‐
29]. Quality problems with buffer and accessories could reinforce this type of incorrect practice, ultimately leading providers to substitute non-quality assured RDTs or return to poor-quality microscopy or presumptive treatment.
Putting more emphasis on quality assessment of buffer and accessories in WHO laboratory evaluations and WHO PQ manufacturer dossier and site assessments could diminish the likelihood of such problems. This might spur manufacturers to more closely monitor the quality of accessories and RDT components which are often supplied by third parties. FIND, WHO and the other private sector project partners have developed a troubleshooting guide and problems protocol to assist supervisors and end-users in investigating frequently observed anomalous results and errors [
30,
31]. The publications offer guidance documenting problems with RDTs, buffer, and accessories and reporting them to NMCPs, national regulatory authorities, and WHO PQ. Timely reporting can help trigger corrective actions such as the previously described WHO PQ notice of concern. Reporting should include investigation to verify that anomalies are due to quality problems rather than end-user errors. The WHO PQ website also accepts problem reports related to malaria and other IVDs [
32].
FIND’s expanded assessment has now been updated to include verifying the volume of buffer, the number of accessories and the moisture of alcohol swabs. Because this expanded assessment is highly labour intensive, however, WHO-FIND laboratories can only conduct it by special request on a limited number of lots. A shortened assessment is now included in routine WHO-FIND RDT lot testing, the main difference being that buffer volume uniformity is checked visually rather than measured. Lot testing reports now summarize results and include photos of the accessories. A lot with insufficient buffer for preparing the test will fail the assessment.
The aforementioned interventions would likely reduce buffer- and accessory-related anomalies and provide a mechanism for addressing them at a system level when they occur. They would not, however, respond to the immediate needs of front-line private sector providers who encounter problems with buffer or accessories and need defective items replaced rapidly. Many private providers operate on very narrow financial margins and cannot wait the several months required to resolve an issue through international channels. Similarly, many lack capital to purchase replacement supplies while waiting. During the private sector project, project partners or their distributors could resolve problems more rapidly by intervening directly with international entities and manufacturers. Manufacturers supplying the project generally responded quickly and positively. One manufacturer replaced 40,000 alcohol swabs and BTDs while confirming that individual buffer vial volume, though varied, fell within acceptable limits. Two others also replaced buffer vials after receiving complaints. PSI, MC, and their distributors delivered replacements directly to affected providers.
Outside a project setting, an individual provider facing such problems may lack the wherewithal to contact a manufacturer or regulator directly. An individual provider might likewise have difficulty determining whether a problem was limited or widespread or even whom to contact locally to report one. Guidelines such as those in the aforementioned RDT problems protocol should make it easier for a local or regional distributor to compile provider complaints, pass them on to regulators and manufacturers, and serve as a go-between to ensure that provider concerns are addressed. This could protect private providers from unsustainable losses and reduce provider attrition.
The data reported here have many limitations. Information compiled from the private sector project’s M&E and supervisory visits is based on small non-random samples of providers. Visits included direct qualitative examination of test accessories but did not follow a structured protocol or attempt quantitative measurements (e.g., of blood or buffer volume). The systematic laboratory-based assessment of buffer vials, instructions for use, and accessories, though based on random sampling within lots, included only six lots and two test brands procured for project use in Nigeria and Uganda. Lots for other project countries were procured before the accessories assessment was in place. Further, while the systematic assessment made it possible to quantify the volume of blood transferred to each test, all tests evaluated used the inverted cup BTD, widely considered the most accurate and easiest to use device currently available [
21]. Testing with other types of BTD would have provided a more representative picture of the accuracy of devices currently in use. It should also be noted that the assessment is not fully representative of real life since the blood used in testing the BTDs came from a frozen quality control sample, typically less viscous than fresh blood, and was transferred to Whatman filter paper which has a different structure and density than the RDT sample pad. Nevertheless, the procedure allows for a standardized approach to assessing BTDs. Systematic assessment enables quantitative measurement of buffer volume, but leaves open the question of whether the lower-than-specified volume observed is due to inadequate quality control in manufacturing, evaporation during storage, or some other cause. To resolve this, manufacturers would need to carry out root cause analysis. During initial accessory assessments, alcohol swabs were not checked for moisture, lancets were not examined, and accessories were not counted to ensure that their numbers were sufficient. Subsequently, assessment procedures have been updated to address some of these shortcomings, and a shortened procedure for accessories assessment is now incorporated into routine RDT lot testing [
13].
Taken together, these limitations preclude generalizable conclusions about the extent of quality issues related to buffer and accessories or the effect of such issues on test performance and acceptability. However, quality issues clearly exist and manufacturers must apply good quality control standards to buffer vials and test accessories as well as to the test devices themselves. Both private and public entities that procure RDTs should be aware of these issues and have a system in place to document, troubleshoot, and rapidly resolve them. This should include a mechanism for replacing defective products on a timeline acceptable to front-line providers. Programmes that evaluate rapid test performance should include assessment of buffer and accessories as an integral part of routine quality control procedures.
Authors’ contributions
SAH conceived of the topic of the paper, helped assemble and interpret the data on which this paper was based and took principal responsibility for preparing and revised the manuscript, incorporating input from all other authors. SI helped assemble and interpret the data on which this paper was based, helped develop the methods for assessing rapid diagnostic test quality described in the paper, and contributed to writing and revision of the manuscript. NM helped analyse the data and contributed to the manuscript. CL helped assemble and interpret the data and contributed to the manuscript. ES directed project implementation in Uganda and Nigeria, made the initial request for assessment of RDT accessories and buffer that led to the findings reported here, and contributed to the writing and revision of the manuscript. SD served as deputy director of the 5-country project, managed day-to-day project operations including the monitoring and evaluation that contributed to the findings reported here, and contributed to the writing and revision of the manuscript. NC supervised and helped carry out the evaluations of RDT accessories and buffer reported here. She also contributed to the writing and revision of the manuscript. DK supervised and contributed to accessory and buffer assessment in Uganda and Nigeria, contributed to developing the SOPs for accessory and buffer assessment in the project as a whole, and contributed to the writing and revision of the manuscript. RM was the Uganda country lead for project implementation, contributed to assessment of accessories of buffer in Uganda, and contributed to the writing and revision of the manuscript. GN was the Nigeria country lead for project implementation, contributed to assessment of accessories of buffer in Nigeria, and contributed to the writing and revision of the manuscript. NN was the Kenya country lead for project implementation, contributed to assessment of accessories of buffer in Kenya, and contributed to the writing and revision of the manuscript. RR was the Madagascar country lead for project implementation, contributed to assessment of accessories of buffer in Madagascar, and contributed to the writing and revision of the manuscript. JC helped develop the standard operating procedures for testing many aspects of RDT quality including quality of assessments, participating in field visits to project sites to gather data on problems with accessories and buffer and made substantial contributions to drafting and revising the manuscript. All authors read and approved the final manuscript.