The use of dried tube specimens of Plasmodium falciparum in an external quality assessment programme to evaluate health worker performance for malaria rapid diagnostic testing in healthcare centres in Togo

The PT activities were planned and implemented by the National Quality Assessment Programme Management (NQAPM) team of the Division of Laboratories of Togo Ministry of Health. Three field teams from the central level conducted the activity in the following health regions: Team 1: Lomé and Maritime health regions; Team 2: Plateaux health region and Team 3: Centrale, Kara and Savanes health regions.

Based on the information from the NMCP and the number of PT samples available for use, a subset, selected as described below, of the total number of health centres using RDT to diagnose malaria were selected to be included in the malaria PT activity. As sites are unevenly distributed within Togo, facilities in locations with high capacities of microscopy diagnosis (e.g. Lomé) were eliminated to give priority to locations with lower level facilities or without a dedicated laboratory. A second review of the list of sites was undertaken to identify those sites with higher throughput of patients and group them geographically to ensure representation of all six health regions of Togo while maintaining ease of access by the field teams. Finally, after applying these criteria, 80 of these sites were randomly selected as participant sites with all workers present on the day of the field team visit being evaluated. The number of facilities meeting these criteria and selected to participate in the PT programme varied across 6 regions, with Centrale Region represented by 10 facilities, Kara Region by 13 facilities, Lomé Region by 7 facilities, Maritime Region by 17 facilities, Plateaux Region by 24 facilities, and Savanes Region by 9 facilities (Fig. 1).

DTS were prepared in the USA as previously described [11]. Briefly, a culture-adapted P. falciparum strain 3D7 parasite line was grown in culture and harvested when parasitaemia reached 2% (~ 100,000 parasites/µL). Parasite preparation was then diluted with washed (in incomplete medium; RPMI1640, Gibco™ Thermo Fisher, Grand Island, NY, USA) parasite negative blood at a similar haematocrit to generate preparations with 0, 100, 200, 500 or 1000 parasites/µL. Parasite negative blood was procured from a US-based blood bank and were collected from US resident donors with no reported overseas travel and confirmed by PCR to have no malaria infections. Blood was determined to be HIV and HBV negative. Blood was also confirmed in the CDC malaria laboratory to not produce a positive test on RDTs before being used to produce DTS. The diluted parasite preparations were then tested on a histidine-rich protein 2 (HRP2)-based RDT brand with performance characteristics that met World Health Organization (WHO) procurement criteria [9]. After confirmation of reactivity, the diluted parasites were distributed in 50 µL aliquots in Sarstedt® Type I microtubes (Sarstedt Inc., Newton, NC, USA) and, with the caps open, air-dried overnight in a biosafety cabinet.

To check that baseline reactivity was not affected by drying, 1 vial of DTS at each of the parasite densities was re-hydrated with a solution of PBS-Tween20 (Sigma Aldrich, St. Louis, MO, USA) and tested on the same RDTs brands as prior to drying. All DTS vials were from the same batch and stored at + 4°C before shipping via commercial air service at ambient temperatures to Togo and, similarly, were stored centrally in Lomé at + 4°C prior to field activities. As previously described, short term storage (up to 4 weeks) at ambient temperatures does not impact the reactivity of the DTS [12].

A batch of SD Bioline® Malaria Ag Pf RDT (lot number 05CD094A, manufactured in Republic of Korea 09/25/2017, expiration date 09/24/2019) used in Togo was obtained from the NMCP for use in preliminary QC of DTS and in the PT. All procured kits were stored centrally until use. This allowed for the same lot number of kits to be used throughout the evaluation and provided an opportunity for baseline assessment prior to using the kits for the PT as well as eliminating the potential impact of variable storage conditions at testing sites. RDTs and DTS were provided to each health centre by the NQAPM team at the time of PT.

Before distributing DTS vials to sites, the DTS panels were evaluated by the NQAPM in collaboration with the NMCP in the malaria reference laboratory in Lomé. Six aliquots of each parasitaemia level (0, 100, 200, 500, and 1000 parasites/µl), 30 vials in total, were reconstituted in 50 µL of PBS-Tween rehydration buffer according to standard procedures provided by the malaria laboratory, CDC. Tubes were allowed to rehydrate for 1.5 h, gently shaken and 5 µL were then tested using SD Bioline® Malaria Ag Pf RDT (Standard Diagnostics, Inc., Abbott, Gyeonggi, Korea) in the same manner as patient blood samples would be tested.

To standardize reconstitution and remove a potential source of error, all DTS were reconstituted by the NQAPM team in the field according to the procedure provided by the CDC malaria laboratory and described in previous paragraph. Each reconstituted DTS contained 50 µl of sample and thus provided sufficient sample to evaluate multiple health workers at a given site.

To evaluate the performance of the health workers using RDTs, PT panels comprised of five reconstituted samples with different parasite densities (0, 100, 200, 500, 1000 parasites/µl) were distributed to health centres and selected laboratories. One facility received only three samples containing 0, 100, and 1000 parasites/µl due to an insufficient number of samples with parasite density of 200 and 500 parasites/µl. DTS vials were rehydrated onsite as described above. A total of 235 health workers participated in the PT activity, with three, two and one health workers testing samples in 76, three and one health facility, respectively. The number of participants at each site was a function of the number of staff on duty at the time of the supervisory visit. All personnel with a role in RDT work and present on the day of the visit participated in the PT. The NQAPM team explained the PT procedure to the health workers prior to providing reconstituted DTS. To record the observations, each evaluated health worker received a results form designed by the NQAPM. Completed forms were collected by the field teams and returned to the NQAPM for analysis.

XLSTAT Basic (version 2019, Addinsoft, France), a statistical application for Microsoft Excel®, was used for statistical analysis of data and Tableau Desktop (Version 2019.4, Tableau Software, Inc., USA) was used for data visualization. The difference between proportions was assessed using Chi-square test, followed by Marascuilo procedure to identify proportions responsible for statistically significant differences. Correlation between proportions of qualified testing personnel and regional performance was assessed using Pearson’s correlation coefficient. Paired Student’s t-test was used to compare the means of two paired samples. P-value of < 0.05 was considered significant. For the assessment of individual tester performance, PT scores were calculated as percentage of samples identified by a tester correctly (that is, negative aliquot content was tested negative and positive aliquot content was tested positive). For the assessment of health centre, district, or regional performance, correct results were aggregated as percentage of tests performed correctly at corresponding level from a total number of tests performed at corresponding level. For this analysis, a level of 80% correct PT result was selected as a value to mark acceptable proficiency. While 80% correct is used as a threshold for malaria microscopy testing and Stepwise Laboratory Quality Improvement Process Towards Accreditation (SLIPTA) 5-star certification, ideally testers would achieve 100% correct results on PT to offer appropriate patient care when conducting malaria testing.

The results of the QC conducted at the malaria reference laboratory showed that all aliquots of DTS containing 0, 500 and 1000 parasites/µl produced expected results with the SD Bioline® Malaria Ag Pf RDT. All DTS with 100 parasites/µl and 50% of DTS with 200 parasites/µl were classified by NQAPM team as non-reactive (Fig. 2). While all parasite densities were distributed and examined during the PT programme, only 0, 500, and 1000 parasites/µl DTS samples were used for evaluation of tester’s performance.

PT scores across all 80 facilities and 235 testers was 100% for 0 parasites/µl, 63% for 500 parasites/µl and 93% for 1000 parasites/µl (Fig. 3). Although health workers were not formally evaluated on their performance of testing the 100 and 200 parasites/µl DTS samples, they received those parasite densities in the PT panel and processed them. As expected and similar to the pre-study QC data, reactivity of samples with parasite density of 100 and 200 parasites/µl presented a challenge and the detection rate for samples with 200 parasites/µl was significantly lower at field sites than for the QC samples tested at malaria reference laboratory (χ² (df = 1; N = 238) = 8.2; p = 0.004; effect size phi = 0.19). Forty-seven (59%) of the 80 testing facilities that participated in the PT programme received a score of 80% or higher on a set of 0, 500 and 1000 parasites/ µl DTS samples.

Health workers of different levels of qualification participated in the PT programme. Ninety-one percent of health workers were classified into groups, with the majority of being state nurses, auxiliary midwives and auxiliary nurses (together 54%) while senior health technicians or medical assistants represented only 5% (Fig. 4). Other qualifications included categories with only 5 or less health workers in each and were combined into a single group identified as “Other” for group performance analysis.

All qualification groups correctly identified negative DTS and performed better in the detection of malaria in samples with parasite densities of 1000 parasites/µl than those with 500 parasites/µl (t-test for two paired samples t = − 9.36, DF = 7, p < 0.0001, two-tailed test) (Fig. 4). There was no significant difference between these groups of health workers in overall performance (χ² (df = 7; N = 702) = 5.6; p = 0.590; effect size Cramer’s V = 0.03) nor when comparing only the results of tests on samples containing 500 parasites/µl (χ² (df = 7; N = 232) = 3.9; p = 0.787; effect size Cramer’s V = 0.05) or those with 1000 parasites/µl (χ² (df = 7; N = 235) = 13.5; p = 0.061; effect size Cramer’s V = 0.09).

The 80 health centres were distributed amongst multiple regions in the country (Fig. 1). The difference in PT results between health regions of Togo was not statistically significant (χ²(df = 5; N = 702) = 9.9; p = 0.078; Cramer’s V = 0.05) (Fig. 5). There was also no difference in performance between qualification groups within regions; and there was no correlation between the percentage of top three performing groups (state nurses, midwives and senior health technicians/medical assistants) and regional PT scores (Pearson’s correlation coefficient r = 0.422, p = 0.405) despite differences, although with small effect size, in the percentage of health workers drawn from these three groups between the regions (χ² (df = 5; N = 235) = 26.4; p < 0.0001; Cramer’s V = 0.15).

When disaggregated by parasite density, the PT scores for 0 parasites/µl DTS samples were 100% correct across all regions. The difference between regions in the testing of 500 parasites/µl specimens was not statistically significant (χ² (df = 5; N = 232) = 6.3; p = 0.278; Cramer’s V = 0.07). While a difference was detected between regions in testing 1000 parasites/µl specimens, the effect size was negligible and the data were not sufficient to identify significantly different regions (χ²(df = 5; N = 235) = 15.0; p = 0.010; Cramer’s V = 0.11) (Fig. 6). The six health regions are further divided into 40 health districts (Fig. 6) represented in this malaria PT by differing numbers of healthcare centres and health workers. Thirty-four (85%) and 12 (30%) of health districts had individual healthcare centres that scored below 80% for 500 and 1000 parasites/µl, respectively. Overall, 47 (59%) of healthcare centres, 28 (70%) of districts and 5 (83%) of regions performed above the threshold of 80%, and 142 (60%) of health workers correctly identified all three samples (Fig. 7).