Background
Diagnosis of malaria based only on clinical grounds leads to significant overestimates of malaria illnesses even when diagnosis is carried out by experienced clinicians [
1,
2]. Misdiagnosis is problematic for multiple reasons: it exposes patients to unnecessary side-effects of drugs and to the risk of overlooking potentially fatal conditions, such as pneumonia [
3] or diseases of epidemic potential (i.e., Lassa fever or Ebola), and it contributes to the emergence of resistance to anti-malarials [
4]. For these reasons the World Health Organization (WHO) recommends that every clinical suspicion of malaria should be confirmed by either microscopy or rapid diagnostic test (RDT) prior to initiating treatment [
5]. Today RDTs have become the method of choice in many contexts since they allow an easy-to-use biological confirmation that can be performed in low-resource settings where good quality microscopy cannot be maintained, such as remote health centres or at community level.
The number of different RDTs on the market has increased considerably over the past few years. In 2008, WHO launched the Product Testing Programme, which evaluates performance of commercialized malaria RDTs against a standardized panel of laboratory-prepared specimens. This programme, which has already completed its fifth round of testing, allows for product comparisons, encourages producers to improve their tests and, in the long run, promotes product quality. WHO acknowledges however, that its results cannot predict test performance in the field, where other factors such as malaria antigen persistence in the blood and malaria endemicity can influence RDT performance [
6].
Most malaria RDTs are designed to detect a single parasite antigen, the histidine-rich protein 2 (HRP2) or the Plasmodium lactate dehydrogenase (pLDH), while others are designed to detect both antigens in a single test.
HRP2 is a protein produced only by the
Plasmodium falciparum species; detection of this protein can therefore confirm only
P. falciparum infections. Many of the HRP2-based RDTs show very high scores in the WHO evaluation [
7]. For instance, the CareStart HRP2 was reported to have a 98.7 % detection rate (a proxy measurement of sensitivity) and a false positive rate (proxy of the inverse of specificity) of 2.4 %. However, these HRP2-based tests performed more poorly in the field, particularly in terms of their specificity, due primarily to the slow elimination of the HRP2 antigen from the bloodstream. This persistence of the HRP2 antigen means that patients continue to test positive with HRP2-based RDTs long after the patient was treated and the parasites eliminated. During this period, these tests therefore cannot distinguish between lingering positivity and de novo infection. This phenomenon is well documented in high-transmission areas where patients frequently experience repeated malaria infections and treatments [
8‐
10]. However, little is known about field performance of these tests in low-transmission areas where malaria infections are rare.
The pLDH is a parasite protein produced by all Plasmodium species; RDTs based on this antigen can therefore detect all Plasmodium species but are generally less sensitive than HRP2 tests. Their main advantage is that pLDH is rapidly eliminated from the blood stream after treatment, and therefore pLDH-based RDTs returns a negative result within a few days after treatment.
The objective of this study was to explore whether and how much three malaria RDTs vary, in terms of sensitivity, specificity and time to become negative, based on transmission setting. The three RDTs chosen were among the highest scoring tests in the WHO Round 3 malaria test performance evaluation [
7] (two HPR2 RDTs and one pLDH RDT). Tests were assessed in parallel in two settings: one with high malaria transmission, the other with low transmission, in southwestern Uganda.
Methods
Study settings
The study was conducted from September 2011 to January 2013 within the Greater Mbarara district at two locations: Mbarara municipality, where malaria prevalence in children below 5 years of age was relatively low (3–4 %), and Kazo, located 80 km north of Mbarara municipality, where malaria prevalence was ten-fold higher (40–60 %) as reported in a previous study [
11]. In Mbarara municipality the study was hosted at the Kakiika Health Centre; seven other health structures located within or just outside the Mbarara municipality also participated by providing eligible patients. In Kazo the study was hosted at the sole health centre present in the sub-county.
The Greater Mbarara district is located between 130 m and 1500 m altitude above sea level and its vegetation is a mixture of bush and short grass. The climate is tropical with a bi-modal rainfall pattern averaging 1200 mm per annum. The rainfall season falls between mid-August to December and mid-February to May. Malaria transmission is on-going throughout the year, with two seasonal peaks occurring 2 months after the periods of heavy rainfall. Plasmodium falciparum is almost the only species documented. Malaria constitutes one of the major causes of illness in the area.
Sample size
The sample size was calculated to respond to the main study objective, the time to become negative of RDTs, and was based on the expected proportions of RDTs that remained positive at each follow-up day. It was estimated that 184 patients were needed to achieve the accuracy of 8 % around a proportion of 50 % positive RDTs and of 3 % around proportions of 5 or 95 %. By expecting 15 % of patients not being positive for all RDTs, lost to follow up or excluded from the primary analysis, 212 patients were estimated to be included in each study site.
For the sensitivity analysis all patients with a positive microscopy with a P. falciparum infection were included until the sample size for the time to become negative analysis was achieved. For the specificity analysis, the number of patients with a negative microscopy depended of the intensity of malaria transmission. A negative/positive microscopy ratio of around 1:1 and of at least 5:1 was expected in the high and low transmission settings respectively.
Patient recruitment and follow-up
The study recruited consecutively all children who presented to any of the participating health centres, were under 5 years of age, weighed 5 kg or more, and either had fever ≥37.5 °C or whose parent reported fever within the previous 2 days. Blood samples obtained by finger prick from children whose parent or guardian provided written consent were tested in parallel at the corresponding study health centre with the three different RDTs and with microscopy. Patients recruited at this stage formed the population sample for the RDT performance analysis.
At each site, a sub-set of 212 children was selected for the time to become negative analysis. Inclusion criteria were: positivity for at least one RDT, having a P. falciparum mono-infection (i.e., no additional species) confirmed by microscopy, and having not taken any anti-malarial drug within the previous 2 weeks (as reported by the parent). Participants stayed with a parent or guardian in an observation room of the health centre for 3 days (day 0–day 2) to receive a full course of artemether-lumefantrine (Coartem®, Novartis) under the supervision of study personnel. When treatment intake was completed, participants were sent home. During their stay at the health centre and at follow-up visits on a fixed schedule (days 2, 3, 5, 7, 14, 21, 28, 35, and 42) patients received a clinical check-up and malaria testing using all three RDTs and microscopy. If an RDT was negative at a given visit, that test was not repeated during subsequent visits. Follow-up was discontinued as soon as all RDTs were negative, or at a maximum of 42 days.
Patients with recurrent parasitaemia during follow-up were treated with artesunate-amodiaquine (AS–AQ Winthrop®, Sanofi Aventis). Patients who did not appear for a follow-up visit after day 3 (when they went home from the centre) and for whom it was not possible to recover any additional information were considered lost to follow-up.
Laboratory techniques
The following RDTs were evaluated: SD Bioline Malaria Antigen P.f (HRP2) (Standard Diagnostic Inc, Suwon, South Korea, catalogue number: 05FK50-02-4), CareStart Malaria HRP2 (Pf) (Access Bio, Somerset, USA, catalogue number: G0141) and CareStart Malaria pLDH (PAN) (Access Bio, catalogue number: G0111). RDTs were performed according to manufacturer’s instructions. All RDTs were read by two blinded, independent readers. The first reading was done at 15 min for the SD Bioline test and 20 min for the two CareStart tests. The first reader wrote the result in a paper form which was handed to the lab supervisor. The supervisor called within 5 min the second reader who wrote the result in a second paper form. The supervisor confronted the results. If results were discordant the supervisor acted as third tie-breaker reader. The third reading occurred typically within 10 min after the first reading. The lot number of each test was recorded and the storage temperature monitored.
For microscopy, thick and thin blood smears were prepared on the same slide and stained with a 10 % Giemsa solution (pH 7.2) for 15 min. Reading was performed using a 100× magnification lens with oil immersion. Parasite asexual forms were counted against a threshold of 200 white blood cells (WBC); if fewer than ten parasites were seen at this threshold, counting was continued until at least 500 WBC. Plasmodium species was confirmed on the thin smear. Two-hundred high-power fields were read before declaring a blood slide negative.
Parasite density was estimated based on a hypothetical leukocyte density of 8000 WBC/µL. The presence of gametocytes was recorded, although a slide with gametocytes but no asexual parasite forms was scored as negative.
Double reading of all slides was performed. Slides with discordant results (positive/negative or parasite density of second reader varying by ≥50 % compared to the first reader) were read by a third reader. The parasite density result was calculated as the mean of the first and second readers, if not discordant, or as the mean of the two closest results, if a third reading was required. Blinding of microscopists to RDT results was ensured by assigning to the slides a different laboratory code, so that only the laboratory supervisor could match the microscopy result to the patient’s identification number that was used for the RDT. All study microscopists had a Kappa score ≥80 % in evaluations of microscopists conducted by the reference laboratory of the Shoklo Malaria Research Unit (Thailand) in 2010, prior to the start of this study.
Data processing and analysis
Data were double-entered and validated in a Voozanoo database (EpiConcept, Paris, France). Data analysis was carried out using Stata version 12.1 (Stata Corp, College Station, TX, USA).
Qualitative variables were assessed by percentages and 95 % confidence intervals (95 % CI). Quantitative variables were described by mean and standard deviation if normally distributed, and by median and interquartile 25 and 75 percentiles range (IQR), if not normally distributed. Comparison of continuous numeric variables was tested with Student t test if normally distributed, and with Mann-Whitney rank-sum test if not normally distributed.
Sensitivity, specificity, positive and negative predictive values, and misclassification rates were estimated for each RDT separately, with microscopy as the gold standard. All patients with P. falciparum (mono- or mixed species infection) were included in the analysis, while patients with an infection that did not include P. falciparum were excluded, since the HRP2 tests in the study detect only P. falciparum.
RDT sensitivity and specificity were defined as the proportion of positive test results among the microscopy-positive blood smears, and the proportion of negative tests among the negative blood smears, respectively. The positive and negative predictive values were defined as the proportion of positive blood smears among the positive tests, and of negative blood smears among the negative tests, respectively. The misclassification rate was defined as the proportion of erroneous results (the sum of false positive and false negative results) among the total number of tests performed.
Performance comparisons between RDTs in the same site, and of the same RDT among sites, were expressed as differences with 95 % CI. McNemar’s test was used to assess statistical significance in performance among RDTs in the same site while Fisher’s exact test was used to assess statistical significance in performance of the same RDT between sites.
Sensitivity of RDTs in three strata defined as low (1–1999 parasites/µL), medium (2000–199,999 parasites/µL) and high parasite density (200,000 parasites/µL or more) were also assessed and compared. For this last analysis, results from the two sites were combined.
The time to become negative was defined as the 1st day when an RDT was reported as negative. Patients with incomplete or incorrect intake of anti-malarial treatment and patients who presented recurrent parasitaemia starting from day 7 (i.e., malaria microscopy was again positive) before all RDTs had become negative were excluded from the analysis. Patients lost to follow-up were censored at their last visit.
The probability of a test remaining positive over time was calculated with the Kaplan–Meier survival function. The same survival analysis was carried out on three strata according to initial parasitaemia. Strata were defined as low, medium and high parasite densities (same cut-off as for RDT performance).
Ethics, consent and permissions
The study protocol was approved by the Ethical Review Board of Médecins Sans Frontières, the Faculty Research Ethical Committee and the Institutional Review Board of Mbarara University of Sciences and Technology, and the Uganda National Council for Sciences and Technology.
All patients were recruited after a parent or guardian provided written consent, and all patients received free health care during the entire period of study participation.
Discussion
All three RDTs evaluated in this study performed better in the low- than in the high-transmission setting, with striking differences especially regarding specificity. While the low specificity of the HRP2 tests (around 80 %) in the high-transmission setting of Kazo confirmed previous studies from similar epidemiological contexts, for example, 73 % in Ghana [
12] and 63 % in Tanzania [
13], the high specificity (almost 99 %) in the low-transmission setting of Mbarara municipality was consistent with the very low false positive rate reported in the WHO Product Testing Programme [
7].
The long-lasting HRP2 antigens, when combined with a high frequency of malaria infections, may lead to quasi-persistent antigen positivity in children living in high malaria transmission areas. A clear demonstration of this situation is the high number of children (86 out of 212) in Kazo that experienced a recurrent malaria infection before clearing the HRP2 antigens of the previous one. This is not the case for children living in low-transmission settings, who are much less likely to have had a recent malaria infection.
Unlike specificity, the sensitivity of all three RDTs was high in both settings; however the pLDH test performed less well than the HRP2 tests. Sensitivity estimates confirm the WHO evaluations [
6,
7] and are in line with previous studies [
14,
15]. Only Nyunt and colleagues found that the sensitivity of the pLDH test was as high as an HRP2 test [
10].
Data of this study clearly indicate that the relatively low sensitivity of the pLDH test was observed only at low parasite density. Nevertheless, this low sensitivity is highly problematic for public health decision makers, who typically favour deploying highly sensitive tests such as the HRP2 RDTs, accepting the associated risk of unnecessarily treating uninfected patients over taking the risk of failing to detect malaria cases. Regarding specificity, the high number of false positives observed with HRP2 tests is also problematic, particularly in high transmission settings where half or more of RDTs are positive. A wrongly positive result may divert clinicians’ attention from other possible aetiologies of febrile illnesses, or may lead clinicians to mistrust the RDT result, diminishing their perceptions of the test’s usefulness. The two HPR2 tests showed much longer times to become negative (median of 35 to >42 days) than expected from most previous reports. Only one study in the Democratic Republic of Congo [
9] reported such a long time to become negative, whereas others report median times of no longer than 21 days [
6,
14,
16].
In contrast, the two-day median time of pLDH antigen persistence generally seems to parallel a decline in the number of viable parasites during treatment [
15,
17,
18]. In some instances, however, the pLDH antigen may persist for longer. Nyunt and colleagues reported in their study in Myanmar a median time of 6 days [
10]. Njama-Meya and colleagues reported that 7 % of patients were still positive at 28 days. In this study no patient was positive beyond 21 days. The reasons for this difference remain unexplained.
While time to become negative did not vary substantially according to the intensity of transmission, it differed based on initial parasite density: the higher the initial parasite density, the more time was required to clear parasite antigens [
8]. This finding supports the hypothesis that the level of antigens in the bloodstream is proportional to parasite density [
19].
The study had a number of limitations. Some negative microscopy results may represent real infections with a sub-microscopic parasite density. In this case, if the RDT was positive, results may have been wrongly classified as false positive instead of true positive. Considering the high sensitivity of HRP2 tests in low parasitaemias, this circumstance may have affected a higher proportion of HRP2 test results than pLDH results.
Double reading of all slides and the inspection of 200 fields before scoring a blood smear as negative were performed to ensure high quality results. Nevertheless, it may be possible that some low parasite densities went undetected and were therefore wrongly classified as negative [
20].
The PCR molecular assay is proved to be more sensitive than a well performed microscopy [
21] and would have provided more comparable results with recently published studies [
22]. Nevertheless, while the high sensitivity provided by the PCR is important in contexts where it is crucial to detect infections with low parasite density—e.g. in a malaria elimination programme—its added value in clinical diagnosis is arguable, since the clinical meaning of a child’s sub-microscopic infection in a high-transmission area is still debated.
Children with recurrent parasitaemia during follow-up were excluded from the time to become negative analysis. This choice had the consequence of reducing the size of the analysable patient cohort. Reduction was limited in the low-transmission setting (eight patients in Mbarara), but led to exclusion of one-third of all patients at the Kazo site (89 patients). The characterization of polymorphic markers to distinguish a re-infection from a recrudescence, a standard method in anti-malarial efficacy studies, would have allowed to censor, rather than to exclude, these children.
The time to become negative for a given RDT was determined on the basis of the first negative result during follow-up, and assumed that any subsequent result with the same RDT was also negative. It may be possible that, in some few instances, a subsequent RDT could have been positive. This definition may therefore have biased estimates of time to become negative towards shorter times.
Authors’ contributions
FG, ALP, MDS and JFE designed the study. FG and CN implemented and supervised the clinical field activity. DN and YB implemented and supervised the laboratory field activity. MB contributed to the data analysis. FG carried out the analysis and drafted the manuscript and all authors provided critical comments to the final manuscript. All authors read and approved the final manuscript.