Factors associated with malaria microscopy diagnostic performance following a pilot quality-assurance programme in health facilities in malaria low-transmission areas of Kenya, 2014

In 2013, approximately 198 million cases of malaria and 584,000 deaths occurred globally, and 90% of the deaths were in Africa [1]. In 2012, Kenya had an estimated malaria mortality rate of 27.7 per 100,000 people [2]. Malaria accounted for almost 9 million outpatient visits in Kenya in 2012, which represented approximately 21% of all outpatient consultations [3].

Parasitological diagnosis is recommended by the World Health Organization (WHO) for all patients in whom malaria is suspected as part of the ‘test, treat, track’ strategy [4, 5]. Both microscopy and malaria rapid diagnostic tests (RDT) are recommended malaria diagnostic methods by the Kenya National Malaria Control Programme (NMCP) [6‐8]. Although over 90% of public health facilities in Kenya had the capacity to diagnosis malaria, the proportion of facilities performing malaria microscopy, approximately 50%, has not changed in recent years [9]. Despite the high proportion of health facilities offering malaria diagnostic services, only 31% of malaria cases were confirmed by parasitological diagnosis in Kenya in 2013 [3].

Limited microscopy services in health facilities in Kenya and across sub-Saharan Africa have been attributed, in part, to limitations in the availability of equipment, supplies, working environment, training, and supervision [10‐12]. Increasing and sustaining access to prompt diagnosis and effective treatment for at least 80% of the population across all levels of the health care system and epidemiological zones is a key objective of the Kenya National Malaria Strategy 2009–2017 [6]. Implementation of the national strategy included providing health facilities with microscopes and laboratory supplies and improving the skills of microscopists through formal microscopy refresher trainings at microscopy centres of excellence [6, 13].

From June to December 2013, the NMCP in coordination with the Malaria Diagnostics Center, Walter Reed Army Research Institute, initiated a pilot to operationalize the laboratory quality assurance (QA) policy and plan for malaria diagnostics in health facilities in malaria low-transmission areas [8, 14]. Malaria low-transmission areas were prioritized because of concerns surrounding over-diagnosis of malaria due to poor microscopy practices. Laboratory QA programmes have been shown to improve the diagnosis of malaria and, in particular, microscopy accuracy [13, 15]. Components of the 7-month pilot QA programme included 4 1-day visits by trained QA laboratory officers, who promoted internal QA/quality control (QC) processes, provided supportive supervision and on-job training, and cross-checked at least 10 malaria microscopy slides at each visit [8, 14, 16]. The pilot QA programme implementation is described in detail elsewhere [16]. Independently in 2013, there were other laboratory-strengthening activities ongoing in Kenya, such as malaria microscopy refresher trainings and the WHO Stepwise Laboratory Improvement Progress Towards Accreditation (SLIPTA) programme. The WHO SLIPTA framework was established to improve the quality of public health laboratories in developing countries through standardized processes to meet international accreditation [17]. In early 2014, a survey was conducted to identify factors associated with accurate malaria diagnosis by microscopy in 42 health facilities in malaria low-transmission areas of Kenya.

All selected heath facilities agreed to participate in the survey. Participating health facilities were located in 10 (38%) of 26 low-malaria transmission counties. Among surveyed facilities, 58% were primary care facilities (i.e., dispensaries [10%] and health centres [48%]) and 42% were hospitals (i.e., primary [38%] and secondary or referral [4%]) (Table 1). More QA-pilot facilities were in urban settings (48 vs 19%), participated in an external laboratory-strengthening program (i.e., SLIPTA) (19 vs 14%), and had microscopes in good optical condition (95 vs 86%) compared to non-QA pilot facilities. The number of microscopists per facility and daily malaria slide workloads were similar across surveyed facilities (Table 1). As shown in Table 2, more microscopists in QA-pilot facilities had completed recent refresher training (68 vs 29%), had worked in a malaria high-transmission area (63 vs 21%), and had knowledge of national malaria diagnostic and treatment guidelines (84 vs 39%).

A total of 756 malaria slides were collected from the health facilities surveyed; 204 (27%) slides were read as positive for malaria by health-facility microscopists and 103 (14%) as positive by expert microscopists. In Fig. 1, slides are stratified by facility QA-pilot programme participation and recent training status (i.e., formal initial or refresher microscopy training within the year prior to the survey) of the microscopists. More microscopists (68%, 38 of 56) had completed recent refresher training in the QA-pilot facilities compared to non-QA pilot facilities (29%, 24 of 82) (p < 0.01). In QA-pilot facilities, recently-trained microscopists read 285 (75%) slides compared to 176 (47%) in the non-QA pilot facilities (p < 0.001). Recently-trained microscopists in QA-pilot facilities performed better on all microscopy performance measures with 97% sensitivity and 100% specificity compared to recently-trained microscopists in the non-QA pilot facilities with 69% sensitivity and 98% specificity (p < 0.01). Microscopists without recent microscopy refresher training performed the same regardless of facility participation in the QA-pilot programme.

Overall as shown in Table 3, QA-pilot facilities performed significantly better on measures of diagnostic accuracy (i.e., sensitivity, specificity, PPV and NPV) against expert reference compared to non-QA pilot facilities. The overall inter-reader agreement between QA-pilot facilities and expert microscopy was κ = 0.80 (95% CI 0.74–0.88) compared to κ = 0.35 (95% CI 0.24–0.46) in non-QA pilot facilities (p < 0.001). Table 3 also shows the diagnostic performance measures stratified by service level; only primary hospitals participating in the pilot QA programme performed statistically better on all diagnostic accuracy measures compared to non-QA pilot facilities. In total, 351 (93%) slide results were read as concordant with expert reference from QA-pilot facilities compared to 292 (77%) in the non-QA pilot facilities (p < 0.001) (Table 3).

In unadjusted logistic regression analysis shown in Table 4, all the microscopist characteristics were associated with accurate malaria diagnosis except initial level of training, but only pilot QA programme participation and good optical condition of microscopes were institutional factors associated with accurate malaria diagnosis. In adjusted multivariable logistic regression analysis, recent microscopy refresher training (prevalence ratio [PR] = 13.8; 95% CI 4.6–41.4), ≥5 years of work experience (PR = 3.8; 95% CI 1.5–9.9), and pilot QA programme participation (PR = 4.3; 95% CI 1.0–11.0) were the only factors significantly associated with accurate malaria diagnosis.

This observational study demonstrated that diagnostic accuracy of malaria microscopy was positively associated with recent microscopy refresher training and ≥5 years of experience for microscopists and health facility participation in the pilot QA programme. The findings are consistent with other studies from Kenya and elsewhere that have shown both laboratory QA programmes and microscopy refresher trainings improve malaria microscopy performance [13, 15, 23‐25]. In 2013, the NMCP independently started both formal refresher trainings for microscopists at a malaria microscopy centre of excellence and the pilot QA programme for malaria diagnostics at 83 health facilities; both the refresher trainings and pilot QA programme were intended to improve malaria diagnosis by microscopy. However, implementation of the two diagnostic strengthening components was not coordinated or systematic in health facilities, across service-provision levels or administrative zones, which hindered independent evaluation of the pilot QA programme.

Recent microscopy refresher training at the individual level was more strongly associated with accurate malaria diagnosis than health facility participation in the pilot QA programme. However, microscopists who had recently completed refresher training and worked in a facility that was part of the pilot QA programme had the best performance for all measures of diagnostic accuracy. These findings suggest that synergies exist between formal microscopy refresher training and the pilot QA programme. Implementation of both diagnostic strengthening components together appear to produce the best performance results. Therefore, the NMCP and partners should consider systematically implementing formal microscopy refresher training and the QA programme together as a package of interventions to improve parasitological diagnosis of malaria by microscopy in accordance with national and WHO guidance [8, 10, 14, 26].

Malaria microscopy refresher training was an important confounder in the study. The study was powered to detect differences at the health-facility level rather than at the individual microscopist level, and malaria microscopy refresher training was not uniform across surveyed facilities. Twice as many microscopists from QA-pilot facilities had recent refresher training compared to non-QA pilot facilities. Three-quarters of the malaria slides from QA-pilot facilities were read by microscopists who had recently completed malaria microscopy refresher training compared to less than half of the slides from non-QA pilot facilities. In addition, there were other general laboratory-strengthening activities ongoing, such as SLIPTA, in a minority of facilities that were included in the survey. Although participation in the WHO SLIPTA programme was not significantly associated with accurate malaria microscopy diagnosis, the programme might have contributed to overall laboratory improvements that were not specifically measured [17].

Overall in QA-pilot facilities, the sensitivity, specificity and NPV were very high at over 90%. The PPV was much lower, but lower PPVs and higher NPVs would be expected because all the surveyed facilities were located in malaria low-transmission counties. These counties have community malaria parasitaemia prevalences by microscopy of between 1 and 3% during peak malaria transmission season [27]. A 2014 national health-facility survey for malaria infection found that 3.4% of outpatients who reported a history of fever within the last 48 h had a positive malaria RDT in seasonal low-transmission counties in Kenya [28]. Malaria slides were collected in January and February for the survey, which is not the peak malaria transmission season in Kenya. Therefore, most persons presenting to health facilities, even if febrile, were unlikely to have malaria at the time of the survey. In malaria low-transmission settings, the low PPV findings translate into a large number of false-positive results. Persons misdiagnosed as having malaria when they do not are at risk of not being treated for their actual illness, which can lead to increased morbidity and potentially mortality. In addition, treating people who do not have malaria with relatively expensive artemisinin-based combination therapy wastes limited resources and can contribute to the development of artemisinin resistance [4, 7, 10, 26].

Hospitals require expert microscopy for the management of complicated patients with severe malaria and co-morbidities. Expert microscopy is the gold standard for identifying mixed infections, treatment failures, and quantifying parasite density [8, 10, 26]. Hospitals generally have more substantial laboratories and resources available to maintain at least adequate, if not expert, diagnostic microscopy programmes compared to outpatient health centres and dispensaries. Outpatient health centres and dispensaries generally have high patient workloads, which makes labour-intensive diagnostics, such as malaria microscopy, challenging. Historically in Kenya, programmes and training cascaded from the highest service-provision levels to the lowest and often did not reach dispensaries due to limited resources and lower prioritization. In 2010, Kenya prioritized dispensaries to receive malaria RDTs for parasitological diagnosis since expert microscopy services were not expected at this level [3].

However, the strategy for utilizing malaria RDTs and microscopy concurrently to improve diagnostic performance across service levels and malaria epidemiologic zones is not clear in the national diagnostic and treatment guidelines [6‐8].

This study has a number of limitations. Although health facilities were randomly selected for the survey, the facilities selected to participate in the pilot QA programme were a convenience sample. Thus, the surveyed facilities are not representative of all public health facilities in Kenya, which limits the generalizability of the findings. A baseline evaluation of microscopy performance was not conducted prior to the start of the refresher trainings or the pilot QA programme. Microscopists and facilities selected for participation in the diagnostic strengthening components might have performed better at baseline compared to those not selected. Therefore, the association between microscopy performance and refresher training and the pilot QA programme might have been overestimated. Additionally, when health facilities consented to participate in the survey, they were asked to store slides during a specific time interval for later retrieval. Facilities might have preferentially stored slides for which they felt confident about the results, and microscopists might have performed better during this period because they were aware of the survey (i.e., Hawthorne effect) [29, 30]. Both situations would have resulted in an overestimation of diagnostic accuracy, but the potential bias should be non-differential across all facilities.

Another important limitation was that slide preparation quality, including the stain type and adequacy, was not evaluated. Although both NMCP and WHO recommend Giemsa preferentially for malaria microscopy, the use of both Giemsa and Field stains was common in health facilities [8, 10, 16]. Slides were not matched on parasite density either. Thick films were examined for the presence or absence of parasites; no thin films were examined for parasite density or speciation [8, 10]. Slides from QA-pilot facilities might have had higher parasite densities, which would make malaria easier to identify correctly. However, urban areas generally have a substantially lower parasitaemia prevalence compared to rural areas and a greater percentage of QA-pilot facilities were located in urban areas [21, 22]. Therefore, it is possible that persons who presented to QA-pilot facilities in urban areas would have had lower parasite densities overall; if this represented the true situation, then QA-pilot enrolled facilities would have performed better than estimated compared to non-QA pilot facilities.

Diagnostic accuracy of malaria microscopy was positively associated with recent microscopy refresher training and ≥5 years of experience for microscopists at the individual level and pilot QA programme participation at the health-facility level. Microscopists who had recently completed refresher training and worked in a QA-pilot facility had the best performance for all measures of diagnostic accuracy. Therefore, formal microscopy refresher training and the QA programme should be systematically implemented together to improve parasitological diagnosis of malaria by microscopy in Kenya.