Background

HIV/AIDS is still one of the world’s most devastating pandemics, with sub-Sahara Africa the most affected region [1,2]. HIV testing remains a critical entry point for prevention, treatment and care. Since the mid 1980’s when an accurate and reliable antibody test became available, HIV testing has been offered mainly through client-initiated voluntary counseling and testing (VCT) services [3,4]. Uptake has been very low despite high willingness to be tested [5,6] and stigma has been identified as one of the strongest barriers to VCT [7]. As antiretroviral treatment (ART) became available in low-income settings, Provider Initiated Testing and Counseling (PITC) or Routine opt-out Testing and Counseling (RTC) has been recommended to be implemented in countries with generalized epidemics. Despite a steady increase in test rates over the years [8,9], uptake is still unacceptably low and distributed in an inequitable way [10].

To meet the increasing need for HIV testing, testing methods have shifted from sophisticated techniques such as enzyme linked immunosorbent assays (ELISAs), which are usually performed in a traditional laboratory by highly trained laboratory professionals, to rapid tests (RTs) [11,12]. RTs are simple to perform, are accurate, give results without the need for laboratory equipment and have allowed provision of test results and post-test counseling in a single visit [13–16]. This has allowed non-laboratory personnel such as lay counselors and nurses to perform HIV tests [11–13], thus in keeping with the WHO recommendation of task-shifting of HIV testing services [17,18]. Concerns have been raised, however, on the effects of this expansion on quality of testing and accuracy of test results [12,13]. Given the high volume of testing, even a small error rate can result in a high number of misdiagnosed cases. For example a 5% error rate in testing 2 million people could result in 100,000 erroneous diagnoses. This calls for coherent quality assurance (QA) systems to regulate and monitor performance of HIV testing [11,12,19].

Countries such as Uganda, Zimbabwe, China and India have reported the establishment of national QA programs through national reference laboratories (NRLs). These programs assess quality of testing and ensure accuracy and reliability of test results [11,20–22]. QA monitors all aspects of the testing process (pre-analytic, analytic and post-analytic) and is implemented in mainly two ways: 1) Internal Quality Control (IQC) and 2) External Quality Assessment (EQA). IQC is used to evaluate and ensure that the test precision is optimal, while EQA provides additional external checks to the testing process [23–25]. EQA is usually implemented through a combination of 3 approaches: 1) Re-testing 2) Site supervisory visits (SSV) 3) Proficiency testing (PT). Due to the huge volume of testing required, re-testing has proved to be neither practical nor achievable where human and financial resources are limited [12]. SSVs, which involve an on-site review of all aspects of the quality system, require a large pool of trained personnel to conduct costly visits. PT, a generally effective EQA approach [12,25], is a tool where simulated specimens issued by the NRL are tested by participating HIV testing sites and individual site performance is assessed by comparison to the NRL expected results and collective performance of all participants [26]. PT establishes the degree of accuracy of test results, evaluates technical competence and identifies staff training needs [13,19,27].

Though HIV RTs are simple to perform, inadequate training and experience of test performers may present a limitation to achieving accurate diagnostic testing [11,14,19]. Test kit manufacturer’s instructions often only illustrate examples of clear-cut results along with strong-intensity test and control lines. Thus operators without in-depth training and experience may have difficulties in interpreting weakly reactive results, or in understanding and resolving indeterminate (inconclusive or discrepant) [14,19] or invalid (un-interpretable) results. Some studies have suggested that difficulties of this nature may occur more among non-laboratory personnel if not properly trained [28,29]. Comprehensive hands-on training and on-going performance monitoring has been suggested to be essential for all test performers [12,14,30]. In addition to training, other factors such as adherence to testing procedures, policy development, test kit availability, supervision, skilled human resource availability and workload need to be addressed to ensure quality of testing and the successful expansion of the service [11].

Adult HIV prevalence in Zambia has been estimated at 13.5% [2]. An overall decline in HIV incidence has been observed since the mid-1990s but with marked geographical differences [31,32,33] and has been positively associated with education attainment [34,35]. HIV testing services have been available in Zambia since the late 1980s and with a country-wide scale up of VCT from 1998. In recent years, VCT services have expanded from about 650 sites in 2006 to about 1689 sites in 2010 [36]. With this expansion, a sharp rise in reporting ever tested for HIV has been observed [37,38]. The Ministry of Health (MOH) and its collaborating partners provide HIV rapid testing training to prospective testers of various professional backgrounds [39]. Through the University Teaching Hospital Virology Laboratory (UTHVL), the NRL, the MOH established the Zambia National Quality Assurance Program (ZANQAP) in 2009, with the aim of monitoring the performance of HIV testing across the country. PT and SSVs are the two main EQA approaches being practiced [40]. Despite the rapid scale up of HIV testing services in high HIV prevalence countries, documentations in the literature assessing accuracy of test results are few. The aim of this study was to assess the overall level of accuracy in HIV rapid testing in Zambia and variation by different groups of testing personnel and to examine factors that are associated with accuracy.

Methods

Data analysis

Data was entered using EpiData 3.1 and MS excel 2007 and analyzed using IBM SPSS version 20 for windows for overall and trends analysis.

Tester profession was categorized into four groups as: lay counselors, nurses, laboratory personnel (scientists, technologists and assistants) and others (microscopists, Environmental Health Technologists, clinical officers and doctors). Responses to having received HIV rapid testing training was categorized into four groups as seen in Table 1. HIV testing work experience was dichotomized as ‘less’ or ‘more’ than one year. The number of staff performing HIV testing at a site was categorized into three groups as 0–5, 6–10 and 11 or more. Responses to if IQC is performed at the site was dichotomized as ‘yes’ and ‘no’. Adherence to the national HIV testing algorithm during the PT cycles was dichotomized as ‘yes’ and ‘no’.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. General background and proficiency testing characteristics by tester profession.

https://doi.org/10.1371/journal.pone.0146700.t001

Assessing level of accuracy.

Accuracy was expressed as a percent score for each tester/site, tester group and each PT [42] (Table 2). Accuracy ranged from 0% (no specimen correctly identified) to 100% (all five specimens correctly identified) for each PT. Mean scores were calculated and reported with 95% confidence intervals (CIs). The Kruskal-Wallis one-way analysis of variance (ANOVA) by ranks test was used to assess differences in accuracy between the tester groups. Post hoc multiple comparisons using a series of Mann-Whitney U tests with Bonferroni adjustment were used for pairwise comparisons to determine which groups were statistically significantly different from the other. Similar analyses were performed to assess whether differences in accuracy between PT1 and PT2 overall, and among sites that participated in both PT cycles were statistically significant.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Proportion of tester groups who achieved a specified level of accuracy in PT1 and PT2.

https://doi.org/10.1371/journal.pone.0146700.t002

DTS panel tests results.

The DTS coded as A1 to A5 (PT1) and B1 to B5 (PT2) were analyzed to obtain frequencies of correct, false (positive and negative) and discrepant results (Table 3), as well as to determine the level of agreement with the expected results. The reported false-negative, false-positive and indeterminate (discrepant) results were further investigated to determine which categories of testers reported them as such (Table 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Expected and reported results for each DTS specimen in PT1 and PT2.

https://doi.org/10.1371/journal.pone.0146700.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Distribution of reported false and indeterminate results by tester profession in PT1 and PT2.

https://doi.org/10.1371/journal.pone.0146700.t004

Factors associated with accuracy in HIV rapid testing.

Multiple linear regression analysis was used to examine factors associated with accuracy. The regression analyses were performed step-wise. Bivariate associations were presented first followed by the multivariate associations in four steps as seen in Table 5. Preliminary analyses were conducted to ensure no violation of the assumptions of normality, linearity and multicollinearity. No interactions were identified. All analyses were done first for the whole group then stratified by profession of tester. The dependent variable ‘accuracy’ was used as a continuous variable and was coded 0 to 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Factors associated with accuracy in HIV rapid testing among all tester groups in PT2.

https://doi.org/10.1371/journal.pone.0146700.t005

Results

Discussion

The majority of the testers were lay counselors and nurses in both PT rounds. The overall accuracy level was 93.1% and 96.9% (p = 0.000) in 2009 and 2010 respectively. Further, among sites that participated in both PT cycles, accuracy level was 91.4% and 96.7% (p = 0.005) in PT1 and PT2 respectively. Differences in accuracy were seen between tester groups in 2009, with laboratory personnel being more accurate than lay counselors, nurses and others (p = 0.005), while in 2010, no statistically significant differences were seen. Supposing that the 2010 results represent the national accuracy level and that at least 3 million tests were performed [43], nearly 100,000 individuals would have received an incorrect HIV test result. The seemingly small error rate therefore has substantial implications for many individuals. Having received the standard HIV rapid testing training and adherence to the national HIV testing algorithm were positively associated with testers’ accuracy.

Lay counselors and nurses together constituted the largest group of testers, indicating a rapid implementation of task-shifting in HIV testing services [44] and hence a need to concentrate efforts in these groups to ensure higher levels of accuracy. In the first round, laboratory personnel were substantially more accurate than the non-laboratory personnel. However, no significant variation in accuracy between the tester groups was seen in the second round due to a great improvement among all non-laboratory tester groups and particularly among lay counselors. We see this as an indication that there is a great potential for these groups to attain higher accuracy levels and perform testing as well as laboratory personnel. One possible explanation as previously suggested is that lay counselors may be more receptive to learning new procedures that are outside their routine occupations [11]. Although laboratory personnel showed a stable performance in both PT cycles, there is a need for continuous training to improve competencies [12,25]. Though a specimen reconstitution step that is not normally required in routine testing was introduced, the fact that it was found to be easy by nearly all the testers showed that they were comfortable with DTS as PT material for EQA [12].

A significant improvement in performance was revealed from PT1 to PT2 overall, and among sites that participated in both PT cycles. Coupled with the individual tester group results, these findings suggest that PT implementation has a positive impact on performance. Possible reasons for this improvement could have been the site supervisory visits (SSVs) by the NRL experts [45], as well as improved adherence to testing procedures, which could be directly linked to increased training. These findings suggest the need for EQA programs in collaboration with their MOHs to prioritize investment in SSVs as well as training of testers, regardless of setting or profession, for enhanced performance.

Our study revealed some difficulties in interpretation of results in both PT rounds, with the occurrence of false-negative, false-positive and indeterminate interpretations. The problems tended to occur more among lay counselors and nurses than laboratory personnel and others. According to the national HIV testing algorithm, specimens that give discrepant results between the screening and confirmatory tests should be subjected to the tie-breaker test. On the other hand, specimens that test negative on the screening test are not tested further, thus false-negative interpretations are not defined in the testing algorithm. The occurrence of false-negative interpretations was of concern, and highlights the need to further investigate the ability of testers to correctly interpret test results and the potential need for further training [19]. We cannot exclude the possibility that the occurrence of false-positives could have arisen from cross-contamination due to poor handling techniques during the processing of the DTS, i.e. at the point of reconstitution since only one transfer pipette was provided per PT package, or at the point of testing. These findings support previous publications suggesting that even though HIV rapid tests may be easy to perform, they may be fraught with difficulties in interpretation of results particularly if testers have limited training and experience [14,19]. Further, our findings confirm previous observations that non-laboratory personnel may have more difficulties of this kind [28,29]. Based on our findings, a review of the current training and the development of a curriculum that is tailor-made to profession of testers may be helpful. The training duration could be increased to include more practical sessions. A previous home based VCT study in rural Zambia found that with more comprehensive training and hands-on practical experience, lay counselors were able to perform testing proficiently and achieved a 100% accuracy score during a PT exercise [10].

We assumed that the level or type of training would affect testing accuracy [14,19]. Our results revealed accuracy to be associated only with exposure to HIV rapid testing training as a focused training, as compared to having received other trainings. The standard national HIV rapid testing training is a three-day training focused on principles of HIV rapid testing and practical sessions for hands-on experience [12,39]. Other training programs have various components included in their training curricula, with some components such as counseling being more emphasized than testing [40]. Our findings suggest offering all testers the standard HIV rapid testing training, regardless of any other training they may have previously received. In addition, reports that were given by the testers during the PT exercises revealed a multiplicity of trainers (about 40 organizations) that had provided HIV rapid testing training of varied content and duration ranging from 2 hours to 6 months. This shows the need for the establishment of a central coordinating body e.g. the NRL or equivalent institution by the MOH to ensure training is standard for all testers [12].

In this investigation, having more than one year work experience was not associated with accuracy, contrary to previous publications [12,19]. Similarly, the data did not support an association between accuracy and having more numbers of skilled staff performing HIV testing. Inadequate numbers of skilled staff in a facility could translate into a higher workload for the few available staff. This has been found to have a negative impact on quality of service, particularly in public institutions [46–48]. Many HIV testing facilities report high staff turnover and attrition, with rural public facilities having the lowest numbers of skilled staff as compared to urban facilities [46,49]. Further, many rural sites are located in remote areas which may pose a challenge to quality service provision with regards to easy access to test kits and reagents, training as well as external supervision, communication and networking with other testing sites at regional or national level [24]. Strategies for staff retention and improved service provision conditions need to be made a central issue in health policy [46].

Adherence to the national HIV testing algorithm was found to be positively associated with accuracy. An algorithm involves the use of categories of tests in a particular sequence which have been selected and validated through an evaluation process [12,50]. Thus an alteration in sequence could have a negative impact on test sensitivity and specificity. Non-adherence to the national testing algorithm was defined as performing parallel testing, confirmatory testing on specimens that are non-reactive on the screening test, reporting positive results based on one test (screening test) or confirming a reactive specimen with the wrong test (tie-breaker). Though parallel testing and confirming non-reactive specimens did not have a negative impact on accuracy, such practices in a serial testing strategy may lead to wastage of test kits and increase the cost of reagents [12]. However, reporting positive results based on one test and confirming a reactive specimen with the wrong test were of concern. The reasons for such practices could have been limited training and understanding, or non-availability of confirmatory test kits.

A global concern on the use of tie-breakers in routine testing has been raised. Tie-breakers have been found to not always resolve the HIV status with accuracy [12]. Some studies have found that due to antibody cross-reactivity, i.e. the production of antibodies by various foreign antigens and infectious agents that non-specifically cross-react with antigens in the test kits, the use of a tie-breaker has sometimes led to false-positive diagnosis. This has led to individuals being wrongly enrolled on ART programs, as well as devastating individual consequences. Some studies have suggested re-testing of individuals already on ART programs, as well as samples testing positive on the tie-breaker with confirmatory tests such as Western blot or PCR. A few studies have also indicated a need for ART and prevention programs to plan for quality assurance in HIV testing [51,52,53]. Other studies however have found high accuracy in HIV testing using an algorithm that includes a tie-breaker [54]. The outcome thus depends on the test kit that is selected to be used as a tie-breaker. In some countries, a blood specimen is collected from the individual with discordant results and sent to a facility capable of performing further testing to confirm the HIV status. But often, rather than using a tie-breaker, clients are re-tested after 4 weeks to resolve the HIV status. This approach has been found to be more likely to provide a correct diagnosis, though clients may not return for follow-up testing. Appropriate counseling may help ensure a return visit [12].

Selection bias might have been one of the possible biases in this study. The sample was not a random sample of all the sites in the country. However, sites were selected to represent all types and levels of facilities in both rural and urban areas, i.e. public, private and mission. Therefore, though only a few sites (30%) of the total 1689 registered sites countrywide were reached by the program by PT2, the findings are likely to reasonably well portray the national situation. However, the selection of rural sites could to some extent have been biased towards ease of access, thereby possibly leaving out those sites most challenged in provision of quality service and thus we are likely to have over-estimated rural accuracy. Further, non-participation, which was due to hard-to-reach sites not receiving the PT materials or reporting results to the NRL on time, might have been another possible source of bias. This type of non-response might have also led to over-estimation of accuracy. Non-response was substantially lower in PT2 than in PT1, with the involvement of the national courier and MOH partners in the distribution of the PT materials and the return of the results to the NRL for analysis. Therefore generalization of rural or remote area accuracy may be restricted to the selected sites and from which reports were received. Self-reports of training, experience and adherence to procedures are subject to recall and reporting biases which might to some extent have diluted the estimated associations. PT involves only a few specimens and the test results may not represent routine test performance. This may partly be due to the greater care that is taken by testers in handling PT specimens [50]. Further, the person assigned to test the specimens may be the most experienced at the site. Accordingly, the direction of these biases would be an over-estimation of accuracy, but the magnitude was not possible to estimate.

A particular limitation in this study was that each PT round differs from the previous with the addition of new sites and mostly with different testers participating due to high staff attrition. A more optimal design would have been a cohort design to observe the same testers and sites and measure accuracy over a period of time, thus the impact of quality assurance systems. The use of retrospective data has its limitations in that the data was collected for routine program purposes and not particularly for this study and therefore we were restricted to the available data. In addition, the areas of missing data for some of the variables could to some extent have biased the associations. Another limitation is that due to limited resources, PT exercises are currently conducted in a few sites of the total sites countrywide, and are conducted only once a year thereby limiting coverage and closer monitoring of performance. Possible suggestions to reduce some of the limitations may include the use of a decentralized approach in combination with the MOH partners and other local organizations that could be trained to run the PT program and provide coverage to sites in specific geographical areas. Provincial reference laboratories could be set up to oversee and compile data for overall analysis and coordination by the NRL [12].

Conclusions

The study showed that there was an improvement in tester groups and overall accuracy level in HIV rapid testing from the first to the second exercise. Further improvement is urgently needed, however, and the national HIV PT system could be an important tool in this regard which will need to be strengthened and given higher priority in terms of investment. The findings showed that task-shifting in HIV testing services has taken effect in Zambia, with lay counselors and nurses constituting the majority of testers. The type of training testers received and adherence to testing procedures appeared critical in achieving higher levels of accuracy. A regulatory framework for training is needed to ensure central coordination of training that is standard.

Acknowledgments

We would like to acknowledge the Zambia National Quality Assurance Program (ZANQAP) Team at the Virology Laboratory: Goodness Ngwele, Caroline Chisenga, Barbara Mwenechanya and Chisenga Musonda. We also acknowledge the Centres for Disease Control and prevention (CDC), the Ministry of Health (MOH) and collaborating partners for the logistical and technical support to the ZANQAP, and the University Teaching Hospital (UTH) Management together with the UTH HIV/AIDS Program (UTH-HAP) secretariat for providing logistical support.