Usability of a point-of-care diagnostic to identify glucose-6-phosphate dehydrogenase deficiency: a multi-country assessment of test label comprehension and results interpretation

Point-of-care (POC) diagnostic testing can be transformative for the management of disease, particularly in resource-constrained settings where laboratory infrastructure is inadequate or overwhelmed by demand. Decentralized testing close to the patient can improve quality of care and better inform treatment options as well as increase the efficiency of health care through more targeted referrals, lower costs, and faster results [1, 2]. This is particularly true in the context of malaria case management, where POC rapid tests have come to replace or supplement the use of microscopic examination of stained blood smears, which is considered the standard of care [3, 4]. While challenges with test accuracy, performance, supply, and operational use persist, these tests have significantly expanded access to malaria diagnosis where it is most needed [5].

POC testing for glucose-6-phosphate dehydrogenase (G6PD) deficiency represents another significant transformation in malaria case management. For patients with Plasmodium vivax malaria, widespread screening for G6PD deficiency has the potential to improve health outcomes, align diagnosis and treatment options in remote settings with globally recommended clinical practices, and reduce P. vivax transmission through the expanded use of 8-aminoquinolines, including primaquine and tafenoquine (Kozenis®) [6]. G6PD is a critical enzyme in red blood cells. Those with G6PD deficiency express a variant of the enzyme with reduced activity, which can make red blood cells vulnerable to oxidative damage [7]. The approximately 500 million people with this genetic enzymopathy are at a greater risk of experiencing haemolytic anaemia and subsequent clinical complications, particularly when exposed to certain drug triggers, including primaquine [7‐10]. Because of the risks associated with G6PD deficiency and primaquine use, the malaria treatment guidelines of the World Health Organization (WHO) recommend that “the G6PD status of patients should be used to guide administration of primaquine” [11]. However, due in part to test availability and operational challenges with G6PD testing at the point of care, national-level policies and practices do not routinely align with these recommendations [12‐14].

Until recently, G6PD testing capacity was largely restricted to moderate-to-high throughput tests, only available at national reference laboratories, and some use of near-patient fluorescent spot tests, which still require some laboratory infrastructure and skilled technicians. The recent commercialization of POC G6PD diagnostic tests has begun to address the global gap in test availability [15]. These tests include qualitative rapid diagnostic tests and quantitative or semi-quantitative biosensors that can be readily used at the same levels of the health system where malaria is routinely managed. Qualitative rapid tests may be adequate to differentiate severe deficiency, but they do not reliably differentiate between intermediate and normal levels of G6PD activity; specifically classifying heterozygous females with G6PD normal and deficient allele and intermediate G6PD activity as normal [16].

For this, biosensors show particular promise for their ability to distinguish between the clinically relevant classifications of G6PD status and provide reliable results in both male and female populations [17‐19]. Kozenis is indicated for individual with normal G6PD activity (greater than 70%) and, therefore, requires a G6PD test that can differentiate individuals with intermediate activity from those with normal activity [20, 21]. The exclusion of those with intermediate G6PD status along with more severe deficiencies can only be achieved with semi-quantitative and quantitative biosensor tests that can reliably differentiate G6PD deficient, intermediate and normal individuals.

However, challenges persist in the operationalization of biosensor tests. Some of the primary concerns regarding the adoption of G6PD testing at the point of care and the integration of these tests into malaria case management centre around user training and supervision [22]. Although G6PD biosensor diagnostics are portable and rapid, they are moderately more complex than traditional lateral flow rapid tests, as they involve an instrument component as well as a sample preparation step. In addition, enzymatic assays present specific challenges related to maintaining adequate specimen quality, accounting for the instrument’s sensitivity to operating and storage temperatures and ensuring user adherence to test workflows.

Further, effective use of these tests will require a diverse cadre of intended users—some with limited education and training—to not only run the tests but also to interpret the test result outputs. In the case of quantitative or semi-quantitative biosensor tests that provide a numeric result, one or two numbers, potentially with a decimal point, must be compared against predetermined thresholds that classify patients as having either normal, deficient, or intermediate G6PD status. As such, understanding device usability among intended end users is essential to inform appropriate test introduction, uptake, and ultimately, the ability of the diagnostic to improve health outcomes.

Many user challenges and risks can be mitigated in part by effective and clear training, job aids, test labelling, and instructions for use (IFU). In particular, IFUs are a foundational resource in developing a user’s understanding of a test and can thus either help or hinder user proficiency. Best practices for designing IFUs are well documented, and regulatory authorities, including the WHO, have provided considerable guidance on how to evaluate and refine the content of product labels and instructions [23, 24].

In this study, WHO guidance on methods for qualification of usability of POC G6PD tests was applied to understand the usability of the STANDARD G6PD Test (SD Biosensor, Suwon, South Korea). This study sought to explore the following questions: Can the same cadre of health workers who are currently responsible for malaria diagnosis adopt an additional biosensor test? Does the test labelling and IFU support effective operational use of the test? What additional training methods or materials should be developed and adopted to support real-world use of the test?

Accordingly, the goals of this study were (1) to assess the comprehension of the STANDARD G6PD Test packaging and labelling among intended users and (2) to assess users’ ability to interpret the Standard G6PD Test result.

SD Biosensor has developed a quantitative POC diagnostic test for G6PD deficiency—the STANDARD G6PD Test (Fig. 1). This test measures G6PD enzymatic activity and total-haemoglobin concentration in fresh human whole blood specimens (both capillary and venous) based on reflectometry assays and is intended to aid in the identification of people with G6PD deficiency to inform treatment decisions for P. vivax at the point of care. The test is registered for use in multiple malaria-endemic countries and has been validated in both malaria-endemic and non-endemic settings [17, 18].

The test includes an instrument, referred to as an analyzer and disposable strips, called test devices. It requires unitized buffer tubes and a unique sample collector called an EziTube+ as well as a lot-specific code chip to calibrate the instrument. The workflow of the test involves collecting a blood sample, using an EziTube+ to mix the blood and buffer, and then transferring the mixed sample to the test device inserted in the analyzer. After 2 min, a quantitative measurement of both G6PD activity and haemoglobin concentration appear on the analyzer screen (Figs. 2, 3).

Overall, the results of the multiple-choice questions indicated that most participants were able to understand information regarding the test’s intended use, materials, and safety concerns based on the training provided and the resources made available during the assessment.

The most commonly missed questions included (1) What is the operating temperature range for test operation? (2) After mixing the buffer and the sample, how much of the mixed specimen should be added to the test strip? and (3) When do you first insert the test strip into the analyzer? All of which were classified as non-critical.

The most missed question, regarding the operating temperature range, addresses the risk of an operator unknowingly using the test in an environment where the temperature exceeds the acceptable operating range. However, given the test’s wide range of acceptable operating temperatures (15 to 40 °C), the risk of this error occurring is low. It is also likely that the fact that this question was frequently missed was an artefact of the content emphasized during the training: the trainers may not have sufficiently highlighted this aspect of the test.

The second question addresses a critical aspect of the test procedure which, if misunderstood, could potentially impact the result of the test or the ability of the user to obtain a result. However, the instrument design and software include some design features to prevent and lower the risk associated with these types of operator errors. For example, the test includes an error code that would signal to the operator that insufficient sample had been applied to the test strip. The third question addresses a noncritical error, as the instrument will produce an error of the test strip was not fully interested into the analyzer at the beginning of the procedure.

Data from the results interpretation section indicated that participants of all literacy levels can read and record numbers with decimals from the Hindu-Arabic numeral system with very few errors. At site B, where lower-quality printouts of the simulated test readouts were used on the data collection forms, it is possible that the higher rate of transcription errors was a result of this poor print quality. However, misrecording of G6PD test results has been observed and reported both for laboratory-based quantitative tests [27] and the fluorescent spot test [28], so it is important to recognize this risk, but not necessarily as a characteristic specific to the Standard Test. Overall, participants were able to correctly identify the differences between the G6PD activity result and the haemoglobin result and transfer these numbers accurately to paper forms. Most of the missing points in this section were due to blank responses and could be a result of inconsistent instructions on the part of the test proctor or some uncertainly from the participants regarding the application of the thresholds. Results suggest that participants could benefit from clear and reinforced messages around how to apply the thresholds to male and female patient results.

Differences between the unweighted and weighted analysis are minor, with questions weighted by critical or noncritical status resulting in only two additional participants meeting the acceptance criteria. This suggests that in the current data, participant errors are distributed across both critical and noncritical error. Even with a focus on critical errors, most participants (82%) were able to comprehend key aspects of the test procedure and interpret results.

There are limitations to this study. Principally, the sampling approach has some limitations in terms of its application to a quantitative methodology and the degree to which the participants included in this study are representative across malaria case management settings. Given the wide range of potential end users of POC G6PD tests globally and across different levels of the health system, the results of this usability assessment may not be generalizable outside of the study sites. The sampling methodology, which involved development of locally-relevant definitions of POC G6PD test end user(s) and then recruiting a convenience sample from health workers at the site that met that definition may further limit generalizability. The modest sample size (n = 45 across 3 countries and 4 sites) was designed in accordance with WHO technical specifications for the quantification of usability but does not support robust sub analyses by site or specific user types, which can be difficult to quantify and compare across contexts. This usability assessment was conducted in view of one specific use case for POC G6PD testing, malaria case management, and with a group of representative end users specific to this context of use.

Even within this use case, participant’s literacy levels and background knowledge varied and either adaptations to the questionnaire based on these differences or data analysis segmented by participants’ literacy and knowledge in the future would be useful. A key limitation to the study is that important differences between the baseline knowledge and medical literacy of participants were not included in the training delivery and data analysis. Additional research, with a larger sample size of each user group, is warranted to better understand the impact of participants baseline knowledge on these findings.

Participants’ results are also highly dependent on the quality of the training provided. Due to differences in language, facilitators, and duration, there was some variation in the training content and delivery across all sites. In Brazil, the training and questionnaire were translated into and delivered in Portuguese whereas in Ethiopia and India the training and questionnaire were delivered in English.

In addition to the differences in training, there were also variations in how the study team proctored the assessment. Emphasis on certain content over other content and logistical issues such as how questions appeared on the test itself might have impacted the results. For example, at some sites, the proctor provided reminders not to leave responses blank. This practice was not consistent across all the study sites and may have impacted study results. Finally, this assessment provides insight only into a user’s comprehension and ability at a single point in time, shortly after the training was conducted. This study does not reflect a user’s ability to interpret results or comprehend the label after weeks or months of cumulative practice with the test. Finally, the scope of the data collection instrument is limited, and was designed as a pragmatic tool to assess key aspects of user comprehension. As POC G6PD testing expands into operational research and routine use, additional user challenges may be highlighted and could be incorporated into further adaptations of this tool.

These data provide insight into the usability of the SD Biosensor Standard G6PD test as performed by a range of potential end users working in varied roles, across multiple facility types, and from different geographies, to advance the transition toward routine G6PD testing at the point of care. First, these data may serve to inform programs and key decision-makers as to the expected usability of a POC G6PD test in the context of malaria case management. Second, these data indicate what additional user resources are needed to support effective and widespread adoption of decentralized G6PD testing. Finally, these data and the process of data collection offer insights into how these types of user assessments can be adapted; integrated into training, monitoring, and supervision plans; and used to assure user proficiency with decentralized G6PD testing across diverse health workers and levels of the health system.

The use of POC G6PD tests in the context of malaria case management will require laboratory technicians and health care providers working in diverse settings, some with limited education and training, to execute a moderately complex workflow, to interpret results and use them to inform patient care, and to understand the key risks and limitations associated with POC tests. The assessments conducted in Brazil, Ethiopia, and India suggest that this is feasible. All new health technologies, including POC diagnostic tests, will require some training before being rolled out and used across a health system. The results presented here inform where additional training and supervision could be best targeted and how laboratories and clinics can plan for successful introduction of POC G6PD testing.