Diagnostic performance evaluation of hepatitis B e antigen rapid diagnostic tests in Malawi

A serological survey was conducted in Ndirande township from December 2016–April 2018 based on a demographic census as part of the Strategic Typhoid alliance across Africa and Asia (STRATAA) study [18]. Ndirande is a large urban township located in northeast Blantyre. Participants from the census were selected for participation in the serosurvey using single-stage random sampling with replacement for refusals at the household level. If a replacement was not available in the household, we made an additional random selection for replacement from the census population. We tested 6073 randomly selected participants from the serosurvey for hepatitis B surface antigen (HBsAg). We returned to households to invite all HBsAg positive individuals aged over 16 years to participate in a detailed study of HBV disease burden. We additionally invited the household contacts of HBsAg+ individuals for testing; HBsAg+ household contacts were included in this analysis. Samples were collected in community centres close to participants homes and following collection, were stored in temperature monitored cool boxes between 2 and 8 °C prior to transportation to the study laboratory.

For the hospital study, we prospectively recruited consecutive patients from a tertiary referral hospital (Queen Elizabeth Central Hospital) in Blantyre who had cirrhosis or hepatocellular carcinoma (HCC). Between November 2017 and April 2019, research nurses screened all patients on weekdays in medical and surgical wards, the endoscopy unit and medical clinics using screening criteria to elicit symptoms or signs suggestive of liver disease or HCC. Potentially eligible individuals underwent transient elastography with a study nurse and ultrasound performed by AS. Eligible participants had liver stiffness ≥9.5 kPa after fasting for 3 h (Fibroscan 430 Mini, Echosens, Paris, France) or a hepatic lesion consistent with hepatocellular carcinoma based on ultrasound features using a standardised protocol. We excluded participants with falsely elevated liver stiffness measurements that were unlikely to represent cirrhosis, comprising those with evidence of transaminitis (ALT> 2x upper limit of normal) without ultrasound features of chronic liver disease, patients with right heart failure (IVC dilated > 2.5 cm at level of 1 cm below the cavoatrial junction and/or dilated hepatic veins on ultrasound with clinical evidence of cardiac failure), obstructive jaundice with a dilated biliary system visible on ultrasound and hepatic lesions not consistent ultrasonographically with HCC.

All laboratory work was conducted at the Malawi-Liverpool-Wellcome Trust Clinical Research Laboratories in Blantyre, Malawi. All kits and reagents, including those of the reference ELISA test, were shipped to the testing facility using temperature-controlled shipments and stored in a temperature-monitored cold room at 4 °C until use, within the manufacturer specified expiry date of the kits. Following collection, participant serum tubes (Vacutainer SST II, BD, Franklin Lakes, New Jersey, USA) were centrifuged at 1500×g for 10 min, and dipotassium ethylenediaminetetraacetic acid (K2 EDTA, BD) plasma tubes at 1200×g for 10 min, separated and stored at − 80. Participant sera were tested for HBsAg using the Monolisa HBsAg ULTRA commercial plate ELISA (Bio-Rad, Marnes-la-Coquette, France) in accordance with the manufacturer’s instructions. All HBsAg positive results were confirmed by retesting in duplicate.

HBV DNA was quantified using an in-house quantitative real time PCR calibrated with the 4th International WHO standard. DNA was extracted from 200 μl of plasma using Qiamp DNA mini (Qiagen, Hilden, North Rhine-Westphalia, Germany) and eluted into 60 μl of Tris-EDTA buffer with carrier RNA added at a concentration of 10 ng/μl. A pipetting robot (Qiagility, Qiagen) was used to transfer 15 μl of extracted DNA template, primers (400 nM) and probe (200 nM) and 25 μl of mastermix (Taqman Universal Mastermix, Applied Biosystems, Foster City, CA, USA) onto a 96 well 0.2 ml PCR plate. Primers and probes were as previously published [19] with a FAM reporter dye and a ZEN/IowaBlack FQ quencher probe (Integrated DNA Technologies, Leuven, Belgium). Real time PCR was conducted using the following PCR conditions: 95 °C for 10 min, then 42 cycles of 95 °C for 15 s and 60 °C for 1 min with ROX background adjustment using a Quantstudio Flex 7 (Thermofisher Scientific, Waltham, MA, USA). The lower limit of quantification (LLQ) of the assay was determined as 34 IU/ml and the assay linear range was 1.58 log to 8.58 log₁₀ IU/ml. All HBsAg positive samples were tested for HBeAg using the Monolisa HBeAg plus commercial plate ELISA (Bio-Rad). The assay has an analytical sensitivity of 0.64 IU/ml [0.49–0.84] according to manufacturer data, assessed by dilution of the Paul-Ehrlich Institute reference standard. Sera were initially tested qualitatively for HBeAg in accordance with manufacturer instructions. To quantify HBeAg concentration, the 1st WHO International Standard for HBeAg (Paul-Ehrlich Institute, Langen, Germany) was serially diluted 1:2 (from 100 to 0.4 IU/ml) using human serum negative for HBsAg and HBeAg, and tested alongside patient samples using Monolisa HBeAg plus (Bio-Rad). Samples exceeding the range of the standard were diluted and repeated until within range of the standard dilutions. Known concentrations of serial dilutions of the WHO standard were plotted against absorbance at 450/620 nm and a five parameter logistic (5PL) regression curve was fitted to the data [20]. Absorbance at 450/620 nm of HBeAg-positive patients samples plotted on the 5PL standard regression curve was used to quantify HBeAg concentration.

Three commercial RDTs for HBeAg were assessed: i) SD BIOLINE HBeAg (Product code 01FK30, Alere, Kempton Park, Gauteng, South Africa), ii) HBeAg Serum Rapid Test (Cassette), (Catalogue DTS382, Creative Diagnostics, Shirley, NY, USA) and iii) HBeAg Rapid Test (Catalogue RAPG-HBeAg-001, Biopanda Reagents, Belfast, United Kingdom). Participant sera was restored to room temperature prior to testing by two investigators (AS and NS) using each of the three RDTs in accordance with the manufacturers’ instructions. Investigators were blinded to the results of the reference test. The results of each test were recorded by each investigator independently (blinded to the other investigator’s scoring), after which any discordant results were resolved by discussion to achieve consensus.

To evaluate diagnostic test performance with anticipated sensitivity and specificity of 95%, as reported by test manufacturers, with precision of 10%, and based on an anticipated cohort HBeAg prevalence of 20%, we estimated a sample size requirement of 91 participants [21].

Performance characteristics were calculated using exact binomial confidence intervals. Inter-observer agreement was calculated using the Cohen’s kappa [22]. The association between HBeAg concentration and HBV DNA was assessed using Spearman’s ρ statistic. Analyses were performed using Assay Fit Pro (AssayCloud, Nijmegen, Gelderland, The Netherlands). The packages diagt [23] and kappaetc in Stata v16.1 (Statacorp, College Station, TX, USA) were used to calculate sensitivity, specificity, positive and negative predictive values and diagnostic accuracy.

The study was conducted in accordance with the Standards for Reporting Diagnostic Accuracy Studies 2015 and QUADAS-2 criteria for diagnostic accuracy studies [24, 25].

We evaluated 194 HBsAg positive participants, comprising 100 individuals who tested HBsAg positive within a community serosurvey study and 94 hospitalised patients with cirrhosis or HCC (Fig. 1 for study recruitment flowchart). Among the community and hospital populations 24 (24%) and 27 (29%) were HIV positive, respectively. Characteristics of study participants are shown in Table 1. The hospitalised population comprised predominantly men (64/94, 68.1%) and showed a HBV DNA concentration of median 4.2 log₁₀ IU/ml, and a median of 6.2 log10 IU/ml among HBeAg positive individuals. Conversely the community population comprised men and women in equal proportions and had a HBV DNA load of median 1.9 log₁₀ IU/ml and a liver stiffness of median 4.9 kPa. By qualitative ELISA, 11/100 (11%) of community participants and 36/94 (39%) of hospitalised patients were HBeAg positive.

Characteristics of the RDTs and reference ELISA used for HBeAg testing are shown in Table 2. HBsAg Good inter-observer agreement was observed for all RDTs with Cohen’s κ statistic ranging from 0.71 to 1.0. Discrepancies occurred more commonly with the Biopanda Reagents HBeAg rapid test with disagreement on 6/194 tests (3%), relative to 2/194 disagreement (1%) with Creative Diagnostics and no discordance with SD Bioline. In all cases, disagreement was due to faint result bands.

Diagnostic sensitivity of RDTs relative to the reference plate ELISA was consistently poor on evaluation of all three assays with an overall sensitivity of 28% (95% CI 16–43), 53% (95% CI 38–68) and 72% (95% CI 57–84) for the SD Bioline, Biopanda and Creative Diagnostics tests, respectively (Table 3). The upper bound of 95% confidence intervals for sensitivity for each of the three assays was below 85%. Sensitivity ranged from 36 to 64% in community samples and 25 to 75% in inpatient samples. No association between test sensitivity and HIV status was observed among the three assays (odds ratio 1.3 (95% CI 0.3–5.0, p = 0.7), 1.4 (95% CI 0.4–5.8, p = 0.6) and 1.1 (95% CI 0.3–3.5, p = 0.9), respectively. Overall specificity exceeded > 95% for all assays (Table 3). Cross-tabulated raw data are shown in the Additional file 1. HBeAg detection rate by RDTs was associated with the HBeAg concentration. (Figs. 2 and 3). For the SD Bioline, Creative Diagnostics and Biopanda HBeAg RDTs, the minimum threshold HBeAg concentration yielding consistent HBeAg reactivity was 3.1, 2.2 and 2.6 log₁₀ IU/ml respectively.

We considered the application of HBeAg as a surrogate for identifying individuals with a HBV DNA > 200,000 (5.3 log₁₀) IU/ml, in keeping with the recommended threshold for use of antiviral therapy for prevention of mother to child transmission in WHO guidelines [12]. Among HBeAg positive individuals, a weak correlation was observed between HBeAg concentration and HBV DNA concentration (Spearman’s ρ = 0.35, p = 0.02). The sensitivity of HBeAg RDTs for detection of HBV DNA above 200,000 IU/ml for the HBeAg RDTs was 22.2% (95% CI 11.2, 37.1) for SD Bioline, 53.8% (95% CI 37.2, 69.9) for Creative Diagnostics and 48.7% (95% CI 32.4, 65.2) for Biopanda RDTs. By comparison, the sensitivity for the reference test HBeAg ELISA for the detection of HBV DNA > 200,000 IU/ml was 76.9% (95% CI 60.7, 88.9). When this analysis was restricted to the 65 women of childbearing age (aged between 16 and 45 years; median age 34 (IQR 30–38)) included in this study, the sensitivity of HBeAg RDTs for detecting HBV DNA > 200,000 IU/ml was 16.7% (95% CI 0.4–64.1), 50.0% (95% CI 11.8–88.2) and 33.3% (95% CI 4.3–77.7) for SD Bioline, Creative Diagnostics and Biopanda, respectively. The sensitivity of HBeAg RDTs for detecting the treatment threshold of HBV DNA of 20,000 IU/ml as recommended in WHO treatment guidelines, was 19.0% (95% CI 9.9, 31.4) for SD Bioline, 44.8% (95% CI 31.7, 58.5) for Creative Diagnostics and 41.4% (95% CI 28.6, 55.1) for Biopanda RDTs. By comparison, the sensitivity for the reference test HBeAg ELISA for the detection of HBV DNA > 20,000 IU/ml was 62.1% (95% CI 48.4, 74.5).

We next evaluated treatment eligibility using the TREAT-B score, comparing the effect of using RDTs in place of the reference ELISA assay. A total of 43/143 participants (30.1%) met the TREAT-B criteria for starting HBV treatment (with a TREAT-B score ≥ 2), excluding the 51 individuals living with HIV. This comprised 41/67 (61.2%) hospital patients and 2/76 (2.6%) of community patients. Using RDTs instead of the reference assay, a total of 8/43 (18.6%), 5/43 (11.6%) and 3/43 (9.3%) of patients who met TREAT-B treatment criteria, were misclassified as not requiring treatment on the basis of the SD Bioline, BioPanda and Creatine Diagnostics RDTs, respectively. By contrast, 0/101 (0%), 2/101 (1.9%) and 1/101 (1.0%) were classified as requiring treatment based on the SD Bioline, BioPanda and Creatine Diagnostics RDTs, respectively, when evaluated using the reference HBeAg ELISA test, treatment was not required.