Background
According to the most recent WHO global estimates, four curable sexually transmitted infections (STIs) were responsible for 374 million cases in 2020 [
1]. Among these, infection with
Chlamydia trachomatis (CT) and
Neisseria gonorrhoeae (NG) respectively accounted for an estimated 128 and 82 million cases [
1]. Once detected, these two infections can be treated successfully with a short course of antibiotics. However, antibiotic resistance in NG has increased globally and concerns that it may become untreatable in certain circumstances have been raised [
2,
3]. Many cases of both chlamydial and gonococcal infection are asymptomatic in both genital and extragenital sites (such as anorectal and oropharyngeal) representing a significant reservoir for infections spreading and enhancing the risk of HIV transmission if one of the partners has an untreated or not fully suppressed HIV infection [
4‐
6]. In addition, if untreated, CT and NG can cause epididymitis, prostatitis and, although still under debate, infertility in the male population [
7‐
9].
In the past decade, the prevalence rates of both CT and NG have been increasing amongst populations at risk, although they have varied significantly in terms of both type of infection and anatomical sites. In 2016 a systematic review assessed the prevalence of pharyngeal and anorectal CT and NG infection among women and men [
10]. For men who have sex with men (MSM), the median prevalence of NG infection was 4.6% (min 0.5%, max 16.5%) in the pharynx and 5.9% (min 0.2%, max 24%) in the rectum, while for CT the median prevalence was 1.7% (min 0%, max 3.6%) and 8.9% (min 2.1%, max 23%) for the two anatomical sites respectively [
10]. Other studies have shown that MSM carry a high burden of CT and NG [
11,
12] and should be offered comprehensive targeted screening programmes [
7,
13,
14], although, at the time the study commenced, there were no STI molecular diagnostics approved for detection of STI pathogens in extragenital anatomic sites. In addition, where they exist, screening programmes are far more common amongst women than men and are largely restricted to sampling genital sites [
15]. To globally address this issue, the WHO in 2016 developed a strategy promoting the use of Point-Of-Care Tests (POCTs) and near-to-the patient tests as a tool for improving the continuum of STI services. The objective is to engage individuals as early as possible, retain them in care, minimizing loss to follow-up and guarantee appropriate treatment [
16].
To better guide POCTs implementation, their performance characteristics and usability should be evaluated against gold standard laboratory-based tests. These evaluations should be conducted in a range of settings with different patient populations, disease prevalence and infrastructures [
17]. Early POCTs for gonococcal and chlamydial infections such as lateral flow immunochromatographic tests (ICTs) or optical immunoassays (OIAs), have demonstrated poor performance, especially in terms of sensitivity [
18‐
21]. They require a significant number of steps for sample preparation and test performance dependent on subjective human visual interpretation. More recently, CT/NG POCTs using nucleic acid amplification test (NAAT) technologies have been introduced [
22‐
25]. Despite the high cost, these new diagnostic assays could expand their use in the field to include the detection of extra-genital NG and CT infections. A WHO laboratory validation study was carried out supporting the use of GeneXpert CT/NG POCT (hereafter termed as CT/NG POCT) for both urogenital, anorectal, and oropharyngeal specimens [
26]. While the CT/NG POCT has been widely evaluated in clinical settings in genital anatomic sites in symptomatic men and women, less data are available on its performance when used to detect largely asymptomatic extragenital infections particularly when samples are collected and tested at peripheral clinical sites. The objective of this study was to determine the performance characteristics and usability of the GeneXpert® CT/NG assay on the GeneXpert® system (Cepheid) for screening for CT and NG at genital and extra-genital sites (anorectal and oropharyngeal) among MSM attending primary sexual health services when compared to gold standard laboratory based NAATs. In addition, the operational characteristics and acceptability of this near-patient technology to clients were assessed.
Methods
Study design
This clinic-based evaluation was a multi-site, observational study with a consecutive enrolment of MSM presenting at sexual health clinics for HIV/STI screening. Detailed study procedures and testing methodologies were based on a WHO standardised core protocol and have been presented elsewhere [
27].
Study sites and population
The four study sites details (country, enrolment period, reference tests, participants, prevalence) are presented in Table
1. They were all STI-clinics for both general and key/vulnerable populations. The study protocol was approved by the Research Project Review Panel (RP2) of the WHO Department of Sexual and Reproductive Health and Research and by the WHO Ethics Review Committee (ERC) and locally by the site-specific ethics boards.
Table 1
Enrolment sites and reference testing in Italy, Malta, Peru
Italy | Infectious Diseases and Tropical Medicine Unit | 14-Feb-18; 14-May-19 | COBAS 4800 NG/CT test | 299 | 10 | 10 |
Malta | GU clinic, Mater Dei Hospital | 20-Sep-19; 26-Feb-21 | HOLOGIC Panther | 742 | 9.2 | 11.1 |
Peru | Cerits Alberto Barton | 16-Sep-19; 29-Jan-21 | HOLOGIC DTS (in-house) | 267 | 10 | 12 |
Peru | Centro Comunitario Epicentro | 10-Sep-19; 20-Jan-21 | HOLOGIC DTS (in-house) | 394 | 10 | 12 |
Total | 1702 |
Inclusion / Exclusion criteria
The target population comprised MSM presenting at sexual health clinics asking for HIV/STI screening. The term MSM is used to describe those males who have sex with other males, regardless of whether or not they have sex with women or have a personal or social identity associated with that behaviour, such as being ‘gay’ or ‘bisexual’. MSM assigned male at birth of all gender identities were included in the study, information on gender identity was not collected.
All participants had to be at least 18 years old to participate.
MSM who refused to give consent, were younger than 18 years, and/or had previously participated in the study were excluded.
Study procedure, specimen collection and testing procedures
Enrolled MSM received a unique study number. A structured questionnaire was used by all sites to collect data on demographics, behavioural characteristics, STI history and acceptability of the POCT approach. For each study participant, three types of specimens were collected: i) a sample of first void urine, ii) two oropharyngeal swabs collected by trained healthcare staff and iii) two anorectal swabs self-collected by enrolled respondents according to study standard operational procedures and manufacturer’s instructions. In addition, other biological samples were collected as part of routine clinical care. One swab per anatomical site and an aliquot of urine was used to perform the CT/NG POCT at the clinics, while the remaining swabs and urine specimen were sent to the reference laboratories. Swabs were collected in no particular order, and the manufacturer’s sample collection kits were used for the POCT and each commercial NAAT. At the laboratories, specimens that could not be processed immediately were stored at 4°C and processed within 4 days.
The GeneXpert Systems were installed at the clinics and samples were tested directly after collection by specifically trained clinic staff. Specimens generating an invalid, error or no POCT result were retested if adequate residual specimen was available. Results were recorded on a paper form.
Treatment was based on a site’s routine laboratory tests results. In the event of positive results of laboratory tests, participants received further information about the infection(s) and were treated following post-test counselling in line with international guidelines and local standards.
POCT under evaluation
The test evaluated in this study was the GeneXpert CT/NG assay (Cepheid, Sunnyvale, California, United States) whose characteristics have been detailed elsewhere [
27].
Although CT/NG POCT was originally approved for use with genital and urine specimens (first catch urine in men, urine, endocervical specimens and vaginal swab specimens in women), the U.S. Food and Drug Administration (FDA) cleared its use also with extragenital specimens (pharyngeal and anorectal swabs) in May 2019 [
28].
Reference laboratory tests
The laboratory NAATs used as reference for this study were both FDA and CE approved as required by the master protocol [
29,
30].
In all reference laboratories, trained staff processed and tested the specimens according to the manufacturers’ instructions.
CT/NG assays from two manufacturers (performed on three different diagnostic systems) were used as laboratory-based reference NAATs (Table
1). Briefly, at the Italian reference laboratory, the cobas CT/NG assay on cobas 4800 system (Roche Diagnostics, Indianapolis, USA) was used. This assay uses real-time polymerase chain reaction (PCR) technology on a fully automated system for qualitative detection of CT and NG in patient specimens. Two different targets each for CT and NG are detected, i.e. to ensure high sensitivity and specificity [
29‐
31].
The Aptima Combo 2 assay (Hologic/Gen-Probe Inc., San Diego, USA), detecting CT and NG, was performed on Tigris DTS system at the Peruvian central laboratory and on the Panther system at the WHO Collaborating Centre for Gonorrhoea and Other STIs in Sweden where the samples from Malta were tested. It has to be noticed that whereas the Panther system is fully automated, the DTS is semi-automated, requiring manual procedure from sample preparation to result reading. The Peruvian laboratory staff was specifically trained at the Hologic company in San Diego (CA) before the study started.
The Aptima Combo 2 assay is based on target capture of rRNA (23S rRNA for CT and 16S rRNA for NG) followed by transcription-mediated amplification (TMA) and a hybridization protection assay (HPA) for semi-quantitative detection of CT and NG [
29,
31].
Quality Assurance
Both the clinical sites and reference laboratories participated in internal quality control (IQC) and external quality assessment (EQA) programmes as detailed elsewhere [
32].
IQC tests were conducted monthly using dry swabs with known bacterial loads (i.e. double negative, double positive) at the POCT sites. POCT and laboratory EQA tests were performed twice during the study period using a panel of 5 dry swab samples [
32].
Statistical Analysis
Sample size calculation
For NG/CT, a specificity of 98% and sensitivity of 80% was used to estimate the sample size for the study, with a 5% confidence interval around the point estimate. A conservative approach was adopted, assuming a POCT sensitivity lower than that both declared by the manufacturer [
33] and proposed by WHO target product profiles for NG and CT POCTs [
34]. The formula used for the sample size calculation, based on the 2006 WHO/TDR expert panel document on the evaluation of new diagnostic methods and techniques [
35], yielded a target indicative recruitment for all sites of 245 positive results for both NG and CT.
Data analysis
Subjects’ demographic, STI-history and examination data, operational characteristics and acceptability of index tests data were summarised using descriptive statistics for aggregate and site level data. For all statistical tests the significance level was 5%. All statistical tests assumptions were assessed before carrying out the analysis and nonparametric alternative techniques were used in case of assumption violations. All analyses assumed that a true disease status by the reference test is known.
Sensitivity, specificity, positive and negative likelihood ratios (LR), positive predictive value (PPV) and negative predictive value (NPV) for the index test in each anatomical site were estimated by comparing the Xpert CT/NG results with those of the gold standard laboratory test [
35,
36]. Test accuracies were reported following the Standards for Reporting Studies of Diagnostic Accuracy (STARD) guidelines [
37].
A meta-analysis of diagnostic tests was used to analyse the results aggregating data from the four participating sites (Italy, Malta, Peru 1 and Peru 2). A Random Effects bivariate model with logit transformation was used to estimate pooled sensitivity, specificity, PPV, NPV, as well as likelihood ratio (LR) and diagnostic Odds Ratio accounting for between site heterogeneity. Hierarchical models were used to estimate tests diagnostic accuracy summary curve if thresholds for positivity vary across sites. Fixed effects models were used when there was low variability among sites because lack of convergence of the main model.
Predictive values were calculated from the summary sensitivity and specificity for a range of prevalence scenarios: the minimum and maximum of the observed sites’ prevalence and these values respectively minus (if possible) or plus 5%.
For data management, STATA Version 16.1 was used (College Station, TX: StataCorp LP) while SAS version 9.4 (SAS Institute Inc., Cary, NC, USA) was used for the statistical analysis.
Discussion
CT and NG infections are becoming increasingly prevalent amongst MSM. Considering that the majority of these infections are asymptomatic, screening should be carried out at extragenital sites with reliable and accurate screening tests for extragenital specimens [
7]. In fact, over the last two decades, the screening for NG and CT infections has become easier mainly due to the introduction of NAATs that currently represent the diagnostic golden standard [
29,
30].
Unfortunately, laboratory-based NAATs can be very complex, expensive and are not widely available in many healthcare settings, particularly in low- and middle-income countries (LMICs) [
38]. As a consequence, these laboratory tests are not widely accessible to the majority of people in the world [
38] and, therefore, only a limited number of individuals can benefit from this high standard of diagnosis and subsequent treatment. This limits the possibility of an equitable coverage of services, particularly for disadvantaged and stigmatized populations. To overcome this disparity, in 2016 the WHO reaffirmed the importance of the POCT approach in improving the continuum of STI services provision with the aim of engaging individuals as early as possible, retaining them in care and minimizing any loss to follow-up [
16]. The development of the GeneXpert platform and other potential near-patient platforms has enhanced the possibility of providing NAAT diagnostics at POC, thus permitting appropriate treatment at an initial consultation.
The study reported here is part of a wider international evaluation of POCTs and near-patient technologies for the diagnosis of the most prevalent and clinically challenging STIs conducted by WHO and its international partners (The ProSPeRo Network) [
39]. In this study, we evaluated the analytical performance characteristics of the Xpert CT/NG assay compared to laboratory-based NAATs and the acceptability of the near-patient platform to end users when used to screen for NG and CT in both genital and extra-genital anatomical sites among MSM attending sexual health clinics in Italy, Malta and Peru. To the best of our knowledge, all previous evaluation studies, even if they were based on prospectively collected clinical specimens, were aimed at evaluating the performance of GeneXpert system in specialized laboratories. In our study, the instruments were made available at the clinics where patients were evaluated and the testing was performed on specimens directly after their collection by non-laboratory clinic staff.
The results reported here show reasonably good agreement with those previously reported in laboratory-based evaluation studies on genital specimens [
20,
21,
40]. Considering male urine, in our real-life clinical study, the Xpert sensitivity was 91.43% (CI 95% 76.54–97.21) for NG, that resulted slightly lower compared to the laboratory evaluation (98%—CI 95% 88.4–99.9) [
21]. For CT, the sensitivity was found significantly lower (84.82%—CI 95% 67.76–93.69) compared to figures previously published (97.5%—CI95% 91.4% to 99.7%) [
20]. Despite these differences in sensitivity, the PPVs resulted above 90% with the highest observed prevalence for NG (2.68%, PPV 90.97%) and CT (6.02%, PPV 93.62%).
As for CT/NG POCT performance on pharyngeal swab, our findings are in line with a meta-analysis published by Bristow and Colleagues in 2019 [
41]. Estimated with a Bayesian approach, in their paper the positive per cent agreement ranged from 50 to 100% for CT and from 77.8% to 97.3% for NG [
41]. In our study, the sensitivity was 67.79% (CI95% 42.92–85.49), and 75.87% (CI95% 66.90–83.03) for chlamydial and gonococcal infection respectively. Considering prevalence scenarios, even with a minimum observed prevalence of 1.68% the PPV for CT was above 90%, whereas for NG this value was reached only with the highest prevalence scenario (14.74%, PPV 91.43%). This may result in a significant number of undiagnosed infections if the diagnosis is based solely on test results from a single extragenital site.
As for the anorectal site, where both gonococcal and chlamydial infections were more frequently detected, the sensitivity point estimates of our study are lower than those already published [
42,
43] although looking at the confidence interval they fall within the 95%CI of all studies. In particular, our estimates were 78.07% (95%CI 71.57–83.44) and 89.68% (83.03–93.92) for CT and NG respectively, substantially in line with those described by Goldenberg et al. (CT: 86%—95%CI 72.1–94.7; NG: 91.1%—95% CI, 80.4–97) [
42].
When positive predictive values are considered, while NG was consistently above 90% even in the lowest observed prevalence (5.03%, PPV 91.35%), CT detection showed a PPV of 93.25% only when the highest observed prevalence (12.53%) was applied.
In line with the literature, the CT/NG Xpert specificity for both pathogens in all anatomical sites was very high with the lowest estimate of 98.77% (95%CI 97.94–99.27) for NG at the pharynx. Also considering the prevalence scenarios, the NPVs were consistently above 95%, with the lowest value for CT at rectal site when the highest prevalence (95.51% with 17.53% prevalence) was used.
There may be several reasons to explain the suboptimal performance of the Xpert CT/NG assay reported in our evaluation when compared with laboratory reference NAATs.
The real-life setting, with the majority of the samples collected from asymptomatic individuals, might have had an impact on index test performance as many published results are based on evaluations conducted using clinical specimens that are from symptomatic individuals [
40,
42‐
44], which may have a higher bacterial load than asymptomatic patients. In addition, the GeneXpert platform was used in a laboratory setting by well-trained laboratory staff that might have lowered the chance of human error and therefore have increased the POCT performance [
40,
42‐
44]. Our study is one of the few, if not the only one, in which the GeneXpert system was evaluated in a non-laboratory setting by non-laboratory technicians (although specifically trained by representatives of the manufacturer) and this might have had an impact on test performance evaluation. However, the study methodology was specifically designed to guarantee the correct implementation of testing procedures in a non-lab setting. The EQA results across all study sites confirm that procedures were followed consistently [
32]. In addition, the healthcare staff involved in the study was properly trained by Cepheid and WHO trainers in the proper use of the GeneXpert system. Clinic staff followed manufacturer’s instructions for both specimen collection and testing. In addition, internal and external quality controls were routinely performed during the whole study period, ensuring that no systematic errors in the Xpert CT/NG procedures were made. Notwithstanding these discrepancies in sensitivity, it is important to describe POCT performance in terms of likelihood ratio and test efficacy rather than in terms of sensitivity and specificity alone [
45].
Another factor that may have impacted POCT sensitivity might be due to anorectal self-sampling, but the literature suggests the same quality as healthcare provider collection) [
46].
An additional factor that may have impacted on the sensitivity could be the moderate sample size of positive specimens that is also reflected in the confidence intervals although, for the sample size of the study, a conservative approach was used considering 80% sensitivity of the CT/NG Xpert, which is lower than the one declared by the manufacturer [
33] and published in other studies [
40,
41].
The study confirms that a CT/NG POCT negative result can exclude the infections and therefore guide clinicians as to whether to prescribe additional tests and/or antibiotics. The high negative likelihood ratio of the Xpert CT/NG assay can rule out the presence of both infections regardless of the anatomical site (Figs.
1 and
2).
As for the positive likelihood ratio and the positive post-test probability, these vary largely across anatomical sites and infection prevalence. Considering that, as stated by WHO in its POCT target product profile [
34], the minimal acceptable sensitivity for POCT is 90% for NG and higher than 90% for CT, our study shows that the Xpert CT/NG assay seems to have adequate performance for NG only in rectal specimens (Fig.
1) while for CT, this is the case in urine and pharyngeal swabs (Fig.
2). However, when highest infection prevalence scenarios are considered, for both NG and CT, a positive POCT result proved to be more reliable for all anatomical sites, with the only exception of NG at the pharynx where the positive post-test probability resulted slightly lower than 90% (Figs.
1 and
2).
Diagnostic tools should be considered and evaluated not only in terms of analytical characteristics, but in the broader context of clinical utility [
47]. Based on anatomical sites and country prevalence scenarios, clinicians and public health officers can better understand the meaning and the limitations of the POCT in guiding resource allocations, diagnostic pathways and antibiotic prescriptions.
NG and CT prevalence
In the past ten years, many international guidelines have recommended both genital and extragenital screening for NG and CT amongst asymptomatic MSM [
7,
13,
14,
48,
49] and, in this context, the WHO endorsed the use of POCTs to improve screening strategies and reach those never or not sufficiently tested for STIs [
16].
Testing strategies are based on infection and AMR prevalence, proportion of symptomatic infections, long-term consequences at the individual level and transmission rate. Both CT and NG infections are often asymptomatic and highly transmittable and many studies have demonstrated that a significant proportion of infections would be missed if only genital sites were tested [
10,
50,
51], particularly for specific populations such as MSM.
In the most recent literature, however, there is a lack of consensus on NG/CT testing frequency and its consequences [
52,
53]. In fact, according to Kenyon, extensive screening and subsequent treatment may be linked to a temporary decrease in bacterial STI prevalence, but an increase of resistant strains due to a much higher use of antibiotics [
53].
Across the geographical sites, the prevalence rates of CT and NG infection found in our study appear to be consistent with those among MSM reported elsewhere [
10,
51]. It should be noted that previous studies that reported higher NG and CT rates of infection enrolled many symptomatic subjects [
54‐
56]. If MSM are sampled for NG and CT from the urethral site only (using first void urine or urethral swabs), a significant number of rectal and pharyngeal CT/NG infections would be missed [
10,
50,
51] which increases the transmission risk of HIV if one partner has an undiagnosed or not fully suppressed infection and therefore perpetuating a reservoir of CT/NG that potentially further spreads these infections in the community. It is also important to bear in mind that, considering the infection and coinfection rates of CT and NG amongst MSM [
51], the availability of a valid dual CT/NG POCT allows the concomitant screening for both infections with just a single cartridge and a subsequent reduced impact of screening costs.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.