Ten years of external quality assessment (EQA) of Neisseria gonorrhoeae antimicrobial susceptibility testing in Europe elucidate high reliability of data

Gonorrhoea is the second most common bacterial sexually transmitted infection (STI) worldwide, with a global estimate of 78 million new cases among adults in 2012 [1]. If untreated, gonorrhoea can result in complications and sequelae such as pelvic inflammatory disease, ectopic pregnancy and infertility [2]. In the absence of an effective vaccine, antimicrobial treatment along with appropriate prevention, diagnostics and surveillance, is the mainstay in the clinical and public health management of gonorrhoea and prevention of these complications. However, due to the emergence and spread of antimicrobial resistance in the causative agent, Neisseria gonorrhoeae, most previously used therapeutic agents can no longer be recommended for first-line treatment [3]. Dual antimicrobial therapy, mainly ceftriaxone 250–500 mg plus azithromycin 1–2 g, is the current recommended empirical first-line treatment for gonorrhoea in many countries [4]. As strongly emphasized in the WHO global action plan [5] and the European response plan [6] to control multidrug-resistant N. gonorrhoeae, enhanced worldwide, quality-assured surveillance of gonococcal antimicrobial susceptibility is crucial in order to ensure the effectiveness of the recommended empiric treatment, to monitor antimicrobial resistance trends, and to identify new emerging resistance.

In general, confidence in any diagnostic and antimicrobial susceptibility testing data is provided by appropriate and regular quality assurance (QA) procedures. These include validations of testing methods used, internal quality controls, and quality assessments such as internal quality assessment and, importantly, external quality assessment (EQA). In Europe, the European Gonococcal Antimicrobial Susceptibility Programme (Euro-GASP) has been monitoring the antimicrobial susceptibility in N. gonorrhoeae since 2004 [7‐11]. Isolates are tested either centrally or via a decentralised testing model where antimicrobial susceptibility testing is performed in laboratories in participating countries after fulfilling set quality criteria. Criteria include acceptable performance in the EQA and good comparability between the laboratories own national susceptibility testing data and susceptibility data generated by centralised susceptibility testing [11]. Euro-GASP includes an EQA scheme as an essential component for a quality-assured laboratory-based surveillance programme [12]. This EQA scheme aims to ensure high-quality, accurate and comparable susceptibility data between and within testing laboratories. Furthermore, successful performance in the EQA is one of the quality criteria (introduced in 2010) required for Euro-GASP to include susceptibility data generated by laboratories performing decentralised testing [11]. Participation in the EQA scheme enables any problems with the performed antimicrobial susceptibility testing to be identified and addressed, feeds into the curricula of laboratory training organised by the Euro-GASP network, and assesses the capacity of individual laboratories to detect emerging new, rare and increasing antimicrobial resistance phenotypes.

The first Euro-GASP EQA gonococcal strain panel was distributed to the newly created Euro-GASP network in 2003 [13], before the first sentinel susceptibility study in 2004 [9]. The results showed an overall susceptibility category agreement of only 70% and a crucial need for enhanced standardisation of the susceptibility testing methods used in Europe. In 2007, the Euro-GASP EQA scheme was re-established and subsequently run until 2009 as part of the European Surveillance of Sexually Transmitted Infections (ESSTI) programme [14]. Since 2010, the EQA has been co-ordinated by the European Centre for Disease Prevention and Control (ECDC).

The aims of the present study were to evaluate the performance of the Euro-GASP EQA over a 10-year period (2007 to 2016) and to compare the results of the first 5 years (2007–2011) with the latter 5 years (2012–2016; no EQA in 2013) in order to assess whether Euro-GASP data provide a high-quality and valid picture of gonococcal antimicrobial resistance in the European Union/European Economic Area (EU/EEA), i.e. on which treatment recommendations can be based. These time periods were selected to additionally assess the impact of the 2012 European Union case definitions for the reporting of antimicrobial susceptibility (http://​eur-lex.​europa.​eu/​LexUriServ/​LexUriServ.​do?​uri=​OJ:​L:​2012:​262:​0001:​0057:​EN:​PDF).

An increasing number of laboratories participated in the ECDC Euro-GASP EQA; 27 in 2016 compared with 16 in 2009. Etest was the most common methodology and GC agar base the most frequently used agar. In the last 5 years, there was a marked shift among participants to the use of EUCAST [17] breakpoints from the CLSI [16] breakpoints, most likely influenced by Euro-GASP and the publication of the EU case definitions in August 2012 (http://​eur-lex.​europa.​eu/​LexUriServ/​LexUriServ.​do?​uri=​OJ:​L:​2012:​262:​0001:​0057:​EN:​PDF), which include definitions of antimicrobial resistance and state that EUCAST clinical breakpoints [17, 18] should be adhered to. However, the lack of a recommended methodology for N. gonorrhoeae susceptibility testing by EUCAST might result in some laboratories continuing to use the CLSI breakpoints [16], which are inherently linked to the CLSI methodology, which may impact the interpretation. The harmonisation of antimicrobial susceptibility testing methods in the latter five-year period (2012–2016) in Europe is a substantial improvement compared with when the first EQA was performed and generally no common methods were used [13]. However, even though methods and SIR breakpoints have increasingly been harmonised, the overall impact on the validity of results has been very limited since 2007, as the percentage susceptibility category agreement and MIC concordances changed very little between the two five-year periods. Nevertheless, Euro-GASP will work together with EUCAST to establish best practice so laboratories across Europe can use standardised antimicrobial susceptibility testing methods for N. gonorrhoeae.

The United States Food and Drug Administration (FDA) recommends that essential agreement (within one MIC log₂ dilution of the modal MIC) and antimicrobial susceptibility category agreement should be at a minimum 90% for each antimicrobial agent [19]. In the present study, the overall concordance of antimicrobial susceptibility categories in each year reached this target (≥91%), which is a major improvement compared with the first EQA performed in 2003 (> 70%). This demonstrates an improved level of comparability of antimicrobial susceptibility results, despite the diversity of methods. In future Euro-GASP EQAs, categorical agreement using a known number of non-susceptible strains and the number of very major, major and minor errors will be established, as recommended by CLSI [20]. The comparability of Etest and agar dilution observed in this study, has also been observed elsewhere, particularly for cephalosporins [21‐23]. MIC differences in our study could be due to agar media and inoculum size differences as established previously [24, 25], along with varying levels of comparability between different disk diffusion methods and agar dilution and/or Etests [26‐30]. Identified agar media differences in this study (data not shown) in general agree with data presented from other studies, including that cephalosporin MICs were generally slightly higher from agar dilution with GC agar compared with Etests [21‐23], MICs from Etests with chocolate agar were higher [31], as well as differing MIC variation depending upon which media was used for agar dilution [32, 33]. It was suggested by Liao et al. [31] that laboratories should adhere to CLSI media recommendations (GC agar base with 1% growth supplement) [16], however the lack of commercial, off-the-shelf options of this medium makes this challenging for laboratories who do not have in-house agar-plate pouring facilities.

Discrepancies in susceptibility categories were generally because the MICs for EQA panel isolates were on or very close to the breakpoints, particularly for azithromycin in 2016 (68%), as well as for ciprofloxacin in 2014 (89%) when a triplicate was composed of isolates with MICs exactly on the resistance breakpoint. The use of triplicates/duplicates allows laboratories to investigate their own intra-laboratory concordance. However, if strains with MICs exactly on or very close to a breakpoint are selected, the overall category agreement for that antimicrobial agent can be highly affected. For this reason comparisons over time are difficult, so the inclusion of the same strains over the years, as achieved in this EQA and with comparable results, is an important aspect to consider when analysing EQA performance. Even though strains with MICs close to a breakpoint can impact on susceptibility category agreement, they are clinically relevant, effectively challenge the antimicrobial susceptibility testing in participating laboratories, and should be included in EQAs. However, when interpreting susceptibility category results it is important to consider the actual MIC for individual strains in respect to patient management and the MIC distribution for isolates contributing to susceptibility surveillance data, so isolates near or on breakpoints can be identified and appropriate caution issued. Differences in breakpoints will also impact on susceptibility category agreement, for example ten different breakpoint schemes for azithromycin were used by EQA participants from 2007 to 2012, and the breakpoints for cefixime were less harmonised during earlier years, which may have contributed to the lower concordance in 2012 (88%).

High comparability of results was also demonstrated by the high proportion of isolates tested over the 10 years that were within one (≥90%) MIC log₂ dilution and two MIC log₂ dilutions (≥97%) of the modal MIC. The overall lower MIC concordance for ceftriaxone (85%) may be due to the smaller dilution scales due to the mostly lower ceftriaxone MICs and requiring more precision, e.g. at 0.004, 0.008 and 0.016 mg/L, as compared with other antimicrobial agents with higher MICs in the dilution scales such as 4, 8, 16 mg/L. The lower MIC concordance for azithromycin (89%) was likely affected by the fact that MIC testing for azithromycin is very sensitive to minor differences in methodologies, in particular the medium used and pH (which is affected by the concentration of CO₂), as has been demonstrated previously [33, 34]. Full concordance in antimicrobial susceptibility categories and MICs will likely never be possible, due to the inherent inter-assay variation of any testing method, and particularly not before there is a complete harmonisation of antimicrobial susceptibility testing methods.

The use of the same WHO strains over the years allowed the measurement of variability over time, which was shown to be very low in this EQA. The present study has shown that the inter-laboratory reproducibility was high amongst participating laboratories, comparable in different distributions and years, and the use of standardised quality control strains [15] allows improved comparison of results over time and between as well as within laboratories.

The Euro-GASP EQA revealed high levels of competence and capability in recovering and testing strains of unknown phenotype. The high level of comparability over the 10 years of the EQA indicates that high quality data are produced by the Euro-GASP participants and gives confidence in decentralised testing and comparison of antimicrobial susceptibility surveillance data in the EU/EEA. The results from this EQA compared well with similar national schemes in Canada (> 90% for MIC and interpretation concordance) [35], India (82% interpretation concordance) [36] and Australia (3.1% error rate in respect to penicillin MICs) [32], even though the Euro-GASP EQA is regional with many different participating countries, which by default means more variability in methodologies. A quality control comparison programme for the Latin America and the Caribbean GASP region recently reported that most participants had acceptable results and the impact of the different methods on the results was also highlighted [37].

It should be noted that the Euro-GASP laboratories are frequently experienced national reference laboratories with a high level of expertise and access to training and advice from the Euro-GASP coordinators. In contrast, the global GASP coordinated by the WHO includes antimicrobial susceptibility data from both experienced as well as less experienced laboratories. It would be exceedingly valuable to implement a global EQA scheme, particularly in regions not participating in existing schemes, to monitor and support comparability of antimicrobial susceptibility surveillance data from different countries and laboratories globally. In addition, the use of a global EQA could support primary diagnostic laboratories that perform antimicrobial susceptibility testing for patient management and local surveillance studies to ensure adequate quality. The crucial need for this was illustrated in a national survey in the United Kingdom [38], where low levels of QA in gonococcal antimicrobial susceptibility procedures were identified. Confidence in the reporting of patient related antimicrobial susceptibility results is essential to avoid administering inappropriate treatment. A global EQA would additionally allow the global dissemination of important reference strains for QA and clinical strains with interesting/emerging resistance profiles or diagnostically challenging characteristics, and provide a further opportunity for laboratories to achieve accreditation standards.