Diagnostic accuracy of glycated hemoglobin for gestational diabetes mellitus: a systematic review and meta-analysis

Search strategy and data sources

We searched PubMed (MEDLINE), Embase, SCOPUS and ClinicalTrials.gov, with assistance from our Institution’s library professionals, for papers published up to October 2018 using search terms related to GDM, HbA_1c and diagnosis combined. Details of all search terms are presented in Supplementary Material. From the papers retrieved, a manual search of their references was conducted. Articles published before 1996, duplicate articles and those which were not complete were removed and the remaining articles were assessed for eligibility. The revision of titles was followed by reading the abstracts for relevance. Finally, the identification of eligible studies was carried out, based on a full reading of the articles selected by at least two researchers.

Study selection

The inclusion criteria were: (1) cross-sectional or cohort studies that assessed the performance of HbA_1c (index test) and 75 g OGTT (reference test) for the diagnosis of GDM; (2) the HbA_1c method certified by the National Glycohemoglobin Standardization Program (NGSP; http://www.ngsp.org/, 13 December 2017, date last accessed) and/or the International Federation of Clinical Chemistry (IFCC) [17], [18]; (3) studies that included pregnant women without DM prior to pregnancy or with GDM already diagnosed. Exclusion criteria were: (1) studies that did not perform the 75 g OGTT for the GDM diagnosis; (2) review articles; (3) comments, letters and/or editorials; (4) studies with a language other than English, Spanish or Portuguese; (5) articles published before 1996, as from this date on was when the standardization for the HbA_1c methods started [16], [17], [18]. Three independent reviewers (PBR, FCC and JRTT) decided which studies were included based upon the eligibility criteria. First, we screened the titles of all papers resulting from the search to identify potentially relevant articles. Afterwards, we evaluated the abstracts of these studies, and relevant articles had their full-text reviewed. Finally, the reviewers selected articles qualified for inclusion and performed data extraction from all the included reports. Any disagreements concerning study eligibility or data interpretation were resolved through discussion or, if required, a fourth reviewer was consulted (JLC or ALP).

Data collection and analysis

A data extraction form was developed and the following pieces of information were extracted from each report: (1) study details (author, publication year, country of origin); (2) study design; (3) sample size; (4) GDM incidence; (5) participant characteristics (age, gestational age, HbA_1c results); (6) test methods (details of methodology and equipment description for the HbA_1c test and the OGTT); and (7) test results (true-positive [TP] cases; false-positive [FP] cases; true-negative [TN] cases; and false-negative [FN] cases). We also attempted to contact authors for further information when data to construct a 2×2 table was unclear or additional data were required. When data were not available from the authors, the study was excluded.

Quality assessment

At least two reviewers independently assessed the quality of primary studies by evaluating the risk of bias and applicability, using the Quality Assessment of Diagnostic Accuracy Studies tool QUADAS-2, a questionnaire containing 14 questions assessing risk of bias and applicability concerns [32]. Disagreements were resolved by consensus or by involving a third reviewer (JLC or ALP). We also evaluated if the articles were presented according to Standards for Reporting of Diagnostic Accuracy (STARD) initiative guidelines [33].

Statistical analysis and data synthesis

We followed the standard methods recommended for diagnostic accuracy meta-analysis studies [34]. For each study, 2×2 contingency tables were constructed with data extracted for TP, TN, FP and FN rates. By a bivariate model using a random effects approach [35] indexes of HbA_1c test accuracy were computed: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR) and diagnostic odds ratio (DOR). PLR >1 for a positive test result is associated with the presence of disease, and NLR <1 for a negative test result is associated with the absence of disease [36]. The DOR is a single indicator that summarizes the diagnostic accuracy of a test, and higher values indicate a better test performance [37]. The overall diagnostic accuracy for the HbA_1c test for GDM diagnosis was determined by a summary receiver operating characteristic curve (SROC) for the main cut-off points discussed in each study. Afterwards, hierarchical summary ROC curves (HSROC) were used to summarize the HbA_1c performance for specific cut-offs if four or more studies presented data for the same cut-off [38]. The Fagan nomogram was applied, considering a pre-test probability of 18% for GDM, based on external data from the Metzger et al. study [6], to calculate posttest probabilities for GDM using different HbA_1c cut-offs [39]. The heterogeneity among studies was evaluated by chi-square and Cochran Q analysis, I² (measure of inconsistency, when I² has a value above 50%, it is considered that there is moderate heterogeneity, 25% is low and 75% is high) and by visual inspection of forest plots. When the studies are reasonably homogeneous, accuracy indexes from individual studies will lie within or near the interval of the pooled accuracy estimate. Deviations may indicate possible heterogeneity or outlier studies [40], [41]. We also ran the meta-analyses over again while removing studies one at a time to determine whether a particular study accounted for the heterogeneity. In addition, when data was available, we carried out subgroup analysis by specific cut-off points, HbA_1c methods and country of origin. The presence of publication bias was tested by using Deeks’ funnel plots [42]. A p-value <0.05 was considered statistically significant in all analyses, except for Deeks’ test, where a value of p<0.1 was considered statistically significant. Analyses were carried out in Meta-Disc Version 1.4 (Universidad Complutense, Madrid, Spain) and Stata Version 12.1 software (Stata, College Station, TX, USA) by METANDI command. The forest plots were constructed using Review Manager Version 5.3 (Cochrane Collaboration, Oxford, UK). All studies selected for this review were previously approved by an Ethical Review Board and consequently ethical approval was not required by this review study.

Study selection and study characteristics

With this strategy 2927 records were identified. Of those, 49 studies were assessed for eligibility. After full-text reading, 40 articles were excluded (one for different language, nine for insufficient data, four for different reference test criterion, three studies performed the diagnostic test in the first gestational trimester and 23 did not meet the research question). Lastly, nine studies met our inclusion criteria [27], [29], [43], [44], [45], [46], [47], [48], [49], and of these, only one article [43] was excluded from the meta-analysis due to a lack of relevant information to allow a proper extraction of data as it was not clear which diagnostic criterion was used to perform the ROC analysis. Nevertheless, it was included in the qualitative analysis. Eight studies were eligible for systematic review and meta-analysis [27], [29], [44], [45], [46], [47], [48], [49] (Figure 1).

Figure 1:

Flowchart of the article selection process (from [30]).

All studies included in this review totaled 6848 pregnant women, who performed the OGTT and HbA_1c test in the second or third trimesters of pregnancy for GDM diagnosis, of those 1128 were diagnosed with GDM (15.2%). Table 1 summarizes the characteristics of all selected studies. Three studies had a prospective design [43], [47], [48], one was a retrospective study [49] and five were cross-sectional studies [27], [29], [44], [45], [46]. All studies were written in English and published between 2005 and 2017. Four studies were from India, while the Arab Emirates, Australia, Brazil, China and Turkey contributed with one study each.

Quality assessment

The quality assessment of the studies by QUADAS-2 criteria is summarized in Table 2. Most studies presented a low risk of bias and applicability concerns. One study [44] presented a high risk of bias in the patient selection, flow and timing; in this study 1459 pregnant women participated, 33 of which were in the first trimester of pregnancy while the remaining women were in the second trimester of pregnancy. Another study [43] had a high risk of bias in the reference standard; this study used two different diagnostic criteria for the diagnosis of GDM and it was not clear which criterion was used in the analyses. For this reason, we did not perform the data extraction. One study [27] presented an unclear risk of bias in flow and timing, as 120 pregnant were diagnosed using the WHO 1999 diagnostic criteria and 142 were diagnosed using the IADPSG criteria. Only one article followed the recommendations and was presented according to the STARD guidelines [27].

Table 2:

Quality assessment using QUADAS-2 criteria.

Study	Checklist QUADAS-2
	Risk of bias				Applicability concerns
	Patient selection	Index test	Reference standard	Flow and timing	Patient selection	Index test	Reference standard
1. Agarwal 2005				?
2. Bhavadharini 2017
3. Khalafallah 2016
4. Rajput 2012
5. Renz 2015				?
6. Saxena 2017
7. Servket 2014
8. Soumya 2015
9. Ye 2016

1. 
   , low risk;
   
   , high risk; ?, unclear risk.

Meta-analysis

Overall diagnostic accuracy

For this analysis we considered the main HbA_1c cut-offs discussed in each article [27], [29], [44], [45], [46], [47], [48], [49]. HbA_1c thresholds ranged from 5.4% (36 mmol/mol) to 6.0% (42 mmol/mol). A total of 6406 pregnant women were included in this analysis, of those, 1044 were diagnosed with GDM. Using data from these eight studies, DOR was 6.97 (95% CI 4.17–11.65) and the area under the curve (AUC) was 0.825 (95% CI 0.751–0.899); indicating a good level of overall accuracy (Figure 2).

Figure 2:

Summary receiver operating characteristic curves (SROC) of HbA_1c for all eight studies.

Effect of the HbA_1c threshold on diagnostic accuracy

The forest plot in Figure 3 shows the sensitivity and specificity of HbA_1c for the detection of GDM across all eight included studies. For studies reporting accuracy at more than one threshold, 2×2 tables were built for each cut-off. The cut-offs 5.4% (36 mmol/mol), 5.7% (39 mmol/mol), 5.8% (40 mmol/mol) and 6.0% (42 mmol/mol) were reported by at least four studies and their data were included in the forest plots. Table 3 summarizes the accuracy measures for these cut-offs.

Figure 3:

Forest plot of the sensitivity and specificity of HbA_1c cut-offs in the diagnosis of GDM.

TP, true positive; FP, false positive; FN, false negative; TN, true negative. The blue square depicts the sensitivity and specificity for each study and the horizontal line represents the corresponding 95% confidence interval for these estimates.

Table 3:

Summary of findings.

Review question: What is the diagnostic accuracy of HbA1c in the GDM diagnosis? Population: Pregnant women in prenatal care, without previous DM, in the third trimester of pregnancy. Studies: Cross-sectional diagnostic test accuracy studies, cohort studies, reporting 2×2 data. Index test: HbA1c Reference standard: 75 g OGTT
Subgroup HbA1c, % [mmomol]	Studies (n)	Pooled sensitivity (95% CI)	Pooled specificity (95% CI)	AUC	Interpretation for every 1000 pregnant women tested prevalence=18%
5.4 [36]	4	0.503 (0.25–0.76)	0.837 (0.67–0.93)	0.779	91 cases of GDM will be detected, 89 cases will be missed, and there will be 134 false GDM diagnoses
5.7 [39]	5	0.247 (0.10–0.48)	0.955 (0.86–0.99)	0.741	44 cases of GDM will be detected, 136 cases will be missed, and there will be 37 false GDM diagnoses
5.8 [40]	4	0.107 (0.06–0.19)	0.987 (0.96–0.99)	0.626	19 cases of GDM will be detected, 161 cases will be missed, and there will be only 11 false GDM diagnoses
6.0 [42]	5	0.129 (0.05–0.27)	0.987 (0.98–0.99)	0.927	23 cases of GDM will be detected, 157 cases will be missed, and there will be 11 false GDM diagnoses

1. AUC, area under the curve; CI, confidence interval.

HbA_1c ≥5.4% (36 mmol/mol) for the diagnosis of GDM

Four studies evaluated the cut-off of 5.4% (36 mmol/mol) [27], [29], [44], [45], totaling 2808 pregnant women. The HSROC curve showed an AUC of 0.779 (95% CI 0.739–0.819; Figure 4A). The DOR was 5.20 (95% CI 3.33–8.12; I²=57.6%). Sensitivity ranged from 26% to 86% and specificity from 61% to 96% (Figure 3). The pooled sensitivity for these studies was 50.3% (95% CI 24.8%–75.7%) and the pooled specificity was 83.7% (95% CI 67.5%–92.7%) (Table 3). After re-running the meta-analysis by removing one paper at a time, when removing the study by Bhavadharini et al. [44], no DOR heterogeneity was found (I²=0%), pooled sensitivity decreased and the pooled specificity was the same (39% [95% CI 33%–44%] and 83% [95% CI 81%–84%]), respectively. However, after carefully reviewing this study, we were unable to explain the reasons why it contributed to the increase in heterogeneity for this cut-off and the results from the primary meta-analysis were considered.

Figure 4:

Hierarchical summary receiver operating characteristic curves of HbA_1c at different cut-off points for GDM.

(A) 5.4% (36 mmol/mol); (B) 5.7% (39 mmol/mol); (C) 5.8% (40 mmol/mol) and (D) 6.0% (42 mmol/mol).

HbA_1c ≥5.7 (39 mmol/mol) for the diagnosis of GDM

Five studies presented data for cut-off of 5.7% (39 mmol/mol) [27], [45], [47], [48], [49], totaling 3540 pregnant women. The HSROC curve showed an AUC of 0.741 (95% CI 0.675–0.807; Figure 4B). The DOR was 7.03 (95% CI 4.50–10.96; I²=55.7%). Sensitivity ranged from 9% to 73% and specificity from 76% to 100% (Figure 3). The pooled sensitivity for these studies was 24.7% (95% CI 10.3%–48.5%) and the pooled specificity was 95.5% (95% CI 85.7%–98.7%) (Table 3). After re-running the meta-analysis by removing one paper at a time, no article explained the moderate DOR heterogeneity for this cut-off and we were unable to explain the reasons for this heterogeneity.

HbA_1c ≥5.8% (40 mmol/mol) for the diagnosis of GDM

Four studies evaluated the threshold of 5.8% (40 mmol/mol) [27], [44], [45], [49], totaling 4160 pregnant women. The HSROC curve showed an AUC of 0.624 (95% CI 0.482–0.766; Figure 4C. The DOR was 8.54 (95% CI 4.89–14.90; I²=38.3%). Sensitivity ranged from 6% to 27% and specificity from 95% to 100% (Figure 3). The pooled sensitivity for these studies was 10.8% (95% CI 5.7%–19.41%) and the pooled specificity was 98.7% (95% CI 96.2%–99.5%) (Table 3). This meta-analysis showed low heterogeneity thus sensitive analysis was not carried out.

HbA_1c ≥6.0% (42 mmol/mol) for the diagnosis of GDM

Five studies reported data at the threshold of 6.0% (42 mmol/mol) [27], [29], [44], [45], [46], totaling 3608 pregnant women. The HSROC curve showed an AUC of 0.927 (95% CI 0.840–1.014; Figure 4D). The DOR was 11.40 (95% CI 5.34–24.36; I²=77.0%). Sensitivity ranged from 4% to 47% and specificity from 97% to 100% (Figure 3). The pooled sensitivity for these studies was 12.9% (95% CI 5.5%–27.5%) and the pooled specificity was 98.7% (95% CI 97.6%–99.3%) (Table 3). After re-running the meta-analysis by removing one paper at a time, by removing the Saxena et al. [46] study, the DOR heterogeneity was 1.4%. After a careful evaluation, this study was the only one using the WHO 1999 criteria to diagnose GDM instead of the IADPSG criteria, this fact could explain the DOR heterogeneity in this subgroup meta-analysis. However, pooled sensitivity and pooled specificity for HbA_1c ≥6.0% (42 mmol/mol) after excluding this study were practically unchanged and were 10.2% (95% CI 7.6%–13.2%) and 98.8% (95% CI 98.3%–99.2%), respectively.

Post-test probabilities

Considering the pre-test probability of 18% for GDM and the PLR and NLR for cut-offs 5.4% (36 mmol/mol), 5.7% (39 mmol/mol), 5.8% (40 mmol/mol) and 6.0% (42 mmol/mol) we calculated the post-test probabilities for GDM applying the Fagan’s nomogram (Figure 5). The post-test probabilities were 40% and 12% for HbA_1c ≥5.4% (36 mmol/mol) and <5.4% (36 mmol/mol); 55% and 15% for HbA_1c ≥5.7% (39 mmol/mol) and <5.7% (39 mmol/mol); 64% and 17% for HbA_1c ≥5.8% (40 mmol/mol) and <5.8% (40 mmol/mol); 69% and 16% for HbA_1c ≥6.0%(42 mmol/mol) and <6.0% (42 mmol/mol); respectively.

Figure 5:

Fagan’s nomograms for HbA_1c test, showing post-test probabilities for GDM.

(A) HbA_1c ≥5.4% (36 mmol/mol); (B) 5.7% (39 mmol/mol); (C) 5.8% (40 mmol/mol) and (D) 6.0% (42 mmol/mol).

Summary of the main results

In this meta-analysis we included eight studies, covering 6406 pregnant women, and of those 1044 were diagnosed with GDM. The diagnostic accuracy of the HbA_1c test was reported at different thresholds ranging from 5.4% (36 mmol/mol) to 6.0% (42 mmol/mol). The AUC was 0.825 (95% CI 0.751–0.899) with a Q* value of 0.758, indicating a good level of overall accuracy of the HbA_1c test. Four studies evaluated the cut-off of 5.4% (36 mmol/mol) [27], [29], [44], [45], totaling 2808 pregnant women. The pooled sensitivity and specificity for these studies was 50.3% (95% CI 24.8%–75.7%) and 83.7% (95% CI 67.5%–92.7%), respectively. For a cut-off of 5.7% (39 mmol/mol), five studies presented data [27], [45], [47], [48], [49], totaling 3540 pregnant women. The pooled sensitivity and specificity for these studies was 24.7% (95% CI 10.3%–48.5%) and 95.5% (95% CI 85.7%–98.7%), respectively. Four studies evaluated the threshold of 5.8% (40 mmol/mol) [27], [44], [45], [49], totaling 4160 pregnant women yielding a pooled sensitivity and specificity of 10.8% (95% CI 5.7%–19.41%) and 98.7% (95% CI 96.2%–99.5%). Five studies reported data for the threshold of 6.0% (42 mmol/mol) [27], [29], [44], [45], [46], totaling 3608 pregnant women. The pooled sensitivity and specificity for these studies was 12.9% (95% CI 5.5%–27.5%) and 98.7% (95% CI 97.6%–99.3%), respectively.

Our results compared with other reports

As far as we know, this is the first meta-analysis including a multi-ethnic population to evaluate the accuracy of the HbA_1c test in the diagnosis of GDM. A recent study in pregnant Chinese women [25] that aimed to establish the overall accuracy of the HbA_1c test for the diagnosis of patients with GDM, after a systematic review, included 5918 controls and 2812 patients with GDM. Meta-analyzed data in this report showed sensitivity of 0.762 (95% CI 0.746–0.777), specificity of 0.917 (95% CI 0.910–0.924) and an AUC of 0.93 with a Q* value of 0.841, indicating a high level of overall accuracy for the HbA_1c test in the diagnosis of GDM.

In a prospective study that enrolled 1989 pregnant Taiwanese women [50], the AUC was 0.70 and the optimal HbA_1c cut-off point to predict GDM was 5.7% (39 mmol/mol) (sensitivity=45.2% and specificity=84.1%). However, the reference test adopted in this study was two-step OGTT recommended by National Health Institute (NHI). The results are in agreement with this review, showing low sensitivity and relative high specificity for HbA_1c to diagnose GDM. Additionally, the study by Li et al. [51] reported a positive correlation of HbA_1c with blood glucose in pregnancy affected by GDM. They showed an AUC for HbA_1c of 0.854 (p<0.01). When HbA_1c was 5.43% (36 mmol/mol), sensitivity and specificity were 0.832 and 0.764, respectively. Hanna et al. [52] examined the concordance between different criteria for GDM diagnosis and observed an increased proportion of women with an HbA_1c ≥6.0% (42 mmol/mol) in the discordant cases. They then evaluated the performance of this HbA_1c threshold in the diagnosis of GDM and found a similar sensitivity and specificity of HbA_1c, around 22% and 97%, respectively, irrespective of the criteria used to diagnose GDM. They concluded that the HbA_1c test alone is unlikely to replace the OGTT in GDM diagnosis. Indeed, an optimal test to diagnose GDM is still desired. The recent study by Farrar et al. [53] evaluated through a systematic review a different test strategy for the diagnosis of GDM and concluded that there is insufficient evidence to suggest which strategy is best for diagnosing GDM, although HbA_1c data were not included in this study.

Strengths and weaknesses of the review

This study was conducted through an extensive and systematic literature search; we included papers from different countries that analyzed different populations of pregnant women. At least two independent reviewers extracted the data and the overall quality of original studies was checked by a QUADAS-2 tool to perform quality assessments and most studies presented a low risk of bias and applicability concerns. As limitations for this study, we highlight: First, although we only included studies that measured HbA_1c with standardized methods, the individual performance of each laboratory was not available. Second, we observed a high heterogeneity among the studies, mainly regarding data for HbA_1c sensitivity. Despite our efforts to analyze and exlpain the heterogeneity among studies, scarcity of data regarding interferent factors, such as anemia, iron supplementation and the presence of variants of haemoglobin in the original papers limited our analyses. However, we were able to draw attention to the likely effect of ethnicity and the use of different criteria for OGTT interpretation. One study used the WHO 1999 criteria [44] and after its exclusion a low heterogeneity was observed. Heterogeneity was also low when we pooled only studies from Asia [29], [44], [46], [47], [48], [49] and absent when we evaluated non-Asian studies [27], [45], pointing to the effect of ethnicity on HbA_1c values in different populations [20], [21]. All these possible interferences might have affected in different ways the HbA_1c levels measured by the method used in the primary studies. Third, only one article [27] followed the recommendations and was presented according to the STARD guidelines [33] which may have affected the quality of reporting of the other studies.

Applicability of findings to the review question

To make sense of the results of the meta-analysis and to assess the false-error rates, we calculated the post-test probabilities for GDM applying the Fagan’s nomogram, we considered the test performance estimates based on external data from the Metzger et al. study [6], with a pre-test probability of 18% for GDM and the PLR and NLR for cut-offs 5.4% (36 mmol/mol), 5.7% (39 mmol/mol), 5.8% (40 mmol/mol) and 6.0% (42 mmol/mol). The post-test probabilities for a positive test were 40%, 55%, 64% and 69% for HbA_1c ≥5.4% (36 mmol/mol), HbA_1c ≥5.7% (39 mmol/mol), HbA_1c ≥5.8% (40 mmol/mol) and HbA_1c ≥6.0% (42 mmol/mol); respectively. The post-test probabilities for a negative test for these cut-offs were low and ranged from 12 to 17%, like the pre-test probability of 18%. HbA_1c results ≥5.4% (36 mmol/mol) increase at least two-fold the probabilitiy for GDM whereas HbA_1c results <5.4% (36 mmol/mol) do not alter the initial probability of GDM.