Validity of type 2 diabetes diagnosis in a population-based electronic health record database

According to the World Health Organization (WHO), type 2 diabetes mellitus is caused by the body’s ineffective use of insulin and is mainly the result of excess body weight and physical inactivity. In 2014 it was estimated that the prevalence of type 2 diabetes around the world was 9% among adults over 18 years old, and in 2012 diabetes was responsible for1.5 million deaths [1]. Observational studies have found that diabetes is associated with an increased risk of cardiovascular diseases and all-cause mortality [2‐4].

The WHO recommends continuous surveillance of prevalence and incidence of the most common noncommunicable diseases: cardiovascular diseases, cancer, respiratory diseases and diabetes [5]. Several approaches have been used to monitor type 2 diabetes, including the use of mortality statistics, surveys with or without laboratory tests, diabetes registers, electronic health records (EHRs), diabetes medications prescriptions, claims, diabetes diagnosis in hospital discharge and laboratory tests. In recent years, some algorithms have been developed for ascertaining type 2 diabetes in adults and children using administrative and clinical databases, for example in Canada [6‐10], the UK [11] and Spain [12‐14], among others. The use of EHR data for surveillance does not require bespoke data collection or patient recruitment [15], provides data for large populations and could be cost-effective [16]. However, it is necessary to evaluate the validity of this data, obtained for clinical reasons, before using them for surveillance or research.

In Navarra, a region with 640,000 inhabitants, a single primary care EHR database covers >97% of the population. This database is administered by a software based in OMI-AP [17] and is structured around a list of episodes (problems in the bio-psycho-social sphere, reasons for consultation, etc.) coded according to the International Classification of Primary Care, Second Edition (ICPC-2) [18]. Coding gathered in this database distinguishes type 2 diabetes (T90) from type 1 diabetes (T89), impaired fasting glycemia (A91 descriptive term “impaired fasting glycemia”), glucose intolerance (T99 descriptive term “Glucose intolerance”) and gestational diabetes (W85).

The aim of this study was to assess the validity of the primary care EHR data for the surveillance of incidence and prevalence of type 2 diabetes in the Navarra population.

The ability of EHR to accurately identify incident cases of diagnosed type 2 diabetes was investigated in a cohort from Navarra that had been included in a large prospective type 2 diabetes case-cohort study nested within the European Prospective Investigation into Cancer and Nutrition (EPIC-InterAct study) [19]. This is a large multi-center study to investigate how genetic and lifestyle behavioral factors, particularly diet and physical activity, interact in their influence on the risk of developing type 2 diabetes. The Navarra EPIC cohort included 8084 participants (3908 men and 4176 women) aged 45–65 years at the time of enrollment (1992–1995). Most of the participants were blood donors (75%), and the rest were civil servants and general population. More detailed information about the EPIC study methods have been described elsewhere [20, 21]. A sensitive approach was used with the aim of identifying all potential incident diabetes cases between the recruitment and December 2007 using multiple sources: self-reported diabetes or use of diabetes medication in a follow-up survey carried out 3 years after recruitment, diabetes diagnosis in the hospital discharge databases, type 2 diabetes (T90), type 1 diabetes (T89) and T99 (descriptive term “glucose intolerance”) diagnosis in primary care EHRs, prescription of antidiabetic drugs and cause-of-death register. A team of trained health professionals reviewed the clinical data to verify if the cases fulfilled the criteria proposed by the American Diabetes Association (ADA) in 2003: 1) Symptoms of diabetes (e.g.: polyuria, polydipsia, and unexplained weight loss) plus casual plasma glucose concentration ≥200 mg/dL (11.1 mmol/L). Or 2) Fasting plasma glucose (FPG) ≥126 mg/dL (7.0 mmol/L). Or 3) 2-h PG ≥200 mg/dL (11.1 mmol/L) during an oral glucose tolerance test (OGTT). In the absence of unequivocal hyperglycemia with acute metabolic decompensation, these criteria should be confirmed by repeat testing on a different day [22]. We excluded from the analysis 262 prevalent type 2 diabetes cases at recruitment, 5 participants who died before 2003 when EHR use was universalized in Navarra and 130 without primary care EHRs, resulting in a final sample of 7687 (3654 men and 4033 women).

The completeness of prevalence type 2 diabetes data recorded in EHRs was studied using “comparison of rates” [11, 23] methodology. To this end, type 2 diabetes prevalence in primary care EHR database in 2005 was compared with the estimations obtained from a health survey carried out in a census sample from the Navarra adult population in 2003 [24]. With the exception of <3% of the population (which was covered by private health insurance), the EHR database encompassed the entire Navarra population’s use of primary care centers.

A total of 1285 charts of potential diabetes cases were reviewed and 598 cases of type 2 diabetes were confirmed in the EPIC-InterAct cohort. The T90 code for type 2 diabetes in the EHR database had a high sensitivity (98.2%), specificity (99.3%), PPV (92.2%) and NPV (99.8%) (Tables 1 and 2). The degree of global agreement measured with the kappa index was very high: ƙ = 0.946 (p <0.001).

The difference between the date of diagnosis reported in the EHRs (date of the T90 code) and the date of diagnosis according to the gold standard was less than 12 months in 3 out of 4 patients (Table 3).

Type 2 diabetes prevalence estimates based on EHR data were comparable to those obtained from the health survey in all age groups. Overall prevalence of type 2 diabetes diagnosed in the EHRs among adults (35–84 years) was 7.2% (95% CI 7.2–7.3) and 5.9% (95% CI 5.8–5.9) in men and women, respectively, very similar to the prevalence estimated from the health survey: 8.5% (95% CI 7.1–9.8) in men and 5.5% (95% CI 4.4–6.6) in women (Table 4 and Fig. 1).

Most of the type 2 diabetes diagnoses in the primary care EHRs were well recorded when compared against the “gold standard,” showing a high sensitivity, specificity, PPV and NPV, and also a very high agreement. Moreover, this study showed that the information obtained from the EHRs provides a good estimation of type 2 diabetes prevalence in population with 35–84 years of age.

The incidence study in the EPIC-InterAct cohort showed that code T90 has a high sensitivity (98.2%), higher than that published in the outpatient records from a large health system, 79% [25]. The percentage of false negative cases in our study was 1.8%, and they corresponded mainly to cases that were coded as type 1 diabetes, impaired fasting glycemia or glucose intolerance. Similar sensitivity, 99.5%, was found in one study published in Spain using an internal validation method [14]. Another study conducted in Spain by Gil-Montalban et al. [12] using a similar approach as ours found a sensitivity of 83.5%. The sensitivity of administrative data ranged from 46 to 97% (median 81.5%) in a systematic review conducted by Saydah et al. [26].

In our study, we found a high specificity for the diagnosis of type 2 diabetes: 99.3%. The percentage of false positive cases was only 0.7% and most of them corresponded to patients that had impaired fasting glycemia, patients with adverse effects to some medications that cause hyperglycemia and type 1 diabetes cases. Other studies conducted in Spain have also reported specificities over 98% for the primary care EHRs [12, 14]. The review of studies based on administrative data carried out by Saydah et al. [26] found that the specificity was consistently high and ranged from 95 to 100% (median 99%).

The PPV and NPV that we found in our study were 92.2 and 99.8%, respectively. These were higher than the values observed by Gil-Montalban et al. [12] at 78.5 and 98.7%, and very similar to those found by Burgos-Lunar et al. [14] at 91.2 and 99.9% for PPV and NPV, respectively. Additionally the review of Saydah et al. [26] found that the PPV ranged from 60 to 98% (median 92%). Finally, the agreement found between the T90 codes in the EHRs and the gold standard was high at ƙ = 0.94 (p >0.001), as in other studies conducted in Spain, 0.99 and 0.79 (14,12). Additionally, Saydah et al. [26] found that the kappa index of the studies included in their review ranged from 67 to 96% (median 83%).

The prevalence of type 2 diabetes that we found in Navarra (7.2 and 5.9% in men and women ≥35 years, respectively) were quite similar to those found in adults over 30 years of age from Madrid, Spain (8.5 and 5.9% in men and women, respectively) registered in the primary care EHR database [12]. Additionally, the standardized prevalence of diabetes estimated with data from Spain’s 2006/2007 National Health Survey (NHS) [27] was 6.6% (CI95% 6.1–7.2) among men and 5.6% (CI95% 5.2–6.0) among women. The prevalence of the NHS was lower that the ones we found in our EHRs because they included adults above 16 years of age and our prevalence was calculated with older people (≥35 years), therefore our results are not fully comparable.

One limitation of the present study is the fact that the survey conducted in 2003 did not register the specific type of diabetes (1 or 2). However, the prevalence of type 1 diabetes represents only around 4% of total diabetes cases in Spain [28]. The strengths of the study are the exhaustive examination of clinical data in each case of potential type 2 diabetes identified in multiples source, the use of the whole population of Navarra for the prevalence study validation and the large number of participants included for the incidence validation study.

To the best of our knowledge, this is the first study that validates diabetes diagnosis in primary care EHRs using a multi-source approach including self-reported diabetes, hospital discharge diagnoses, drug prescription records, cause-of-death registers and several diabetes-related codes from the EHRs. Moreover, all potential diabetes cases were reviewed to confirm their fulfillment of the criteria proposed by the ADA in 2003. Implementation of algorithms that include other variables from the EHRs, especially prescription of antidiabetic drugs, would identify cases that need review to improve the accuracy of estimation as has been shown in a recent study from Spain [28].

The validation methodology used in this study might also be useful for validating EHR diagnosis of those diseases that need continuous surveillance of prevalence and incidence such as: cardiovascular diseases, cancer and respiratory diseases.

With the results of this validation study, we can conclude that the accuracy and completeness of type 2 diabetes diagnoses in the primary care EHR database proved it to be a valid source for epidemiological surveillance and quality care monitoring in our population.