Concordance between administrative health data and medical records for diabetes status in coronary heart disease patients: a retrospective linked data study

The current study was performed in the state of Western Australia (WA) which is representative of the major sociodemographic and health economic indicators for Australia [11]. The population of WA in 2004, the latter period of the study, was 1.99 million, with 75% residing in the capital city, Perth [12]. Indigenous people comprise 3.5% of the WA population [13], with around 65% living in regional, rural or remote areas [14]. Data was sourced from the population-based electronic linked health database (WA Data Linkage System) which is managed by the Department of Health WA and has been used extensively for health-related research [15]. The current study used two of the system’s core databases - the Hospital Morbidity Data Collection (HMDC) and the Mortality register. Statutory requirements mean that all hospitalizations and deaths in WA are recorded within these collections. The datasets are linked by probabilistic matching based on name, date of birth, gender and address, with manual clerical checking of uncertain links, and are regularly audited for quality [16]. Hospital discharge diagnoses are coded in the HMDC by trained coders using the prevailing ICD version and relevant modifications (ICD-9 from 1978, and ICD-10 from July 1, 1999).

The study sample was selected from two existing projects: Monitoring CHD in the Modern Era (Study 1), and More Informed Action to Improve Aboriginal Heart Health in WA (Study 2). The sampling frames for these studies have been described elsewhere [17]. A stratified sample of patients aged 35–79 years with a hospital discharge diagnosis of any cardiac condition or chest pain in 1998 or 2003, admitted to a major public or private metropolitan hospital, was identified from a linked dataset containing all cardiovascular (CVD) morbidity and mortality records. The second study similarly identified all Indigenous patients and a sample of non-Indigenous patients, aged 25–79 years, admitted to any metropolitan or rural hospitals in 2002–04. Hospital record review for these samples was undertaken and information collected and stored in a medical records database. Because of the overlap in time period between the 2003 sample in Study 1 and the Study 2 sample (2002–04), 134 patients appeared in both sampling frames. These were included only once in the medical records database.

Patients in the medical records database were included in the current study if they had a principal discharge diagnosis of CHD recorded in the HMDC (ICD-9-CM 410–414, ICD-10-AM I20-I25), because of the high recording accuracy of CHD in the principal compared with secondary diagnosis fields [18]. The first CHD admission for each patient in each time period was defined as the ‘index admission’ and selected for inclusion in the study. The administrative data for the CHD patients were linked to the medical records database via a unique identification number assigned to every hospital admission in WA. Because of known underestimation of identification of Indigenous status in hospital discharge data [19], a patient was included as Indigenous if 25% or more of all of their HMDC records since 1980 were recorded as Indigenous.

Trained research assistants collected data from medical records. Thirty-nine admissions could not be reviewed due to missing medical notes. Data were obtained from admission notes from the emergency department and inpatient medical records, and each comorbidity documented as present, absent, or not recorded. Treatment of diabetes with insulin or oral hypoglycaemic drugs was identified from inpatient and discharge drug records for the admission under investigation. Patients were classified as having diabetes if it was documented as ‘present’ in the medical notes or if drug treatment for diabetes was identified. Patients with ‘not recorded’ as their diabetes status and no diabetic drugs recorded (n = 66) were classified in the no diabetes group, and a sensitivity analysis with these patients removed showed minimal difference in all concordance measures across the two samples.

Data quality was initially assessed by review of three medical records by all research assistants within two weeks of commencing data collection. Medical records of a total of 11 patients were subsequently assessed by all data collectors in each study. The observed agreement between data collectors in Study 1 was high for selected medical history (92%) and drugs (100%) and similarly for Study 2 (93% and 87% respectively).

ICD-9-CM was in use in WA at the time of the 1998 admissions, and ICD-10-AM at the time of the 2002–04 sample. Diabetes (Type 1, Type 2, other specified or unspecified diabetes mellitus) was identified in hospital discharge data for the CHD sample if coded in any of 21 diagnosis fields (ICD-9/ICD-9-CM 250, ICD-10-AM E10-E14), using a range of lookback periods for each individual patient – index CHD admission only, and 1, 2, 5, 10 and 15 years prior to the CHD admission.

Approval for this study was obtained from the Ethics Committees of The University of Western Australia and the Department of Health WA, and from the Western Australian Aboriginal Health Ethics Committee.

The crude prevalence of diabetes in the CHD patient sample was calculated for the index admission and each lookback period using the administrative data, and from the medical records database. Observed agreement, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and Kappa were used as measures of concordance between administrative and medical records data, with the latter designated the reference standard. Concordance measures were calculated for each lookback period and also for sub-group and supplementary analyses. The percentage of under or overestimation of diabetes recording in hospital discharge data was calculated by (Sensitivity/PPV - 1) ×100 [17]. All analyses were stratified by period, with the 1998 sample corresponding to ICD-9, and the 2002–04 sample to ICD-10. Differences in prevalence between medical records and administrative data were tested using McNemar’s test, and concordance measures between the two ICD samples were tested using a Pearson chi-square or z-test (for under/overestimation).

Because of the possible impact of sociodemographic and clinical factors on concordance, and the potential for changing criteria for admission to hospital for CHD, variables derived from the administrative data on the index CHD admission were examined for their association with false positives and false negatives, separately for the ICD-9 and ICD-10 samples. Univariable associations were analysed using the Pearson chi-square test (or Fisher’s exact test where cell counts were small). The variables tested were length of stay (1–2, 3–5, ≥6 days), age group (25–50, 51–65, 66–79 years), gender, admission type (elective versus emergency), Indigenous status, Charlson Comorbidity Index (excluding diabetes; 0, 1–4, ≥5), number of comorbidities on the index admission (excluding diabetes; 0–3, 4–7, ≥8), hospitalization in the previous 90 days, hospital location (metropolitan versus rural), hospital type (public versus private), transfer in or out on index CHD admission, in-hospital death, principal diagnosis (myocardial infarction, unstable angina, other CHD), and first-ever versus recurrent CHD admission. First-ever CHD admission was identified where there was no CHD admission in the previous 15 years. Variables were tested using binary values unless otherwise indicated and all analyses undertaken separately for each ICD period. Significant univariable variables in each period were entered into a multivariable logistic regression model and odds ratios with 95% confidence intervals (CI) were calculated. All data analyses were undertaken using SAS (version 9.3, Cary, NC, USA) and statistical significance for all analyses was set at p < 0.05.

In the ICD-9 sample, there was a small but significant difference between diabetes prevalence from medical records (22.5%) and index administrative data (21.5%, p < 0.0001) (Table 2). The difference was also significant (p < 0.0001) in the ICD-10 sample (34.9% from medical records compared with 29.7% from the administrative data), but prevalence increased to 34.2% using all previous hospital admissions to 15 years. There was a similar pattern in the Indigenous sample, although absolute prevalence levels were higher in this patient group.

Observed agreement was high and Kappa very good in both samples and across all lookback periods for the recording of diabetes in the two data sources (Table 3). Sensitivity was significantly lower in the ICD-10 compared with ICD-9 sample from the index admission (81.5% versus 91.1%, p < 0.0001), but improved to 89.6% using 10 years of hospitalization history. NPV was significantly higher in the ICD-9 than ICD-10 sample (p < 0.0001 for all lookback periods), although the difference diminished with increasing lookback. Specificity was high in both samples, with a small decrease as lookback period increased. PPV declined with use of an increased lookback period, but there was no statistical difference between the two time periods. Diabetes status was underestimated by 15% from the hospital discharge index admission in the ICD-10 sample, which reduced to 2.2% using a 10-year lookback period (Table 3).

Because of the oversampling of Indigenous patients in Study 2, the ICD-10 sample was stratified to compare concordance for the ICD-10 patients from Study 1 only with the total ICD-10 sample (Study 1 plus Study 2) (see Additional file 1). There was little difference between the restricted and full sample for all concordance measures, with a similar pattern of increasing sensitivity and a small drop in PPV with increasing lookback period.

Sensitivity was higher in Indigenous compared with non-Indigenous people for every lookback period, although the difference was only significant for lookback periods of two years or more (Table 4). Maximal sensitivity was achieved with 10 years lookback in Indigenous patients (93.6%). Specificity was lower in Indigenous than non-Indigenous patients, with an increasing differential with increasing lookback period (p < 0.05). Diabetes status was underestimated from the hospital discharge index admission in Indigenous people by 13.3% reducing to 0.3% using a 10-year lookback period.

Significant univariable predictors in both periods for false negatives were elective admission, in-hospital death, and non-acute CHD principal diagnosis (Table 5). These remained significant after multivariable adjustment in both periods. A lower number of comorbidites (0–3) were associated with higher odds of a false negative in the ICD-10 but not the ICD-9 period. The level of false positives was low in both the ICD-9 (n = 22, 1.3%) and ICD-10 (n = 40, 1.8%) samples, with no significant univariable association with any of the variables tested, and therefore no multivariable analyses were undertaken.

The generalizability of our results to other hospitalized conditions, particularly non-cardiac conditions, is uncertain because concordance has been specifically measured in a sample of CHD patients. However, within a restricted population hospitalized with CHD, administrative data appear to reliably detect diabetes status. Although we have used recording of diabetes status in the medical records as a reference standard, there are potential limitations in this data source. Patients with less severe diabetes who are treated with diet alone may be less likely to be recorded in medical records as diabetic. There is also the possibility of inaccurate transfer of comorbid conditions to the discharge summary, but review of the whole medical record including drug charts limits the impact of this on identifying diabetic patients. Our results may not be generalizable to patients aged over 80 years due to the age range selected in our sample. However, the lack of significant association between increasing age and under-recording of diabetes suggests that any difference in concordance measures in the very elderly may be small. The differences between the study samples could conceivably contribute to the differences in concordance measures demonstrated in our results, however, stratification by Indigenous status and by study source clearly demonstrate a similar pattern as seen in the main results. We were unable to demonstrate whether concordance measures have remained consistent since the latter period of our study.