A data quality assessment to inform hypertension surveillance using primary care electronic medical record data from Alberta, Canada

The Canadian Primary Care Sentinel Surveillance Network (CPCSSN) is a collaboration of eleven practice-based research networks (PBRN) across Canada who manage the extraction, cleaning and processing of de-identified EMR data from primary care settings [13]. At present, over 1200 primary care providers and 1.8 million patients contribute data from eight provinces and territories [14]. National CPCSSN data have been previously used to report on the epidemiology of many conditions in primary care, such as hypertension [5], diabetes [15], depression [16], osteoarthritis [17], dementia [18], chronic obstructive pulmonary disease [19], and others. The CPCSSN organization and data extraction and processing have been described elsewhere [13, 20].

This data quality assessment utilized primary care EMR data obtained by the two PBRNs in the province of Alberta – the Northern and Southern Alberta Primary Care Research Networks (NAPCReN and SAPCReN, respectively). Because healthcare in Canada is organized and delivered separately within each province or territory, only one province (Alberta) was chosen for the data quality assessment in order to minimize variation in the data due to interprovincial differences such as healthcare delivery and practice, drug coverage, health information legislation, EMR uptake and extent of use, types of EMR systems available, and many other factors [21, 22].

In Alberta, there were 323 providers (mostly family physicians with a small proportion of nurse practitioners and community pediatricians) participating from 53 primary care practices. This represents slightly over 5% of the total number of family physicians in Alberta [23]. As of June 2018, de-identified EMR data were extracted from 397,518 patients in total; this reflected approximately 9.2% of Alberta’s general population of 4.3 million people [24]. The CPCSSN data has previously been found to overrepresent older adults and women [25], but this is typical of primary care populations.

Currently, CPCSSN in Alberta extracts from five distinct EMR systems – Wolf, Med Access, Practice Solutions Suite, Accuro and Healthquest. The earliest (or ‘start’) date of information in the CPCSSN database varies by clinic and by patient, depending on when a clinic first implemented their EMR system, as well as when the patient first attended the clinic.

Adult patients (18 years and older) who had at least one primary care encounter in the previous two years (July 1, 2016 to June 30, 2018) were included, in order to establish an ‘active’ patient population. Any patient who was recorded as ‘deceased’ or ‘inactive’ in the EMR was excluded, as were any patients or providers who had explicitly requested to opt out of the CPCSSN database. The data quality assessment focused specifically on patients with hypertension who were identified using a CPCSSN-developed definition [26]. The hypertension definition consisted of a combination of International Classification of Disease version nine (ICD-9) codes (401, 402, 403, 404, 405) and medications located throughout the EMR: a minimum of two physician billing codes within two years or any occurrence of a diagnosis in the Problem List/Profile or prescription for an anti-hypertensive medication (with medication criteria alone being insufficient if other specific diagnoses exist, such as heart failure or diabetes) [26]. The definition was validated using chart reviews as the reference standard and demonstrated good sensitivity (84.9%) and specificity (93.5%) [26].

The data quality assessment was a cross-sectional, descriptive evaluation guided by reporting recommendations for distributed data networks [27]. Data elements were selected based on their potential use and relevancy for hypertension surveillance, as well as availability in the CPCSSN data. These included: patient demographics; physical examinations (weight, height, body mass index [BMI], and systolic and diastolic blood pressure); laboratory values (high density lipoprotein [HDL] cholesterol, low density lipoprotein [LDL] cholesterol, total cholesterol, triglycerides, fasting blood glucose, glycated hemoglobin [HbA1C]), anti-hypertensive medications (defined using categories of the relevant groups of Anatomical Therapeutic Chemical [ATC] codes: C02*, C03*, C07*, C08*, C09*); and risk factor records for smoking and alcohol use. Only the CPCSSN-processed/coded values were used, as these are typically the data elements that are accessible from CPCSSN for secondary purposes. A full description of all data elements can be found in the CPCSSN Data Dictionary online [14].

Summary statistics were reported for continuous variables, which included range, mean, and median. Proportions (restricted to the three most frequent values) and number of unique values were described for categorical variables. Missingness was reported as a proportion of patients without a recorded data element (e.g. height) or record (e.g. medication, smoking); missingness of specific items within a record was also reported (e.g. dose in medication record). Data completeness was also represented visually by clinic and EMR type.

Several temporal aspects of the data were examined – the proportion of patients who had at least one physical exam measurement or laboratory value documented in the previous year (July 1, 2017 to June 30, 2018) was reported, in addition to the proportion of risk factor (i.e. smoking and alcohol) and medication records that contained a stop/end date prior to the start date. An exploration of patient-level weight values over time were visualized by plotting the difference between subsequent weight measurements and the length of time (days) between subsequent measurements for individuals with at least two weight measurements.

External validity was evaluated by comparing the most recent crude hypertension prevalence estimates from three national population-level sources: administrative data from the Canadian Chronic Disease Surveillance System (CCDSS), consisting of physician billing claims, hospitalizations and prescription drug records [28]; the Canadian Health Measures Survey (CHMS), which defines hypertension based on standardized, direct BP measurements and health-related interviews [29]; and self-reported high BP from the Canadian Community Health Survey (CCHS) [30]. Hypertension prevalence estimates from the national CPCSSN data [5] were also used as a comparison to the regional-level (Alberta) data.

RStudio version 1.1.456 was used for the analysis, which was conducted in 2019. This study was approved by the University of Calgary’s Conjoint Health Research Ethics Board (REB17–1825) and the University of Alberta’s Health Research Ethics Board (Pro00079372).

Birth year was complete for all patients, as was sex (with the exception of two patients). However, nearly all socio-demographic information on patients was mostly incomplete (Table 1). For those who had some information recorded in ethnicity, occupation, or education fields, the data were highly inconsistent – for instance, over 3500 unique entries were recorded for occupation and more than 75 distinct entries were found for ethnicity.

Approximately 10% of patients were missing a height or BMI value and even fewer patients were missing weight (Table 2). Males had a median of four measurements for height, weight, or BMI and females had five, with these measurements showing a skewed distribution. From the lower and upper ranges of the height, weight and BMI values, it appears that data errors are present. For example, female weight values ranged from 1.8 to 477 kg, which is biologically unlikely. When plotting patient-level height and weight values (Fig. 2), those located outside the main cluster of points visually identify specific data errors. For instance, the vertical line of points approaching 0 on the x (weight) axis might indicate a data entry error (e.g. weight entered as 10 instead of 100) or swapped height and weight values (e.g. height in metres entered in the weight field). Another observable area of atypical points was between 150 and 200 on the x (weight) axis, which potentially represents height and weight values that were entered in the wrong fields (e.g. weight = 175 and height = 100 recorded instead of weight = 100 kg and height = 175 cm).

Figure 3 investigates possible errors in weight values using successive patient-level weight measurements for those who had at least two weight values recorded in their EMR (n = 39,202). It would be expected that changes in individual weight might demonstrate more variability over time (e.g. patient weight recorded 10 days apart should have minimal difference, whereas weight measurements taken several years apart might show a more significant change). Two peaks centred around 100 and − 100 on the y-axis emerged as potentially problematic data: in a relatively short time period between measurements, the difference between successive weight measurements was approximately 100 kg for patients clustered around those two peaks. This likely represents inconsistencies in the unit of measurement (e.g. kilograms versus pounds) for subsequent weight measurements for a given individual. However, the extent of the problem was not substantial – the majority of weight values (94.8%) occurred within two standard deviations of the central peak (mean − 0.29) and at least one potential data error (i.e. outside two standard deviations) at any time was detected in the records of 18.4% of patients.

BP measurements were well-recorded in terms of completeness (99%) and the majority of patients (85%) had at least one measurement recorded in the previous year (Table 2). However, BP values at the minimum and maximum end of the range may indicate data errors (Table 2). These values could be biologically possible, but would be very unlikely in an outpatient setting; for instance, a systolic BP of 52 might indicate shock and a systolic BP of 290 would be an emergency event. In addition, CPCSSN also sets limits to BP values when processing the raw EMR data (50–300 mmHg for sBP; 20–200 mmHg for dBP), which would underestimate the true range of values.

Male patients had a median of 16 total BP measurements recorded in their EMR and females had slightly more (median = 18). A small proportion of patients had large sums of annual BP measurements – for instance, 4.2% of females and 4.0% of males were above the 95th percentile for number of BP measurements (greater than 10) in 2017 (data not shown).

Of the laboratory tests measuring blood glucose, HbA1C values were present in the EMR more often than fasting glucose (88% versus 79% of patients), and more patients had an HbA1C test result in the previous year compared to the fasting glucose test (Table 2).

The lipid values included in this assessment (LDL, HDL, total cholesterol, triglycerides) were available for the majority of patients in this cohort (at least 91%, varying by lab type), with a median of 4–5 values for each patient in the EMR (Table 2). Female patients were observed to have a slightly fewer lipid values present in their EMR compared to male patients (Table 2).

For all types of lab results, the upper and lower limits were unlikely to be seen in an outpatient setting (i.e. primary care) and many values were beyond a biologically plausible range (e.g. HDL and LDL lower value = 0). This points to likely data errors at the upper and lower ends of the range of values, however, it was only for a very small proportion of lab values.

The vast majority of males (92%) and females (93%) with hypertension had at least one recorded anti-hypertensive prescription, with a median of six anti-hypertensive medication prescriptions per person (Table 3). The medication records themselves were fairly complete; all records contained a start date and most contained a stop date, strength, dose, frequency, duration, and count. Drug Identification Number (DIN) and ‘reason for medication’ mostly incomplete, with DIN missing in over half of medication records and ‘reason’ missing in over three-quarters of records.

Within the Risk Factor section in the EMR, nearly 80% of patients had a smoking status recorded, with ‘Unknown’ and ‘Never’ as the most frequently recorded categories (Table 4). However, after excluding the indiscriminate ‘Unknown’ smoking status, a total of 31,976 patients (66.1%) and 68,110 records remained across three categories: ‘Current’, ‘Past’ or ‘Never’ (data not shown). Males and females had a similar number of smoking records per person (median = 1; mean = 3). All start and end dates were missing from the records.

More males than females had their alcohol use recorded (47 and 40%, respectively) and these records were primarily for ‘Current’ users (Table 4), indicating that alcohol use is likely recorded differentially between users and non-users. Patients had a mean of 2 records in their EMR (median of 1) and no records contained start or end dates.

Of note, a ‘Date Created’ field exists for both smoking and alcohol records. This field indicates when the record was created in the EMR system but does not necessarily correspond to the start of the risk behaviour. ‘Date Created’ was present in 80.6% of smoking records and 76.6% of alcohol use records.

The overall crude estimate for Alberta-specific hypertension prevalence in the CPCSSN data (23.6%) were similar to the 2014–15 physical measure survey (CHMS) (23.3%) and was also comparable to the national CPCSSN estimate (22.8%) (Table 5). The largest discrepancy was seen in the self-reported CCHS, with hypertension prevalence estimated at 17.7%. Male patients in the CPCSSN database had a higher hypertension prevalence (26.1%) than all other sources, while the prevalence for female patients (21.6) in the CPCSSN data was similar to the health measures survey (22.0%) and slightly lower than the CCDSS (25.6%).

This paper provides a quality assessment of select CPCSSN data elements deemed to be important for hypertension surveillance or research, but it was not possible to examine and report on all variables contained in the CPCSSN database in a single manuscript, nor was it possible to examine discrete cardiovascular outcomes related to hypertension (for example, hospitalization for myocardial infarction), as this information is usually contained in other databases external to CPCSSN or captured in the EMR in an inaccessible format (e.g. PDF document, free text notes). It was also not feasible to quantify the true accuracy of data elements, other than appraising the plausibility of values through descriptive means. Secondly, during the CPCSSN processing and data transformation stages for physical exam measurements and some lab types, restrictions are introduced for out-of-bounds values and thus, the summary statistics presented in this paper may not reflect the full variation of values originating from the source EMR. In addition, any changes or improvements made to the CPCSSN processing may result in slight differences in the CPCSSN EMR database between each extraction cycle. Thirdly, although the CPCSSN definition for hypertension demonstrated high sensitivity and specificity, a potential for misclassification still exists. This may have underestimated the number of patients with hypertension or produced a patient sample that is biased towards a greater severity of illness. Lastly, the quality was described specifically for CPCSSN data from Alberta and within the context of hypertension. This is not a population-level data source and only constitutes a sample of participating providers and patients who have sought care. Thus, the overall findings may not be representative of the wider Alberta population or for other provinces or territories that participate in CPCSSN, and may also differ in other disease-based contexts. However, CPCSSN has developed uniform data extraction, processing, and standardization methods across the country, which may allow for other regional networks to compute the same data quality assessment for comparison.