Validating laboratory defined chronic kidney disease in the electronic health record for patients in primary care

Optimizing primary care clinical management of chronic kidney disease (CKD) remains a critical step to reduce overall disease burden [1]. The widespread adoption of electronic health records (EHR) in the United States has garnered excitement about accomplishing this goal by transforming the clinical environment into a “learning health care system” [2‐4]. In this context, a learning healthcare system is one that can collect and store accurate information for each patient, and in turn, use these data to inform and support improvements in clinical care [5, 6]. Pragmatic randomized trials using the EHR to identify individuals with CKD and quantify the gaps in their care, followed by interventions to improve outcomes, can rigorously test the promise of these learning healthcare systems. In fact, the National Institutes of Health (NIH) has published several requests for applications explicitly focused on design and evaluation of information technology-based tools and interventions to improve CKD care [7, 8].

Necessary pre-requisites to the successful design, implementation, interpretation and dissemination of the results of such pragmatic trials include data accuracy and adequate quantification of gaps in care [9]. Specifically, prior to the design and implementation of EHR-based pragmatic trials, investigators must be confident in the ability of the existing data to identify patients with CKD who are at risk for complications, deliver an intervention with a high probability of improving care, and ascertain relevant outcomes [10]. Accurate phenotyping (i.e. the ability to accurately classify disease status) is important because misclassification can introduce bias and limit interpretation of results [11, 12]. The few pragmatic studies that have used EHR data designed to improve care in individuals with CKD have not reported detailed methodology regarding phenotyping process or ascertainment of the clinical gaps prior to design and implementation [13‐17]. Prior registry studies that have validated CKD diagnostic codes to classify CKD status were primarily conducted in inpatient hospital settings, and even fewer have used clinician chart review as a gold standard [18, 19]. One of the largest validated EHR-based CKD registries did not specifically focus on patients actively seen in primary care practices [20]. Investigators must also be able to quantify the clinical practice gap(s) and the potential for resolution of that gap(s) to improve outcomes (i.e. “actionable gap”) [21]. For example, the lack of PCP awareness of CKD has been cited as a major barrier to optimal CKD care, but the degree and importance of this lack of awareness may vary greatly by setting [22, 23]. Thus we see a gap in knowledge between the specific methods used to identify CKD patients in the EHR and the evaluation of the appropriate guideline-driven care practices among the identified patients.

In this methodologic study, we set out to investigate the use of historical laboratory data to identify a cohort of primary care patients with CKD, and, in a subset of the cohort, have clinical nephrologists confirm a CKD diagnosis and the presence or absence of comorbidities known to influence CKD via manual chart review. Additionally, we evaluated the prevalence and usefulness of CKD documentation in the EHR problem list as a surrogate for primary care provider (PCP) awareness of CKD status by assessing its association with guideline-driven care [24].

In this methodologic study, we show findings with important implications for design and implementation of future pragmatic studies that leverage the EHR to deploy interventions to improve management of individuals with CKD in primary care. First, we found that using a CKD definition based on historical eGFR_creat values only may be too sensitive, since we found that CKD status was confirmed by nephrologist chart review in only 68% of cases. Female gender, White race and higher eGFR_creat were more common among persons with CKD that was not ultimately confirmed by chart review. We also show that CKD on the problem list is a relatively good surrogate for PCP awareness, as it has high concordance with both CKD status and PCP awareness as ascertained by nephrologist chart review. In the primary care practice included in this study, providers have only moderate rates of CKD awareness (44%), defined as CKD present on the problem list. However, awareness increased significantly with severity of CKD. Importantly, CKD awareness was significantly associated with higher odds of guideline-concordant CKD testing, and the use of ACEi/ARB and statins.

Eight prior studies have reported misclassified CKD diagnoses from administrative data [18‐20, 23]. However, five of these studies were conducted in an inpatient hospital setting or used patients who were hospitalized [18]. Only three of these studies sampled from outpatient primary care settings and included expert clinician chart review as a gold standard [19, 20]. We identified only one study that focused on patients who were receiving active and ongoing outpatient primary care [23]. Narrowing the focus to these patients is critical because these are the individuals who are most likely to be included in intervention studies. Recently, the eMERGE consortium published an algorithm that maximizes sensitivity and specificity of CKD associated with hypertension and diabetes, compared with chart review [20]. Appropriate implementation of the algorithm requires data handling procedures that natural language processing of text, which are unlikely to be feasible for most primary care practices [20]. Our study adds value to the literature as it shows that CKD ascertained only by two historical eGFR_creat values < 60 ml/min/1.73m² at least 90 days apart may be too sensitive when the goal is to deploy interventions for those individuals with CKD at highest risk for complications who are most likely to benefit from interventions. Our finding that almost a third of patients identified as having CKD by eGFR_creat did not have CKD confirmed by expert physicians has important implications because it is likely that future pragmatic trials will need to include steps to re-test and further risk stratify patients (e.g., with albuminuria and cystatin C testing) to confirm CKD before deploying interventions.

In our study, combining problem list and encounter diagnoses to identify the comorbidities most likely to be relevant in studies of CKD patients correlated well with expert physician chart review. One important exception was presence of heart failure, a finding similar to prior studies showing that presence of heart failure correlates relatively poorly with administrative codes alone [27]. This suggests that, if researchers aim to include or exclude individuals with certain comorbidities from intervention studies, administrative codes and problem list are suitable for most of the conditions we tested, but not for heart failure where additional variables or chart review will be required.

We also found that CKD listed on the problem list may be a good surrogate for PCP awareness of this diagnosis. Low PCP awareness of CKD has been cited as one of the major barriers to improve kidney care in the U.S. [28]. Yet strategies to ascertain the degree of awareness are limited. We showed that, in this setting, CKD documented on the problem list had high concordance with CKD awareness when ascertained by expert chart review. Importantly, we also confirmed findings from prior studies that CKD awareness by PCP remains limited [22, 23, 28]. While only 44% of patients had CKD listed, which is higher than some previously reported estimates, CKD awareness did significantly increase with severity of CKD [19, 22, 23]. This suggests that CKD documentation may be a useful additional variable to increase the specificity of EHR data identification of CKD status. It is possible that the relatively lower prevalence of CKD listing among those patients with higher eGFR_creat represents PCP discomfort with labeling persons with a disease when the eGFR_creat is close to guideline definition cut-points.

Finally, we found that CKD awareness, defined as CKD on the problem list, was significantly associated with higher rates of testing for albuminuria and CKD complications. It was also associated with higher prevalence of ACEi/ARB and statin prescription. These findings highlight improving CKD awareness as a “low hanging fruit” actionable gap in CKD care. Thus, pragmatic trials that include interventions to improve CKD awareness have the potential to improve some important processes of care.

Performing secondary analyses of EHR data to explore associations between patient characteristics, clinical measurements, delivery of care, and CKD may be useful and appropriate for hypothesis generation or risk prediction. Given the higher likelihood of CKD problem listing with lower eGFR_creat, it may be particularly useful in studies of patients with more advanced disease. However, if the nature of the investigation is to identify patients with CKD who are most likely to benefit from an intervention, researchers should expect that CKD will need to be confirmed.