Original Article
Comorbidity information in older patients at an emergency visit: self-report vs. administrative data had poor agreement but similar predictive validity

https://doi.org/10.1016/j.jclinepi.2007.07.009Get rights and content

Abstract

Objectives

The objectives of this study were (1) to ascertain the level of agreement between the Charlson Comorbidity Index (CCI) based on self-report vs. administrative records, and factors affecting agreement and (2) to compare the predictive validity of the two indices in a sample of older emergency department (ED) patients.

Study Design and Setting

The study was a secondary analysis of data from a randomized trial of an ED-based intervention. The self-report and administrative CCI were compared using the intraclass correlation coefficient (ICC). Factors examined for effect on agreement included health service utilization, age, and sex. The predictive validity of the indices was compared using subsequent health services utilization and functional decline as outcomes. Participants (n = 520) were recruited at four university-affiliated Montreal hospitals. Eligibility criteria included 65 years of age or older, able to speak English or French, and discharged to the community.

Results

Agreement between the two sources was poor to fair (overall weighted ICC 0.43 [95% confidence interval [CI]: 0.40, 0.47]). The predictive validity was similar for the two indices (area under the receiver-operating characteristic curve 0.51–0.66, depending on the outcomes).

Conclusion

Agreement between self-report and administrative comorbidity data is only poor to fair but both have comparable predictive validity.

Introduction

In 1987, Charlson et al. published and validated a weighted chart-based comorbidity index whose purpose was to predict 1-year mortality and quantify the chronic disease load in patients participating in therapeutic trials [1]. The Charlson Comorbidity Index (CCI) became widely used in epidemiologic studies to simplify data analysis [2] and has been used in targeted clinical interventions [3].

Because of the high cost of medical chart reviews [3] and increasing reliance of patients on outpatient care [4], the original CCI has since been adapted for use with hospital administrative data [5], [6] and medical claims data [7]. Cost-efficient self-report comorbidity scales [8] have also been developed; one such scale predicted 1-year mortality comparably to a comorbidity index based on administrative data [9]. To our knowledge, however, no study has yet compared self-report and administrative data in an emergency setting. Both data sources have potential limitations: self-report data suffer from recall bias, and administrative data are limited by diagnostic and coding errors.

Older emergency department (ED) patients have a high number of ED visits, high risk of adverse outcomes [10], and poor accuracy of ED diagnoses [11], [12]. Comorbidity indices, however, do not appear to perform as well among elders, a population with a high comorbidity burden [13], [14].

Thus, the objectives of this study were (1) to compare comorbidity data derived from self-report with administrative data in a sample of elders in the ED, (2) to determine the effect on agreement between these two data sources of sociodemographic, medical, and other factors, and (3) to compare the predictive validity of the CCI derived from the two sources.

Section snippets

Methods

This study is a secondary analysis of data from a randomized trial of a two-step intervention for older ED patients conducted in four Montreal EDs [3]. Recruitment methods have been described previously [3]. Patients aged 65 years and above, ready to be discharged home from the ED were screened using the Identification of Seniors at Risk (ISAR) screening tool [15], a short self-report screening questionnaire designed to identify patients at risk for functional decline, other adverse health

Results

Characteristics of the study population are reported in Table 1. The presence of each of the 18 comorbid conditions differed in a statistically significant way by data source (administrative vs. self-report) for nine of the conditions (data not shown). Four conditions were reported more frequently in the self-report data (myocardial infarction, ulcer disease, diabetes with end organ damage, and connective tissue disease), whereas five were reported more frequently in the administrative data

Discussion

This study examined the agreement between self-reported comorbidity conditions and administrative records in a sample of 520 older ED patients, with oversampling of frailer patients (ISAR score of 2 or more). Comparison of the CCI scores computed from administrative and self-report data indicated low levels of agreement, even in the subgroup with prior hospital admissions. There were also systematic differences in the scores, with those patients with greater prior service utilization having

Acknowledgments

Author contributions were as follows: Stephanie Susser developed the study design, carried out the literature review and the data analysis, and wrote the initial draft of the manuscript; Jane McCusker provided the data for the study, supervised the conduct of the study, and edited the manuscript; Eric Belzile assisted with the statistical analyses and with checking the data presented in the manuscript. The study sponsor had no input into the design, methods, subject recruitment, data

References (29)

  • S. Schneeweiss et al.

    Consistency of performance ranking of comorbidity adjustment scores in Canadian and U.S. utilization data

    J Gen Intern Med

    (2004)
  • J. McCusker et al.

    Rapid emergency department intervention for elders reduces risk of functional decline: results of a multi-centre randomized trial

    J Am Geriatr Soc

    (2001)
  • S. Schneeweiss et al.

    Use of comorbidity scores for control of confounding in studies using administrative databases

    Int J Epidemiol

    (2000)
  • J.N. Katz et al.

    Can morbidity be measured by questionnaire rather than medical record review?

    Med Care

    (1996)
  • Cited by (35)

    • Patient-Reported Morbidity Instruments: A Systematic Review

      2020, Value in Health
      Citation Excerpt :

      Several studies also analyzed the relationship between patient-reported comorbidity instruments and healthcare utilization outcomes (eg, [re]hospitalizations, emergency room visits, medical costs).9,11,26,27,63 Susser et al9 estimated an AUC of 0.68 and AUC of 0.67 in predicting the number of hospital days and emergency room visits, respectively. Katz et al11 estimated weak correlations between the patient-reported comorbidity CCI and healthcare utilization outcomes, and Sangha et al27 reported similar weak associations with the SCQ.

    • Immunogenicity and safety of high-dose versus standard-dose inactivated influenza vaccine in rheumatoid arthritis patients: a randomised, double-blind, active-comparator trial

      2020, The Lancet Rheumatology
      Citation Excerpt :

      The strengths of this study include the randomised design, its focus on the understudied but at-risk population with rheumatoid arthritis, and the fact that clinical data collection was based on patient-reported outcomes that are reproducible, sensitive to change, and ensure a comprehensive assessment of possible adverse events associated with the trial intervention.33 Indeed, we selected tools with tested predictive validity such as the Charlson comorbidity index,34 and confirmed that the reported frequencies of specific comorbidities (ie, diabetes, cancer) in our study were similar to those in much larger studies using administrative data (data not shown).5 We also confirmed the patient-reported rheumatoid arthritis treatments and the doses of drugs of interest with an updated pharmacy list.

    • A systematic review identifies valid comorbidity indices derived from administrative health data

      2015, Journal of Clinical Epidemiology
      Citation Excerpt :

      Three studies compared the predictive ability of CCI adaptations derived from self-reported data with the same index derived from administrative data [44–46]. Two studies [44,45] found that self-reported data and administrative data adaptations had similar ability to predict various outcomes. Ronksley et al. [46] found that self-report of comorbid conditions had varying levels of agreement with those derived from administrative data, ranging from poor to substantial agreement depending on the comorbid condition (κ = 0.14–0.79).

    View all citing articles on Scopus
    View full text