Elsevier

Journal of Clinical Epidemiology

Volume 60, Issue 2, February 2007, Pages 155.e1-155.e11
Journal of Clinical Epidemiology

Original Article
Diagnosis-dependent misclassification of infections using administrative data variably affected incidence and mortality estimates in ICU patients

https://doi.org/10.1016/j.jclinepi.2006.05.013Get rights and content

Abstract

Objective

To determine the accuracy of hospital discharge diagnoses in identifying severe infections among intensive care unit (ICU) patients, and estimate the impact of misclassification on incidence and 1-year mortality.

Study Design and Setting

Sepsis, pneumonia, and central nervous system (CNS) infections among 7,615 ICU admissions were identified using ICD-9 and ICD-10 diagnoses from the Swedish hospital discharge register (HDR). Sensitivity, specificity, and likelihood ratios were calculated using ICU database diagnoses as reference standard, with inclusion in sepsis trials (IST) as secondary reference for sepsis.

Results

CNS infections were accurately captured (sensitivity 95.4% [confidence interval (CI) = 86.8–100] and specificity 99.6% [CI = 99.4–99.8]). Community-acquired sepsis (sensitivity 51.1% [CI = 41.0–61.2] and specificity 99.4% [CI = 99.2–99.6]) and primary pneumonia (sensitivity 38.2% [CI = 31.2–45.2] and specificity 98.6% [CI = 98.2–99.0]) were more accurately detected than sepsis and pneumonia in general. One-year mortality was accurately estimated for primary pneumonia but underestimated for community-acquired sepsis. However, there were only small differences in sensitivity and specificity between HDR and ICU data in the ability to identify IST. ICD-9 appeared more accurate for sepsis, whereas ICD-10 was more accurate for pneumonia.

Conclusion

Accuracy of hospital discharge diagnoses varied depending on diagnosis and case definition. The pattern of misclassification makes estimates of relative risk more accurate than estimates of absolute risk.

Introduction

Evaluations of health care delivery and effective planning of clinical trials in intensive care require nationwide estimates of incidence rates and diagnosis-specific mortality. Specialized national databases of intensive care unit (ICU) patients exist, but this is a costly and labor-intensive method for these purposes. An alternative, low-cost approach would be to use readily available administrative databases to extract information required to calculate incidence rates and diagnosis-specific mortality. However, detailed knowledge of the nature and size of diagnosis misclassification is imperative for correct design and interpretation of such studies.

The accuracy of diagnoses in administrative databases has been found to be highly variable, and depend on the diagnosis and clinical setting, as well as the method used for evaluation [1], [2], [3], [4], [5], [6]. Severe infections are major issues in intensive care research but may be difficult to study using available sources of administrative data. Sepsis and pneumonia may be primary diagnoses, or they may be secondary complications of the patient's main condition. A previous study of the accuracy of hospital discharge diagnosis of sepsis for the identification of bacteremia in a general hospital population showed an excessively high degree of misclassification [7], whereas another study found high accuracy in identifying patients with meningococcal disease from two nationwide registers [8]. Different strategies can be applied to increase the accuracy of case definitions. Adjusting restrictions in the sample definition [9] or adding secondary data sources [5], [10] can improve the accuracy. It is also essential that patient transfer between departments and hospitals can be traced to avoid overestimation of disease frequency [11].

One way of increasing the accuracy of data from a national hospital discharge register (HDR) would be to study a defined subpopulation of patients, such as ICU patients. These patients most often suffer from dramatic and life-threatening diseases or complicating events, which are likely to be obvious to the person who is responsible for coding hospital discharge diagnoses. This restriction might increase accuracy, but it also reduces the external validity as compared to a population-based approach.

The aim of this investigation was to estimate the accuracy of a national HDR in identifying severe infections among intensive care patients, and to evaluate the influence of the source of diagnosis on estimates of 1-year mortality and incidence proportion.

Section snippets

Methods

The study was approved by the Human Ethics Committee of Uppsala University Hospital.

Infections of the CNS

Fifty admissions (0.7%) had an ICU diagnosis of CNS infection and 49 (98%) of these had a CNS infection as the main diagnosis. The HDR tended to overestimate the incidence proportion (Table 1). The overall accuracy in identifying these patients was very high (Table 2) and was not significantly improved (P = 0.07) by restricting the case definition to only the main diagnosis in the HDR.

Incidence proportion and mortality

Three hundred and sixty-five admissions (4.8%) had an ICU diagnosis of sepsis and 285 of these cases (78%) had

Discussion

The accuracy of hospital discharge diagnoses in identifying ICU patients with severe infections was found to depend on the diagnosis and case definition used. CNS infections were identified with very high accuracy, whereas sepsis and pneumonia were more difficult to capture. Positive likelihood ratios generally indicated acceptable accuracy, which could be enhanced by more restrictive case definitions, but often at the cost of substantial reductions in sensitivity and increased negative

Acknowledgment

The project was supported by a grant from the Department of Surgical Sciences at Uppsala University Hospital. The authors are also grateful for statistical advice from Biostatistician Johan Lindbäck at the Uppsala Clinical Research Center, Uppsala University Hospital.

References (32)

  • H.T. Sorensen et al.

    A framework for evaluation of secondary data sources for epidemiological research

    Int J Epidemiol

    (1996)
  • K.M. Madsen et al.

    Can hospital discharge diagnosis be used for surveillance of bacteremia? A data quality study of a Danish hospital discharge registry

    Infect Control Hosp Epidemiol

    (1998)
  • H.T. Sorensen et al.

    Identification of cases of meningococcal disease: data quality in two Danish population-based information systems during a 14-year period

    Int J Risk Saf Med

    (1995)
  • S. Mattke et al.

    Evaluating the role of patient sample definitions for quality indicators sensitive to nurse staffing patterns

    Med Care

    (2004)
  • M. Riley et al.

    Validation study of the Victorian Birth Defects Register

    J Paediatr Child Health

    (2004)
  • D.A. Bloch

    Comparing two diagnostic tests against the same “gold standard” in the same sample

    Biometrics

    (1997)
  • View full text