You are here

Article Text

Article menu

Download PDFPDF

XML
Research methods
Research
Validation of discharge diagnosis codes to identify serious infections among middle age and older adults
  1. Andrew D Wiese1,
  2. Marie R Griffin1,2,
  3. C Michael Stein3,
  4. William Schaffner1,
  5. Robert A Greevy4,
  6. Edward F Mitchel Jr1,
  7. Carlos G Grijalva1,2
  1. 1 Department of Health Policy, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
  2. 2 Mid-South Geriatric Research Education and Clinical Center, VA Tennessee Valley Health Care System, Nashville, Tennessee, USA
  3. 3 Department of Pharmacology, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
  4. 4 Department of Biostatistics, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
  1. Correspondence to Dr Andrew D Wiese; andrew.d.wiese.1{at}vumc.org

Abstract

Objectives Hospitalisations for serious infections are common among middle age and older adults and frequently used as study outcomes. Yet, few studies have evaluated the performance of diagnosis codes to identify serious infections in this population. We sought to determine the positive predictive value (PPV) of diagnosis codes for identifying hospitalisations due to serious infections among middle age and older adults.

Setting and participants We identified hospitalisations for possible infection among adults >=50 years enrolled in the Tennessee Medicaid healthcare programme (2008–2012) using International Classifications of Diseases, Ninth Revision diagnosis codes for pneumonia, meningitis/encephalitis, bacteraemia/sepsis, cellulitis/soft-tissue infections, endocarditis, pyelonephritis and septic arthritis/osteomyelitis.

Design Medical records were systematically obtained from hospitals randomly selected from a stratified sampling framework based on geographical region and hospital discharge volume.

Measures Two trained clinical reviewers used a standardised extraction form to abstract information from medical records. Predefined algorithms served as reference to adjudicate confirmed infection-specific hospitalisations. We calculated the PPV of diagnosis codes using confirmed hospitalisations as reference. Sensitivity analyses determined the robustness of the PPV to definitions that required radiological or microbiological confirmation. We also determined inter-rater reliability between reviewers.

Results The PPV of diagnosis codes for hospitalisations for infection (n=716) was 90.2% (95% CI 87.8% to 92.2%). The PPV was highest for pneumonia (96.5% (95% CI 93.9% to 98.0%)) and cellulitis (91.1% (95% CI 84.7% to 94.9%)), and lowest for meningitis/encephalitis (50.0% (95% CI 23.7% to 76.3%)). The adjudication reliability was excellent (92.7% agreement; first agreement coefficient: 0.91). The overall PPV was lower when requiring microbiological confirmation (45%) and when requiring radiological confirmation for pneumonia (79%).

Conclusions Discharge diagnosis codes have a high PPV for identifying hospitalisations for common, serious infections among middle age and older adults. PPV estimates for rare infections were imprecise.

  • coding algorithms
  • medicaid
  • older adults
  • serious infections

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-020857

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of the study

    • This study examined the performance of diagnosis coding algorithms to identify hospitalisations due to serious infections among middle age and older adults enrolled in a state Medicaid programme using a systematic and representative sample of records from hospitals of different sizes and in distinct state regions.

    • The reference criteria to identify true infections were based on the previous literature and clinical expertise but may be imperfect. Nevertheless, identifying microbiologically confirmed infections is difficult due to the low sensitivity of culture-based diagnostic methods often used in clinical practice.

    • Diagnosis codes were based on the International Classifications of Diseases, Ninth Revision (ICD-9) coding system only. These findings will continue to be helpful for retrospective studies that encompass periods of ICD-9 use, yet additional studies evaluating the performance of ICD-10-based codes would be beneficial.

    • Our coding algorithms to identify serious infections had a high positive predictive value overall, and will be useful in ongoing and future research using administrative data.

    Introduction 

    Infectious diseases remain a leading cause of morbidity and mortality in the USA and elsewhere.1 Middle age and older adults, in particular, are at high risk for serious infections and their long-term consequences.2 3 Among older adults, community-acquired serious infections (including pneumonia, sepsis and meningitis) often require hospitalisation and represent a substantial burden on the US healthcare system.4–7 The incidence of community-acquired pneumonia is very high among adults >=50 years of age (248 cases per 100 000 adults) with an even higher burden among adults >80 years of age (1643 cases per 100 000 adults).8 Sepsis, cellulitis and pyelonephritis are also very common (sepsis: 100 cases per 100 000 and cellulitis/pyelonephritis: >150 hospitalisations per 100 000 adults) with an increasing incidence of severe sepsis with increased age.9–11 Meningitis and endocarditis are relatively rare (around two to three cases per 100 000), although the case fatality rate is very high.12 13 Therefore, it is important to monitor the incidence of these infections, identify important risk factors and determine the impact of preventative policies (eg, vaccination) on these diseases among middle age and older adults.14–16

    Large-scale epidemiological studies using administrative data often use serious infections as outcomes.17–21 However, few studies have evaluated the performance of diagnosis codes to identify serious infections among middle age and older adults. Most previous studies that have assessed the performance of coded discharge diagnosis codes to identify serious infections have focused mainly on common infections (eg, pneumonia or sepsis), specific populations (eg, patients with rheumatoid arthritis) or on healthcare-associated or hospital-acquired infections.22–31 Nevertheless, the performance of coded discharge diagnoses for accurately identifying infections requiring hospitalisation among middle age and older adults is unclear. Therefore, we sought to determine the positive predictive value (PPV) of specific discharge diagnoses for identifying infections that required hospitalisation among middle age and older adults.

    Methods

    Data sources

    TennCare is the managed Medicaid programme in the state of Tennessee that provides healthcare insurance to those who are Medicaid eligible (around 20% of the Tennessee population).32 The adult TennCare population consists of low-income pregnant women and individuals who are elderly or have a disability (over 600 000 annually).32 We used data from TennCare, supplemented with data from the Tennessee Hospital Discharge Data System (a registry for all hospitalisations in Tennessee) and pharmacy information from Medicare part D for those that were dual eligible, to identify a retrospective cohort of TennCare enrollees >=50 years of age with pharmacy benefits (2008–2012). We restricted the hospitalisations for serious infection to those occurring from 2008 to 2012 to only include more recent hospitalisations for which medical records are more likely to be available. Cohort members had at least 180 days of baseline continuous enrolment before cohort entry, and were also required to be free of certain life-threatening conditions known to increase the risk of infection (solid organ transplantation, end-stage renal disease, HIV/AIDS, malignancy and serious kidney, liver and respiratory disease) that may limit longitudinal follow-up and impact the assessments of patients’ exposures and their risk of infections. Cohort members were also required to have evidence of at least one pharmacy prescription fill and evidence of at least one healthcare encounter during baseline (to ensure use of benefits so that if a healthcare encounter for an infection occurred, it would be detected). Follow-up started on the earliest date the inclusion criteria were met and continued through the earliest of the following: study end date (31 December 2012), the day prior to diagnosis of a serious life-threatening condition that would have precluded entry to the cohort, loss of enrolment or date of death. From this retrospective cohort, we identified possible hospitalisations for serious infections (see below) for our validation study. To avoid including infections that may have originated due to a previous hospital stay, we excluded hospitalisations for infections that occurred in the 30-day period after discharge from a previous hospitalisation.

    Identification of hospitalisations for serious infection

    Clinical knowledge and a literature review were used to identify primary discharge diagnosis codes that have been used previously to identify specific serious infections that require hospitalisation (study infections), including pneumonia (alone or with a primary diagnosis of bacteraemia/sepsis), bacteraemia/sepsis, pyelonephritis, meningitis/encephalitis, osteomyelitis/septic arthritis, endocarditis and cellulitis.31 33–35 Specific International Classifications of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes used to identify possible hospitalisations for each infection type are presented in table 1. As the objective of our study was to determine the PPV of coding algorithms to identify serious infections that required hospitalisation, we focused only on primary diagnoses of infection to reduce the possibility of detecting concurrent infections that may not have led to hospitalisation or nosocomial infections that developed during the course of the hospitalisation.35

    View this table:
    Table 1

    Discharge diagnosis code definitions (ICD-9-CM) for hospitalisations for serious infection

    Sampling strategy

    We used stratified random sampling to select a representative subset of study infection hospitalisations from among all possible cases identified in the retrospective cohort from among hospitals within 200 miles of Vanderbilt University Medical Center (VUMC). Since larger hospitals would be over-represented in a purely random sampling, and because there may also be regional variability in coding practices and infection prevalence, we constructed a sampling framework where hospitals were stratified based on their geographical region in Tennessee (West, Central and East), and tertiles of reported discharge volume (low, medium and high) during the study period.36–38 From this sampling framework, we randomly selected three hospitals from each of these nine sampling strata, and retrieved their medical records for review and validation (figure 1). This strategy, relative to a purely random sample, ensured better representation of infections identified in smaller hospitals and those in more rural regions of the state of Tennessee. If a hospital refused to participate, it was replaced by another hospital randomly selected from the same sampling stratum.

    Figure 1
    Figure 1

    Sampling strategy for identifying potential hospitalisations for serious infection.

    The overall goal was to review and validate 675 hospitalisations for serious infection from 27 hospitals (25 hospitalisations for each of the three hospitals comprising a stratum, yielding 75 hospitalisations for each of the nine strata) (figure 1). We conservatively assumed that up to 80% of records requested would be available for review, and so we requested 32 records per hospital to receive an average of 25 records from each (figure 1). To ensure that we reviewed sufficient rare infections, we preferentially selected any identified possible hospitalisations for meningitis/encephalitis, osteomyelitis/septic arthritis and endocarditis from each hospital in the sample. We randomly selected the remaining set of possible hospitalisations for other serious infections based on the proportional distribution of common infections at each hospital (pneumonia, bacteraemia/sepsis, pyelonephritis and cellulitis) until 32 infections were identified. For hospitals with fewer than 32 infections during the study period, all infections were requested.

    Abstraction of medical records

    Relevant clinical information was abstracted from the medical record (transfer notes, emergency room summary, admission summary, physical/history, pharmacy, laboratory, microbiology and radiology information, and discharge summary) of each hospitalisation with a primary discharge diagnosis code indicative of infection using a standardised and customised Research Electronic Data Capture instrument hosted at Vanderbilt University.39 As we were interested in infections that led to hospitalisations, we focused our reviews on clinical, microbiological and radiological information from the 2 days prior to the admission date through 2 days after admission to limit the possibility of identifying infections that developed during the hospitalisation (ie, nosocomial infections). In preparation for this study, the case report form was pilot tested among a separate, convenience sample of 354 possible infections identified in the cohort from three hospitals in the same city as Vanderbilt University. This separate sample of hospitalisations was used only for pilot testing the case report form, and was not included in the current study. One trained medical reviewer abstracted the relevant information for all selected records using the case report form. During the abstraction process, the lack of a particular finding in the medical record was treated as a lack of evidence for that finding, and so no information was considered missing after abstraction.

    Adjudication of medical records

    All records received were reviewed, abstracted and adjudicated. We made the final determination of whether a hospitalisation represented a confirmed infection or not using a priori definitions of clinical, radiological and/or microbiological findings compatible with infection for each infection type. Previous validation studies and expert clinical knowledge were used to define these specific a priori definitions for each infection type (online supplementary appendix).31 35 40

    Supplementary file 1

    Statistical analysis

    We calculated the PPV of the ICD-9-CM discharge diagnosis codes for identifying hospitalisations for serious infection using the results of the a priori definitions applied to the information abstracted from the medical records as the reference (ie, the proportion of cases identified with discharge diagnosis codes that were determined to be true cases after adjudication of the medical record information). We calculated 95% CIs for the PPV using Wilson’s formula.41 Secondary analyses assessed the PPV for hospitalisations for serious infection across hospitals of different sizes and in different geographical regions of Tennessee.

    We also assessed the reliability of the abstraction process. A second trained medical reviewer abstracted relevant information from a subset of selected records, which included all meningitis and endocarditis records, and a random selection of 10% of each of the remaining infection types. Each reviewer conducted the process independently and blinded from one another. For the subset of records abstracted by both reviewers, inter-reviewer agreement for the adjudication of a true or misidentified infection was assessed using the Gwet’s first agreement coefficient (AC1).42–44 Since Cohen’s kappa statistic can be unreliable when the prevalence of the event and the level of observer agreement are high in the study sample, we used Gwet’s AC1 as a reliability measure unlikely to be affected by these concerns.44–46

    In planned sensitivity analyses, we first assessed the impact of excluding hospitalisations that occurred after the individual was transferred from another healthcare facility, as initial documentation and details of the infection could be missing or incomplete in the receiving hospital.40 We also assessed the impact on the PPV for all infections when requiring microbiological identification of a pathogen (excluding common contaminants) from a sterile site within 2 days before or after the hospitalisation admission date. A final sensitivity analysis among hospitalisations for possible pneumonia assessed the PPV when radiological evidence of pneumonia was required (ie, pneumonia, opacity or infiltrate mentioned in a chest X-ray or CT scan report) (online supplementary appendix). All analyses were performed in Stata-IC, V.15.1.

    Patient and public involvement

    No patients were involved in the development of the research question, the outcome measures, or the design or conduct of the study. As we conducted a retrospective study using administrative data, we have no plans to disseminate the results of the research to study participants.

    Results

    Cohort characteristics

    Among a retrospective cohort of 129 465 adults >=50 years of age enrolled in TennCare, 9769 hospitalisations for serious infection were identified during the study period (2008–2012) among 7770 unique patients (figure 2). Cohort members were primarily female (57.8%) with a median age of 54 years (mean: 57 years; range: 50–110). For efficiency considerations, our medical chart review activities then focused on hospitalisations for serious infection (n=8322) that occurred at hospitals within 200 miles of VUMC. Pneumonia, cellulitis and bacteraemia/sepsis were the most common infections identified using discharge diagnosis codes (54.3%, 20.5% and 18.4%, respectively), followed by pyelonephritis (3.8%) and septic arthritis/osteomyelitis (2.5%). Fewer than 1% of hospitalisations were due to meningitis/encephalitis (n=30) and endocarditis (n=18).

    Figure 2
    Figure 2

    Identifying a retrospective cohort of patients >=50 years of age without serious/life-threatening conditions, Tennessee Medicaid (2008–2012).

    Collection, review and adjudication of selected medical records

    Of the 27 hospitals that were initially selected for the sample, 21 (78%) were able to participate. We selected seven additional hospitals to replace the six non-participants to achieve the desired sample size, including an additional small hospital in the East region due to a large number of unavailable records from a single participating hospital.

    We received 716 (88.6%) of 808 requested records from 28 participating hospitals (table 2). Record availability from participating hospitals was lower in medium size hospitals (81.8%) compared with small (93.5%) and large hospitals (91.7%), but did not differ by geographical region. Record availability by infection type was greater than 86% for all infection types, with the exception of hospitalisations for the rare endocarditis cases (57.1%; only four of seven cases).

    View this table:
    Table 2

    Positive predictive value (PPV) of coded discharge diagnosis definitions for hospitalisations for serious infections among adults >=50 years of age enrolled in Tennessee Medicaid, 2008–2012

    The sample of hospitalisations for serious infection included patients who were primarily female (63.6%), with a median age of 60 years (mean: 64 years; range: 50–101) at the time of hospitalisation. There was evidence of transfer from a prior healthcare facility for 21.8% of the hospitalisations for serious infection (highest percentage of transfers for bacteraemia/sepsis (38.5%) and pneumonia (25.1%)). The most common healthcare facility source was a nursing home/skilled nursing facility (84.6%), and included group home sources (7.7%), other sources (4.5%) (assisted living facility, mental health centre) and another acute care hospital (3.2%). There was evidence of an emergency department visit within 7 days prior to admission date for the serious infection hospitalisation in 4.8% of the records.

    Performance of discharge diagnosis codes

    A total of 646 (PPV: 90.2% (95% CI 87.8% to 92.2%)) of the hospitalisations for serious infection identified using ICD-9-CM primary discharge diagnosis codes were confirmed by applying the a priori definitions to the abstracted data. The PPV was highest for pneumonia and cellulitis (96.5% (95% CI 93.9% to 98.0%) and 91.1% (95% CI 84.7% to 94.9%), respectively), and was > 75% for bacteraemia/sepsis, pyelonephritis, septic arthritis/osteomyelitis and endocarditis. The PPV was lowest for meningitis/encephalitis (50.0% (95% CI 23.7% to 76.3%)), although the precision was limited due to a low number of available records for review (table 2). Among the 10 potential cases of meningitis/encephalitis, seven cases were meningitis/meningoencephalitis and three were encephalitis. The respective PPVs for meningitis/meningoencephalitis and encephalitis were 71.4% (95% CI 35.9% to 91.8%) and 0%, respectively.

    When performance was evaluated across stratification sampling parameters, no apparent differences were observed in the PPV for records from hospitals in different geographical regions of Tennessee. Although the PPV was high for all three discharge volume groups, the PPV was significantly lower in high-volume hospitals (84.6% (95% CI 79.6% to 88.5%)) compared with low-volume hospitals (93.9% (95% CI 90.0% to 96.3%); PPV difference: −9.3% (95% CI −14.7% to −3.9%)) and medium-volume hospitals (92.7% (95% CI 88.6% to 95.4%); PPV difference: −8.1% (95% CI −13.7% to −2.6%)) (table 2). This was likely driven by the different distributions in the types of infections selected for review in the hospital groups. Large hospitals had a higher proportion of non-pneumonia infections (70.4%) compared with medium and small hospitals (49.4% and 36.1%, respectively). Importantly, the PPV for pneumonia was similar in each discharge volume group (range: 96.0%–96.6%), whereas the PPV was smaller for non-pneumonia infections in large hospitals (79.8%) compared with medium (88.7%) and small (89.2%) hospitals.

    In the 82 records independently abstracted by two reviewers to assess reliability, there was 92.7% (95% CI 86.9% to 98.4%) agreement for identifying true hospitalisations for serious infection. The inter-rater agreement was also high when assessing reliability, independent of the outcome prevalence, with an AC1 of 0.91 (95% CI 0.84 to 0.99). Of the six discordant cases, three were meningitis/encephalitis (one meningitis/meningoencephalitis and two encephalitis), with one each of bacteraemia/sepsis, pyelonephritis and septic arthritis. The main reason for a discrepancy between reviewers was whether or not treatment for the infection of interest occurred within 2 days of the admission date, which was one of the major criteria for adjudication (see online supplementary appendix).

    Sensitivity analyses

    The PPV was virtually unchanged when excluding the 21.8% of hospitalisations that occurred as transfers from another healthcare facility (90.1% (95% CI 87.4% to 92.3%)). Microbiological evidence of the specific infection type was found in 47.6% of records, leading to reduced PPVs when requiring microbiological evidence (45.4% (95% CI 41.8% to 49.1%)). Microbiological evidence of infection was highest in hospitalisations for suspected pyelonephritis (94.4%), but was <60% for every other infection type (pneumonia (42.7%); cellulitis/soft-tissue infections (58.5%); bacteraemia/sepsis (26.1%)). When requiring radiological confirmation of pneumonia, the PPV for coded diagnoses was 78.8% (95% CI 74.2% to 82.8%). Approximately, 95.6% of possible hospitalisations for pneumonia had at least one documented chest X-ray or CT-scan. Among those patients with a chest X-ray or CT-scan report available (n=325), 83.4% had a finding compatible with pneumonia. The main findings among the 54 patients with possible pneumonia and a radiological report available, but without radiological confirmation of pneumonia included atelectasis (n=6), interstitial pneumonitis (n=3), chronic heart failure with pulmonary oedema (n=1) and no radiological findings of any kind (n=44).

    Discussion

    Discharge diagnoses for identifying hospitalisations due to serious infections among middle age and older adults had an overall PPV of 90.2%, with the highest values for the identification of common serious infections. PPVs were consistently high across different hospital types and regions of Tennessee. Furthermore, the PPV was similar after exclusion of hospitalisations for serious infection that were the result of a transfer from another healthcare facility (eg, acute care hospital, skilled nursing facility). Microbiological confirmation was available for fewer than 50% of patients admitted with possible hospitalisations for serious infections, and as expected, the requirement resulted in a low PPV for all infections, with the exception of pyelonephritis. Importantly, the PPV for pneumonia hospitalisations remained relatively high even when requiring radiological confirmation.

    The PPV for hospitalisations for pneumonia in previous smaller validation studies has ranged from 72% to 86% in different healthcare systems, but those studies were not focused on middle age and older adults.31 47–49 In our study of hospitalisations among middle age and older adults, we found that coded discharge diagnoses have a higher PPV for pneumonia compared with previous studies. The PPV for bacteraemia/sepsis was also on the higher range of previously reported PPVs for diagnosis codes to identify bacteraemia/sepsis from administrative data in other populations (reported range from 45% to 97.7%), and for septic arthritis/osteomyelitis compared with a previous study conducted among patients with diabetes (63.9% vs 75.9% in our study).23 50 51 Overall, the observed PPV for all infections in our study was comparable to two previous comprehensive validation studies of bacterial infections, one among patients with rheumatoid arthritis in a single hospital system and another among patients in one of the Veteran’s Affairs integrated service networks.35 40 Compared with these two previous studies of ICD-9 codes, we abstracted and adjudicated a larger number of records while using a more systematic sampling strategy to retrieve and review records for hospitalisations from multiple regions and hospital types as opposed to a single hospital or healthcare system. However, some of the PPVs for individual infections were less precise and less similar to these previous studies. This was especially true for rare infections, as would be expected due to the low numbers of rare infections in our study and across previous studies.35 40 The results of our study are also similar to previous validation studies that used corresponding ICD-10 diagnosis codes to identify hospitalisations for serious infection.52 53

    One limitation to consider in our study was that it was not designed to estimate the sensitivity and specificity of the coding algorithms. This would have required the identification, review and adjudication of a sample of hospitalisations that did not fulfil our algorithm (ie, presence of the ICD-9 primary discharge diagnosis codes indicative of infection). However, when the prevalence of an outcome is low, the PPV approximates the specificity.54 Importantly, any non-differential outcome misclassification between exposure groups resulting from the use of imperfect but highly specific measurements would attenuate the impact of the misclassification on the relative risk estimates.55 In addition, we found that the PPV of coded discharge diagnoses for serious infections remained high across hospitals of different sizes and across different geographical areas of Tennessee, which may have different rates of hospitalisations for serious infection.56 Although our study applied a systematic sampling strategy to assure the representation of different settings in our population, our population was restricted to middle age and older adults enrolled in a state Medicaid programme. Therefore, caution is warranted when extrapolating the study findings to other populations.

    Another limitation is the use of available clinical information to operationalise definitions for adjudication of true hospitalisations for infections. It is possible that some procedures, laboratory findings and diagnoses that informed the final diagnosis of infection were not fully recorded in the medical records, and thus, were not available for our review and may have contributed to the observed PPV for some infections. Although we used previous validation studies and clinical information to build prespecified definitions for the adjudication of true infections, our reference criteria may be imperfect, considering the retrospective nature of our determinations and potential variability in clinical practice. Nevertheless, we also assessed how the availability of selected findings (ie, microbiological and radiological information) in the medical record impacted the overall and infection-specific PPV. We demonstrated that relying on highly specific clinical diagnostics, such as microbiological and radiological information, to confirm true infections would result in lower PPVs for identification of infections in administrative data. Requiring microbiological confirmation to confirm true infections is challenging because of the known low sensitivity of culture-based diagnostic methods (most commonly used in clinical practice), which may lead to misclassification.57 58 In addition, requiring radiological evidence compatible with pneumonia within 2 days of hospital admission did lower the observed PPV for pneumonia hospitalisations. Nevertheless, the observed PPV remained close to 80%, which should reduce concerns about using diagnosis codes to identify hospitalisations due to pneumonia. Finally, the coding algorithms were based on the ICD-9-coding system only. Although these findings will be helpful for retrospective studies that encompass periods of ICD-9 use, additional studies evaluating the performance of ICD-10-based codes would be useful to complement our findings.

    Our study demonstrated that discharge diagnosis codes can be used to accurately identify hospitalisations for serious infections among middle age and older adults. The highest PPVs were observed for the most common infections, and the PPV for pneumonia remained high when requiring radiological confirmation. Importantly, consistently high PPVs were observed across different hospital sizes and regions. However, the estimated PPV was lower and less precise for very rare infections (eg, encephalitis). This should be an important consideration for studies specifically focused on those less frequent outcomes, especially when strict microbiological confirmation is required. Taken together, these findings support the use of discharge diagnosis codes for infections to identify outcomes in ongoing and future epidemiological studies among middle age and older adults.

    Acknowledgments

    We are indebted to the Tennessee Bureau of TennCare of the Tennessee Department of Finance and Administration, which provided data for the study. We are also indebted to the Tennessee Department of Health for providing data for the study. Statements in the report should not be construed as endorsement by the US Department of Health and Human Services, the Department of Veterans Affairs or the Tennessee Department of Health.

    References

    1. 1.↵
      2. Khabbaz RF ,
      3. Moseley RR ,
      4. Steiner RJ , et al
      . Challenges of infectious diseases in the USA. Lancet 2014;384:53–63.doi:10.1016/S0140-6736(14)60890-4
      OpenUrlCrossRefPubMedWeb of Science
    2. 2.↵
      2. Grohskopf LA ,
      3. Sokolow LZ ,
      4. Broder KR , et al
      . Prevention and Control of Seasonal Influenza with Vaccines. MMWR Recomm Rep 2016;65:1–54.doi:10.15585/mmwr.rr6505a1
      OpenUrlCrossRefPubMed
    3. 3.↵
      2. Kim DK ,
      3. Bridges CB ,
      4. Harriman KH
      . Centers for Disease Control and Prevention (CDC) Advisory Committee on Immunization Practices (ACIP) ACIP Adult Immunization Work Group. Advisory committee on immunization practices recommended immunization schedule for adults aged 19 years or older--United States, 2015. MMWR Morb Mortal Wkly Rep 2015;64:214–2.doi:10.7326/M14-2755
      OpenUrl
    4. 4.↵
      2. Jeon MH ,
      3. Chung JW ,
      4. Choi SH , et al
      . Pneumonia risk factors and clinical features of hospitalized patients older than 15 years with pandemic influenza A (H1N1) in South Korea: a multicenter study. Diagn Microbiol Infect Dis 2011;70:230–5.doi:10.1016/j.diagmicrobio.2011.01.005
      OpenUrlPubMed
    5. 5.↵
      2. Pilishvili T ,
      3. Bennett NM
      . Pneumococcal Disease Prevention Among Adults: Strategies for the Use of Pneumococcal Vaccines. Am J Prev Med 2015;49(6 Suppl 4):S383–90.doi:10.1016/j.amepre.2015.09.008
      OpenUrl
    6. 6.↵
      2. Sjoding MW ,
      3. Prescott HC ,
      4. Wunsch H , et al
      . Longitudinal Changes in ICU Admissions Among Elderly Patients in the United States. Crit Care Med 2016;44:1353–60.doi:10.1097/CCM.0000000000001664
      OpenUrl
    7. 7.↵
      2. Crotty MP ,
      3. Meyers S ,
      4. Hampton N , et al
      . Epidemiology, Co-Infections, and Outcomes of Viral Pneumonia in Adults: An Observational Cohort Study. Medicine 2015;94:e2332.doi:10.1097/MD.0000000000002332
    8. 8.↵
      2. Jain S ,
      3. Self WH ,
      4. Wunderink RG , et al
      . Community-Acquired Pneumonia Requiring Hospitalization among U.S. Adults. N Engl J Med 2015;373:415–27.doi:10.1056/NEJMoa1500245
      OpenUrlCrossRefPubMed
    9. 9.↵
      2. Filbin MR ,
      3. Arias SA ,
      4. Camargo CA , et al
      . Sepsis visits and antibiotic utilization in U.S. emergency departments*. Crit Care Med 2014;42:528–35.doi:10.1097/CCM.0000000000000037
      OpenUrl
    10. 10.↵
      2. Christensen KL ,
      3. Holman RC ,
      4. Steiner CA , et al
      . Infectious disease hospitalizations in the United States. Clin Infect Dis 2009;49:1025–35.doi:10.1086/605562
      OpenUrlCrossRefPubMedWeb of Science
    11. 11.↵
      2. Ginde AA ,
      3. Moss M ,
      4. Shapiro NI , et al
      . Impact of older age and nursing home residence on clinical outcomes of US emergency department visits for severe sepsis. J Crit Care 2013;28:606–11.doi:10.1016/j.jcrc.2013.03.018
      OpenUrlCrossRefPubMedWeb of Science
    12. 12.↵
      2. Iung B ,
      3. Vahanian A
      . Epidemiology of acquired valvular heart disease. Can J Cardiol 2014;30:962–70.doi:10.1016/j.cjca.2014.03.022
      OpenUrlCrossRefPubMed
    13. 13.↵
      2. Thigpen MC ,
      3. Whitney CG ,
      4. Messonnier NE , et al
      . Bacterial meningitis in the United States, 1998-2007. N Engl J Med 2011;364:2016–25.doi:10.1056/NEJMoa1005384
      OpenUrlCrossRefPubMedWeb of Science
    14. 14.↵
      2. Simonetti AF ,
      3. Garcia-Vidal C ,
      4. Viasus D , et al
      . Declining mortality among hospitalized patients with community-acquired pneumonia. Clin Microbiol Infect 2016;22:567.e1–7.doi:10.1016/j.cmi.2016.03.015
      OpenUrlCrossRef
    15. 15.↵
      2. Rivero-Calle I ,
      3. Pardo-Seco J ,
      4. Aldaz P , et al
      . Incidence and risk factor prevalence of community-acquired pneumonia in adults in primary care in Spain (NEUMO-ES-RISK project). BMC Infect Dis 2016;16:645.doi:10.1186/s12879-016-1974-4
      OpenUrl
    16. 16.↵
      2. Park JY ,
      3. Park S ,
      4. Lee SH , et al
      . Microorganisms Causing Community-Acquired Acute Bronchitis: The Role of Bacterial Infection. PLoS One 2016;11:e0165553.doi:10.1371/journal.pone.0165553
    17. 17.↵
      2. Jackson ML ,
      3. Walker R ,
      4. Lee S , et al
      . Predicting 2-Year Risk of Developing Pneumonia in Older Adults without Dementia. J Am Geriatr Soc 2016;64:1439–47.doi:10.1111/jgs.14228
      OpenUrl
    18. 18.↵
      2. Clement RC ,
      3. Haddix KP ,
      4. Creighton RA , et al
      . Risk Factors for Infection After Knee Arthroscopy: Analysis of 595,083 Cases From 3 United States Databases. Arthroscopy 2016;32:2556–61.doi:10.1016/j.arthro.2016.04.026
      OpenUrl
    19. 19.↵
      2. Weinert BA ,
      3. Edmonson MB
      . Hospitalizations at Nonfederal Facilities for Lower Respiratory Tract Infection in American Indian and Alaska Native Children Younger than 5 Years of Age, 1997-2012. J Pediatr 2016;175:33–9.doi:10.1016/j.jpeds.2016.03.019
      OpenUrl
    20. 20.↵
      2. Akiyama T ,
      3. Chikuda H ,
      4. Yasunaga H , et al
      . Incidence and risk factors for mortality of vertebral osteomyelitis: a retrospective analysis using the Japanese diagnosis procedure combination database. BMJ Open 2013;3:e002412.doi:10.1136/bmjopen-2012-002412
      OpenUrlAbstract/FREE Full Text
    21. 21.↵
      2. Graversen ME ,
      3. Dalgaard LS ,
      4. Jensen-Fangel S , et al
      . Risk and outcome of pyelonephritis among renal transplant recipients. BMC Infect Dis 2016;16:264.doi:10.1186/s12879-016-1608-x
      OpenUrl
    22. 22.↵
      2. Abou Zahr Z ,
      3. Spiegelman A ,
      4. Cantu M , et al
      . Perioperative use of anti-rheumatic agents does not increase early postoperative infection risks: a Veteran Affairs' administrative database study. Rheumatol Int 2015;35:265–72.doi:10.1007/s00296-014-3121-0
      OpenUrl
    23. 23.↵
      2. Carnahan RM ,
      3. Herman RA ,
      4. Moores KG
      . A systematic review of validated methods for identifying transfusion-related sepsis using administrative and claims data. Pharmacoepidemiol Drug Saf 2012;21(Suppl 1):222–9.doi:10.1002/pds.2322
      OpenUrlPubMed
    24. 24.↵
      2. Condell O ,
      3. Gubbels S ,
      4. Nielsen J , et al
      . Automated surveillance system for hospital-acquired urinary tract infections in Denmark. J Hosp Infect 2016;93:290–6.doi:10.1016/j.jhin.2016.04.001
      OpenUrl
    25. 25.↵
      2. Gedeborg R ,
      3. Furebring M ,
      4. Michaëlsson K
      . Diagnosis-dependent misclassification of infections using administrative data variably affected incidence and mortality estimates in ICU patients. J Clin Epidemiol 2007;60:155.e1–11.doi:10.1016/j.jclinepi.2006.05.013
      OpenUrlCrossRef
    26. 26.↵
      2. Goto M ,
      3. Ohl ME ,
      4. Schweizer ML , et al
      . Accuracy of administrative code data for the surveillance of healthcare-associated infections: a systematic review and meta-analysis. Clin Infect Dis 2014;58:688–96.doi:10.1093/cid/cit737
      OpenUrlCrossRefPubMed
    27. 27.↵
      2. Shaklee J ,
      3. Zerr DM ,
      4. Elward A , et al
      . Improving surveillance for pediatric Clostridium difficile infection: derivation and validation of an accurate case-finding tool. Pediatr Infect Dis J 2011;30:e38–40.doi:10.1097/INF.0b013e3182027c22
      OpenUrlCrossRefPubMed
    28. 28.↵
      2. Wright SB ,
      3. Huskins WC ,
      4. Dokholyan RS , et al
      . Administrative databases provide inaccurate data for surveillance of long-term central venous catheter-associated infections. Infect Control Hosp Epidemiol 2003;24:946–9.doi:10.1086/502164
      OpenUrlCrossRefPubMedWeb of Science
    29. 29.↵
      2. Greenberg JA ,
      3. Hohmann SF ,
      4. Hall JB , et al
      . Validation of a Method to Identify Immunocompromised Patients with Severe Sepsis in Administrative Databases. Ann Am Thorac Soc 2016;13:253–8.doi:10.1513/AnnalsATS.201507-415BC
      OpenUrl
    30. 30.↵
      2. Olsen MA ,
      3. Young-Xu Y ,
      4. Stwalley D , et al
      . The burden of clostridium difficile infection: estimates of the incidence of CDI from U.S. Administrative databases. BMC Infect Dis 2016;16:177.doi:10.1186/s12879-016-1501-7
      OpenUrl
    31. 31.↵
      2. Grijalva CG ,
      3. Chung CP ,
      4. Stein CM , et al
      . Computerized definitions showed high positive predictive values for identifying hospitalizations for congestive heart failure and selected infections in Medicaid enrollees with rheumatoid arthritis. Pharmacoepidemiol Drug Saf 2008;17:890–5.doi:10.1002/pds.1625
      OpenUrlCrossRefPubMedWeb of Science
    32. 32.↵
      Administration DoHCF. TennCare Fiscal Year 2015-2016: Administration DoHCF, 2016. Annual Report.
    33. 33.↵
      2. Wiese AD ,
      3. Griffin MR ,
      4. Stein CM , et al
      . Opioid Analgesics and the Risk of Serious Infections Among Patients With Rheumatoid Arthritis: A Self-Controlled Case Series Study. Arthritis Rheumatol 2016;68:323–31.doi:10.1002/art.39462
      OpenUrl
    34. 34.↵
      2. Grijalva CG ,
      3. Kaltenbach L ,
      4. Arbogast PG , et al
      . Initiation of rheumatoid arthritis treatments and the risk of serious infections. Rheumatology 2010;49:82–90.doi:10.1093/rheumatology/kep325
      OpenUrlCrossRefPubMedWeb of Science
    35. 35.↵
      2. Schneeweiss S ,
      3. Robicsek A ,
      4. Scranton R , et al
      . Veteran’s affairs hospital discharge databases coded serious bacterial infections accurately. J Clin Epidemiol 2007;60:397–409.doi:10.1016/j.jclinepi.2006.07.011
      OpenUrlCrossRefPubMedWeb of Science
    36. 36.↵
      2. Brenner H ,
      3. Gefeller O
      . Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med 1997;16:981–91.doi:10.1002/(SICI)1097-0258(19970515)16:9<981::AID-SIM510>3.0.CO;2-N
      OpenUrlCrossRefPubMedWeb of Science
    37. 37.↵
      2. Kim HM ,
      3. Smith EG ,
      4. Stano CM , et al
      . Validation of key behaviourally based mental health diagnoses in administrative data: suicide attempt, alcohol abuse, illicit drug abuse and tobacco use. BMC Health Serv Res 2012;12:18.doi:10.1186/1472-6963-12-18
      OpenUrlCrossRefPubMed
    38. 38.↵
      2. El-Ghitany EM ,
      3. Farghaly AG ,
      4. Farag S , et al
      . Validation of EGCRISC for Chronic Hepatitis C Infection Screening and Risk Assessment in the Egyptian Population. PLoS One 2016;11:e0168649.doi:10.1371/journal.pone.0168649
    39. 39.↵
      2. Harris PA ,
      3. Taylor R ,
      4. Thielke R , et al
      . Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81.doi:10.1016/j.jbi.2008.08.010
      OpenUrlCrossRefPubMedWeb of Science
    40. 40.↵
      2. Patkar NM ,
      3. Curtis JR ,
      4. Teng GG , et al
      . Administrative codes combined with medical records based criteria accurately identified bacterial infections among rheumatoid arthritis patients. J Clin Epidemiol 2009;62:321–7.doi:10.1016/j.jclinepi.2008.06.006
      OpenUrlCrossRefPubMedWeb of Science
    41. 41.↵
      2. Niesner K ,
      3. Murff HJ ,
      4. Griffin MR , et al
      . Validation of VA administrative data algorithms for identifying cardiovascular disease hospitalization. Epidemiology 2013;24:334–5.doi:10.1097/EDE.0b013e3182821e75
      OpenUrl
    42. 42.↵
      2. Govatsmark RE ,
      3. Sneeggen S ,
      4. Karlsaune H , et al
      . Interrater reliability of a national acute myocardial infarction register. Clin Epidemiol 2016;8:305–12.doi:10.2147/CLEP.S105933
      OpenUrl
    43. 43.↵
      2. Wongpakaran N ,
      3. Wongpakaran T ,
      4. Wedding D , et al
      . A comparison of Cohen’s Kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Med Res Methodol 2013;13:61.doi:10.1186/1471-2288-13-61
      OpenUrlCrossRefPubMed
    44. 44.↵
      2. Gwet KL
      . Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 2008;61(Pt 1):29–48.doi:10.1348/000711006X126600
      OpenUrlCrossRefPubMed
    45. 45.↵
      2. Lantz CA ,
      3. Nebenzahl E
      . Behavior and interpretation of the kappa statistic: resolution of the two paradoxes. J Clin Epidemiol 1996;49:431–4.doi:10.1016/0895-4356(95)00571-4
      OpenUrlCrossRefPubMedWeb of Science
    46. 46.↵
      2. Feinstein AR ,
      3. Cicchetti DV
      . High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 1990;43:543–9.doi:10.1016/0895-4356(90)90158-L
      OpenUrlCrossRefPubMedWeb of Science
    47. 47.↵
      2. Drahos J ,
      3. Vanwormer JJ ,
      4. Greenlee RT , et al
      . Accuracy of ICD-9-CM codes in identifying infections of pneumonia and herpes simplex virus in administrative data. Ann Epidemiol 2013;23:291–3.doi:10.1016/j.annepidem.2013.02.005
      OpenUrlCrossRefPubMed
    48. 48.↵
      2. Aronsky D ,
      3. Haug PJ ,
      4. Lagor C , et al
      . Accuracy of administrative data for identifying patients with pneumonia. Am J Med Qual 2005;20:319–28.doi:10.1177/1062860605280358
      OpenUrlCrossRefPubMedWeb of Science
    49. 49.↵
      2. Kern DM ,
      3. Davis J ,
      4. Williams SA , et al
      . Validation of an administrative claims-based diagnostic code for pneumonia in a US-based commercially insured COPD population. Int J Chron Obstruct Pulmon Dis 2015;10:1417–25.doi:10.2147/COPD.S83135
      OpenUrl
    50. 50.↵
      2. Iwashyna TJ ,
      3. Odden A ,
      4. Rohde J , et al
      . Identifying patients with severe sepsis using administrative claims: patient-level validation of the angus implementation of the international consensus conference definition of severe sepsis. Med Care 2014;52:e39–43.doi:10.1097/MLR.0b013e318268ac86
      OpenUrlCrossRefPubMedWeb of Science
    51. 51.↵
      2. Newton KM ,
      3. Wagner EH ,
      4. Ramsey SD , et al
      . The use of automated data to identify complications and comorbidities of diabetes: a validation study. J Clin Epidemiol 1999;52:199–207.doi:10.1016/S0895-4356(98)00161-9
      OpenUrlCrossRefPubMedWeb of Science
    52. 52.↵
      2. Sahli L ,
      3. Lapeyre-Mestre M ,
      4. Derumeaux H , et al
      . Positive predictive values of selected hospital discharge diagnoses to identify infections responsible for hospitalization in the French national hospital database. Pharmacoepidemiol Drug Saf 2016;25:785–9.doi:10.1002/pds.4006
      OpenUrl
    53. 53.↵
      2. Holland-Bill L ,
      3. Xu H ,
      4. Sørensen HT , et al
      . Positive predictive value of primary inpatient discharge diagnoses of infection among cancer patients in the Danish National Registry of Patients. Ann Epidemiol 2014;24:593–7.doi:10.1016/j.annepidem.2014.05.011
      OpenUrlCrossRefPubMed
    54. 54.↵
      2. Schneeweiss S ,
      3. Avorn J
      . A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol 2005;58:323–37.doi:10.1016/j.jclinepi.2004.10.012
      OpenUrlCrossRefPubMedWeb of Science
    55. 55.↵
      2. Rodgers A ,
      3. MacMahon S
      . Systematic underestimation of treatment effects as a result of diagnostic test inaccuracy: implications for the interpretation and design of thromboprophylaxis trials. Thromb Haemost 1995;73:167–71.
      OpenUrlPubMedWeb of Science
    56. 56.↵
      2. van Walraven C ,
      3. English S ,
      4. Austin PC
      . Administrative database code accuracy did not vary notably with changes in disease prevalence. J Clin Epidemiol 2016;79:86–9.doi:10.1016/j.jclinepi.2016.05.009
      OpenUrl
    57. 57.↵
      2. Mancini N ,
      3. Carletti S ,
      4. Ghidoli N , et al
      . The era of molecular and other non-culture-based methods in diagnosis of sepsis. Clin Microbiol Rev 2010;23:235–51.doi:10.1128/CMR.00043-09
      OpenUrlAbstract/FREE Full Text
    58. 58.↵
      2. Niederman MS
      . The argument against using quantitative cultures in clinical trials and for the management of ventilator-associated pneumonia. Clin Infect Dis 2010;51(Suppl 1):S93–9.doi:10.1086/653055
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • Contributors ADW planned the medical record collection and statistical analysis, analysed and interpreted the data, and drafted and revised the paper. MRG, WS, CMS and RG planned the statistical analysis, interpreted the data and revised the paper. EFM prepared the data and revised the paper. CGG designed the project, acquired the data from TennCare, planned the medical record collection and statistical analysis, interpreted the data and revised the paper.

    • Funding This study was funded by the NIH (R03-AG-042981 and R01-AG-043471-01A1) and the TL1 award TL1TR000447.

    • Competing interests CGG has received consulting fees from Pfizer and Merck, and received research support from Sanofi-Pasteur, Campbell Alliance, the Centers for Disease Control and Prevention, National Institutes of Health, The Food and Drug Administration and the Agency for Health Care Research and Quality. WS has received personal fees from Pfizer, Merck, Novavax, Dynavax, Sanofi-Pasteur, GSK, Seqirus and received research support from the Centers for Disease Control and Prevention.

    • Patient consent Not required.

    • Ethics approval The study was approved by the Institutional Review Boards of Vanderbilt University and the Tennessee Department of Health, and by the Bureau of TennCare.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional unpublished data are available from the study. The study protocol and statistical code are available from the corresponding author, ADW (andrew.d.wiese.1@vumc.org).