Figures
Abstract
Measures of morbidity and comorbidity are frequently used for the control of confounding, particularly in health services research. Several proposals for those measures are defined with ICD-coded diagnoses available in hospital routine data. However, a measure that makes use of the ICD structure is missing. Objective of this work was to elaborate the power of the ICD structure for defining morbidity and comorbidity measures. Routine data from three German hospitals with inpatients discharged 2008 were used for model development; routine data from 36 German hospitals with inpatients admitted and discharged 2010 were used for model evaluation. Two different risk models were developed, one based on ICD-10 chapters, the other based on ICD-10 groups. The models were transformed into sum scores using whole-number weights. Models and scores were compared with the Charlson Index and the Elixhauser Comorbidities using the receiver operating characteristic. Dependent variable was hospital death. Logistic regression was used to derive the new models. Charlson Index and Elixhauser Comorbidities were mapped to the German ICD-10. According to the receiver operating characteristic, the quality of the measures based on the structure of the ICD-10 was superior compared with the Charlson Index and the Elixhauser Comorbidities. The best result was achieved with the measure based on ICD-10-groups with an area under curve of 0.910 (95% confidence interval = 0.907–0.913). The sum scores showed a comparable performance. The developed new measures may be used to control for confounding.
Citation: Stausberg J, Hagn S (2015) New Morbidity and Comorbidity Scores based on the Structure of the ICD-10. PLoS ONE 10(12): e0143365. https://doi.org/10.1371/journal.pone.0143365
Editor: Eugenio Paci, ISPO, ITALY
Received: January 10, 2015; Accepted: November 1, 2015; Published: December 14, 2015
Copyright: © 2015 Stausberg, Hagn. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: Data are not made publicly-available due to legal and ethical restrictions on patient-sensitive information. However, data availability requests for interested researchers can be made to Dr. Bidjan Sobhani, DRK Kliniken Berlin, for model development data. Evaluation model data requests are addressed to Qualitätsindikatoren für Kirchliche Krankenhäuser - QKK e. V. Any other data is contained within the paper.
Funding: The development of the models was supported by ID Information und Dokumentation im Gesundheitswesen GmbH & Co. KGaA, Berlin, Germany. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: There is not any relationship from both authors with the company ID Information und Dokumentation im Gesundheitswesen GmbH & Co. KGaA in terms of “employment, consultancy, patents, products in development, marketed products, etc.”. There are not any restrictions concerning sharing of data/or materials due to the funding or any other relationship with that company. The funding does not alter the authors' adherence to PLOS ONE policies on sharing data and materials.
Introduction
Measures of morbidity and comorbidity are used, when the overall burden of diseases is regarded as a confounder. This is particularly relevant in health services research, in quality management, in pay-for-performance approaches, and in health economics. Four reasons had been mentioned for measuring morbidity [1]:
- control of confounding,
- identification of effect modifiers,
- prediction of outcome, and
- improvement of statistical efficiency.
Morbidity can be defined several fold [2]:
- absence of physical and mental well-being,
- a sick individual,
- diseases of a sick individual, or
- the duration of a disease.
For this work, definition three is applied. If morbidity is the set of diseases of an individual, comorbidity might be different. Firstly, comorbidity can be regarded as diseases that exist additionally to one or several diseases of interest [1, 3]. In routine data of inpatient care, the principle diagnosis is frequently taken as the disease of interest, the set of secondary diagnoses as comorbidity. Secondly, comorbidity could be defined as the coexistence of several diseases, without the necessity to exclude a disease of interest [4]. Thirdly, complications could be separated from preexisting diseases and those summarized as comorbidity [5, 6]. All three approaches for the definition had been applied in works about comorbidity measures.
The calculation of morbidity and comorbidity measures from diagnoses coded with the International Statistical Classification of Diseases and Related Health Problems (ICD) has reached great interest (cf. Birkmeyer et al. [7] as application example and Quan et al. [8] as methodological work). In most of the developed countries that reimburse inpatient cases with diagnosis related groups (DRGs), routine data can be used without the necessity of additional data acquisition at least for inpatient care. Diagnoses coded with the ICD are part of the routine data. Several proposals for the definition of morbidity or comorbidity measures based on ICD-coded diagnoses had been made [8–12]. These definitions make use of two different comorbidity measures, the Charlson Index published in 1987 [13] and the Elixhauser Comorbidities published in 1998 [10]. Both measures and their derivatives were developed through a combination of expert opinions and statistical analyses. Meanwhile, several reviews are available comparing the performance of the different measures in different settings [14, 15]. Yurkovich et al. showed that there is room for improvement [15]. Predicting hospital mortality, analyses based on the Charlson Index received very heterogeneous results with an area under curve (AUC) between 0.608 and 0.860, scores based on the Elixhauser Comorbidities with an AUC between 0.632 and 0.878.
As far as the author knows, the feasibility of the native ICD structure as basis of morbidity or comorbidity measures has not been analyzed yet. Therefore, this analysis intended to make use of the ICD structure for defining such measures, not only to code expert opinions about those measures with the ICD as in previous work. Furthermore, the quality of respective measures was compared with ICD-based definitions of the Charlson Index and the Elixhauser Comorbidities. In-hospital death was used as the dependent variable. The study was split up in the development and the validation of the measures using different convenient samples of inpatient cases.
Materials and Methods
Data sets
Anonymized routine data of 51,898 inpatient cases discharged in 2008 from three acute-care hospitals were used for the development of the model (cf. Table 1). The hospitals belonged to one hospital group in a large city of Germany. Newborns were excluded from the analysis. Hospital mortality was 2.4%; the mean number of secondary diagnoses was 3.55 with a standard deviation of 3.33. For evaluation, anonymized routine data of 435,076 inpatient cases admitted and discharged in 2010 from 36 acute-care hospitals in Germany were included. Those hospitals participated in a nationwide quality benchmarking project of ecclesiastical institutions. Hospital mortality was 2.3%; the mean number of secondary diagnoses was 4.61 with a standard deviation of 4.56. All diagnoses, i.e. principal as well as secondary ones were available, coded with the ICD-10 German Modification 2008 (ICD-10-GM 2008) or ICD-10-GM 2010. A present on admission flag was not available in both data sets. It is still missing in hospital routine data in Germany. Furthermore, a patient identifier was missing too. Therefore, it was not able to assign different inpatient cases to an individual.
In alphabetical order, the five most frequent principal diagnoses in the develeopment data set were atherosclerosis of arteries of extremities Fontaine stage IIA (ICD-10-GM code I70.21), left ventricular failure with severe limitations (I50.14), obstructive sleep apnoea syndrome (G47.31), other forms of angina pectoris (I20.8), and overlapping lesion of bronchus and lung (C34.8). The five most frequent principal diagnoses in the evaluation data set were concussion (S06.0), left ventricular failure with severe limitations (I50.14), other primary gonarthrosis (M17.1), singleton, born in hospital (Z38.0), and syncope and collapse (R55).
Model and score development
For the new measures concerning the ICD-10-structure, two risk models were developed using logistic regression with hospital death as dependent and the ICD-10-chapters or ICD-10-groups as independent variables. In the final models, non-significant chapters or groups were excluded. Both models were developed using the principal diagnosis (PDX) as well as secondary ones (SDX). For each patient, the set of secondary diagnoses was trimmed as follows. Additional codes were excluded. On the one hand additional codes were identified by the dagger and asterisk-system that is used in the ICD-10 internationally. Typically, dagger codes describe the etiology and asterisk codes the localization of a disease. The use of dagger codes is obligatory, the additional use of asterisk codes optional. On the other hand additional codes are flagged by an exclamation mark in the ICD-10-GM nationally. Those codes are not part of the international version of the ICD-10. A disease occurring left and right simultaneously (e.g. both right hip and left hip arthrosis) was counted only once. Furthermore, a code recorded several times was not counted but once.
The coefficients of the models were transformed into a score according to Perneger et al. [16] and Sullivan et al. [17]. The regression coefficient with the minimal absolute value received the weight “1” or “-1” depending on its prefix. The other regression coefficients were transformed by dividing their absolute value by the minimal absolute value rounded to a whole number preserving the prefix. The score of an individual case is then calculated as the sum of the weights counting a group or chapter only once.
Other variables as sex and gender were not included into the models, even if they influence hospital mortality. The goal was not to find the best model explaining hospital mortality applying the whole range of patient characteristics. This is typically the case in risk models used to calculate a hospital standardized mortality ratio (HSMR) [18, 19]. Those risk models include a comorbidity score as one out of several independent variables. It would be contradictory if the comorbidity score itself includes other variables.
Measures for comparison
Charlson et al. calculated the mortality risk of specific diseases using univariate statistical procedures [13]. An exclusion of a disease of interest is not reported. A relevant risk was identified for 19 conditions. A weight of 1, 2, 3 or 6 was given to each condition. The Charlson Index is the sum of the weights. In this analysis, a reduced version with 17 conditions with the original weights of Charlson et al. was used [9]. The definition of the conditions with the ICD 10th Revision Canadian version (ICD-10-CA) by Quan et al. [8] was mapped to the ICD-10-GM 2010.
Elixhauser et al. published a list of 30 diseases coded with the ICD 9th Revision Clinical Modifications (ICD-9-CM) that showed a risk for length of stay, costs, and mortality [10]. The principal diagnosis was excluded. Later on, van Walraven et al. [20] added a scoring algorithm based on 21 out of the 30 Elixhauser Comorbidities with weights between -7 to 12. The occurrence of negative weights is an artifact resulting from the statistical methodology. A negative weight does not denote diseases preventing patients from dying during hospital stay. The score is then calculated as the sum of the weights. The definition of the comorbidities with the ICD-10-CA by Quan et al. was also mapped to the ICD-10-GM 2010.
For evaluation, each case was characterized by values of six different models: Charlson Index, score of Elixhauser Comorbidities, risk according to the model based on ICD-10-chapters, risk according to the model based on ICD-10-groups, ICD-10-chapter-score, and ICD-10-group-score. All six models were applied to the data once excluding and once including the PDX leading to 12 different measures.
Statistics and software
The accordance between the measures was checked using the non-parametric correlation coefficient of Spearman. The methodological quality of the measures was analyzed calculating the AUC of the receiver operating characteristic (ROC) with hospital death as the dependent variable. Significance was assumed with a p < = 0.05 (both sides). Goodness of fit was analyzed with Nagelkerke’s R2. The data were maintained with the relational database management system Microsoft Access. The correlation analysis and the logistic regressions were done with IBM SPSS Statistics. All ICD-10-chapters were considered in a first regression analysis (method ENTER). Non-significant chapters were excluded from the final model in a second regression analysis. The logistic regression with ICD-10-groups was run forward stepwise (method FSTEP). Therefore, only significant variables remained in the model.
Results
Measures based on the ICD structure
The first measure using the ICD structure was defined on the level of chapters. The ICD-10-GM is divided into 22 chapters. Five chapters were excluded:
- Chapter I “Certain infectious and parasitic diseases“, because it includes many additional codes.
- Chapter XV “Pregnancy, childbirth and the puerperium“, because it does not describe diseases primarily.
- Chapter XX “External causes of morbidity and mortality”, because it includes solely additional codes.
- Chapter XXI “Factors influencing health status and contact with health services”, because the conditions that are classified here are usually not regarded as a disease.
- Chapter XXII “Codes for special purposes”, because it includes solely undefined codes.
The second measure using the ICD structure was defined on the level of groups. In some parts of the ICD-10, groups are further structured hierarchically. In those parts, only the uppermost group was considered:
- C00-C97 Malignant neoplasms
- M00-M25 Arthropathies
- M40-M54 Dorsopathies
- M60-M79 Soft tissue disorders
- M80-M94 Osteopathies and chondropathies
Groups belonging to the chapters I, XV, XX, XXI, and XXII (see above) were excluded as well. In the calculation of both measures, codes including death as part of their definition were excluded:
- Codes within group R95-R99 Ill-defined and unknown causes of mortality
- I46.1 Sudden cardiac death, so described
- O95 Obstetric death of unspecified cause (excluded due to the chapter selection as well)
- O96 Death from any obstetric cause occurring more than 42 days but less than one year after delivery (excluded due to the chapter selection as well)
- P95 Fetal death of unspecified cause
From 241 ICD-10-GM-groups, 165 remained for the measure based on ICD-10-groups.
The final models included 13 ICD-10-GM-chapters (S1 Appendix) and 42 ICD-10-GM-groups (S2 Appendix). The model based on ICD-10-chapters received an AUC = 0.863 (confidence interval (CI) = 0.853–0.872), the model based on ICD-10-groups an AUC = 0.916 (CI = 0.909–0.923). The ICD-10-chapter-score included weights between -6 and +7 with a maximal sum of 32; the ICD-10-group-score included weights between -131 and +6 with a maximal sum of 71.
Nagelkerke’s R2 was 0.229 for the model based on ICD-10-chapters and 0.330 for the model based on ICD-10-GM-groups.
Evaluation
Tables 2 and 3 show the results of the correlation analysis. All bivariate comparisons were statistical significant with p< = 0.01. The maximum of the correlation coefficient for different models was 0.734 between the Charlson Index and the Elixhauser Comorbidities including the PDX and 0.711 excluding the PDX. The minimum was 0.488 between the ICD-10-group-score and the Charlson Index including the PDX and 0.418 excluding the PDX. The correlation coefficients were equal and higher than 0.990 for the comparisons of the scores with the risks recalculated using the original ß-coefficients. Having correlation coefficients between 0.451 and 0.539, the new measures are less concordant among each other than the Charlson Index and the Elixhauser comorbidities.
Table 4 shows the results of the ROC-analysis. All results were different from the null hypotheses (AUC = 0.5) with p<0.001. The initial performance of the new models was confirmed with an AUC = 0.856 (CI = 0.852–0.859) for the model based on ICD-10-chapters and an AUC = 0.910 (CI = 0.907–0.913) for the model based on ICD-10-groups. According to their 95%-confidence limits, the scores performed equally to the raw models. Inclusion of the PDX did not change the performance of the model based on ICD-10-chapters; the model based on ICD-10-groups showed a slightly higher AUC = 0.910 (CI = 0.907–0.913) with PDX than without PDX (AUC = 0.894, CI = 0.891–0.897). To check for a bias due to the extreme negative weight of group K35-K39 “Diseases of appendix” the scores based on ICD-10-groups were additionally analyzed excluding diagnoses codes of that group (results not shown in Table 4). The results were identical to the original scores with an AUC = 0.909 (CI = 0.906–0.911) with PDX and AUC = 0.891 (CI = 0.888–0.895) without PDX.
Including the PDX, the Charlson Index received an AUC = 0.792 (CI = 0.788–0.797) and the Elixhauser Comorbidities an AUC = 0.829 (CI = 0.825–0.833), being significantly lower than all results achieved with the new models based on the ICD structure.
Discussion
Two new risk models for morbidity and comorbidity based on the ICD structure were developed using logistic regression analyses with hospital death as dependent variable. One of them was based on 13 out of 22 chapters of the ICD-10-GM, the other one on 42 out of 241 groups. The performance results of the development stage were confirmed in the evaluation using a totally independent data set of more than 400,000 inpatient cases. Moreover, the sum score for both models showed a comparable performance to the application of the raw model. This is important to provide an easy-to-use-approach for the application of the models in future projects. Depending on the use case, the scores are applicable with and without the PDX. Including the PDX will address morbidity, excluding the PDX will address comorbidity. The weight determined for ICD-10-group K35-K39 “Diseases of appendix” appears as an artifact. Nevertheless, diagnoses from that group could be excluded without lowering the score’s performance.
The concordance of the new measures among each other and with the Charlson Index and the Elixhauser Comorbidities was weaker than the concordance between the Charlson Index and the Elixhauser Comorbidities. Charlson Index and Elixhauser Comorbidities might share a common tradition that is not addressed by the new measures. Surprisingly, the concordance between the ICD-based measures was lower than the concordance between their competitors. The formal hierarchy of the ICD-10 might not correctly reflect the clinical relationship between diseases. As a consequence, the meaning of a branch in terms of morbidity or comorbidity changes between chapters and groups.
In the ROC-analysis, the measure based on ICD-10-groups showed the best performance, independently of the exclusion or inclusion of the PDX. As far as the authors know, an AUC of 0.910 (CI 0.907–0.913) for hospital death is the best value published for ICD-based measures ever, if the measure is applied without co-variables like sex and age. A superior result was published by Quail et al. combining the Elixhauser Comorbidities with a base model [21]. They calculated an AUC of 0.913 (CI 0.910–0.916) for death in the year following the base year in a general population cohort of 662,423 people in Canada.
The superiority of the Elixhauser Comorbidities to the Charlson Index was recognized in several studies before. E. g. Southern et al. [22] received an AUC of 0.704 for the Charlson Index in the adaptation of Deyo et al. and an AUC of 0.793 for the Elixhauser Comorbidities. In that study, the principal and the secondary diagnoses coded with the ICD-9-CM had been included. Ou compared both comorbidity measures defined with the ICD-10 [23]. The Charlson Index received an AUC of 0.860 in the prediction of hospital mortality, the Elixhauser Comorbidities an AUC of 0.870, including all diagnoses for both measures. In the work of Chu et al. [24], results between 0.708 and 0.737 were reached for both measures in the prediction of hospital mortality for acute myocardial infarction and chronic obstructive pulmonary disease. The results derived from the German dataset for the international measures are similar. This might be an indicator of the high quality of diagnoses documentation for inpatient care in Germany [25]. Together with the USA and Belgium, the mean number of secondary diagnoses in German hospitals is the highest in 14 member states of the Organisation for Economic Co-operation and Development (OECD), all using some kind of a DRG-like reimbursement system [26]. Therefore, a general underreporting of diagnoses could be ruled out. However, reimbursement incentives could induce an improper up-coding [27]. As far as we know, that did not happen in Germany due to a strong quality control of the medical documentation on behalf of the statutory health insurances [28, 29].
The risk models based on the ICD structure reached reasonable results. The model applying the chapters of the ICD-10 achieved results that are reasonably higher than those of the Charlson Index and the Elixhauser Comorbidities. A further differentiation of the chapters to the groups of the ICD-10 then improved the model’s quality yet again. Obviously, ICD-10-groups cover comorbidity relevant for hospital mortality quite better than the chapters. A further automatic differentiation is difficult due to the high number of about 14,000 terminal codes in the ICD-10-GM, a number too large as to include them as independent variables into the analysis. The results presented in this paper demonstrate the promising approach of using the native structure of the ICD-10 as basis for morbidity and comorbidity measures.
The presented scores might not be appropriate for every setting. Studies focusing on homogeneous groups of patients suffering from an index disease might benefit from a disease specific score. Furthermore, the presented scores are justified by hospital mortality as proxy for morbidity. If other proxies are favored, scores developed specifically with those proxies as dependent variable might be better.
Study limitations
The presented study made use of diagnoses recorded and coded in hospitals routine data to estimate the level of morbidity and comorbidity of inpatients. An evaluation of other sources—for example a subjective assessment of the patients’ comorbidity level by experts—was not part of the study. Furthermore, there seems to be a mismatch in some ICD-10 groups and chapters between the weights and the perceived clinical relevance. Those ICD-10 groups and chapters might indicate situations going beyond the specific diseases. Further research could elaborate options to keep the presented approach based on the structure of the ICD-10 while adapting the model to the clinical appraisal.
One might claim that the international approaches perform inferior in Germany due to differences in case mix, etc. This concern can be rejected because the results achieved in German data are equal to the results published in the literature.
Morbidity and comorbidity as well as the respective measures are complex constructs that could be operationalized in several ways, for example:
- diagnoses could be coded differently with the same classification (see the differences between [9] and [12]),
- conditions can be included or excluded (see the smaller version of the Charlson Index proposed by Halfon et al. [11]),
- different weight scores could be used (see [30] for alternative weights for the Charlson Index),
- the principal diagnosis can be included or excluded,
- complications could be considered separately (as in [31]) and
- the dependent variable could be changed.
Complications were not excluded from the models and the data. At least the exclusion of complications from model development is sometimes mentioned as an important feature. However, there is first evidence, that not the frequency of complications determines hospital mortality but rather the handling of complications [32]. If this is true, the exclusion of complications in risk adjustment could lead to wrong conclusions, if a score is applied in research concerned with mortality as adverse event. Therefore, further research should focus on the methodological role of those characteristics for measuring morbidity and comorbidity. Furthermore, the applicability of the scores in other research settings like outpatient care and its performance regarding prospective mortality (e. g. one year) should be analyzed in subsequent studies.
The use of routine data that are primarily recorded for reimbursement purposes might be regarded as problematic due to several reasons. Diagnoses are not necessarily clinically verified, routine data might exaggerate the disease burden due to an inadequate up-coding, routine data might be incomplete because information is missing that is financially not relevant, and diagnoses might be imprecise due to the coding by administrative staff.
Supporting Information
S1 Appendix. Risk model based on ICD-10-chapters.
Table A shows the variables along with their coefficients and weights of the risk model based on ICD-10-chapters.
https://doi.org/10.1371/journal.pone.0143365.s001
(DOCX)
S2 Appendix. Risk model based on ICD-10-groups.
Table B shows the variables along with their coefficients and weights of the risk model based on ICD-10-groups.
https://doi.org/10.1371/journal.pone.0143365.s002
(DOCX)
Author Contributions
Conceived and designed the experiments: JS SH. Performed the experiments: JS SH. Analyzed the data: JS SH. Contributed reagents/materials/analysis tools: JS SH. Wrote the paper: JS SH.
References
- 1. de Groot V, Beckerman H, Lankhorst GJ, Bouter LM. How to measure comorbidity: a critical review of available methods. J Clin Epidemiol 2003; 56: 221–229. pmid:12725876
- 2.
Porta M. A dictionary of epidemiology. New York: Oxford University Press; 2008.
- 3. Feinstein AR. The pre-therapeutic classification of co-morbidity in chronic disease. J Chron Dis 1970; 23: 455–468. pmid:26309916
- 4. Valderas JM, Starfield B, Sibbald B, Salisbury C, Roland M. Defining comorbidity: implications for understanding health and health services. Ann Fam Med 2009; 7: 357–363. pmid:19597174
- 5. Luft HS. Methodological commentary. Health Services Research 1997; 32: 239–242.
- 6. Pompei P, Charlson ME, Ales K, MacKenzie CR, Norton M. Relating patient characteristics at the time of admission to outcomes of hospitalization. J Clin Epidemiol 2008; 44: 1063–1069.
- 7. Birkmeyer JD, Siewers AE, Finlayson EV, Stukel TA, Lucas FL, Batista I, et al. Hospital volume and surgical mortality in the United States. N Engl J Med 2002; 346: 1128–1137. pmid:11948273
- 8. Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 2005; 43: 1130–1139. pmid:16224307
- 9. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992; 45: 613–619. pmid:1607900
- 10. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care 1998; 36: 8–27. pmid:9431328
- 11. Halfon P, Eggli Y, van Melle G, Chevalier J, Wasserfallen J-B, Burnand B. Measuring potentially avoidable hospital readmissions. J Clin Epidemiol 2002; 55: 573–587. pmid:12063099
- 12. Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol 1993; 46: 1075–1079. pmid:8410092
- 13. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987; 40: 373–383. pmid:3558716
- 14. Mansour TAS, Aylin P, Bottle A. Systematic review of comorbidity indices for administrative data. Med Care 2012; 50: 1109–1118. pmid:22929993
- 15. Yurkovicha M, Avina-Zubieta A, Thomas J, Gorenchtein M, Lacaill D. A systematic review identifies valid comorbidity indices derived from administrative health data. Journal of Clinical Epidemiology 2015, 68: 3–14. pmid:25441702
- 16. Perneger T, Rae A-C, Gaspoz J-M, Borst F, Vitek O, Héliot C. Screening for pressure ulcer risk in an acute care hospital: Development of a brief bedside scale. J Clin Epidemiol 2002; 55: 498–504. pmid:12007553
- 17. Sullivan LM, Massaro JM, D'Agostino RBD. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Statistics in Medicine 2004; 23: 1631–1660. pmid:15122742
- 18. Jarman B, Gault S, Alves B, Hider A, Dolan S, Cook A, et al. Explaining differences in English hospital death rates using routinely collected data. BMJ 1999; 318: 1515–1520. pmid:10356004
- 19. Pouw ME, Peelen LM, Lingsma HF, Pieter D, Steyerberg E, Kalkman CJ, et al. Hospital standardized mortality ratio: consequences of adjusting hospital mortality with indirect standardization. PLoS One 2013; 8: e59160. pmid:23593133
- 20. van Walraven C, Austin PC, Jennings A, Quan H, Forster AJ. A Modification of the Elixhauser Comorbidity Measures into a point system for hospital death using administrative data. Med Care 2009; 47: 626–633. pmid:19433995
- 21. Quail JM, Lix LM, Osman BA, Teare GF. Comparing comorbidity measures for predicting mortality and hospitalization in three population-based cohorts. BMC Health Services Research 2011; 11: 146. pmid:21663672
- 22. Southern DA, Quan H, Ghali WA. Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data. Med Care 2009; 42: 355–360.
- 23.
Ou HT. Performances of comorbidity measures in healthcare related behaviors and outcomes in Type 2 Diabetes. Dissertation for the degree of Doctor of Philosophy (Social and Administrative Sciences). University of Michigan. 2010.
- 24. Chu Y-T, Ng YY, Wu S-C. Comparison of different comorbidity measures for use with administrative data in predicting short and long-term mortality. BMC Health Services Research 2010; 10: 140. pmid:20507593
- 25. Stausberg J. Quality of coding in acute inpatient care. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2007; 50: 1039–1046 [in German]. pmid:17676418
- 26.
Drösler S, Romano P, Wei L. Health care quality indicators project: patient safety indicators report 2009. OECD Health Working Papers No. 47. Paris: OECD; 2009.
- 27. Simborg DW. DRG creep—A new hospital-acquired disease. N Engl J Med 1981; 304: 1602–1604. pmid:7015136
- 28. Gaertner T, Lehr F, Blum B, van Essen J. Consistency and reliability of MDK expertise examining the encoding in the German DRG system. Gesundheitswesen. 2015 Mar 11. [Epub ahead of print] [in German].
- 29. Huber H, Brambrink M, Funk R, Rieger M. Quality assurance in coding expertise of hospital cases in the German DRG system. Evaluation of inter-rater reliability in MDK expertise. Gesundheitswesen 2012; 74: 645–650 [in German]. pmid:22411547
- 30. Schneeweiss S, Wang PS, Avorn J, Glynn RJ. Improved comorbidity adjustment for predicting mortality in medicare populations. Health Serv Res 2003; 38: 1103–1120. pmid:12968819
- 31. Roos LL, Stranc L, James RC, Li J. Complications, comorbidities, and mortality: improving classification and prediction. Health Serv Res 1997; 32: 229–238. pmid:9180617
- 32. Ghaferi AA, Birkmeyer JD, Dimick JB. Variation in hospital mortality associated with inpatient surgery. N Engl J Med 2009; 361: 1368–1375. pmid:19797283