Background
Healthcare institutions increasingly emphasize improved outcomes and performance and focus efforts to improve quality of care and patient safety while lowering costs [
1,
2]. They cannot determine whether their efforts are satisfactory without tracking outcomes and comparing with peers, so benchmarking is widely applied within healthcare organizations to improve their clinical performance and management of operations [
3‐
5]. Hospitals have also started Clinical Documentation Improvement (CDI) programs to improve documentation quality; these programs ensure better patient outcomes, optimized data quality and accurate reimbursement [
6,
7].
Vizient is the largest member-driven health care performance improvement company in the U.S. and it provides services to about 95% of the nation’s academic medical centers and more than 50% of the nation’s acute care health systems [
8]. Using data collected by Vizient, members can benchmark many key performance indicators such as Case Mix Index (CMI), Length of Stay (LOS), Expected Risk of Mortality (EROM), and Severity of Illness (SOI) [
9‐
12]. SOI describes the disease severity in hospitalized patients and measures the physical effects of disease on a patient. It is a powerful tool to track resource consumption and to track patient outcomes. In addition, SOI is also closely related to cost, revenue, and profit [
13]. The admission and discharge SOI are created by the 3 M coding algorithm. The admission SOI is important for hospitals to measure the health status and severity of illness of a patient when he/she is admitted. Hospitals can use admission SOI to estimate resource distribution. Discharge SOI can be used for prospective payment and risk adjustment in quality reporting. Benchmarking SOI helps hospitals better evaluate their clinical performance and distribution of resources by comparing them to peers. The SOI levels presented in Vizient data come from the All Patients Refined Diagnosis Related Groups (APR DRG) classification system developed by 3M [
14,
15] Each APR DRG has four categorical severity levels: minor, moderate, major and extreme. The SOI subclasses are related to the APR DRG grouper that is updated annually by 3M. However, there are several limitations using the current SOI system. Firstly, cross-category comparison of disease severity is less meaningful. The same SOI level from different APR DRG does not mean the same level of disease burden. Secondly, it is hard to compare severity among institutions because of each institution’s unique patient mix. Lastly, it is difficult to track yearly clinical performance using longitudinal data, given that the grouper is updated annually. Therefore, we sought to develop a novel measure of SOI that is grouper independent.
To find the appropriate predictors, we compared several models targeting Elixhauser comorbidities, body systems for chronic condition indicators, and complication or comorbidity (CC) or major complication or comorbidity indicators (MCC) [
16‐
19]. The Elixhauser comorbidities are a comprehensive set of measures to identify different pre-existing conditions based on secondary diagnoses listed in hospital administrative data. The system was developed by Anne Elixhauser using all adult, nonmaternal inpatients from acute care hospitals in California in 1992 [
20]. It includes 30 comorbidity measures that are associated with considerable increases in LOS, hospital charges, and mortality. The comorbidities are usually not directly related to the primary reason for the inpatient stay, but they have a possible effect on outcomes used to assess the quality of care. The Agency for Healthcare Research and Quality (AHRQ) has created a powerful Healthcare Cost and Utilization Project (HCUP) tool called Elixhauser Comorbidity Software Refined for ICD-10-CM, which can be applied to ICD-10 diagnosis codes to create a comorbidity profile [
16,
17]. AHRQ also created another tool to categorize ICD-10-CM codes into 18 body systems [
19]. Body systems allow us to correct for regional differences in patient mix and comorbidity-driven DRG modifiers, so that we can compare the intensity of severity that is independent on types of diseases. These tools provide the potential indicators for predicting the severity of illness. Another valuable resource is the list of all of the ICD-10 codes that are defined as either a CC or MCC diagnoses, released by Center for Medicare & Medicaid Services (CMS) [
21]. We combined CC/MCC levels with body systems and created a 3-level indicator for each body system, indicating whether the body system has any CC or MCC diagnosis code. In this study, the Elixhauser comorbidities and 18 body systems with CC/MCC indicators were used as predictors to evaluate case severity. Instead of predicting four categorized SOI levels, we aimed to develop a model to better predict high and low illness severity that is independent of APR DRG assignment. Additionally, the probabilities generated from the model would be used as a quantitative measurement of SOI.
Discussion
In this study, we sought to develop a consistent measure of severity that is independent from the APR DRG grouper. To achieve this goal, we mapped diagnosis codes to Elixhauser comorbidities, CCI body systems, and CC/MCC indicators to create predictors and then applied orthogonal polynomial regression models to predict case severity of illness. This method can be used as an alternative way to estimate patients’ severity. We compared three models by evaluating their performance through ROC analysis, prediction accuracy, and the counts of predictors. Eventually, the body system model with CC/MCC indicators was considered the best because of higher AUC, higher accuracy rate and fewer variables. The probabilities calculated from this model were named “J_Scores,'' which serve as a measure of severity. Then, we compared the severity obtained from our model with APR DRG SOI levels acquired from the Vizient database and found that the proportions of high severity cases were similar, indicating that our model had great value in benchmarking (Supplemental Fig.
1). Plus, the large sample size of patients from 21 facilities ensured reliable results.
Although J_Scores exhibit great values in evaluating severity, the methodology differs from APR DRG SOI. The SOI subclasses developed from 3 M are determined from 3 phases with 18 steps in total after the APRDRG is assigned to a patient, incorporating the secondary diagnoses, the impact of principal diagnosis, age, OR procedure, non-OR procedures, and multiple OR procedures [
31]. Each APR DRG has four subclasses of SOI. However, the severity scores generated from our model are based on affected body systems and their complication and comorbidity (CC) levels, which are not specified for APR DRG grouper. One caveat of our model is that the optimistic and promising predictive ability for severity is based on adult inpatient cases, and it needs to be validated in pediatric inpatients because pediatric cases do not have the same patterns of comorbidities as adults. Additionally, the model needs to be refreshed or re-evaluated annually to make sure it incorporates the latest version of the diagnosis codes from CMS and the updates on the body system assignment from AHRQ.
In our institution, we applied J_Scores to Vizient data to benchmark severity of illness and integrated J_Scores in CDI analysis. It was found that the average J_Score_POA of cases selected for review by CDS were much higher than the J_Score_POA of cases not selected, suggesting that our CDS were targeting cases with higher severity for review. We also envision the utility of J_Scores in audit processes, given that effective post-coding audits include a review of high-risk MS-DRGs, SOI, and risk of mortality.
Conclusions
In this study, we developed a novel method to measure SOI using diagnoses with body system and CC/MCC indicators. It is independent from APR DRG and can be used to better evaluate or compare severity in patients from different disease categories. The results demonstrated that J_Scores generated from the body system model offer reliable predictability of patients’ illness severity on admission and at discharge. Overall, this new scoring system provides a useful tool for hospitals to benchmark SOI, assess CDI programs and direct case review to improve clinical performance and quality.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit
http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (
http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.