Background
Outcome measures for quality of care are increasingly used to monitor and compare hospital performance, with the aim to identify areas for improvement. Three outcome measures that are commonly used in various countries to evaluate quality of care in hospitals are in-hospital mortality, readmissions, and long length of stay (LOS) [
1]. However, these outcomes are interrelated which will affect the interpretation of hospital outcomes. For example, patients who die in hospital cannot be readmitted, so that in theory hospitals may have low readmission rates due to relatively high mortality. Such mechanisms need to be understood for outcome measures to be able to reflect quality of care.
A previous study among Medicare patients hospitalized for acute myocardial infarction, heart failure or pneumonia did not find an association for mortality and readmission, except a weak association for heart failure patients [
2]. However, this study only examined hospital-level and not patient-level associations. Both types of associations need to be considered to evaluate whether interventions to reduce mortality may raise readmission rates or whether the two measures have shared underlying processes.
Furthermore, to obtain a comprehensive picture of quality –especially for between-hospital comparisons - it is attractive to jointly report outcome measures as e.g. mortality may be very low in some patient groups but readmissions are common. In addition, combining multiple outcome measures has the advantage of an increased number of events per hospital, resulting in more reliable estimates (lower statistical uncertainty) and thereby more reliable comparisons of hospitals [
3].
In this study we aim to 1) disentangle the relationship between mortality, readmission and long LOS both at patient and hospital level, and 2) to develop a measure to jointly report the three outcomes for a more comprehensive, unambiguous and more reliable estimate of hospital-specific quality of care to be used within a hospital over time and between-hospital comparisons.
Methods
Data
We used data from the Global Comparators Project in which hospitals from various countries collaborate and share their routinely collected administrative admission data. Participating hospitals were large academic medical units, likely to be fairly comparable with respect to their (complex) patient population. Diagnoses were combined into Clinical Classification Software (CCS) groups, procedures into groups representing major surgical specialties and comorbidities defined for the various coding systems as described elsewhere [
1]. For the present study, data from 26 hospitals in six countries (USA, Netherlands, UK, Italy, Belgium and Australia) were included who agreed their data to be used for research.
Within the Global Comparators Project, clinicians and other professionals work in Global Outcomes Accelerated Learning (GOAL) groups to use the observed variation in outcomes to initiate inquiries to identify best practices that participants may implement at their own institutions. Therefore, in addition to studying all patients, we focused on 3 specific GOAL areas: patients with stroke, heart failure or colorectal surgery as their principal diagnosis or procedure during admission. We expected these clinical areas to differ in having relatively high mortality rates (e.g. stroke), or that readmission would be a more relevant quality indicator (e.g. heart failure). All patients discharged in the years 2007 to 2012 were included in the analysis.
Definition of variables
We considered three outcomes: mortality, readmission, and long LOS. Mortality was defined as death in hospital during the index admission. Readmission was defined as an unplanned (emergency) readmission to the same hospital within 30 days after discharge. Long LOS was defined as a LOS greater than the 75th percentile for the specific diagnosis or procedure group (upper-quartile LOS). In a sensitivity analysis, long LOS was defined as greater than the 90th percentile LOS (upper-decile LOS) to assess whether this affected the results.
Statistical analysis
For case-mix adjustment, separate logistic regression models were developed for each of the 259 diagnosis and 32 procedure groups for each of the three outcome measures, using the inbuilt backward elimination procedure in SAS with all case-mix variables included and retaining variables with
p < 0.1. Case-mix variables were used as described previously [
1]: age-group, sex, method of admission (planned / unplanned), transferred in from other hospital, urgent admission in previous month, Elixhauser comorbidity score, year, diagnosis (sub)group or procedure group (for analysis of all patients). Statistical interactions between age and Elixhauser comorbidity score, and between method of admission and transfer, were included as candidates, based on a priori beliefs [
1]. To aid model convergence, age groups with fewer than 10 events were iteratively combined with the immediately older group [
4]. This resulted in expected probabilities per outcome for each patient. At a hospital level, these probabilities were summed to obtain the expected number for a particular outcome. By dividing the observed number by the expected number and multiplying by 100, standardized ratios of mortality, readmission, and long LOS per hospital were calculated.
At a patient level, we first assessed the correlation between outcomes using logistic regression models with mortality and readmission as dependent variables and upper quartile LOS as independent variable, both unadjusted and adjusted for center (as fixed effect) and case-mix. The resulting odds ratios indicate the increase in odds of mortality or readmission for patients in the upper quartile LOS. At a hospital level, we assessed the correlation between standardized mortality, readmission (survivors) and long LOS (survivors) using Pearson correlation coefficients.
We then created a composite outcome measure in which the different combinations of outcomes were ordered from best to worst outcome: 1) alive, no long LOS, no readmission, 2) alive, long LOS, no readmission, 3) alive, no long LOS, readmission, 4) alive, long LOS, readmission, 5) death. This ordering was based on previous research showing that adverse outcomes occurring during admission, thereby prolonging LOS, did not affect patients’ evaluation of quality of care [
5]. Adverse outcomes that occurred after discharge, often resulting in readmission, negatively affected patients’ evaluation of quality of care. This motivates our ordering with a readmission being worse than long LOS. The ordering was presented at a meeting of about 100 experts, clinicians and CEOs involved in the Global Comparators Project, for agreement. The composite outcome measure with 5 levels was analyzed with ordinal logistic regression with the case-mix variables and hospital as a fixed effect to estimate a coefficient per hospital, representing the standardized effect of hospitals on the composite outcome. This coefficient was then used to calculate the standardized rate, by calculating the average of all hospitals coefficients and then calculating the difference of each hospital coefficient compared with that average. Exponentiating this difference will give an odds ratio which can be interpreted as higher or lower than the average. We assessed the correlation between the standardized rates of the individual versus the composite outcome.
Finally, we assessed whether this composite measure would enable us to better discriminate between hospitals in terms of their apparent performance. We thereby evaluated the reliability of ranking the hospitals (the rankability) [
3]. This measure has been proposed to quantify the signal (i.e. differences in outcome) versus noise in hospital rankings. Rankability consists of two elements: 1) The magnitude of the between-hospital differences, defined by tau [
2], the variance of the random hospital effects, and 2) the uncertainty in the individual hospital estimates, defined as the median sigma, the squared standard error of the fixed effect hospital estimates. The rankability is a percentage and can be interpreted as the part of the total between-hospital differences that is due to ‘true’ differences in outcome (rather than being due to statistical uncertainty). It is thus a characteristics of the group of hospitals we aim to compare, not of each single hospital. A logistic regression model was fitted with hospital included as a random effect to estimate tau [
2]. To estimate the second element, the same regression model was fitted with hospital included as a fixed effect. This was done both for the individual and the composite outcome.
Discussion
In this study, using a large international database, we have explored the interrelations between three common outcomes and derived a new ordinal composite measure. We found that hospital mortality and LOS rates were positively correlated at patient and hospital level. High readmission rates did not correlate with either mortality or long LOS rates. Our composite measure provides a rank-ordered view of the three outcomes and has comparable or better rankability than the individual outcomes, indicating that hospital comparisons with this composite measure are more reliable and stable.
Our results are consistent with the results from Krumholz et al. showing no association between hospital readmission and mortality rates [
1]. Whereas they found a weak negative association for heart failure, the association in the present study was not significant among heart failure patients. Mortality and readmission rates may reflect different processes of care [
6]. Early intervention and coordination of care in a hospital may be particularly important for mortality and length of stay, whereas the outpatient clinic and patient education may be factors influencing chances of readmission. At a patient level our results are consistent with previous studies with respect to an increased mortality and readmission risk in heart failure patients with longer length of stay [
7] Contrary to previous studies, we examined both patient- and hospital level associations. These were particularly relevant for interpreting outcomes in stroke patients. It was shown that the hospitals with many patients dying were the same hospitals where the surviving patients had longer LOS. The same correlation was shown at both patient- and hospital-level for the other patient groups. So our results, together with the findings from Krumholz et al., suggest rather consistent patterns. Further, our results confirm that correlations between outcomes may differ between hospital and patient level [
8], which argues against post-hoc combination different indicators at hospital level.
The proposed composite measure has substantive and statistical advantages. It combines multiple outcomes and therefore gives a more comprehensive view of quality of care. It incorporates the interrelations between the three outcomes, which prevents ‘gaming’ the single outcomes e.g. when hospitals receive incentives or penalties when individual outcomes are unfavorable. On the other hand, when hospitals use the composite measures to monitor their own performance, assessment of the constituent outcomes is always needed to identify areas for improvement, as with every composite measure. This adds to previous approaches to create composite measures e.g. combining outcomes with structure and process measures [
9] [
10], and that multiple outcomes are used, and that these are ordered [
11].
The statistical advantage is that the composite outcome is ordinal and thus is more sensitive to between hospital differences than a dichotomous outcome. It contains more information; e.g. not only whether patients survived but also whether they were not readmitted. This explains why the rankability of the composite outcome was generally higher than of the single outcome. The rankability is a function of the differences between hospitals and the uncertainty in the estimates of the outcome per hospital. The latter is lower with the composite outcome, compared with single outcomes. The variation between hospitals might be smaller. In our example the variation in LOS was larger than in the composite measure, which explains that the rankability of LOS was somewhat higher. The variation in mortality was even larger. But as mortality is less frequent, the uncertainty in the estimates per hospital is large, and rankability is lower. Given a ‘true’ between-hospital variation, the rankability of composite ordinal outcome will always be higher compared with a dichotomous outcome.
The rankabilty in our study was high for the combined data but lower for some outcomes in the different diagnosis groups. Nevertheless rankability is substantially higher than in previous studies, showing the value of a large international database. Previous studies hardly ever reported rankability above 50% [
3] [
12] [
13] [
14] indicating that numbers of patients and events are typically just too low to reliably compare hospitals based on outcome. In the composite outcome, the most prevalent outcome, in this case LOS, will implicitly be weighted more. We performed sensitivity analysis with the upper decile LOS rather than the upper quartile LOS so that event rates were more equal to e.g. readmission rates. Similar results were found, although rankabilty was on average lower. Another possibility would be to define per diagnosis when LOS is considered too long and use that as a cut-off. Another possibility would be to put an explicit weight on the different outcomes. However, there is no evidence on how much more e.g. mortality should be weighted above readmissions after a long LOS, and in practice it will be difficult to determine explicit weights that appropriately represent the preferences of all different stakeholders.
The best category of this composite, an event-free hospital admission, obviously is what patients aim for. We also showed that within our collaboration as a whole, the percentage of patients with an event-free admission has increased over time particularly due to decreased mortality and shorter LOS. In addition, the variation between hospitals decreased over time. This is noteworthy as not all hospitals are faced with reimbursement penalties based on e.g. readmission rates, which may have caused variation to increase rather than to decrease over time. Part of the explanation may be the exchange of best practices within a collaboration among professionals, which occurred independently of these reimbursement policies.
Our study has some limitations, mostly due to the use of administrative data. Severely ill patients have higher chances of mortality, readmission and long LOS, and some hospitals may treat more of these severely ill patients than other hospitals. Our adjustment for severity of illness may have been insufficient, as suggested by others [
15]. Part of the correlations between the outcomes that we observe might actually represent insufficient adjustment. With perfect adjustment the correlations are likely to be smaller. The same holds for using in-hospital mortality rates instead of 30-day mortality, which is known to affect hospital standardized mortality ratios [
16]. If hospitals discharge patients early and they die outside of the hospital, this would result in shorter LOS, and lower mortality rates. Not counting these post-discharge deaths might thus have resulted in an overestimation of the relation between long LOS and mortality at a hospital level. We were only able to count readmissions to the same hospital, which will lead to underestimation of the readmission rate. Further, we did not explicitly study between-country differences while these may explain some of the between-center differences. But this was beyond the scope of the study.