Adverse event rates as measures of hospital performance

Abstract

Objectives

Adverse event or complication rates are increasingly advocated as measures of hospital quality and performance. Objective of this study is to analyse patient-complexity adjusted adverse events rates to compare the performance of hospitals in Victoria, Australia. We use a unique hospital dataset that routinely records adverse events which arise during the admission. We identify hospitals with below or above average performance in comparison to their peers, and show for which types of hospitals risk adjusting makes biggest difference.

Methods

We estimate adverse event rates for 87,790 elective and 43,771 emergency episodes in 34 public hospitals over the financial year 2005/06 with a complementary log–log model, using patient level administrative hospital data and controlling for patient complexity with a range of covariates.

Results

Teaching hospitals have average risk-adjusted adverse event rates of 24.3% for elective and 19.7% for emergency surgical patients. Suburban and rural hospitals have lower rates of 17.4% and 17%, and 16.1% and 15.7%, respectively. Selected non-teaching hospitals have relatively high rates, in particular hospitals in rural and socially disadvantaged areas. Risk adjustment makes a significant difference to most hospitals.

Conclusion

We find comparably high adverse events rates for surgical patients in Australian hospitals, possibly because our data allow identification of a larger number of adverse events than data used in previous studies. There are marked variations in adverse event rates across hospitals in Victoria, even after risk adjusting. We discuss how policy makers could improve quality of care in Australian hospitals.

Introduction

Adverse events (AEs) are incidents in which harm resulted to a person receiving health care, generally resulting in additional treatment, prolonged hospital stay, disability at the time of discharge, or death [1], [2]. In recent years, innovative ways of identifying AEs in routine administrative hospital data have been developed [3], [4], [5], [6], [7], including for example the Patient Safety Indicators of the Agency for Healthcare Research and Quality [8]. The accessibility, low costs and representativeness of administrative data may make it possible to use AE rates as indicators of hospital quality and performance, supplementing conventionally used measures such as mortality or readmission rates [9], [10], [11], [12].

One of the main stumbling blocks for using AE rates as performance indicators is the difficulty of distinguishing complications which were already present on admission from ones acquired during the hospital stay [13], [14], [15], [16], [17]. If hospital-acquired conditions cannot be uniquely differentiated from community-acquired conditions, hospitals may be held accountable for poor outcomes which are beyond their control. Therefore, the use of administrative hospital data is usually limited to identification of a narrow range of adverse events which unambiguously constitute medical errors, such as medication errors or instruments left in the body after surgery. However, many of the more common adverse events, such as urinary tract infections or pressure ulcers, can both be acquired in the community or in hospital, and are therefore only used with great caution as indicators of quality, if they are used at all [18], [19], [20]. Our study is one of the first to compare the performance of hospitals based on Australian administrative hospital data which allow identification of a wide range of hospital-acquired conditions. Australian data are among the few hospital datasets in the world which flag complications arising during the patients’ stay in hospital [21]. This allows to unambiguously attribute AEs to the treating hospitals, a necessary condition if they are to be used as measures of performance.

We examine adverse event rates for surgical inpatients in 34 public hospitals in the state of Victoria, Australia, using patient level administrative hospital data for the financial year 2005/06. An important step in using AE rates as measures of performance is to separately identify the hospitals’ management effort in improving quality of care from systematic differences in patients’ medical complexity (casemix) across hospitals. We carefully adjust for casemix, following the literature which commonly adjusts mortality rates for casemix if they are analysed and compared across hospitals [see for example 22]. We use fixed effects complementary log–log models to represent the probability of AEs as a function of observable patient and episode characteristics, and hospital fixed effects. Our research has three objectives. First, we identify hospitals with markedly higher or lower AE rates than their peers, conditioning on the complexity of the patients they treat. This allows Australian policy makers to conduct further and targeted investigation into hospitals which diverge markedly from average performance. Second, we compare AE rates for groups of hospitals with particular characteristics. This provides some interesting clues as to why some hospitals may have higher or lower AE rates. Third, we evaluate how important it is to risk adjust observed AE rates, and for which types of hospitals risk adjusting makes the biggest difference. This gives an indication of the extent of misinformation when reporting unadjusted adverse event rates, as is sometimes done in the public media.

Our study supplements current, mostly clinical and qualitative research on AEs [23]. Usually, a team of medical experts analyse patient records retrospectively to judge whether an AE has occurred, and whether it could have been prevented. Due to the subjective nature of this process, record reviews are said to have only modest reliability in identifying AEs [24], [25]. Because they are expensive and time consuming, record reviews usually focus on one or two hospitals and/or particular patient groups [26] and are unsuited to compare performance of all hospitals in a country or region. This requires a quantitative approach such as ours which is based on the population of all hospitals and readily available data.