Implementing performance improvement in New Zealand emergency departments: the six hour time target policy national research project protocol

In May 2009, the New Zealand government announced a new policy aimed at improving the quality of Emergency Department (ED) care and whole hospital performance. The 'Shorter Stays in ED' target stipulates that "95% of patients will be admitted, discharged or transferred from an ED within six hours by July 2009" [1]. This target was supported by the National ED Advisory Group, made up of senior ED clinicians and managers [2]. They advised the policy initiative based on observational studies from overseas showing that overcrowding or long waits in ED were associated with poorer outcomes for patients such as increased mortality [3‐5]. Similar ED time targets have been recommended by health reformers in Australia [6] and follow the lead of the English National Health Service which introduced time targets for ED in 2001 [7]. In the UK, official monitoring indicated that these targets were successfully met at the national level by 2004 [8]. This contrasted with the lack of improvement in ED waiting times in Scotland, Wales and Northern Ireland over the same time period, where such targets were not introduced [9].

Nevertheless, there has been widespread debate about whether targets in general, and the English ED targets in particular 'do more harm than good' [10]. Advocates of a target approach claim that ED targets can act as an effective catalyst for quality improvement across the whole hospital, and even the wider health system. Taking a systems view of a hospital, the ED is only one part of the patient journey. The pathway for ED patients into in-patient beds is determined by bed availability, inpatient care delivery and discharge practices. These have been shown to improve through attention to quality initiatives [11]. Therefore, from a whole system perspective, using the ED waiting time as a performance measure has the potential to see improvements in parts of the health system beyond ED. This has been demonstrated via application of dynamic system model simulations [12, 13] and such simulations have become a cogent influence on policy makers.

However, instead of acting as a catalyst for stimulating and improving broader health system performance, the imposition of targets may have significant adverse consequences. Two types of adverse consequences are typically identified. The first is gaming; or reactive subversion, such as 'hitting the target but missing the point'. Examples are ambulances being made to wait outside the ED to delay arrival time, [14] or re-designating patients as short stay admissions at or around the target time to avoid 'breaching' the target [15, 16]. The second is substitution effects; the potential diversion of attention from other important clinical areas and possible distortion of clinical and management priorities. Organisations, in their efforts to meet targets, turn resources and attention away from other important dimensions of performance and quality, and in doing so, shift systemic problems to other parts of the system or organisation [17]. For example, the pressure of admission from ED to an already stretched hospital service with close to 100% bed occupancy may serve to precipitate inappropriate discharge of patients. It is also possible that pressure to meet time targets for patient discharge may exacerbate inequities of care due to clinicians implicit biases [18].

As such, it is possible that the introduction of targets may ultimately detract from health service quality of care. Relevant measures of quality and performance include mortality, hospital length of stay (LOS), readmission rates, and more specific, condition-related measures such as time to reperfusion for those with acute myocardial infarction [19]. Some of these outcomes have been associated with ED overcrowding, [5, 20, 21] however the associations are not consistent [22‐24] and there is little evidence to support the assumption that introduction of an explicit 'ED stay' performance measure will improve clinical markers of quality of care [25].

Whatever relationships there may be between the use of targets and broader clinical and hospital outcomes, these are almost certainly shaped by organisational factors. The literature on organisational studies has demonstrated that the application of performance targets drives organisational attention and resources towards achieving these targets. This has been illustrated in manufacturing, [26, 27] retail, [28] and education [29]. Relevant factors include the motivational role of the Chief Executive Officer, the structure of the senior management team and elements of organisational culture. However, it is not at all clear how targets affect organisational behaviour within the complex and complicated context of a health system. Much of the organisational studies literature has been based in contexts that are relatively less complex, more likely to have an agreed organisational purpose, and where the value system of employees is more aligned [30, 31].

The evidence regarding organisational responses to the English ED target does not yet add up to a coherent picture. A report commissioned by the UK Health Commission, identified a number of organisational factors contributing to delays in patient care, including the level of clinician involvement in ED and hospital management, levels of nurse absenteeism and lower proportion of non-salary expenditure [32]. This study, however, did not link ED performance with broader clinical outcomes. Another recent study sought to investigate the presence and extent of dysfunctional organisational response to the ED time target [17]. The authors undertook an aggregated analysis of the effects of the ED targets on English National Health Service trusts between 2002 and 2007. They concluded that there was no evidence of dysfunctional effects, and that if anything, the impact on quality was beneficial. However, this research has significant limitations. Firstly, only mortality and inpatient admission (itself a rather ambiguous indicator) were included in the study, and the results conflicted with other reports using the same dataset [33, 34]. Secondly, it is doubtful that an exclusively quantitative study of this type would be capable of detecting gaming and effort substitution. Gaming behaviour may be undetectable or statistically insignificant at the aggregate level because the nature and strategies of gaming most likely vary across health care organisations due to differences in organisational context.

The New Zealand health care system is divided into 20 regional District Health Boards (DHB), who manage an extensive system of publicly-owned hospitals that are directly subject to government policy priorities. This provides a context in which it is possible to investigate the effect of an explicit policy directive on the performance of EDs, hospitals and wider health systems.

The introduction of the ED target in New Zealand raises some fundamental questions which can be adequately answered only by undertaking a thorough investigation of implementation processes and impact on the quality of health care delivered. There is very little empirical work that examines how the performance of the wider hospital system is affected by the introduction of a performance target within ED. Both positive and negative consequences of time targets can be best detected by combining qualitative, case-study research into the behaviour and practices of specific hospitals with quantitative data on clinical outcomes and other dimensions of performance.

A priority for this research is to contribute to health equity in New Zealand Emergency Departments. Health inequalities in New Zealand are well documented. In particular, health inequalities by ethnicity have been described as "the most consistent and compelling of health inequalities in New Zealand" [35]. Importantly, evidence also suggests that changes in the political and economic environment, including setting of health targets, may have "unequal impact" and can increase disparities rather than reduce them [36].

Key results of the 2006/7 New Zealand Health Survey [37] describe current inequalities in ED use in New Zealand by ethnicity, gender, age and neighbourhood deprivation. We will therefore consider the impact on the Target via these variables, where data is adequate.

This research therefore aims to investigate the following questions:

This is a mixed methods study with three interdependent research streams. The streams employ a range of quantitative and qualitative methods to address a number of research questions. Each stream is conducted consecutively and concurrently over the study time frame and outputs from each inform the others. This material is united into a final, fourth stream to address the overall aims of the study (Figure 1).

Ethical approval for this study was granted by the New Zealand Multi-Region Ethics Committee (MEC 10/06/060) on 1st July 2010.

The Shorter Stays in Emergency Departments National Research Project team is a collaboration of researchers from the School of Population Health (Faculty of Medical and Health Sciences, University of Auckland) and the Emergency Department Research Team at the Auckland District Health Board. The lead investigators are Peter Jones and Linda Chalmers. The investigators are Shanthi Ameratunga, Toni Ashton, Peter Carswell, Elana Curtis, Papaarangi Reid, Joanna Stewart, Tim Tenbensel and Susan Wells. The research fellows are Alana Harper and James LeFevre, the data manager is Denish Kumar and the research administrator is Jayshree Ramesh-Sukha