Background
The sustainability of Australia’s current health system and level of service it provides is an increasing concern for federal and state governments. Multiple factors are involved, such as increasing availability of, and demand for, advanced technology services, an ageing population (more older people surviving, but with the chronic and multi-morbid diseases of ageing impacting on their independence, and quality of life), rapid advances in technology, and the ongoing issues meeting supply of and demand for healthcare providers. It is projected that under the current system, health care expenditure will increase to between 12 and 15% of the gross domestic product over the next 30 years [
1].
Generally underpinning the health care system are pressures to reduce costs, increase access and affordability of services, and provide greater accountability. There is also an increasing recognition worldwide of the need to examine how healthcare providers practice, and justify their performance and productivity. Measuring and monitoring aspects of the services provided by healthcare practitioners has therefore become a major concern not just in Australia but in many other countries [
2]. Performance evaluation
`seeks to monitor, evaluate and communicate the extent to which various aspects of the health system meet their key objectives’[
3]
. Performance is therefore an important indicator of how well a healthcare system is progressing towards its goals, and helps identify strengths and weaknesses to improve future performance [
4]. There is international evidence to suggest that organisations which do not integrate ongoing performance evaluation into their system tend to experience lower than expected performance improvements, as well as higher dissatisfaction and turnover of staff [
5].
Allied health services have been increasingly highlighted over the last five years as essential primary, sub-acute and tertiary services which could contribute significantly more to Australia’s healthcare system than they are currently doing [
6]. Allied health is an umbrella term used to describe a range of health disciplines, other than medicine and nursing, which provide therapy, organisational and scientific services [
6]. Commonly included under this umbrella are disciplines such as physiotherapy, occupational therapy, podiatry, speech pathology, social work, dietetics and nutrition, psychology, audiology and psychology. Although there have been attempts to define allied health [
7],[
8], there remains a lack of an internationally recognised definition because of the range and complexity of services delivered by the disciplines listed under the allied health umbrella. The lack of definitive definition for allied health precludes a comprehensive understanding of allied health quality service issues [
9].
The design of an effective performance evaluation strategy is fundamental to aligning allied health organisation’s operations with its strategic direction. It involves an ongoing cyclical process of information gathering, analysis and action at different levels-the workforce, consumers of care, and organisation in which the services are provided. However, there is no agreed approach to measuring or monitoring allied health service performance. This appears to be, in part, due to the
diversity of disciplines which fall under the allied health umbrella, the variability in roles and tasks these disciplines undertake, and lack of standard data items, data collection processes, and dedicated support systems to capture the range of services that allied health provides[
6]. Therefore, it is clear that there is no `one size’ fits all approach that can be used to measure allied health service performance. It also highlights the potential challenges and barriers associated with performance evaluation of allied health practitioners, specifically in terms of selection of performance measures, data collection, and implementation of an effective performance evaluation strategy.
The aim of this review was to examine the literature on performance evaluation in healthcare to assist in the establishment of a framework that can guide the measurement and evaluation of allied health clinical service performance. This review determined the core elements of a performance evaluation system, tools for evaluating performance, and barriers and challenges to the implementation of performance evaluation.
Discussion
Performance evaluation is an integral part of health care. Its primary aim is to measure the quality of health services with the ultimate goal of improving health outcomes. Measurement and evaluation of allied health performance and quality of services is in its infancy and the complexity of the services they provide contributes to the challenges associated with the process. In allied health, the individual disciplines have different purposes, ways of operating, stakeholders, outcomes and quality measures. As such, there is no one-size-fits-all approach for performance evaluation that can be recommended to all allied health care settings. However, the literature suggests that there are core elements involved in performance evaluation which include prioritising clinical areas for measurement, setting goals, selecting performance measures, identifying sources of feedback, undertaking performance measurement, and reporting the results to relevant stakeholders. The literature describes performance evaluation as multi-dimensional, requiring information or data from more than one perspective to provide a rich assessment of performance. A range of tools or instruments is available to capture various perspectives and gather a comprehensive picture of health care quality. This review, while it is primarily targeted to allied health services, generates findings that are broad and appear to be generalisable to a wide range of healthcare settings (e.g. medical, nursing, allied health), both locally and internationally. The premise that performance evaluation should be context dependent and designed to meet the unique requirements of the health care system holds true and remains valid, regardless of the healthcare setting being evaluated. Therefore, the findings of this review are not necessarily unique to allied health but may also be used to inform performance evaluation in the wider health system.
Performance evaluation is reported for different levels of the health care system, ranging from individual practitioners to geographical (e.g. regional or state) locations. Performance evaluation of an individual practitioner can promote ownership of the quality of care provided to clients [
43]. However, differences in performance between practitioners are often the result of
`random fluctuations rather than real differences in quality of care[
41].
’ On the other hand, by examining a state level performance, statistical problems, which are common in individual level assessment, are not an issue. At this higher level, however, practitioners do not feel a sense of ownership for their performance and are therefore not likely to be motivated to change or improve their practice [
41]. Marshall and Davies [
41] suggest a mid-level assessment, which involves small functional groups of health professionals, ranging from individual hospitals to groups of practices [
41]. In allied health, for example, an evaluation of the performance of podiatrists in foot screening for diabetic patients in a specific hospital is more ideal than evaluation of podiatry practice across a range of clinical conditions.
The selection of clinical area for evaluation is often dependent on what data or information is readily available, and as such the evaluation process starts by determining what data are accessible within the practice or organization [
35],[
41]. This is then followed by the identification of goals, which align with the available performance data-an approach that seeks to minimise the collection of further data. While this would be cost-effective, it violates the basic principle of having to establish goals prior to the development of the evaluation system or process. If performance evaluation is to be meaningful, the clinical area for performance evaluation should be carefully selected and be based on explicit criteria [
25],[
36]. Loeb [
35] claims that a more rational and appropriate approach would be to define the evaluation goals relevant to the clinical area, and then determine whether reliable data exist to support such goals [
35]. Additional data collection can then be applied when required and if it outweighs the time and costs associated with the process [
35]. It is always important to obtain the commitment of the chief executive and management team, as this is critical to the successful implementation of performance evaluation systems [
45]. The management plays a key role in performance evaluation as they will articulate the system or organisation’s vision of quality, ensure that there is an infrastructure and systematic approach, and make resources available to support the process [
45].
The identification and selection of appropriate performance measures (i.e. whether to use outcome or process measures, or both) is one of the most challenging activities for those who undertake performance evaluation. Mant [
40] argued that in instances where health services have a major impact on outcome, use of outcome measures as performance indicators is appropriate, provided that the data collected can be interpreted reliably [
40]. Conversely, in situations where factors such as lifestyle, co-morbidities, socio-economic circumstances rather than health care play a major role in health outcomes, process measures are preferred [
19],[
40]. This does not mean, however, that outcome data should not be collected, just that it should be collected within context. In other words, what seems to be the best solution is to combine process and outcome measures which are tailored to local circumstances and priorities [
39]. The identification of specific outcome and process indicators are then based on standards of care, or in the absence of scientifically-based evidence, determined by an expert panel. Performance evaluation typically involves a combination of measures, which are comprehensible, clear, valid, reliable, reproducible, discriminative and easy to use. Grimmer et al. [
16] proposed quality measures relevant to allied health therapy services which capture the elements of `what allied health does’, `how allied health does it’, and `what happens [
16]’.
The data or information required to measure process and outcomes could be retrieved from various sources including clinical or medical records, administrative data and patient reports. There is also a range of tools available that can be used to monitor performance [
17],[
20],[
23],[
24],[
27]-[
29],[
53]. The analysis of data and reporting of results should then lead to the recognition of good performance, improvement of poor performance and modification of the performance evaluation system if required. There is no point in conducting a performance evaluation if the results are not followed through. There should be a clear plan of action that needs to be agreed upon by relevant stakeholders in order for performance evaluation to be meaningful and worthwhile. Improvement to the performance evaluation system and attention to barriers and challenges can then facilitate its effective and sustainable uptake by allied health care practitioners and organisations.
As with any other studies, this review has a number of limitations which should be considered when interpreting the results. First, there are limitations to the search and it is possible that articles could have been missed as the search strategy did not include terms which referred to `quality of healthcare’ which was considered in the literature as an important component of performance evaluation. Second, the key concepts or domains identified in this review may have been influenced by the perspectives of the reviewers and it is possible that if a different perspective was obtained, a different set of concepts or domains would have been found; a validation study may be required to confirm the findings. Despite these limitations, the findings presented in this review provide valuable insights to clinicians, managers and health service researchers that can assist with the development of a broad framework for undertaking performance evaluation.
Competing interests
This review was commissioned by the Allied and Scientific Health Office, Department of Health, South Australia to inform the development of a model for performance evaluation in allied health.