Background
The Australian rural health service context
Australia is a vast continent with a population of approximately 22 million people of which 3.5 million are spread across 1500 small rural and remote communities and 7.5 million square kilometres. Isolated rural communities throughout Australia are experiencing demographic ageing, an increasing burden of chronic diseases[
1], and de-population as families move to larger cities. In addition, many of these rural communities are struggling to maintain viable and comprehensive health care services due to lack of service infrastructure, transport difficulties[
2], and health worker shortages due to high levels of staff turnover and difficulties in recruiting new health workers[
3]. In many cases rural communities are either foregoing care, travelling to larger regional centres or depending on irregular visiting services. As a consequence, Australia's rural and remote communities face significant health service disadvantages compared with metropolitan regions[
4].
The vast distances separating small communities throughout rural Australia provide enormous challenges for authorities responsible for servicing population health needs as there are conflicts between ensuring operational efficiency and cost-minimisation, whilst maintaining effective and equitable delivery of accessible health services. Traditional urban health service models are proving to be unsustainable. Undoubtedly, there is no "one-size-fits-all" solution to meeting the diverse health needs of rural Australian residents and the range of service models needed is likely to vary between communities. Hence it is necessary to investigate models of health service delivery to ensure equitable access to care and reduce the health differential between rural and metropolitan people[
5].
Research has shown that a primary health care approach yields the best health outcomes in situations characterised by limited resources[
6]. As a result, the need for sustainable comprehensive primary health care, characterised by multi-disciplinary team approaches, is urgent in rural communities. In response, a wide variety of innovative health care models have evolved and been trialled in rural and remote areas[
3]. However, whilst research has identified the requirements for sustainable rural and remote primary health care services[
7], few rigorous longitudinal evaluations have been conducted to systematise them over time; assess their transferability to other regions[
8]; assess how health services continually evolve to address ongoing changes in the external environment; monitor the effect of comprehensive primary health care on health service utilisation behaviour and health literacy of their communities[
9]; or better understand how or when primary health care services can redress the poorer rural health status.
Given these gaps in knowledge it is vital to understand which rural health services 'work well, where and why' and are sustainable over the long-term in order to inform rural health service policy, assist with planning sustainable health services in other rural communities[
10] and contribute to the equitable delivery of health care services that are likely to bring about improved health outcomes.
This paper describes the study protocol of a comprehensive longitudinal evaluation of a successful primary health care service located in a small rural Australian community to assess its performance, sustainability over time, responsiveness to changing community needs and health system requirements, and its impact on community health behaviours, health outcomes and community viability.
Methods/Design
Setting: The Elmore Primary Health Service (EPHS)
The EPHS is located at Elmore in central Victoria (Australia) 170 kilometres from the capital city, Melbourne. While the population in Elmore in 2006 was only 693[
11], the EPHS provides care to nearly 3,000 patients from surrounding districts up to 125 kilometres from Elmore. The EPHS is a single-entry point, comprehensive primary health care model formed in 2001 by a partnership between the private medical practice and the local, publicly funded, community health services after the closure of the local hospital and loss of the local doctor in 1994. The model combines health care, community coordination and outreach services, is financed by public and private funding and is delivered by a multidisciplinary team of, amongst others, doctors, nurses, physiotherapists, psychologists and podiatrists[
12]. This service has grown in contrast to the rural national trend of service closures[
13] and can be an exemplar for other communities to emulate.
Evaluation framework: domains, components and indicators
The longitudinal evaluation framework evolved from Donabedian's quality of care paradigm linking structure, process and outcome[
14] and a conceptual framework for primary health care performance assessment developed originally by Sibthorpe[
15], as reported in another publication[
16]. This evaluation aims to examine the health service over a six year period in terms of:
-
Health service performance characteristics;
-
The sustainability of the health service organisation and function; and
-
Quality of care that the service provides across the health promotion, treatment and rehabilitation spectrum.
A key aspect of the evaluation was the identification and development of unambiguous sentinel indicators for each component of these key domains that can be reliably and validly operationalised so that data can be routinely collected and the health service monitored over time. The recent health systems reform process being undertaken in Australia has highlighted the difficulties in developing appropriate and valid accountability and performance benchmarks[
17]. Despite potential indicators being easily identified on the conceptual level, it can be extremely difficult to operationalise them in forms for which data can be routinely and unobtrusively collected whilst being beneficial to health outcomes and health services. Hence our indicator selection was guided by significant health services research conducted by the Canadian Institute for Health Information[
18]; the Australian National Health Performance Committee[
19]; the Australian National Health and Hospital Reform Commission[
17]; the Royal Australian College of General Practitioners[
20]; the Australian Institute of Health and Welfare [
21]; and Wakerman et al[
22].
Drawing on this evidence, sentinel indicators for assessing the structural domains: health service performance (Table
1); sustainability (Table
2); and quality of care (Table
3), were developed in accord with the evaluation framework. As described in each table, each domain is divided into components, for which sentinel indicators were selected based on their technical merits and validity as identified in the research literature and their applicability in the health service setting[
23] - that is, the data can be routinely collected, validated, and extracted reliably from the data sources identified for this study (health service record audits, community surveys, interviews and focus group discussions).
Table 1
Health Service Performance Domain
Accessible
| The importance of geographical proximity to services; and timely and affordable access to routine and emergency care. | • Distance to nearest GP | Survey | |
| | • Distance to usual GP | Survey | |
| | • Operating hours | Audit | |
| | • After hours services | Survey | |
| | • Availability of emergency care | Survey/Audit | |
| | • Availability of bulk-billing | Audit | |
Appropriate
| To assess the comprehensiveness of services in dealing with a whole of health approach to health care. | • Availability of disease prevention, health promotion, early identification, sub-acute, acute and rehabilitation services. | Audit | AIHW [ 21] CIHI [ 18] NHHRC [ 17] RACGP [ 20] |
| | • Availability of female GPs | Audit | |
Effective
| To assess extent to which a service's disease prevention interventions are achieving the desired results within an expected timeframe. | • Immunisation coverage | Audit | |
| | • Cervical cancer screening coverage | Audit | |
| | • Use of chronic disease registries for diabetes mellitus, asthma and hypertension for timely planning and disease management. | Audit | Campbell S [ 26]; CIHI [ 18] |
Responsive
| To assess the extent to which respectful care is provided that promotes dignity, privacy, safety and community empowerment. | • Service response to cultural and other specific needs of people utilising the service | Interviews | |
| | • Community input into service planning | Interviews Focus group | |
Continuous
| The importance of providing uninterrupted, coordinated care across programs, providers, and organisations over time | • Choice of GP or nurse | Survey | |
| | • Age-specific health assessments | Audit | |
| | • 45 year old health check | | |
| | • Use of integrated care plans for diabetes, asthma and depression | Audit | |
| | • Use of recall and reminder systems | Audit | |
Efficient
| The importance of achieving desired results with the most cost-effective use of service resources. | • Electronic billing system | Audit | |
| | • Electronic medical records | Audit | |
Table 2
Sustainability Domain
Workforce
| The importance of having a workforce that is appropriate in number, volume and distribution and is responsive to emerging needs. | • Staff profile- Numbers and FTE. • Staff length of stay • Succession planning • CPD activities | Audit Interview Audit | AIHW [ 21] Wakerman[ 22] RACGP [ 20] |
Linkages
| Efficient and effective co-ordination between providers and between services is essential for continuity of care and service sustainability. | • Centralised electronic medical records • Care integrated with external agencies and mainstream programs | Audit/Interview | Wakerman et al [ 22]; CIHI[ 18]; |
Infrastructure
| Infrastructure and ICT needs to be appropriate to the service, its catchment population and monitoring and reporting requirements. | • Uptake of Information and Communication Technology (ICT) | Audit | CIHI [ 18]; Wakerman et al [ 22] |
Funding
| Financing and provider remuneration should be appropriate, sustainable and clearly identified within program budgets to maximise service efficiencies and adequate to meet identified community health needs. | • Funding sources: public, private, other • Service providers' remuneration methods | Audit Audit | Wakerman et al [ 22] CIHI [ 18] |
Governance, management and leadership
| Good governance and leadership have been identified to be integral to service sustainability | • Governance structure and processes and a risk management plan in relation to service sustainability need to be clearly defined, implemented and reviewed. | Interview/documentation | RACGP [ 20], Wakerman et al [ 22] |
| | • Level of accreditation | Audit | |
Table 3
Quality of Care Domain
Cervical cancer screening | Cervical smear tests improve the early detection and treatment of cervical cancer and improve survival and quality of life. | Cervical cancer screening coverage | Audit | |
Immunisation | Immunisation is a very important public health measure that can prevent the spread of common infectious diseases that cause significant morbidity and mortality. | Immunisation coverage: | Audit | |
| | • Children | | |
| | • Older adults (65+years) | | |
| | • Aboriginal and Torres Strait Islanders | | |
Secondary prevention
| | | | |
Recording of modifiable risk factors in medical records | Many common chronic diseases are preventable if the risk factors are identified and managed so that there can be improved health status and reduced health inequalities and need for health care. | • Smoking status | Audit | AIHW [ 21] Broemeling [ 10] |
| | • Body Mass Index | | |
| | • Alcohol use | | |
| | • Blood pressure | | |
Treatment goals and outcomes
| | | | |
Safety | Minimizing or eliminating inappropriate prescribing improves quality of care and health outcomes. | Safety - Risk management plan and use of medication alerts | Audit Audit | |
Diabetes mellitus | The percentage of patients with diabetes mellitus for whom the ideal treatment goal of HbA1c* < 7% is met. | Management of diabetes mellitus (HbA1c readings) | Audit | |
To evaluate the impact of the EPHS on health service utilisation, community viability and satisfaction in Elmore and its hinterland, sentinel indicators include: community participation within health service planning; community experience of and satisfaction with health services; indicators of population composition and growth, employment trends, and health service multiplier effects in the local economy[
24].
Data sources
The quantitative data for indicators of health service performance, sustainability and quality are being obtained by an annual comprehensive health service record audit and biennial community surveys. These quantitative data will be analysed using descriptive statistics that monitor trends over the period of the study.
The comprehensive health service audit is examining medical record data, billing and financial data, human resources records, registry data (e.g. Australian Childhood Immunisation Register and the Victorian Cervical Cytology Register) and government reports to the practice (e.g. Practice Incentive Payments and Service Incentive Payments). All data are being extracted by an EPHS practice nurse and provided in a de-identified and aggregated format to the external university research team to ensure utmost privacy and confidentiality of patient and health service records.
Further quantitative data are to be obtained through a biennial community survey that is to be delivered to all residents in the catchment area and is designed to obtain information in relation to health service utilisation, satisfaction and need, in addition to health risk behaviours.
The qualitative data that explores sustainability of the service and the impact of the EPHS on the local community are being obtained by in-depth interviews with key stakeholders and document analysis. These data will be transcribed and thematically analysed and triangulated with the quantitative data for validation and clarification of the data. As new understandings emerge through this iterative and reflective process, new indicators will be developed or existing indicators may be modified.
Knowledge translation
Engagement with the local community is essential for this study as the current health service grew from community activism in collaboration with the local doctor and regional community health service. A detailed communication strategy, including community forums and articles in the local newspaper, will ensure that all stakeholders and community residents are kept informed of the purpose and conduct of the evaluation study. The conduct of the research and dissemination of findings will be guided by a Reference Group that will be comprised of representatives from the local community (leaders, businesses and community groups), regional health authorities, and general practice support agencies (e.g. Divisions of General Practice and general practice education and training services). The research team will provide information to the community through plain language summaries, oral presentations and presence at key community events.
Ethics
The study has obtained ethics approvals from Monash University Human Research Ethics Committee (CF08/0419- 2008000176; CF08/0238 - 2008000089; CF08/2434 - 2008001256; CF10/2540 - 20100001423).
Discussion
Ongoing monitoring and evaluation of successful rural health service models is needed to provide sound empirical evidence of what makes a primary health care service sustainable and effective in a rapidly changing health policy environment in order to provide an evidence base that can inform health service policy makers and other rural health services.
This iterative evaluation framework and methodology aims to ensure the ongoing monitoring of service activity and health outcomes associated with the service that allows researchers, health service providers and administrators to assess the extent to which health service objectives have been met; the factors that have helped or hindered achievements; what worked or did not work well and why; what aspects of the service could be improved and how; what benefits have been realised and for whom; the level of community satisfaction with the service; and the impact of a health service on community viability.
This evaluation focuses on health service performance and capacity rather than focussing on how well a particular disease is treated. This raises challenges in identifying appropriate indicators as primary health care organisation and function is complex, dynamic and composed of both measurable and unmeasurable elements[
23] that are subject to many external influences that can impede performance, sustainability and quality of care. The scope and breadth of ideal indicators for the components of each domain in the research literature is more extensive than those selected for this study. The selection of indicators was limited principally by the types of data items that are stored in paper records and electronic databases that are reliable, valid and readily extracted.
This study also provides an opportunity to assess, over time, the effect of locally available health promotion, preventive and early intervention programs on health literacy and health status of a small rural community. As it is possible to relate the EPHS catchment to a geographically circumscribed hinterland, this evaluation may examine the associations between sustainable local health services and community viability and prosperity through examining community satisfaction, growth, employment, and multiplier effect indicators. Hence, building on previous studies that highlighted the significance of rural hospitals and health services to the local economy and fortune of rural communities [
25], the findings from this study may assist health authorities and other small rural communities to benchmark what services are appropriate and successful.
The danger in not undertaking such comprehensive longitudinal research to identify "what works well, where and why" is the risk of widening health differential between urban and rural communities; increasing costs of health care treatment to individuals and to government; and reduced implementation of best practice care equitably to all communities.
The need to provide equitable access to quality primary health services for Australia's rural communities is essential and urgent. However, the evidence base for rural health service design remains sparse. This longitudinal evaluation will contribute significant knowledge to how a primary health service can be evaluated and play a role in reducing the Australian rural-metropolitan health disparity.
Acknowledgements
This study is being funded by Australian Rotary Health for the period 2008 - 2010 and the Australian Government Department of Health and Ageing for the period 2011-2013. The funding bodies had no role in the conception or development of the study design, the writing of the manuscript or the decision to submit it for publication. The authors wish to acknowledge the contributions of Karly Smith, Karen Riley and Judith Jones in the development of the study design and ethics applications.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (
http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
RT, JSH and LK conceived and developed the design of the study, obtained the ethics applications and drafted the manuscript. PB obtained the ethics application and contributed to drafting and revising the manuscript. AA and KT contributed to the conception and design of the study and revising the manuscript. All authors read and approved the final manuscript.