Background
Quality networks aim to enhance standards of health care by engaging and supporting clinicians and managers to assess and improve the quality of the care they provide [
1]. By assessing quality of care against recommended standards and providing feedback on service quality to front-line staff, quality networks have much in common with accreditation schemes. However, in quality networks services do not work towards achieving accreditation for the care they provide. Rather, they receive suggestions and support for initiating changes to services and to keep front-line staff engaged in efforts to improve service quality. Over recent years the number of quality networks and accreditation programmes has substantially increased [
2]. Despite this, there is little evidence to judge how effective they are, and the impact they have on patient outcomes is largely unknown.
Data from observational studies of services that participate in quality networks show that they achieve higher standards of care over time [
3‐
5]. However, comparative data from services that do not take part in such networks have rarely been gathered. A greater amount of research has been conducted into the impact of accreditation programmes. A systematic review of the impact of accreditation programs found some evidence from observational studies that such programmes improve staff perceptions of service quality [
6]. Consistent findings about impact on patient outcomes were not found.
To date, two randomized controlled trials have been carried out to explore whether peer-led quality improvement programs enhance the quality of patient care [
7,
8]. The first study was conducted in acute hospitals in KwaZulu-Natal, South Africa. Over the course of two years, greater improvements in standards of care were seen in hospitals that were randomized to participation in an accreditation programme. However, differences in clinical records keeping, hospital sanitation and patient satisfaction with care were not found. While this study demonstrated that experimental studies of peer-led quality improvement initiatives are feasible, its limitations, including a small sample size and inconsistent implementation of the accreditation programme limits generalisability to other clinical settings. In addition, hospitals that were randomized to control treatment appear to have had limited exposure to other quality improvement initiatives. In Europe, North America and other industrialised economies health services are usually involved in a range of other quality improvement initiatives, such as local audits or statutory inspection. In this context, Roberts and colleagues [
8] conducted a randomized trial to explore the added value of participation in a peer-led quality improvement initiative in the United Kingdom. In this study, hospitals admitting patients with acute Chronic Obstructive Pulmonary Disease (COPD) were randomized to the intervention arm of the trial that received a reciprocal peer-review and a control arm that continued to undertake COPD service development within normal processes. A number of quality indicators drawn from national guidelines were used to measure service outcomes as well as four key areas of service provision. Participating services had to complete a self-assessment baseline pro-forma describing the service provision and attainment (met in full, partially met, not met at all) of the quality indicators. This document was used to direct discussions and record observations during the peer-review visit. After each peer-review visit, the intervention units received a final report and agreed action plans for service development. All services were asked to complete a change diary 12 months after all the peer-review visits were completed to provide information about major service changes occurring during the year that could be related to their involvement in the study or not. Follow-up data at 12 months on those aspects of care assessed at baseline were collected as part of the UK National COPD Audit. The results showed that the compliance of the services in the intervention group with the quality indicators assessed was only marginally higher than that shown by the control group. In contrast, qualitative data suggested many benefits of the peer-review in most intervention units and some control teams [
8]. One limitation of this study was that the data collected at baseline relied entirely on the services’ self-assessment. In addition, the team did not assess whether the peer review process had any impact on patients’ experience of their care.
The Royal College of Psychiatrists’ Centre for Quality Improvement runs a range of peer-led quality improvement programmes (see:
http://www.rcpsych.ac.uk/researchandtrainingunit/centreforqualityimprovement.aspx). The setting up of a new quality improvement network starts with the development of service standards that are based on recommendations made by professional bodies and organisations such as the National Institute for Health and Clinical Excellence [
9,
10]. Front-line clinicians and service users and carers are all involved in the development of these standards. Services that choose to participate in a network are sent materials necessary to conduct a self-review (where they indicate whether they believe that they meet or do not meet agreed standards of care). The self-review is followed by an independent assessment led by a multi-professional team of trained peers who work in similar clinical settings. Service users and carers also take part in these assessments [
1]. Discrepancies between the self-review and the peer review data are discussed and feedback on service quality is then provided to each service after each visit. Clinical teams are sent a final report summarising the services’ achievements and areas of improvement and are supported to develop an action plan aimed at making improvements to service quality. Participation in a quality network also offers access to online discussion groups, newsletters, workshops and an annual forum. These activities encourage services to share good practice and find solutions to the challenges they share.
Essentially, network membership entails committing to a culture of openness, sharing and enquiry through a supportive peer-led process with the common goal for all members to improve quality of care. All members are required to engage actively in all stages of the peer-review cycle (self-review, peer-review, attendance of annual forum) and expected to use the results of reviews to develop action plans to achieve year on year improvement. Members are also expected to share their results throughout their services as well as with key stakeholders, including health and local authorities, those making referrals to their services and local patient and carer groups.
Secure inpatient forensic mental health units are specialist services for people with mental health problems who have either committed a criminal offence or whose challenging behaviour requires a level of security that is higher than that provided in mainstream adult mental health services. They are called ‘secure’ because the freedom of people treated on these units is restricted by mental health legislation. Inpatient forensic mental health services aim to treat people’s mental health problems and ensure the safety of patients and the public by monitoring risk, preventing absconsions and providing support and supervision while on the unit and on agreed periods of leave. When the decision was made to set up a new peer-review network for low secure inpatient forensic units, we built in a randomized evaluation in which low secure units were randomly allocated to immediate or delayed participation in the quality network 12 months later. The low-secure network aims to help individual services identify areas where their practice falls below national standards and to share good practice aimed at improving the quality of care they provide. The standards used in the network cover key aspects of physical and relational security, the interventions and treatments patients are offered, the quality of the physical environment, training, supervision and support for staff the governance of services including the way that adverse incidents are reported and managed. We hypothesised that through helping teams focus on these areas of care, membership of the network would help them identity where these elements of care could be improved and that this in turn would impact on the health, well-being and experience of the patients treated on these units.
Aim of the study
The aim of the study is to assess the impact of membership of the quality improvement network for Low Secure Units on the quality of care that people receive.
Hypotheses
-
The primary hypothesis is that the quality of the physical environment and facilities in wards that participate in the network will be higher 12 months after randomization than in those wards that do not participate.
-
The secondary hypotheses are that, 12 months after randomization, wards that participate in the network will have:
ii)
Higher levels of patient satisfaction
iii)
Higher levels of patient mental well-being
iv)
Lower levels of staff burnout
v)
Higher levels of training and supervision for staff
Discussion
This study provides an opportunity to explore the impact of participation in a peer network on the quality of inpatient mental health services. By comparing the quality of the physical environment and facilities of units that do and do not take part in a network we will able to find out what, if any, benefit membership of a peer network delivers. In addition, the study will explore whether participation in a quality network has any impact on staff and patient outcomes. To our knowledge this is the first time that a randomized evaluation of participation in a peer-led network has been conducted that will examine patient and staff safety, patient experience and the clinical effectiveness of care that patients receive.
Quality networks are one of a broad range of initiatives that services may use to try to improve the quality of care they provide. All such initiatives have costs as well as potential benefits, both in terms of direct costs of delivering the programme and time spent by staff completing self and peer reviews. Strengths of this study are that it is adequately powered to examine clinically important differences in service quality and the wide geographical area that the units were recruited from. By integrating the trial into the development of a new quality improvement network we ensured that nearly all eligible services took part in the study.
Our primary outcome is designed to assess the quality of the physical environment and facilities on participating wards. This aspect of services is one which a previous observational study has suggested is amenable to change within the first year of participation in a quality improvement network [
19]. Limitations inherent in the study design are the relatively short follow-up period and changes in patients and staff at wards that may limit our ability to assess the impact of participation in the network on patient and staff outcomes. Staff involved in setting up the network were concerned about withholding membership from control wards. Wards felt that a delay of more than a year would be unacceptable. This means that we will only be able to examine the impact of the first year of membership of the network and we will not be able to examine any benefits associated with longer-term participation. Given turnover in staff and patients at study wards, we are not able to compare changes in staff burnout, or patient mental health and satisfaction with treatment at an individual level. Whilst we will be able to explore aggregate changes in these measures, it is possible that planned changes or random variations in intake of patients will limit our ability of the study to demonstrate the impact of peer-networks on these measures.
Nonetheless, we believe that this study provides a rare opportunity to examine the impact of a peer-led quality improvement initiative on the quality of inpatient mental health services. The findings will also add to our understanding of the impact of this widely used approach to improve the quality of health services.
Status of the trial
Recruitment of the study commenced in June 2012 and ended in July 2014. Seventy-five wards were recruited in total and the data collection for the follow-up assessment was competed in October 2015. Data management and cleaning is currently ongoing and the results will be published by the end of 2016. The results will also be summarised and made available to the members of the participating units on the Royal College of Psychiatrists’ website.
Acknowledgements
We would like to acknowledge all the services that took part in the study, the staff members and the patients that gave some of their time to provide the information needed to complete the study. We also wanted to thank the staff of the Quality Network for Forensic Mental Health Services at the Royal College of Psychiatrists’ College Centre for Quality Improvement for embedding this evaluation study within the network’s review timeline. Last but not least, thanks also to the members of the network’s advisory group for helping with the development of the study outcome measures and for generating helpful discussion on quality improvement in forensic mental health settings.