Background
Crisis resolution teams (CRTs) – otherwise referred to as home treatment or crisis assessment teams – provide rapid assessment in mental health crises and offer intensive home treatment as an alternative to acute admission if feasible [
1]. The introduction of CRTs, mandated by the National Health Service (NHS) Plan in 2000 [
2], has been an extensive change in the English mental health care system. In 2000, few areas had such teams, but at the time of writing, CRTs are available in every Trust in the country, and several thousand mental health professionals have migrated into them [
3]. UK government guidance [
1] recommended that CRTs should provide an easy access, rapid response, 24-hour service; should be multi-disciplinary and able to provide medical, psychological, and social interventions; and should help facilitate early discharge and adopt a ‘gatekeeping’ function of assessing all service users before admission to acute wards to ensure home treatment is presented as an alternative to admission wherever possible [
4]. When CRTs first became national policy, their evidence base was criticised as limited [
5,
6]. However, some positive findings have now been reported from naturalistic studies and a randomised controlled trial, suggesting that CRTs reduce inpatient admissions [
7‐
11] and healthcare costs [
12,
13] and increase service user satisfaction with acute care [
7,
10].
Despite indications of the potential effectiveness of CRTs, considerable reservations have emerged about their delivery in routine settings in the UK, most notably in recent reports by Mind [
14], the Schizophrenia Commission [
15], and the Care Quality Commission [
16]. Both CRT and ward managers still view a significant minority of hospital admissions as unnecessary [
17]. The impact on bed use appears to vary considerably between areas [
11,
18,
19], and the reductions in bed days tend to be less marked than those in admissions [
9,
11]. Rates of compulsory admissions have continued to rise in England since 2000 despite the national implementation of CRTs [
18‐
20], and a recent national audit raised concerns about the number of suicides by service users receiving CRT care [
21]. Service users and carers, whilst mainly positive about receiving care in their own homes, report important areas of dissatisfaction with CRTs [
14,
22,
23], especially regarding the continuity of care, the quality of relationships with staff, and a narrow range of support on offer, which sometimes focuses principally on medication and short-term symptom control.
The UK experience, and similar challenges in Norway, where CRTs are also mandated at the national level but with variable implementation [
24], illustrates that the national mandate, policy, and guidelines have been insufficient to ensure consistent and complete implementation of this complex intervention. A survey of CRTs in 2005/6 [
25] and another in 2011/12 [
26] confirmed considerable variation in CRT resources, organisation, and service delivery. One potential reason for inconsistencies in CRT implementation is that the CRT model and its theoretical basis were not highly specified at the outset [
4,
26], nor has a means of measuring fidelity to a model of good practice been developed and disseminated. Improving acute mental health services is a current UK policy priority [
27], yet available quality improvement resources are lacking for CRTs to promote best practice.
The US National Implementing Evidence-Based Practices Project (EBP) [
28,
29] offers a model for evaluating and promoting quality improvement in complex service-level mental health interventions. Integral elements of the EBP approach are service reviews using a fidelity measure that assesses how far services are achieving a model of good practice and utilisation of a resource kit to support implementation and quality improvement [
30]. Resource kits typically consist of guidance, training materials, and coaching and support for service managers and staff, designed to help services address areas where high model fidelity has not been achieved. Modelling of factors associated with successful implementation of evidence-based practices has helped to theoretically underpin the EBP implementation model [
31,
32]. Attention to workforce development and training may in itself be insufficient to achieve high fidelity implementation of complex mental health interventions: securing leadership support, organising workflow (service structures and organisation) to support implementation, and providing feedback to reinforce implementation successes are also required [
31]. Access to technical assistance that enables understanding of the specific requirements of a complex intervention can also facilitate implementation [
32]. The EBP programme has successfully developed fidelity scales and implementation resources for a range of service-level interventions including supported employment [
33] and assertive community treatment [
34]. Previous studies have found correlations between fidelity to an evidence-based practice and better client outcomes [
35,
36]. CRTs are comparable with models in the EBP project because trial evidence exists for their efficacy in the right conditions. A CRT fidelity scale was developed in an earlier part of this programme of research (The CORE Study) [
37]. This trial will test the effectiveness of the CORE CRT Service Improvement Programme in helping CRTs improve outcomes and achieve high model fidelity.
Methods/design
This cluster-randomised controlled trial is funded by the National Institute of Health Research (NIHR) and has received ethical approval from the Camden & Islington Research Ethics Committee (Ref: 14/LO/0107). It is registered on the ISRCTN registry (Ref: 47185233). The protocol reported in this paper corresponds to the current, ethically approved, version of the trial protocol: V3_25/06/15.
Preliminary work
The lack of a highly specified model for CRTs [
4,
26] led to the development of a fidelity scale by the CORE study team [
37]. Fidelity refers to adherence to the standards defining a specific practice; a fidelity scale is a quantitative measure assessing the degree of adherence. The fidelity scale was developed from qualitative interviews with stakeholders [
39], a systematic review of previous research [
40], and a national survey of CRT managers [
26]. The CORE CRT fidelity scale defines 39 fidelity criteria; each item is scored on a scale of 1–5, with 5 indicating excellent fidelity and 1 very low fidelity; and a total score is yielded that ranges from 39–195. The scale has been piloted and used to survey CRT model fidelity across 75 CRT teams in England in 1-day reviews by three trained external reviewers. Whilst completing the 75 team fidelity surveys, researchers collected best practice resources and case study examples that inform the development of the Service Improvement Programme, which is designed to help CRT services achieve high model fidelity and quality improvement.
Main study
The CORE Trial involves the collection of data from 25 CRTs at baseline and at the 1-year follow-up. The trial comprises four components: (1) quantitative evaluations of service user experience, (2) quantitative evaluations of service use, 3) quantitative evaluations of staff well-being, and 4) an accompanying qualitative and process evaluation. This protocol paper follows the SPIRIT recommendations [
41] for trial protocols. A copy of the SPIRIT checklist, detailing where each recommended element of protocol reporting is included in this protocol paper, is provided as Additional file
1.
Setting
Twenty-five CRTs will be recruited to the trial. CRTs will be selected from NHS Trusts in four regions of England (North London, South London and the South-east, the West of England, and the heart of England) to reflect a range of Trusts and urban and more rural areas. We excluded the CRT teams identified by our earlier CORE fidelity survey as already achieving good model fidelity (defined as having a mean item score of 4 or higher). Fifteen CRT teams were included in the intervention arm of the trial to allow a thorough investigation of the implementation of the intervention in a range of contexts. Ten teams were included in the control group, as this was considered sufficient to represent treatment as usual. The number of participants in each outcome is therefore greater in the intervention than the control arm, although it is equal within each cluster for the primary outcome.
Randomisation
The 25 teams will be randomised to either receive the Service Improvement Programme (n = 15) or the control (n = 10) after the baseline fidelity reviews have taken place for all participating teams within each NHS Trust. Randomisation of the CRTs will be stratified by the NHS Trust to ensure that within each Trust, some CRTs will receive the intervention and some will act as control teams, to address the potential confounding factor of Trusts’ macro-level management and service processes. Randomisation will be conducted by a Priment (a University College London Clinical Trials Unit) statistician who is independent of the study. Randomisations will be conducted for all participating teams within each NHS Trust at the same time, once baseline fidelity reviews have been completed at all sites within the Trust. The CORE trial has made no arrangements for blinding: both participating services and researchers will be aware of the team allocation status. Service user participants providing data for the trial’s primary outcome are not expected to be informed of their team allocation, and the NHS Informatics Teams providing anonymised patient service-use data will not be aware of the team allocation status.
Sample
The four components to this study are presented and described separately.
1) Service user experience
At each CRT, 15 service user participants will be recruited at baseline, and 15 service user participants will be recruited who were discharged in months 10–12 of the 12-month intervention, providing a total sample of N = 375 at each time point. At each service, we will screen and recruit consecutively discharged, eligible, and consenting service users until we reach our target of 15. Eligibility criteria for participants are the use of the CRT service for at least 7 days, ability to read and understand English, capacity to provide informed consent, and do not pose too high a risk to others to participate (including being interviewed on NHS premises or participating by phone, email, or online survey).
2) Service use
Anonymised service data for two cohorts of service users will be collected at two time points – baseline and follow-up: (1) a cohort of all service users admitted to the CRT during a 1-month period ending 6 months prior to the study baseline date and another cohort of all service users admitted during month 7 of the study intervention period at each Trust and (2) all service users admitted to the acute inpatient services during a 6-month period up to the study baseline date and during months 7–12 of the study intervention period at each Trust.
3) Staff well-being
At each CRT, all clinical staff will be invited to complete a set of questionnaires at baseline and outcome (months 10–12) time points to measure staff morale and psychological health.
4) Qualitative and process evaluation
Fidelity reviews
Fidelity reviews will involve individual interviews with CRT managers and a separate focus group with available CRT clinical staff. The CRT team will invite discharged service users (n = 6) and carers (n = 6) to take part in short phone interviews. Managers of other community teams (n = 3), who make referrals to the CRT and ward managers (n = 2) who work with the CRT, will be interviewed. Anonymised case notes of recently discharged service users (n = 10) and CRT routine records, policies, and protocols will be reviewed.
Facilitator monthly updates
The CRT service improvement facilitators (n = 7) will be interviewed each month throughout the 12-month intervention by a member of the research team who will record the implementation activities undertaken by each team (n = 15). Interviews will be conducted with the ten control team managers at 6 months and the end of the study period to check for possible contamination, i.e. take up of service improvement initiatives or shared learning from intervention teams within the same Trust.
Qualitative Evaluation
Six case study sites will be purposively selected following 12-month fidelity reviews to include teams in urban and more rural settings, teams starting from comparatively high and low baseline model fidelity, and teams where an improvement in the fidelity score was achieved and where it was not. Interviews with CRT manager and a separate focus group with up to ten CRT team members per team will be conducted at each case study site. We will seek to include staff representing a range of professional groups, levels of seniority and amount of CRT experience within each focus group. Interviews will also be completed with all CRT facilitators (n = 7) and their clinical supervisor (n = 1).
Sample size
A sample size calculation for the primary outcome measure (service user satisfaction measured using the Client Satisfaction Questionnaire [
38]) determined the size of the service user sample. A sample of 375 participants (225 from 15 CRTs that have implemented the service improvement intervention and 150 from ten CRTs that have not) will give 97 % power to detect half a standard deviation difference in mean satisfaction (3.5 points assuming a typical SD of 7.0), and 80 % power to detect a small difference of just over one-third of a standard deviation, allowing for moderately large within-team clustering (ICC = 0.05).
Procedures
The 1-year study intervention period will be the same for all teams within each participating NHS Trust, defined as starting from the date when the Trust’s CRT facilitator starts in post. Outcome data from Trust patient records will be collected for a 6-month period 6–12 months following the intervention start date. Outcome interviews with service users and staff, and the end-of-study CRT fidelity review, will be conducted between months 10 and 12 of the trial-intervention period.
1) Service user experience
Screening and recruitment
Clinical staff in participating CRTs will be asked to screen and identify consecutively discharged potential service user participants who meet the study’s inclusion criteria. Eligible service users will be approached close to the point of discharge by clinical staff from the CRT that supports them, who will explain the study briefly and ask if they are willing to be contacted by a researcher. A study researcher will then contact potential participants to explain what the study involves, answer any questions, and send a written study information sheet [
48]. The researcher will make contact again to check the participant has understood the information sheet and has continued capacity to consent. Consent to participate and completion of the questionnaire can be done in one of the following ways: at a face-to-face meeting, via post, over the telephone (consent will be audio-recorded and stored securely), or online via a link sent by researcher (using UCL’s secure “Opinio” system). A copy of the consent form is provided as Additional file
2.
The interview will take approximately 15 minutes to complete. This process will continue until 15 service users from each team have completed the questionnaire. Participating service users will be offered a gift of £10 in acknowledgement of their time, given as cash or as an e-voucher, according to the participant preference.
2) Service use
For baseline and study outcome data from patient records, a study researcher will contact the appropriate administrators or informatics team within each NHS Trust. The study researchers will provide a pro forma that specifies clearly the nature of the information and time periods for which data are required. Administrators will be asked to provide the required patient-level data in anonymised form.
3) Staff well-being
A study researcher will visit the CRT team to publicise the study and answer any questions the staff may have about their involvement. The study researcher will assign a study identification number to each staff member. A master document linking CRT staff names to ID numbers will be stored securely at the research study office. An email will then be sent to all CRT staff containing an invitation to participate in the study, a study information sheet, their individual ID number, and a link to the online structured questionnaire hosted on the UCL's secure network. Consent to participate in the study will be provided by staff through completing the questionnaire.
4) Qualitative and process evaluation
Qualitative interviews
At the end of the 12-month intervention, a study researcher will contact all local facilitators and the managers of the six case study CRTs directly to invite them to participate in an individual interview. Staff focus group participants will be identified initially through liaison with managers of case study CRTs. A study researcher will provide potential participants with written information about the study, and the focus group and will include information on how to contact the researchers with any questions about participating. Staff will be informed that participation is entirely voluntary. Written consent will be taken from participants before the focus groups begin. Focus groups will be facilitated by two researchers from the study team. Focus groups and individual interviews will be audio-recorded.
Process monitoring
All monthly monitoring data collected from progress update phone calls with local facilitators will be coded under the EBP implementation categories by a researcher each month. [
31]. Usage of the online resource kit will be monitored using Google analytics.
In addition, phone calls will be made to control team managers at 6 and 12 months: any reported impact of the Service Improvement Programme on their own team’s practice and any other major service improvement initiatives affecting CRTs taking place within NHS Trusts will be recorded.
Data management
Study researchers will develop and manage a secure database for all quantitative study data using SPSS software and will store electronic copies of focus group transcripts using Nvivo qualitative software [Nvivo9:
http://www.qsrinternational.com/product ] on the secure IT network at University College London. The study team will follow advice from Priment, a UCL Clinical Trials Unit, regarding development and maintenance of the study database.
Staff data will be entered by staff themselves, via the secure UCL online questionnaire. Service user data will be entered either by participants themselves via the secure UCL online questionnaire, or by researchers after a phone or face-to-face interview, or receipt of a paper copy of the questionnaire. All data will be transferred to the trial SPSS databases by study researchers. The study Chief Investigator will act as custodian of the data.
Study oversight
The study sponsors, the Camden and Islington NHS Foundation Trust, act as guarantors for the trial, including insurance and indemnity arrangements, and are responsible for overseeing and auditing trial conduct. The study is supported by the Priment Clinical Trials Unit at University College London. Management of the study is coordinated through a trial management group, consisting of the study chief investigator and trial manager, trial statisticians and health economist, senior investigators, and representatives of Priment. Any proposed changes to the trial protocol during the study will be agreed on by the trial management group and submitted for approval to the research ethics committee by the study team. Service user and staff participants providing outcomes data for the trial will all provide informed consent to take part, using ethically approved procedures. Independent advice to the study team and oversight of the study is provided by a trial steering committee, which is independent of the sponsor and will meet at least annually during the trial. The steering committee comprises senior academics, including a statistician and a health economist; clinicians with relevant experience in acute care and/or service improvement; and service user and carer representatives with expertise by experience. A data monitoring committee (DMC) is not planned for this team-level intervention, but the trial steering committee will advise if any role for a separate DMC is indicated during the trial. No interim analyses are planned, and no stopping criteria are pre-set. Any serious adverse events reported to the study team will be screened by the chair of the trial steering committee as an independent reviewer. Any adverse events assessed as study-related will be reported, with the trial steering committee chair’s recommendation, to the study sponsor and the research ethics committee. Annual study progress reports will be provided by the study team to the sponsor and the research ethics committee.
Research implications
The main trial results will provide preliminary evidence regarding whether the CORE CRT Service Improvement Programme is effective in increasing model fidelity and improving outcomes in CRTs. The study will also provide the following:
1.
Evidence regarding the validity of the CORE CRT fidelity scale. We will explore, through multilevel modelling, whether team fidelity scores relate to important service user and staff outcomes.
2.
Evidence regarding the feasibility of implementing the intervention from process data and barriers and facilitators to implementation from the qualitative evaluation, which can help understand any mechanisms of change and inform future implementation programmes.
Implementation represents the biggest challenge to translating research knowledge into patient benefit [
52]. CRTs exemplify this challenge: trial evidence suggests they can be effective in reducing admissions and enhancing service user experience, but observational studies and audits indicate the benefits of CRT implementation on a national scale may be smaller and less consistent than originally expected [
19,
20]. Means of supporting the effective implementation of the CRT model are therefore of high interest to service planners. Through testing a theoretically underpinned, clearly defined, extensive package of implementation resources for CRTs, our trial can provide evidence to address this important issue.
Trial status
The recruitment of service user and staff participants for outcome evaluation and the trial intervention are ongoing at the time of submission of this protocol for publication. The final report of the study to the funders is due for submission by April 2017.
Acknowledgements
Our colleague, Steve Onyett, died suddenly in September 2015. Steve contributed to the design of the CORE trial and led in the development of the trial intervention and supporting its implementation. His contribution to the study and his warmth and generosity as a colleague are greatly missed.
We are grateful for the help and support with the study given by staff, managers, service users, and their families from participating NHS Trusts:
Camden and Islington NHS Foundation Trust
West London Mental Health NHS Trust
North East London NHS Foundation Trust
Sussex Partnership NHS Foundation Trust
Avon and Wiltshire Mental Health Partnership NHS Trust
Barnet, Enfield and Haringey Mental Health NHS Trust
Northamptonshire Healthcare NHS Foundation Trust
Surrey and Borders Partnership NHS Foundation Trust
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
SJ, BLE, GB, RG, DO, GA, LM, RH, SP, OM, SO, CH, NG, KK, DL, FN, and SS all contributed to the trial design and BLE, KF, DL, SO, EJ, OM, KK, DO, and GB contributed to the development of the intervention. SJ led the study. GA and LM led the development of the statistical analysis plans. RH led the development of the health economics analysis plans. KF, DL, and BLE led the writing of this protocol paper. All authors contributed to, read, and approved the final manuscript.