IMPlementation of A Relatives’ Toolkit (IMPART study): an iterative case study to identify key factors impacting on the implementation of a web-based supported self-management intervention for relatives of people with psychosis or bipolar experiences in a National Health Service: a study protocol

Across the world, there is a growing interest in the application of self-management approaches as a clinical and cost-effective way to empower people to manage long-term health conditions. A lot of money is spent on developing and testing new health technologies, but there is little understanding of how they can be successfully implemented in routine clinical practice [2]. This is particularly true for digital health interventions, the development of which has seen a huge explosion in recent years, but the majority of which have yet to be successfully integrated into routine clinical practice.

Governing bodies around the world recognise the benefits and cost savings that can be achieved through harnessing digital technology within clinical settings [3, 4]. Not only can such technology offer a possible solution to address the increasing demand for ever more sophisticated and available healthcare in wealthier nations, but also an invaluable opportunity to improve healthcare delivery in developing nations, where it is estimated that more people have access to mobile phones than to water or sanitation [5, 6]. However, digital interventions present particular challenges to healthcare providers, evident by expensive information technology (IT) failures, most notably the UK National Programme for IT in the National Health Service (NHS) which cost approximately 10 billion pounds and failed to deliver its key aims [7‐10], and more recently, global privacy concerns linked to data breaches [11]. In developing countries, additional challenges include unreliable web and mobile access, lack of technical expertise and infrastructure, and illiteracy [5]. Some of these challenges are linked to the technology itself, but the greatest challenges are in transforming the ways in which people think about delivery of healthcare and developing new ways of working within healthcare systems [12]. Whilst it is perhaps inevitable that such transformation will be difficult and costly, we need to learn from each attempt to better understand the factors that underlie successful implementation.

There is growing evidence for short-term benefits of web-based psychological treatments for depression and anxiety disorders compared with waitlist (WL) controls [13, 14]. Online interventions are being rapidly developed for psychosis and bipolar disorder, where data supports their feasibility and acceptability [15‐18]. Web-based platforms are particularly suited to delivering standardised psychoeducation and facilitating peer support through online forums. Online support may be particularly useful for relatives of people with chronic health conditions [19], due to the flexibility of use and empathy and support from being linked to other carers sharing similar concerns and difficulties [20].

Although some web-based platforms and mobile Apps are designed to be used as standalone interventions by the general public, many are designed as part of broader packages of clinical care and often require staff support to engage people in their use. Thus, implementation of digital interventions is complex, often requiring behaviour change at many levels of the healthcare system. Whilst the importance of organisational factors such as management structures and adequate resources is well recognised, there is a need for more in-depth understanding of how both clinical staff and service users make sense of any new intervention, appraise its worth, engage with it, and monitor its ongoing usefulness [21]. Ideally, these considerations inform the development and design of the intervention from the very outset, but even where this occurs, it may still be necessary to develop of an implementation intervention to facilitate use of the digital intervention within the specific context in which it is being delivered [22]. Attempts to deliver web-based mental health interventions such as Beating the Blues at scale in the UK [23], and as part of routine care in the US [24], have highlighted great difficulties in getting patients to use them or staff to integrate them into practice.

In this study, we aim to understand the key factors impacting on the implementation of the Relatives’ Education And Coping Toolkit (REACT) in UK mental health services and use this understanding to develop an implementation plan. By drawing on implementation theory, we will highlight what can be learnt that is of value to implementation of other digital interventions in different contexts. REACT is a supported self-management toolkit which offers easily accessible evidence-based information and support for relatives of people with psychosis or bipolar disorder. Relatives of people with severe mental health problems provide the vast majority of care, saving the NHS an estimated £1.24bn per year in the UK [25], but this is associated with high levels of distress in relatives [26, 27], significant practical, financial, and emotional burden [28]; stigma; worry; shame and guilt [29]; trauma [30]; and loss [31, 32]. REACT has been shown to be effective in reducing distress and improving perceived support and ability to cope in relatives of people with psychosis in early intervention services [33]. Originally offered in paper form and supported by clinical staff on telephone or email, REACT has since been developed to be available online, broadened to include relatives of people with bipolar disorder, and supported online by REACT supporters who may be either peer relatives or clinicians [34]. REACT is designed to meet the NICE (The National Institute for Health and Care Excellence) recommended quality standard in relation to the provision of carer education and support but is not intended to replace more intensive face to face structured family intervention.

There are several models, frameworks, and theories we could have used to guide our work and ensure our findings are interpreted within a theoretical framework that can support their generalisability. We chose to work with normalisation process theory (NPT) as this facilitates generation of specific hypotheses about the process by which a complex healthcare intervention is implemented, embedded, and integrated (or not); can be tested empirically; and has previously been applied in eHealth settings [21, 35, 36].

NPT began as a model (NPM) of the factors that promote or inhibit the routine work of embedding a new health technology into practice. The key constructs identified were interactional workability, relational integration, skill-set workability, and contextual integration. The model has since been developed into a theory which includes the NPM as constituting “collective action” and adds concepts of “coherence” (how actors make sense of a set of practices), “cognitive participation” (the means by which they participate in them), and “reflexive monitoring” (how these practice are then appraised) [37].

Our research employs a theory-driven multiple case study design [43] using a mixed methods approach integrating quantitative assessments of outcome (delivery and use of REACT) and qualitative assessments of mechanisms including observation, document analysis, and in-depth interviews. Our case is defined as the EIP service in NHS Mental Health Trusts, which are public sector organisations in England that provide early intervention support to people with early signs of psychosis and/or other severe mental health problems (including bipolar disorder) in a particular geographical locality. Depending on the size of the NHS Trust, there are often further embedded units which are locality teams delivering care to distinct geographically defined areas. Data will also be analysed within stakeholder groups, so we can understand the implementation factors impacting on both staff delivering the intervention and relatives receiving it. This will allow us to understand the process of implementation of REACT within a real-world setting and to identify the causal factors impacting on this process. We will first outline a programme theory about the factors we consider will influence successful implementation of REACT. This theory will be based on NPT and facilitated by tools available on the NPT website which can guide this process (www.​normalizationpro​cess.​org/​). We will generate hypotheses about the mechanisms that will lead to successful outcomes and then test and refine these hypotheses using a series of case studies in which data collected in early cases is used to inform development of our implementation plan and tested in subsequent cases. The first version of our implementation plan (IPv1) will be based on our programme theory. Detailed case study data from two of the participating Trusts (wave 1) will then be used to develop a revised implementation plan (IPv2), and a refinement of our implementation hypotheses. These hypotheses will then be tested using the revised IPv2 in 2 further Trusts (wave 2). Data collection and analysis across these two Trusts will then focus on further hypothesis testing, leading to a further iteration of the implementation plan (IPv3). This will be introduced to the final two Trusts (wave 3), and data from these Trusts will inform the final implementation plan (IPv4).

Case studies can provide rich data and are particularly useful when trying to understand the implementation of a complex intervention in a real-world setting in which the process or context cannot be controlled. REACT is a “complex intervention” [44] because it depends on the actions of individuals, across different contexts, and adapting their behaviour over time. It also produces multiple outcomes which need to be understood. Implementation is made more complex by the context in which the intervention is situated, which is dynamic and includes competing demands on the system. A mixed methods approach including quantitative assessments of outcome (use and impact of REACT) and qualitative assessments of mechanisms including behavioural observation, document analysis, and in-depth interviews is therefore required to attempt to understand this complexity. We have also designed the study to have extensive input from stakeholder groups at each of the Trusts to ensure that the implementation plan is collaboratively designed.

We will use this design flexibly, adapting our approach in response to activity on the ground. Whilst maintaining a focus on specific Trusts at each time point, we will maintain good links with all Trusts and listen to what is happening at each. This ensures that we can respond to significant relevant events at all Trusts as they occur and ensure we are able to collect data where and when it is most informative. This design is shown in Fig. 1 below.

EIP teams represent a highly accessible and universal point of access to mental health services for people experiencing first episode psychosis through a range of referral routes including NHS, voluntary sector, and self-referral. They support service users and their relatives. EIP teams were set up in many countries across the world in response to evidence of a “critical period” during the first 3 years of illness during which intervention is thought to be particularly effective in preventing longer term disability [45]. Most teams work with people who have developed symptoms of psychotic illness for the first time, for up to 3 years following first contact (exact criteria vary between services). In the UK, EIP teams generally consist of a mix of psychiatrists, psychologists, care coordinators (social workers, community psychiatric nurses, occupational therapists), and support workers, and aim to deliver services that are consistent with the NICE guidelines for good clinical care [46]. Despite a worldwide recognition of the need for parity of esteem for mental and physical health [47], EIP services have not been immune from the funding challenges faced by all mental health services in the UK [48]. NICE guidelines state that relatives of people with psychosis and bipolar disorder should be provided with information and support and offered structured family intervention to enhance family coping and communication [49, 50]. However, a recent national audit of EIP teams in England showed poor implementation: only 50% of relatives receiving a carer-focussed education and support programme, only 31% offered structured family intervention, and only 12% receiving it [51]. To facilitate implementation of NICE guidelines, the recent “guidance to support the introduction of access and waiting times standards for mental health services in 2015/2016” [52, 53] commits NHS Trusts to ensuring more than 50% of patients experiencing their first episode of psychosis will access NICE concordant care within 2 weeks of referral. Offering REACT should help services to meet the NICE quality standard (NICE QS80) recommendation to offer carers access to an education and support programme, which may therefore facilitate Trust participation and engagement with the IMPART study.

The study will be conducted across two regions (North and South of England), with three Trusts participating from each. A formal power calculation is not appropriate for this design, but careful purposive sampling of Trusts will provide sufficient variability to ensure a widely applicable implementation plan. Participating Trusts have been chosen to reflect a range of population size, urbanity, and ethnicity, increasing theoretical generalisability of findings across NHS Trusts. Staff members within the EIP services will be able to invite any relatives of people within this service to use REACT. A sample of staff and relatives in each Trust will be interviewed to understand their experiences of REACT. We will use role within the Trust and levels of engagement with REACT to select participants who can provide a broad range of views.

Data collection will require a mixed methods approach. All methods have limitations, but used together, they strengthen the validity of the findings [54]. At each site, the following data will be collected to address each of the study objectives.

Data to identify the critical factors impacting on implementation of REACT (objective 2) will be primarily qualitative and will consist of:

The selection of data sources will be informed by the specific hypotheses being tested, i.e. we will seek out data which is best placed to help us test our hypotheses.

Quantitative measures will be used to assess the uptake and use of REACT by NHS EIS teams and relatives (objective 1), investigate the impact of REACT on self-reported relatives outcomes at each site (objective 4), and investigate the resources (and costs) needed for successful implementation of REACT (objective 3). Specifically, we will assess the number of REACT accounts created for relatives to access the REACT site and the level of use of each module using web analytic statistics. We will collect basic demographic information about all relatives using REACT so we can better understand who uses it. Relatives who choose to visit the site will also be invited to take part in the collection of outcome data. Following online consent, they will complete questionnaires at baseline and again after 12 and 24 weeks to assess levels of distress (General Health Questionnaire-28; GHQ [56], wellbeing (Carer Wellbeing and Support Questionnaire [57]), quality of life (EQ-5D-5L [58]), and eHealth literacy (eHEALS [59]), and collect information about their caring role. These measures have been shown to be acceptable and sensitive to change [33]. Those who decline to complete the outcome measures can still receive the REACT intervention and contribute anonymously to the implementation data. No individual personal data will be available for relatives who declined or were not offered the toolkit. All data from relatives will be downloaded to a database held at the Liverpool CTRC.

Resources needed to deliver REACT will be identified using proformas designed specifically for this study. We will generate a list of the likely resources involved as part of Phase 1 and design measures to record this at each site. The proformas will be flexible to accommodate any additional resources identified during data collection but will include staff time, technology costs, and training or promotional materials.

Consistent with the case series design, data will first be analysed and presented within each Trust, before attempting to analyse similarities and differences between Trusts and how these can be explained. Where Trusts are divided into distinct clinical teams, these will be treated as embedded units, and where appropriate, similarities and differences between units will be investigated.

Qualitative data will be analysed using framework analysis, a pragmatic approach useful for applied research in which data is synthesised from different sources [60]. Our initial framework will be derived from both an initial process of familiarisation with the data and informed by our programme theory. We will use the framework flexibly, recognising that emergent data may inform further development of the framework. Reliability will be enhanced by use of regular data review meetings and a regularly updated codebook used across all Trusts. We will seek alternative interpretations of the data to maximise the validity of our findings. This analysis will inform our final report of the factors impacting on implementation.

In addition to this detailed data analysis, we will need to be able to continuously make sense of the data as it is collected, in order to develop our iterative implementation plans within a timeframe that suits the project and the needs of the NHS Trusts. This requires a more agile sense making at a “bigger picture” level to develop, implement, and evaluate each iteration of the implementation plan. As far as possible, this will be linked to specific data sources (interviews, workshops, documents, and meeting observations), but will not require a full thematic analysis or coding of transcribed data. Regular multi-disciplinary research team meetings and SRGs within the Trusts will ensure this remains grounded in the data.

Quantitative data will be analysed via descriptive statistics to summarise use of REACT at each Trust and overall, by staff and relatives. Descriptive statistics supported by graphical representation will be used to compare relatives’ outcomes over time (baseline, 12 and 24 weeks) within and across Trusts. Across the six Trusts, measures of use of the REACT toolkit will be explored as mediators of change over time.

The specific hypotheses generated in phase 1 will be tested by integrating both the qualitative and quantitative data.

Relevant stakeholders are involved in IMPART at all stages. REACT was developed with extensive input from relatives and clinicians over many years [33, 61]. The team that developed the IMPART study design includes a relative of someone with a long history of psychosis and Director of the McPin Foundation that works to promote service users and relatives in mental health research. We have employed relatives to input to key aspects of the study including taking part in our monthly project management group (PMG), co-facilitating SRGs within each Trust, analysing qualitative data within each Trust, and developing each iteration of the implementation plan. NHS staff members are also key stakeholders and sit on the PMG, SRGs, and data analysis groups. We believe that involving relatives and staff will improve the delivery of the project, the experience of relatives in the research process, and how effectively the findings are disseminated.

A Study Steering Committee (SSC) will provide oversight on behalf of the Project Sponsor (Lancaster University) and Project Funder (National Institute of Health Research) and ensure that the project is compliant with the Department of Health’s Research Governance Framework for Health and Social Care and the Guidelines for Good Clinical Practice. Members of the SSC include experts in implementation research, mental health, and service user/relatives from mental health services.