The feasibility of implementing recovery, psychosocial and pharmacological interventions for psychosis: comparison study

This is the first study to assess the implementation feasibility of interventions recommended in the NICE guidelines for people with psychosis, thus offering an approach which can inform both guideline development processes and routine implementation of interventions. We identify five limitations. First, not all recently published studies were included. The evidence for psychosocial and recovery interventions was updated up to the publication date of the NICE guidelines, which was February 2014. This is one reason for the small number of RCT reports of recovery interventions reviewed. Moreover, in contrast to the psychosocial and recovery interventions, the evidence on pharmacological interventions was not updated in the 2014 NICE guidelines. Therefore, for this intervention category, we only included publications up to 2009, which might have introduced a bias to our sample of pharmacological papers.

Second, our analysis of implementation feasibility of interventions was based on published documents, so SAFE ratings may be influenced by reporting quality rather than the actual intervention. We addressed this issue by accessing published manuals, design papers, research protocols and trial registrations alongside trial reports. However, in some cases, these additional documents did not provide all information needed. For instance, information about costs and cost savings was often limited. In combination with the relatively high threshold used in SAFE ratings for an intervention to be identified as costly, this may partly explain why no differences in cost and cost savings were found between intervention categories. Some specific items, such as costs, may therefore benefit from more detailed analysis to inform recommendations. Related to this limitation, reports of RCTs describe what happened in the context of a trial, when implementation might be more standardised and less flexible than in routine clinical practice. For example, in most reviewed pharmacological trials, medication was prescribed either in a fixed dosage (i.e. the same for each service user) or in a combination of partly fixed and partly variable dosage (e.g. a fixed dosage for the first few weeks, and a variable dosage in later stages of the trial), whereas changes to dosage outside of a trial context may be more variable. Our study might therefore have underestimated the actual flexibility of interventions. However, our approach to using RCTs is consistent with the methodology used to develop guidelines, in that the efficacy estimates come from controlled conditions, which may differ from the efficacy within routine implementation.

A third limitation concerns rating the extent to which interventions were manualised. The difference in manualisation between psychosocial and recovery interventions versus pharmacological interventions may be explained by the fact that in almost all pharmacological interventions, the only structured part was the specified medication dosages. Although dosage may be a crucial component in the delivery of a pharmacological intervention, NICE guidelines emphasise that pharmacotherapy is a trajectory including discussion of risks and benefits and careful recording and monitoring of response and side effects. If this trajectory was not structured or specified in most trial reports, we considered the pharmacological intervention not to be manualised. The implication of our approach may be an underestimation of manualisation in pharmacological interventions.

The fourth limitation is that we could not perform quantitative analyses for all comparisons of blocks and enablers because of empty cells (i.e. zero counts for blocks and enablers) for some items. In these cases, we had to rely on the visual profiles shown in Fig. 1. A final limitation is that we rated only the 15 interventions listed in the NICE guidelines with the strongest evidence so did not provide a comprehensive overview of all interventions.

This study offers an approach which may contribute to tackling the current implementation challenges in psychiatry by addressing one of the major implementation domains identified in implementation theory, namely feasibility of the intervention [8]. Assessment of intervention feasibility can inform the development process of clinical guidelines. Current guidelines are primarily based on evidence reviews with a focus on efficacy and cost-effectiveness [13]. We propose that guideline development would benefit from more weight being put on the feasibility of interventions. Incorporating SAFE ratings into the evidence appraisal would provide a metric for guideline developers to take into account those aspects that might hamper or facilitate successful implementation. For instance, if an intervention A has higher efficacy than intervention B but also has a higher risk for doing harm, then this may be a reason to give higher recommendation to intervention B. In addition, if an intervention is cost-effective but has also found to be highly complex in terms of implementation feasibility, this should be balanced in the recommendations. Balanced recommendations can be made which differentiate between settings in which implementation is more or less feasible.

SAFE items were sometimes scored as ‘unable to rate’ because trial reports and manuals did not provide complete information. Information was particularly lacking regarding potential blocks in staff training, time spent on the delivery of the intervention, ongoing supervision and cost-saving potential of the intervention. Future studies might benefit from using SAFE reporting guidelines, which parallel the SAFE assessment items [7], in a way that the PRISMA [14] and CONSORT [15] statements guide the reporting of systematic reviews and RCTs. These reporting guidelines may help researchers to put more emphasis on feasibility issues, both in the initial design and the final reporting of trials.

Finally, feasibility profiles have relevance for service development as they can support service managers in identifying the important and evidence-based feasibility issues that are likely to arise during translation of evidence into practice. SAFE ratings can provide a pre-implementation assessment; an important step in the implementation process of evidence-based interventions into everyday clinical practice [16‐18]. For instance, looking at the feasibility profiles of recovery interventions in Fig. 1, service managers can conclude that, for the included interventions, (1) they can be sure that recovery interventions are applicable to service users with psychosis, that the goals of the intervention match the prioritised goals of the NHS, that there is evidence of effectiveness (these items all have long blue bars indicating that they are rated as enablers), (2) they do not have to worry about negative consequences when the interventions would be stopped (low risk of irreversibility), (3) the implementation of these interventions can benefit from the fact that recovery interventions are often manualised and can be tailored to context and situation, but that (4) they should take into account that additional staff is needed to provide the interventions and that the interventions often consist of multiple interacting components, which might mean that they have to appoint a skilled person to supervise implementation processes. Assessment of potential blocks and enablers, such as staff time to tailor interventions or to manage the interaction between components of a complex intervention, or the availability of manuals, allows for the creation of an implementation plan in which priorities are set and problems can be anticipated. This enables service managers to allocate resources efficiently and to speed the uptake of interventions by clinical staff.