Applying the intervention Complexity Assessment Tool to brief interventions targeting long-term benzodiazepine receptor agonist use in primary care: Lessons learned

Benzodiazepine receptor agonists (BZRAs) are a class of drugs which include both benzodiazepines and Z-drugs (e.g. zopiclone, zolpidem). Benzodiazepines are indicated for numerous conditions, including anxiety and insomnia, whereas Z-drugs are solely indicated for the treatment of insomnia [1]. Clinical guidelines recommend that BZRA prescriptions should be restricted to short-term use (2–4 weeks), in order to reduce the risk of adverse effects [1‐3]. Tolerance can occur over longer periods of use, whereby the patient experiences reduced effects from the BZRA medication with continued use [4]. Dependence can also develop after only a few weeks of regular BZRA use, and can result in withdrawal symptoms if patients attempt to discontinue the medication [3, 5]. Furthermore, long-term BZRA use (> 4 weeks) has been associated with a number of adverse outcomes such as cognitive impairment, unwanted sedation, and psychomotor impairment which, in turn, can lead to falls and fractures [6‐8]. Despite these risks, BZRAs are still frequently prescribed on a long-term basis with many countries reporting either no changes or only small decreases in BZRA prescribing in recent years [9‐11]. Older patients (≥ 65 years), who account for a substantial proportion of BZRA prescriptions, are particularly vulnerable to BZRAs’ adverse effects due to age-related changes in pharmacokinetic and pharmacodynamic processes, such as reduced drug metabolism and elimination, and increased sensitivity to BZRAs’ pharmacological effects [8, 12‐14].

A number of strategies have been used to try to reduce long-term BZRA use [15‐17], including brief interventions [18]. These generally refer to time-limited efforts to deliver advice or information to patients with the objective of eliciting change in their behaviour [19]. A recent systematic review by Lynch et al. [18] examined the effect of brief interventions targeting long-term BZRA use in primary care. The interventions evaluated in this review included written letters issued to patients recommending that they reduce their use of BZRAs, self-help booklets, and short consultations with healthcare professionals such as general practitioners and pharmacists providing information regarding BZRA dosage reduction. The review found that patients who received a brief intervention in primary care were more likely to reduce or discontinue long-term BZRA use after six to 12 months, compared to those who received usual care. This review also examined the intervention’s components using the Behaviour Change Technique Taxonomy (BCTTv1). Behaviour change techniques (BCTs) refer to the active components of an intervention which are designed to change behaviour [20]. BCTTv1 comprises 93 BCTs and can be applied to systematic reviews in order to identify interventions’ active components and explore their impact on intervention effectiveness. Seventeen BCTs were identified across the included studies with the number of BCTs per intervention ranging between 4–8 [18]. The most commonly identified BCTs in these interventions were ‘information about health consequences’, ‘credible source’ and ‘adding objects to the environment’. The review found no correlation between the number of identified BCTs and intervention effect size [18].

There is an increasing focus in health services research on complex interventions with several definitions discussed in the literature [21‐26]. Complex interventions are frequently described as having non-linear causal pathways; this is in contrast with simple interventions which are commonly seen as having a linear pathway which links the intervention and the outcome [21, 26]. Other key aspects of complex interventions that have been identified are that they are context dependent and their components are interdependent [27]. The UK Medical Research Council (MRC) published a framework for developing, evaluating and implementing complex interventions which has been highly influential in this area [22, 27]. The MRC framework defines complex interventions as interventions containing several interacting components and outlines various dimensions of complexity. These include: the number of interacting components, the difficulty of behaviours needed by those receiving and delivering the intervention, the number of organisational levels targeted, variability of outcomes, and the degree to which flexibility or tailoring of the intervention is required [22]. Clarke et al. suggest that an intervention is complex when many factors are interacting dynamically, making it difficult to predict future outcomes based on past performance [25]. Some authors have suggested that no intervention can truly be categorised as ‘simple’ or ‘complex’ and that even seemingly simple interventions are often found to have complex aspects when they are examined closely [21].

The intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) was developed by to provide systematic reviewers with a structured method to assess the complexity of an intervention [27]. The tool allows reviewers to identify and characterise key aspects of intervention complexity. The tool is made up of six core dimensions and four optional dimensions (Table 1). The six core dimensions include: number of active components, number of behaviours targeted, targeted organisational levels, degree of tailoring or flexibility of the intervention permitted, skills required by those delivering the intervention, and skills required by those receiving the intervention. The four optional dimensions are: degree of interaction between intervention components, the degree to which interventions are context dependent, the degree to which recipient or provider factors impact the intervention, and the nature of the causal pathway between intervention and outcome. Each dimension can be graded on one of three levels of increasing complexity [27]. An accompanying guidance document has also been developed which provides an in-depth explanation of each of the tool dimensions to assist reviewers in applying the tool [28].

A small number of worked examples involving application of the iCAT_SR to studies included in systematic reviews have been published to date [29‐32]. For example, members of the research team previously applied the iCAT_SR to interventions aimed at improving appropriate polypharmacy in older people [29]. However, additional worked examples are needed to further enhance and refine the tool, as well as our understanding of how to optimise its application.

As brief interventions targeting long-term BZRA use provide an effective strategy for addressing a long-standing clinical issue, it is important to gain a more detailed understanding of these interventions and how they work. Many of the interventions examined in the previous systematic review were composed of several different components, making them “complex” [18]. However, the complexity of these interventions has not yet been comprehensively evaluated. The aim of this study was to apply the iCAT_SR [27] to studies included in a systematic review of brief interventions targeting long-term BZRA use in primary care [18]. The study objectives were to:

We used the iCAT_SR [27] to assess the complexity of 12 interventions across eight studies [33‐40] included in a recent systematic review of brief interventions targeting long-term BZRA use [18]. The interventions included in this review were targeted at changing patient’s long-term BZRA use behaviour and were carried out in primary care settings. The patients involved in these studies were 18 years and older (no upper age limit) and had been prescribed BZRAs on a long-term basis (≥ 3 months). For the four studies which assessed more than one brief intervention [34‐36, 40], we applied the iCAT_SR dimensions to each intervention separately.

Two reviewers (AB, CC) in our team carried out the intervention complexity assessments independently using the iCAT_SR. For each study, we extracted the key information from the study using a data extraction form (Additional File 1). This form used was designed and applied in a previous related study by Cadogan et al. [29]. We used the iCAT_SR guidance document as a coding manual while carrying out the assessments [28]. This document outlines the criteria for each assessment level and provides relevant examples. We identified the relevant information for each assessment dimension within each of the study reports, assessed the complexity and provided support for our judgements to enhance transparency. Where there were differences in assessments between the two reviewers, we discussed these until consensus was reached. We applied the six core iCAT_SR dimensions to make assessments of each intervention. We also applied the four optional dimensions where possible.

We planned to base our assessments of skill level requirements on the reported details of prior training. However, the eight included studies did not explicitly report information related to the skills-related dimensions of the iCAT_SR (dimensions 5 and 6). Specifically, formal assessments of skill levels relating to BZRA discontinuation by either intervention deliverer or recipient were not documented in the study reports. To ensure consistency in our interpretation and assessment of these skills-related dimensions in the absence of explicit information, we therefore made a priori decisions on how to grade these dimensions. Our decisions were informed by our previous work involving application of the iCAT_SR [29]. We agreed that dimension 5 (level of skill required by the person delivering the intervention) would be classified as “intermediate skill level” for all of the interventions delivered by healthcare professionals, as supporting patients in successfully withdrawing from and discontinuing BZRAs is a challenging task. However, most of the required skills are likely already within the scope of practice of primary care-based healthcare professionals such as general practitioners and pharmacists and thus only modest upskilling would be required in order to meet the intervention objective. We also decided that dimension 6 (level of skill required by those receiving the intervention) would be classified as intermediate level skill across all of the interventions. This, in our judgement, reflects the challenges associated with successfully discontinuing BZRAs following prolonged use. Withdrawal symptoms associated with BZRA discontinuation have been widely reported and it has been estimated that between 15 and 44% of those who take benzodiazepines on a long-term basis will experience these symptoms [41]. Gradual dosage reduction was recommended in all of the interventions in included studies, and while this method may decrease withdrawal symptom severity, it does not prevent their occurrence [42]. We therefore assumed that an intermediate level of skill would have been required from patients in order for them to successfully discontinue BZRA use.

We used a colour scheme to allow visual representation of the different complexity dimensions assessed for each intervention (red = high level complexity, yellow = intermediate level complexity, green = low level complexity). We also applied a scoring system to numerically represent the complexity of the interventions. The scoring system that we used was employed in two recent systematic reviews [30, 31]. In this system, the intervention complexity score was calculated by the sum of the individual rating scores as follows: high level complexity = 3, intermediate level complexity = 2, low level complexity = 1. We only applied this scoring system to the six core dimensions due to challenges in assessing the four optional dimensions (discussed further in the results section below). Each intervention was therefore given a total score out of 18. We used Cohen’s Kappa co-efficient of inter-rater reliability to assess the level of agreement between the two reviewers based on their final individual assessments prior to consensus discussion (< 0 = poor agreement, 0—0.20 = slight agreement 0.21—0.40 = fair agreement, 0.41—0.60 = moderate agreement, 0.61—0.80 = substantial agreement, 0.81—1.00 = almost perfect agreement) [43].

In order to assess the potential utility of this scoring system, we undertook two exploratory analyses using the intervention complexity scores that we calculated. Firstly, to explore the relationship between intervention complexity and intervention effect size, we used Pearson’s correlation (two-tailed). The risk ratio for BZRA discontinuation at six months post-intervention was used as the measure of each interventions’ effect size as this outcome was reported across all eight included studies. These risk ratios were taken from those reported in the original systematic review for the included primary studies [18]. Secondly, we used Pearson’s correlations to explore the relationship between intervention complexity and the number of BCTs identified by Lynch et al. [18]. We conducted all statistical analyses using SPSS Version 27 (SPSS Inc. Chicago, IL, USA).

In this study, we used the iCAT_SR to assess the complexity of brief interventions targeting long-term BZRA use in primary care across eight studies [33‐40]. This builds on the findings from the existing systematic review by Lynch et al. [18] which identified component BCTs but did not examine the complexity of the interventions any further. This study adds to the literature on worked examples of the practical application of the iCAT_SR [29]. Our previous work involved the tool’s application to interventions aimed at improving appropriate polypharmacy in older people [29] which had different targets and were conducted in different clinical settings. In contrast, the interventions that were examined in the current study targeted patients taking BZRAs on a long-term basis in primary care settings and a single behaviour/action (i.e., BZRA discontinuation). This enabled application of iCAT_SR to a more homogenous group of interventions.

This homogeneity across interventions was evident in the assessments relating to two of the tool’s core domains: ‘Organisational levels and categories’ (i.e., patients taking BZRAs on a long-term basis in primary care settings) and ‘Behaviour or Actions’ (i.e., BZRA discontinuation). As noted earlier, we made a priori decisions in relation to the scoring of these two core domains involving skill level requirements. For instance, patients receiving the interventions were deemed to require “intermediate level skills” due to the difficulty associated with discontinuing long-term BZRA use and the possibility of withdrawal symptoms [41, 42]. We view this as an aspect of complexity that arises in all brief interventions with the same primary outcome of reducing or discontinuing long-term BZRA use. While our assessments for these two domains may have limited the full potential to critically discriminate between interventions using iCAT_SR, we believe that our choices are justifiable given the level of reporting in the included studies.

With regard to the domains focusing on active components and level of tailoring, our study highlights some important similarities and differences between the interventions across the included studies. We found that almost all of the interventions that had multiple active components allowed for some degree of tailoring. In most cases these interventions involved a consultation between the patient and healthcare professional where the information and advice could be tailored according to individual patient needs. The only exception was the study by Cormack et al. [34] which involved a three-armed trial whereby patients assigned to either one of two interventions groups were compared against usual care. In the first group, patients received a letter from their GP asking them to consider discontinuing BRZRA use and advising that this should be done gradually. In the second group, patients received the same letter as the first intervention group, followed at monthly intervals by written information sheets giving advice about BZRA dosage reduction. Although the second intervention had multiple components, it was considered to involve a low level of tailoring as all patients received the same information sheets. Flexible interventions are important for promoting BZRA discontinuation, with many studies and guidelines recommending that BZRA dose reduction should be tailored according to the needs and ability of individual patient to tolerate withdrawal symptoms [4, 44, 45].

For most of the studies, it was not possible to assess interventions in terms of the four optional dimensions due to the lack of detailed intervention reporting in the included primary studies – a limitation noted in the original systematic review [18]. This lack of detailed intervention reporting has also hampered efforts to apply these dimensions in previous research involving the iCAT_SR [29]. The need for a more robust approach to the development, evaluation, implementation and reporting of complex interventions, including a theoretical understanding of the causal pathway and how the intervention brings about change, is widely recognised [22, 46, 47]. Enhanced reporting of causal pathways between interventions and outcome would allow greater understanding of how the intervention is intended to work and whether these hypothesised pathways can be substantiated in practice. As many interventions targeting long-term BZRA use are composed of multiple components, it is also important to have an understanding of how intervention components interact with each other and whether or not this has an impact on effectiveness. More detailed reporting on the intervention recipients would aid in identifying patients who are similar to those who benefited from the interventions in the trials, and in establishing which types of intervention are most effective in relation to specific patient characteristics. Furthermore, identifying intervention deliverer factors that impact on intervention effectiveness would help in determining the most appropriate individuals to deliver the intervention. It is well understood that replication requires detailed reporting [24, 48], and consequently, improved reporting of the complexity dimensions for brief intervention targeting BZRA use would be a pivotal step in allowing future implementation and scale up across health systems. We acknowledge, though, that many other factors affect the implementation of health interventions, including the acceptability of these to those involved and their feasibility within the implementation context [22, 49, 50].

Previous work involving iCAT_SR has applied a scoring system to each assessment dimension and level of complexity to quantify intervention complexity. These scoring systems have been used to examine the relationship between intervention complexity and effectiveness [30‐32, 51]. To attempt to provide novel insights into how the iCAT_SR assessments could be used in conjunction with effect estimates and methodological innovations in systematic reviews (e.g., BCT coding), we explored the use of the scoring system applied in previous research [30‐32, 51]. No correlation was detected between intervention effectiveness and complexity across the twelve interventions assessed as part of this study. This is consistent with previous research involving application of iCAT_SR [30, 31]. Similar findings were observed when the relationship between the number of component BCTs and complexity assessment scores was examined. The lack of correlation that we observed may have been attributable to a number of factors. Firstly, the modest number of interventions and narrow range in complexity scores decreased the likelihood of a detectable relationship being found between intervention complexity and intervention effectiveness [52]. Secondly, because we were not able to implement the four optional dimensions, a full assessment of intervention complexity was not possible. This potentially limits interpretation of our findings regarding the relationship between complexity and intervention effect size and between number of component BCTs and complexity. Further methodological research exploring how the BCT taxonomy can aid future refinement of the iCAT_SR may be helpful.

Complexity scores did not vary substantially across the studies based on the current scoring system. This is perhaps unsurprising, as all of the studies were focused on brief interventions for the reduction/discontinuation of long-term BZRA use. The only complexity aspects that differed across the interventions were the number of intervention components and the degree of tailoring of the intervention. As noted above, the provision of information with respect to the four optional dimensions would have enhanced the potential to differentiate between the interventions in terms of complexity.

The validity and usefulness of applying using a scoring system as part of the application of iCAT_SR remains to be determined. Potential limitations of doing so are that such scoring may provide an overly simplistic view of intervention complexity. In addition, each complexity dimension carried equal weighting in the scoring system that was used, and it may be the case that some complexity questions should be given more weight than others. However, scoring the assessments can facilitate efforts to quantitatively explore relationships between intervention effect size and intervention complexity. This, in turn, may be helpful for understanding the extent to which effects are dependent on higher levels of intervention complexity. Further work is therefore needed to explore whether it is possible to validate a scoring system for the iCAT_SR. In the interim, we suggest that future applications of the iCAT_SR avoid an oversimplistic scoring system to reduce the risk of inaccurate conclusions being drawn being drawn regarding the relationship between intervention complexity and intervention effects.

This study provides a detailed overview of the application of the iCAT_SR to brief interventions targeting long-term BZRA use in primary care. An understanding of the aspects of complexity which arise in brief interventions targeting BZRA use may be beneficial for future researchers developing new interventions targeting BZRA use. The iCAT_SR dimensions could also help researchers in conceptualising these interventions and aid intervention description in trial reports.

The findings highlight that more detailed reporting of interventions is needed in order to optimise the application of iCAT_SR and its potential to differentiate between interventions across the full range of complexity dimensions. In addition, our understanding of the usefulness of iCAT_SR in explaining variation in effect estimates across interventions addressing a particular health issue could be advanced by applying the tool within reviews with wider eligibility criteria and therefore greater variation in intervention complexity. However, further work is needed to determine the validity and usefulness of applying a scoring system to iCAT_SR assessments.