Exploring the factors affecting the implementation of tobacco and substance use interventions within a secondary school setting: a systematic review

Adolescence can be identified as a critical development phase and therefore a key stage within the life course. It is a time when adverse health-related behaviours, such as tobacco or substance use, are frequently established and ‘tracked’ into adulthood [1‐4]. Adolescence—or the ‘secondary school years’—thus provides a key period for the delivery of interventions [2, 5‐7] to deter uptake of unhealthy behaviours. The uptake of health risk behaviours are more likely to occur in the later stages of adolescence, between the ages of 15 and 19 years [8, 9]. This is largely due to the fact that a young person in late adolescence is more susceptible to social influences, such as peer pressure, experimentation and rebellion. These social influences are associated with an increased tendency to undertake in risk-taking behaviour, such as drug taking or risky alcohol consumption, and can play a substantial role in influencing long-term health outcomes [8, 10, 11].

As it remains compulsory for young people, in the UK, to engage in academic education until the age of 16 years; the secondary school setting acts as a platform in which to deliver preventative health education and complex interventions to tackle tobacco and substance use. Recent feasibility research exploring the delivery of brief alcohol interventions (ABIs) in secondary schools has proved effective, highlighting the potential as a setting to deliver such interventions [2, 7]. However, the effectiveness of school-based substance use interventions has often proved inconclusive. A specific example of this is a systematic review conducted by Foxcroft and Tsertsvadze, aiming to explore the extent of research around the effectiveness of school-based, alcohol primary prevention programmes [12]. The review identified that some studies showed no evidence of a primary intervention being effective in a school setting, whilst others presented statistically significant results, indicating a degree of effectiveness [12].

Breaking this down further, to assess whether complex substance use interventions have a place within secondary schools, although there remains to be a series of factors affecting the effectiveness of such an intervention, there does appear to be a gap between generating school-based, tobacco and substance use intervention research evidence and the implementation of this research in practice [13]. Very few of these papers offered a significant assessment of the factors affecting the implementation of their substance use interventions, or how varying the implementation process of such an intervention could have the potential to affect the effectiveness. Therefore, this systematic review was proposed as a way in which to collate the available evidence from studies, which present evidence around the factors affecting the implementation of a tobacco or substance use interventions, within a secondary school setting. It builds upon the current work in this area [14, 15], to provide an account of factors specific to secondary school level education and specifically the implementation of tobacco and substance use interventions.

The field of implementation science has been born as a result of recognising the importance of the gap between research and practice [13]. This gap has expedited the use of multitudinous theoretical constructs, aiming to enhance the implementation process, identify the barriers and facilitators and acting as valuable tools in evaluating implementation [16, 17]. Much of the advancing knowledge on barriers and facilitators to implementation has been construed within health care and the provision of primary care, and implementation theory has been frequently employed within this context [18, 19]. The use of theory has been less widely associated with school implementation research [20]. Therefore, this systematic review seeks to interpret and synthesise determinants of implementation in the school setting by using a specific implementation theoretical framework.

The Normalisation Process Theory (NPT) provides an explanation of the factors affecting whether an intervention can be incorporated into practice, with reference to the context in which the work of the new intervention occurs [21]. It focuses on understanding the implementation, embedding and integration of new technologies and organisational innovations by considering four theoretical constructs: Coherence, Cognitive Participation, Collective Action and Reflexive Monitoring [16, 21, 22]. Table 1 presents an overview of the four NPT constructs and its respective subconstructs.

NPT is our chosen framework as it has demonstrated value in synthesising research findings to identify knowledge consistencies and gaps regarding implementation determinants [23, 24]. Although NPT was designed for implementation and integration problems in healthcare, the constructs are transferable and thus can be applied fluidly to consider the review’s focus of factors affecting implementation in the school setting [25]. As this field is currently small and studies of implementation are heterogeneous, NPT offers advantage as a theoretical framework for integrating both qualitative and quantitative findings to develop an assessment of the factors which can affect implementation in this context [25]. To our knowledge, NPT has not previously been used to synthesise findings in the context of secondary school implementation research.

Using a registered protocol (PROSPERO: CRD42016039354), systematic review methods were undertaken to identify eligible literature. Developing specific inclusion and exclusion criteria allowed the selection of papers.

Figure 1 shows a PRISMA diagram displaying the number of papers excluded at each sifting stage [28, 29], whilst Table 3 presents a summary of each paper, their key findings and the results of quality appraisal.

Fifteen papers were deemed eligible for inclusion in the review. Six papers focused on tobacco interventions [30‐35], four focused on drug use interventions [36‐39], three focused on general substance use interventions [40‐42], one focused on an alcohol intervention [43] and one focused on a dual alcohol and drug intervention [44]. All but one paper lacked the use of an implementation theory [44, 45] and no reference was made within the included papers to the use of implementation strategies.

Despite the 15 included papers being heterogeneous, common factors affecting the implementation of tobacco and substance use interventions in the secondary school could be identified. During quality appraisal, the majority of papers were classified as weak or moderate quality. A common weak area was found to be the reporting of confounding and contextual factors affecting study results, which was also identified in the review of healthcare innovation by Greenhalgh et al. [18]. By offering more of a focus to confounding factors, which have the potential to affect implementation, it is likely to add value in providing a richer understanding of the context and facilitate implementation within the secondary school setting.

This is a common thread within the implementation field, and advances in implementation science has led to the identification that implementation studies often display insufficient and inadequate reporting, requiring intervention [46‐48]. Therefore, the recently published Standards for Reporting Implementation Studies (StaRI) Statement was developed as a set of guidelines to increase the transparency and accuracy of implementation study reporting [46]. This would particularly be of use within the school setting, as the reporting was shown in the review to be largely disparate. By employing the StaRI guidelines, within future implementation studies in the school setting, it would likely have a significant impact on the structure and reporting of implementation outcomes, and would not only ensure the delivery of higher quality papers but would increase the comparability and work towards improving implementation in practice in this setting [46].

NPT was used to provide a common interpretative framework to apply across the full set of papers and ensured that a comprehensive assessment of the factors affecting implementation could be made. This sought to be a novel element of this paper as NPT’s use outside of the healthcare setting has been limited, and no previously published work has used it within the secondary school context. This has implications for broader implementation research, as it emphasises the usefulness of NPT in the school setting, and highlights the transferability of NPT in settings outside of healthcare.

Another key factor determining implementation was the providers’ level of comfort with delivery and the topic. This has not been observed in a similar way in general health promotion implementation studies in schools [15] but is largely unsurprising when focusing on tobacco or substance use interventions, as they are often associated with negative stigma [49, 50] indicating a consideration for future work. Results highlighted providers feeling unprepared or that specialist knowledge was required to deliver interventions effectively [44]. This links with the conflict around role identity and the importance of training, which was emphasised in several papers [32, 34, 41, 42]. Comprehensive training can therefore be highlighted as an implementation strategy, which can positively affect implementation, if it is able to address how to deliver controversial topics and leaves providers feeling adequately supported.

Support and provider buy-in were consistently portrayed as factors facilitating and determining implementation and good engagement were seen to positively influence student behaviour [30]. In addition, provider support was linked with the need for an implementation driving force. Due to the disparate nature of the papers and their context, this was explored differently, with students, teachers, project workers and outsider providers acting as implementation drivers. Organisational support, which has previously been identified as a key implementation determinant [45, 51, 52], was also identified as a key factor affecting implementation, with the most effective support being gained pre-implementation and providing long-term maintenance [32].

Another result specific to the school setting included student engagement, which was observed as a factor affecting implementation in Pettigrew et al. [42]. Although the school settings were shown to be highly heterogeneous, this is likely to be common across schools, as individual differences will affect students’ engagement levels.

Moving on to consider the finding around the implementation outcome fidelity, implementation fidelity appeared to be variable across the included papers and was affected by multiple factors. In some papers, providers felt it necessary to modify intervention components, leading to emphasising the importance of establishing which components are essential for implementation and which components should possess flexibility. Implementation fidelity is often considered as being complex and a key source of variability [37, 53, 54]. A specific example from the surrounding literature is within the review of implementation fidelity of school-based drug use interventions by Dusenbury et al. [55]. The idea that school providers can reduce implementation fidelity, but ultimately increase the ‘implementability’ , is an important area to discuss. Although, it was seen as beneficial to possess flexibility, as programs that were too rigid experienced low fidelity, it is important to identify critical intervention components, to ensure that modifications do not affect the intervention’s effectiveness. Therefore, to facilitate future school-based tobacco or substance use intervention implementation, core elements should be identified and complemented with flexible components, in order to be salient for the differing school contexts. This observation is also supported by several papers reporting teachers struggling with adhering to timelines; as although teachers were seen to be appropriate providers, heavy workloads made it difficult to maintain fidelity due to the preparation or time constraints. It is likely to be inappropriate to allocate teachers large implementation activities, and it may prove advantageous, if feasible, to source training or delivery to outside providers.

Even though this review highlights factors unique to the school setting, such as provider factors and pupil engagement, fitting this review into the wider implementation literature context; the findings around organisational host support, adequate resources and the need for appropriate feedback echo the findings of previously conducted implementation work [14, 45, 56, 57]. NPT was useful as an organising framework for synthesising findings from disparate study designs, to not only identify the factors affecting implementation, but also to highlight the knowledge gaps and areas warranting future research and or action in terms of intervention modification.

A unique finding of the review was that few of the included papers reported results indicative of NPT’s Reflexive Monitoring construct. This could have resulted from methodological reasons, such as participants were not asked or the intervention effects were not known, or could simply be a result of the previously discussed limited reporting. However, as evaluations can provide value to implementation studies by identifying ineffective areas, such as provider or host support, it is likely that building in feedback or evaluation components into future work in the school setting would be advantageous.

Other gaps included papers lacking reporting around the use of predefined implementation strategies, which can be complemented by the use of implementation theory. As stated, almost all of the review’s included papers lacked a theoretical driving mechanism. We argue that future school implementation work would significantly benefit from being theoretically driven, and this has frequently been raised when considering existing implementation studies [46, 58‐60]. By employing the use of a conceptual framework to underpin the implementation research in the secondary school setting, it could have facilitated implementation strategies and the reproducibility and clearly highlighted specific areas of improvement for future implementation and sustainability.

This finding has broad implications for future work, and one of the goals of this systematic review has been to inform the development of a school-based intervention implementation model to facilitate the implementation of novel substance use interventions in the secondary school setting. Although the model will be developed with reference to the rapidly advancing knowledge on implementation determinants assessment [61] implementation strategies [62] and progress and outcomes assessment and measurement [63], it will be informed by in-depth qualitative research currently being undertaken with local school staff and key stakeholders in the implementation process to ensure targeting of key challenges in the secondary school setting. This review thus represents initial advancement in understanding the challenges of implementing substance use interventions in the school setting, as part of a programme of work that moves more towards the development and testing of tools for facilitating improved implementation of such interventions. Conceptual and practical developments stemming from this work will therefore be useful in the wider school implementation field and will be publicly available for use in future implementation research in this setting.

A final gap identified was there was little to no focus, within the included studies, around the cost effectiveness of implementation. This could benefit from playing a role in future work as small budgets and cuts to school funding were reported to be factors negatively affecting the implementation of a tobacco or substance use intervention, specifically within the context of UK secondary schools [41]. There remains limited available research evidence investigating how altering the implementation of a such an intervention, could influence the total cost, and which costs can be directly attributable to implementation. Therefore, as the secondary school setting remains to be a financially restricted setting, it highlights a key area of investigation for school-based intervention implementation research and one which will be explored within the future planned work.

This review identified and synthesised factors reported to affect the implementation of tobacco and substance use interventions within the secondary school setting. Key factors affecting implementation that were identified, such as contextual factors, and support and training and provider perceptions, should be understood and addressed when implementing secondary school-based interventions. However, increased exploration should be provided to NPT’s reflexive monitoring construct, the appraisal and evaluation processes of implementing new interventions, as findings around providers reflecting upon components they believe facilitated the implementation process and which aspects could benefit from modifications, were limited and are likely to add value in facilitating improved implementation and sustainability of interventions in the future.

This review sought to reinforce the importance of considering the factors affecting introducing a new intervention into practice. As there were relatively few papers specifically focusing on the implementation of tobacco or substance use interventions in the secondary school, it demonstrated that the school health field could benefit from more work in this area and should build on the findings and lessons from the existing school implementation work. Research should focus on bridging the gap between research and practice, and reflective collaborative working involving educators and practitioners will be conducted, in order to generate an implementation model with the most salience for this setting. Working collaboratively to develop implementation strategies, which employ the use of implementation theory and which comprehensively consider the implementation outcomes, such as adoption, feasibility and acceptability in practice, would be advantageous and would likely contribute to increasing the effectiveness of interventions seeking to reduce tobacco and substance use in adolescents.