The sustainability of public health interventions in schools: a systematic review

A study was included if it:

Interventions were excluded if they provided health-information materials only, created new schools or were primarily family/community-based interventions with a minor school component. Interventions which co-located a health service within schools, with services delivered exclusively by clinical providers, were also excluded. The sustainability of such interventions is likely to differ from those delivered partly/wholly by educators or school employees, for example, greater reliance on schools continuing to commission services or the option of service provision at no cost to the school (i.e. through other funding mechanisms), and differences in clinicians and educators’ commitment to sustainability due to differing professional knowledge/roles, peer support and priorities.

We searched electronic databases for English-language publications between January 1996 and September 2017 (PsycINFO, Social Sciences Citation Index – Social Science & Humanities [Web of Science], British Education Index, PubMed, CINAHL, EMBASE and ERIC). A mixture of free-text and controlled terms was searched in titles/abstracts, and MESH headings where relevant. Synonyms for four concepts were combined: sustainability, school, intervention and public health (see Additional file 1 for full terms used). A comprehensive website search was also carried out (see Additional file 2). School-based studies in Stirman et al.’s review were also screened [20]. The references of included studies were checked, and a citation search was conducted on Google Scholar. Subject-matter experts were contacted to identify unpublished/current research, including authors of included studies (see Additional file 3).

All identified studies were imported into the data-management software EPPI-Reviewer 4 [32]. Fifty articles were initially double-screened by two reviewers (LH, HM) on title/abstract: 94% agreement was achieved and discrepancies were discussed to reach a consensus. Reviewers then worked independently, single-screening on title/abstract. Studies were retained if they met the inclusion criteria or if there was insufficient information in the title/abstract to judge. Full-text copies of potentially relevant papers were retrieved and screened independently by the two reviewers to decide on inclusion. If there was uncertainty, studies were discussed by both reviewers (LH, HM) until a consensus was reached, involving a third reviewer (CB) when necessary.

We extracted data from each included report on study sample/population; description of the intervention (adapted criteria [33]); key dates, study design/methodology and results for the evaluation of effectiveness (or implementation period for non-evaluated initiatives) and sustainability phase; and information needed for quality appraisal (see Additional file 4). Two reviewers (LH, HM) extracted data from two study reports, comparing their results. Pairs of reviewers (LH, HM or LH, TO) independently completed data extraction for each included report. Differences between reviewers were discussed, including a third reviewer (CB) where necessary.

Two reviewers assessed study reliability using an existing checklist [34]: justification for study focus and methods used; clear aims/objectives; clear description of context, sample and methodology; demonstrated attempts to establishing data reliability and validity; and inclusion of original data. Studies were assigned two ‘weight-of-evidence’ ratings [35], one for reliability and one for relevance to answer the review question, rated ‘low’, ‘medium’ or ‘high’. To achieve ‘high’ reliability, at least five criteria had to be met, for ‘medium’ at least four criteria had to be fully or partially met, and all other studies were rated ‘low’. We also downgraded the reliability of retrospective, cross-sectional studies using self-report data for interventions implemented more than 2 years ago. For a judgement of ‘high’ relevance, studies had to describe, with breadth and depth, factors influencing sustainability and privilege participants’ perspectives (Additional file 5 describes quality criteria and ratings). Studies were not excluded from the synthesis based on their reliability, but greater qualitative weight was given to those assessed as ‘medium’ or ‘high’. The quality-assessment tool was piloted on two studies by each pair of reviewers (LH, HM and LH, TO) with results discussed to ensure consistency. Each included study was then independently quality-assessed by each reviewer with discrepancies discussed, where necessary resolved with a third reviewer (CB).

We originally intended to use a meta-ethnographic approach as submitted in the protocol [36]. We anticipated finding qualitative studies that were rich in concepts, metaphors and description. However, only one study went beyond description to interpret participants’ views and experiences, and it was not possible to ‘translate’ and synthesise concepts from one study into another. Instead, we conducted thematic synthesis [37] to develop concepts from the mixture of qualitative, quantitative and mixed studies identified. One reviewer (LH) read and re-read studies and carried out line-by-line coding using NVivo 11 software. Inductive codes were developed from the qualitative data (participants’ verbatim quotes and authors’ interpretations) and from authors’ textual reports of quantitative findings. Each code’s data were checked for consistency of interpretation and re-coded as necessary. We used the General Theory of Implementation (GTI [38]) as a sensitising lens; it explains how implementation proceeds over time, building on normalization process theory [21, 39] (Fig. 1 summarises the theory’s constructs). Memos were used to explain codes, their relationships and their alignment with the GTI. GTI informed the overarching structure of themes and sub-themes that was developed. The reliability of each study was checked and referred to as the overall themes were incorporated into a narrative synthesis. The three other reviewers (HM, TO, CB) commented on and discussed a draft of the themes and sub-themes, and a final version was agreed.

This review was registered on PROSPERO (6.9.17, CRD42017076320, [36]) and follows PRISMA reporting standards (Additional File 6).

Schools’ social norms, staff roles, resources and systems were reported to influence sustainability. Five sub-themes developed from 20 studies of 14 interventions [43‐49, 51‐59, 63‐66].

Teachers, principals and administrators prioritised teaching the academic curriculum, meeting educational standards and regulations. Under time constraints, health promotion was considered dispensable, a theme that arose from nine studies (high and medium reliability) of six interventions focused on physical activity, healthy eating and mental health [43, 44, 46, 48, 49, 52, 56, 65]. A district-level informant from the CATCH study commented:

…if you’re going to prioritize, you’re going to prioritize on academics. ...You always concentrate on academics but there was more room for PE and health and those kinds of things before the state kicked in the really extremely rigorous academic standards. ([48], p. 515)

There were some exceptions where principals or administrators encouraged staff to focus on health [43, 46, 48], but the prevailing norm was to focus on academic attainment.

Staff members’ roles and autonomy were reported to affect whether interventions were sustained at school-level or solely by individual practitioners. Two deeper sub-themes emerged: the importance of the principal and administration, and teachers’ autonomy in the classroom.

Commitment and support from the principal and administration (including the school district in US studies) were considered crucial to ‘pave the way’ for sustainability [46], a sub-theme identified in 12 studies of 11 interventions [43, 45‐48, 52‐54, 59, 63‐65]. Senior staff had the power to stop or continue an intervention at school-level through authorisation [46, 48], re-distributing school funds to or away from interventions [45, 47], allocating time for delivery [43, 46, 47] and providing training for new staff [43, 47, 63] (see sub-theme 4 (i) ‘Staff turnover’ in the ‘Theme 1: Schools’ capacity to sustain health interventions’ section).

Beyond resources, principals/administrators could demonstrate their commitment through integrating the intervention into school policies [43], recruiting new staff who were well-disposed to it [63], giving staff positive recognition [43, 53, 64] and managing staff to ensure that they continued [43]. The principal had a key role in continuing to enrol staff in a community of practice and persuading staff that it was right for them to address health [43]. This sub-theme overlaps with the GTI domain ‘cognitive participation’ under the construct ‘contribution’.

Four studies of four interventions (high and medium reliability) indicated that teachers had autonomy to decide whether to sustain interventions in their classroom, within the bounds of the curriculum and principals’ leadership [43, 44, 48, 65]. Other studies revealed that if teachers sustained interventions, they could adapt them as they deemed appropriate (see sub-theme 1 ‘The workability of the intervention’ in the ‘Theme 3: Intervention adaptation and integration’ section). One teacher from a US study of CATCH reported [48]:

It is an individual decision. The state has a framework of what we are supposed to teach. We are asked to teach the things that the district recommends, but if you have more time, you can teach other things as well. No one has asked us to use the CATCH curriculum since the program ended in our school so it was up to us. ([48], p. 509)

There were some examples of collective action among teachers (reflecting GTI domain ‘collective action’ under ‘contribution’). Two US studies (medium and high reliability) of physical-activity interventions showed teachers working together to plan and develop ideas [44] and to encourage the principal to raise funds for sustainability [45]. There was an example of staff receiving logistical support [46] and providing internal training to other staff [48]. The piecemeal evidence for collective action may reflect the lack of attention given to this factor in the studies or a norm that teachers’ work with an intervention beyond the evaluation of effectiveness is typically independent.

Insufficient funding, equipment, materials and/or physical space could lead to discontinuation, cause logistical challenges [43, 47, 64] or become a reason for adaptation (see sub-theme 1 ‘The workability of the intervention’ in the ‘Theme 3: Intervention adaptation and integration’ section), a sub-theme developed from 16 studies of 11 interventions [45‐49, 51, 52, 54‐59, 63, 64, 66]. A lack of resources could motivate schools to seek out external funds via fundraising, grants or assistance from school-related associations [48, 57, 58, 66], redistribute school budgets [45] or find alternative means such as volunteers or parental payments [47, 57, 66]. As one study (medium reliability) of an all-girls physical-activity intervention reported:

Lack of finances was mentioned as a reason that teachers did not offer guest instructors or hold weekly lunch bunches. Whereas some teachers asked for volunteers to teach yoga or dance, others used videos or asked students to pay a $5 activities fee at the beginning of the class to use for guest instructors’ fees. ([47], p. 5)

Schools needed to retain the knowledge, skills and experience to sustain the intervention. Two deeper sub-themes emerged related to staff turnover and the importance of training.

Fifteen studies of ten interventions described the adverse impact of staff turnover. As staff left, organisational knowledge, enthusiasm and the co-ordination of the intervention could dissipate [43, 46‐49, 51‐53, 55, 56, 58, 59, 63‐65]. A change in principal [43, 48, 63] or loss of a champion (a senior staff member who advocated and assumed responsibility for intervention coordination and integrity) could jeopardise sustainability [46, 58, 62]. New decision-makers did not always share enthusiasm for the intervention or had other priorities, as a clinician from one highly reliable US study of a mental-health intervention explained:

We’ve lost a major senior administrator that is proactive and advocated for the kids’ needs, across the board, regular education and special education. Things have changed. Within the last year, they’re just looking at all the academics right now. ([46], p. 138)

A lack of training for new teachers or booster training was a barrier to sustainability, a sub-theme emerging from 12 studies of nine interventions [43, 46‐49, 51‐53, 56, 59, 64, 65]. One Dutch study (medium reliability) of an intervention to reduce aggressive behaviour found a designated school co-ordinator to train and coach teachers facilitated sustainability [63]. As well as giving staff the skills and knowledge for delivery, training could generate enthusiasm and communicate the intervention’s philosophy [47, 48], as described by a teacher from a US study (medium reliability) of CATCH:

The staff development was interesting and motivated teachers. They learned about nutrition and fitness. They got excited about it and therefore implemented it. And that made it difficult to implement in schools that had not had the training. They missed a real motivational surge and missed looking at the importance and hearing from experts. ([48], p. 515)

Schools’ networks with other schools, community organisations and funding agencies appeared to influence sustainability, a sub-theme emerging from four studies (high, medium and low reliability) of four interventions [43, 45, 48, 58]. Strong social links could give schools access to funding [58] and training [48], and collaborations with community organisations and other schools could motivate schools to maintain and develop interventions [43].

Five sub-themes emerged on staff motivation and commitment to sustain health interventions from 18 studies of 15 interventions [43‐50, 52, 53, 55, 59, 60, 62‐66].

Directly observing the benefits for students’ engagement, wellbeing and behaviour was a strong motivator to continue [43‐50, 52, 63, 65, 66]. No staff referred to the findings of the effectiveness evaluation when discussing the intervention’s value, though a clinician in one study commented seeing a change in students based on a ‘pre and post test’ [46]. Conversely, negative responses from students could be a barrier [48, 55, 64]. For example, a teacher from a Dutch study (medium reliability) of an intervention to reduce aggressive behaviour reported:

It gives the team power. And, especially now, with more children with behavioral problems in the classroom. When you stay on the positive side, almost all children will get along. ([63], p. 85)

Two studies (high reliability) asked students about their experiences of physical activity interventions [45, 65] and found they had little decision-making power over what activities were sustained; they were willing participants, but opportunities were largely dictated by their families or the school. For example, a student commented on a component discontinued due to time constraints (as reported by teachers):

Taylor said, ‘We started these warmups, and then they stopped. I don’t know why, but I wish we had them. It is hard to run the CV day with no warmup.’ ([65], p. 114)

Only four studies (one high, two medium and one low reliability) of four interventions referred to more formal processes to appraise effectiveness [43, 46, 59, 63], overlapping with the GTI domain of ‘reflexive monitoring’ under ‘contribution’. Two studies found no differences in sustainability between schools with procedures for reviewing the intervention and those without [59, 63]. One study (medium reliability) reported principals who sustained the intervention regularly evaluated health-promotion activities.

Staff who had been trained in the intervention felt more confident and better prepared to deliver it [47‐49, 52, 64] (see sub-theme 4 (ii) ‘The importance of training’ in the ‘Theme 1: Schools’ capacity to sustain health interventions’ section). Teachers delivering an intervention outside of their usual expertise were less likely to sustain it [43, 47‐50, 64, 65], for example, PE teachers delivering nutrition education [47] or classroom teachers delivering PE [43, 48‐50, 65]:

Among classroom teachers, feeling inadequately prepared to implement PE was frequently reported; and in many cases, teachers had little interest in gaining the skill. ([49], p. 471)

Five studies noted parent support in a general sense was helpful [43, 45, 52, 59, 64]. Four studies covered parent support in more depth; staff indicated how lack of parent support could reduce their motivation to sustain an intervention [46, 48, 62, 65]. This sub-theme overlaps with the GTI domain ‘coherence’ under ‘contribution’. A teacher from an English study (high reliability) of a physical-activity intervention explained:

I think a lot of it is home life, if the parents don’t push them towards sporting activities then you’re fighting a battle straight away in school. ([65], p. 8)

Belief in the importance of the intervention motivated staff to sustain it, a sub-theme arising from seven studies of six interventions [43, 44, 46‐49, 52, 63] and was related to the importance of training (sub-theme 4 (ii) ‘The importance of training’ in the ‘Theme 1: Schools’ capacity to sustain health interventions’ section) and observing intervention effectiveness (sub-theme 1 ‘Observing and evaluating effectiveness’ in the ‘Theme 2: Staff motivation and commitment’ section). Principals who reported sustaining a 3-year HPS intervention in Norway, which aimed to create a positive school environment for health, were keen to communicate its importance:

School satisfaction and safety are at the bottom of this school. It is under the teachers’ skin and in our walls. We work with this no matter what is on our agenda. ([43], p. 59)

There was limited evidence on the impact of staff perception of the school climate. One highly reliable US study of CATCH suggested climate might differentially impact on different interventions: a positive climate was associated with more teaching hours of the CATCH curriculum but higher levels of saturated fat in school meals [60]. Respondents in two other studies (medium and high reliability) reported that a negative climate meant that sustainability processes were superseded by more critical organisational priorities [46, 63]. One US study (low reliability) of a workplace health-and-safety intervention found no relationship between climate and sustainability.

Schools’ ability to sustain an intervention was affected by its ‘workability’—the degree to which it could be shaped into existing school practices and routines, and its integration into school policies and plans. These two sub-themes emerged from 18 studies of 13 interventions [43‐49, 52‐56, 58, 60, 63‐66].

Three deeper sub-themes transpired: fitting the intervention into the time available, matching the intervention to students’ needs and the need for up-to-date equipment and materials.

Frequently, staff identified that interventions required too much time, time which was primarily devoted to delivering the curriculum (see sub-theme 1 ‘Educational outcomes took precedence over health promotion’ in the ‘Theme 1: Schools’ capacity to sustain health interventions’ section) [44‐46, 48, 49, 52‐56, 63‐65]. Staff dealt with time constraints by reducing or dropping components [45, 47, 64, 65], or making time for the intervention by adapting it to classroom routines [44, 50] or incorporating elements of it into the existing curriculum [48, 52, 53, 56, 58, 65].

Adaptation was also important to match the needs of different cohorts of students, to offer the intervention to different grades [53, 63], better fit students’ learning abilities or make lessons more contextually relevant [43, 54], devote more time to particular activities to ensure students understood a subject or better engage students [46, 64].

Over time, new equipment and materials were needed as equipment grew worn or was lost [49], materials became dated [48, 53, 64], new technological advances emerged [50, 64] or adaptations were needed to meet students’ needs [53, 54, 64].

One Dutch study of an intervention to reduce aggressive behaviour and one Norwegian study of an HPS intervention (medium reliability) reported that schools with greater sustainability more often made reference to it in school policies or plans [43, 63]. Studies suggested formal documentation signalled principals’ and administrators’ commitment to the intervention [63], legitimised it [48, 63], made staff accountable [43] or made the intervention resilient to staff turnover [43] (see sub-theme 4 (i) ‘Staff turnover’ in the ‘Theme 1: Schools’ capacity to sustain health interventions’).

The wider policy context could also affect sustainability, a thematic area positioned outside of the GTI framework, emerging from seven studies of five interventions. Regional or national health policies could support sustainability by legitimising health promotion in schools’ policies [43, 48] (see sub-theme 1 ‘Educational outcomes took precedence over health promotion’ in the ‘Theme 1: Schools’ capacity to sustain health interventions’ section). Over time, health policies could shape social norms: for example, increasing tobacco-control regulations could enhance the sustainability of outdoor-smoking bans in schools [62]. Policy could also provide funding and resources [55, 57], though additional resources could also lead to competing interventions, potentially displacing existing ones [55, 56].

The sustainability of public health interventions after start-up funding and/or other resources end has been relatively uncharted in schools compared to health care. We identified 24 studies assessing the sustainability of school-based health interventions delivered partly/wholly by educators or school-employed health professionals, but quality was not consistently high. None of the interventions assessed were fully sustained; all had components sustained by some schools or staff, bar one that was completely discontinued. Identifying common facilitators and barriers could help researchers and providers optimise the sustainability of school interventions, and consider whether/how the intervention is likely to have a lasting impact on student and staff health. Two key facilitators emerged. First is the central importance of a committed principal and administration that could authorise continuation, allocate resources, integrate the intervention into school policies and enrol new staff into a community of practice. Second is the importance of supporting staff who are confident in delivering health promotion and believe in its value. These facilitators are consistent with studies of the implementation of school health interventions [13, 31, 67], suggesting factors are crucial to both phases.

Many of the facilitators and barriers to sustainability identified for school settings were similar to those in health care: for example, dedicated leaders, the need for continued resources and training, staff turnover and intervention workability [21, 24‐27]. Several factors were more salient for schools. Health encompasses multiple outcomes, some of which may be more obviously relevant to school settings. We identified the sub-theme of educational outcomes taking precedence over physical activity, nutrition and mental health interventions, but not for those focused on anti-social or violent behaviour. This suggests that throughout adoption and implementation, change agents need to convince schools that health interventions can bring education benefits [30, 68‐70].

Student engagement was key to implementation and sustainability at teacher-level. A central role of educators is to engage students [29, 71], and staff were unlikely to sustain interventions that did not draw students in [48]. Sometimes sustainability was prompted by students’ requests for the intervention [44, 45]. Knowing parents encouraged the healthy activities of the intervention outside of school also motivated staff to continue, further supporting the view that schools are complex adaptive systems, where multiple networks of agents act and react to one another [30]. In contrast, only 16% of the 62 sustainability approaches in Lennox et al.’s review [23] included patient involvement, suggesting that most existing tools and frameworks for health care settings do not consider patient support for the intervention critical for sustainability.

Also of particular significance for schools was the need to adapt intervention materials and activities to accommodate other curriculum requirements and the diversity of children’s backgrounds and development [29, 72]. This dynamic context suggests that intervention developers should anticipate the need for adaptation, even for effective, well-implemented and funded school health interventions [21, 30, 73].

Contrary to other studies of sustainability in health care settings [20], we found little evidence that champions helped sustain interventions: like other staff, champions moved to new institutions leaving interventions at risk. We found no discernible relationship between evidence of effectiveness and sustainability, and no school staff mentioned outcome evaluation as an influential factor in sustainability.

Our review was comprehensive and rigorously conducted. It is the first to apply the GTI to the study of sustainability. We found the framework helpful in creating a balance between listing the common enablers and barriers and representing the complexity and context-dependent nature of sustainability in schools. The data aligned well with the constructs of capacity (theme 1), potential (theme 2) and capability (theme 3), while the construct of contribution was implicated within the other themes. It made sense to consider ‘cognitive participation’ and ‘collective action’ under the construct of ‘capacity’ as the ongoing enrolment of staff, the legitimisation of health activities, and whether staff worked independent or collectively appeared significantly affected by schools’ social norms and roles. Under capacity, we included an additional domain of ‘social resources’ which suggested that contact between schools and other organisations could facilitate sustainability through creating opportunities for resource- and knowledge-sharing, while stimulating ongoing interest in the intervention.

Regarding limitations, we did not double-screen full reports and we may have missed reports due to the array of terms used to describe sustainability, despite our sensitive search strategy. We deviated from our original protocol in using thematic synthesis rather than meta-ethnography due to the nature of studies found. We excluded interventions delivered by clinical services co-located in schools, and consequently, our findings may be less representative of the sustainability of targeted or tiered services which typically require a high level of clinical expertise (only 3 of the 24 interventions in the review were targeted). The sustainability of health interventions provided solely by external clinicians is unknown; for example, they could be more sustainable because they do not require educators to expend time gaining additional knowledge and skills, or they may be less because they require sustained funding. There was substantial heterogeneity in study designs, methods and reporting of included studies; many studies were methodologically weak and did not report on the sustainability of all components, in particular reporting for family/community components was poor. Most studies were located in the US, and consequently, our review findings may be most relevant to this setting. Around half of interventions focused on healthy eating/physical activity, with a lack of evidence for the sustainability of other public-health interventions.

Informed by our synthesis, we propose three questions to consider when optimising school health interventions. First, is it important that each component is sustained? Some components, such as needs assessment, may be time-limited stepping-stones. Second (if a component is to be sustained), how would you expect the intervention to be sustained: if there were high staff turnover or the loss of the champion, during time-pressured periods such as exams, with different classes of students with varying needs or if there were no opportunities for regular training updates? Third, do staff understand the key theoretical principles that should underpin any adaptations to intervention activities and resources? Creating forums during the period of the evaluation of effectiveness when these ‘stress-testing’ questions can be discussed with staff could help researchers to understand the likely sustainability of interventions.

Sustainability strategies contributed to our analysis where authors commented on them in papers’ results and discussions [43‐45, 52, 64]. However, several papers referred to specific sustainability strategies in their background sections but did not consider their impact in their analysis of sustainability, including ‘train-the-trainer’ models to spread the intervention across and between schools [58, 63], external consultants exploring adaptations with staff [53] and a staged-approach to implementation [50]. Primary research on the impact of implementation and sustainability strategies and planning would be valuable [74, 84].

Our review suggests regional and/or national school policies and educational standards that promote health and wellbeing and its connection to students’ learning and school enjoyment could enhance sustainability by legitimising staff spending time, effort and resources on continuation, as well as bringing funding and resources to sustain health goals.