Background
The Ottawa Charter defines health promotion as the process of enabling people to increase control over their health, and its determinants, in order to improve health [
1]. Health Promotion Programs (HPP) “improve population health outcomes by reducing preventable disease, injury or death, and taking action on health inequities” [
2]. This highlights the vital role that HPP play in curbing health care costs in the current climate of scarce funding [
3]. While resources for health promotion are limited, the expectation for HPP to generate results remains high [
4]. As such, the health promotion field faces a predicament: to provide quality services that reduce disease burden and improve quality of life, while under budgetary constraints. This gives rise to the necessity for HPP to be sustained, so that health benefits do not end when the program does [
5‐
7].
While sustainability is often thought of within the lens of funding; for instance, finding replacement funding when initial program funding comes to an end [
8,
9], sustainability is more complex than this [
10]. Program sustainability refers to the maintenance of health benefits, the continuation of a program within an organization, and capacity built in the recipient community to continue carrying out the program on its own [
11]. Despite its importance, only 40 to 60% of HPP are sustained [
6,
12], as project funding cycles are short and priorities shift over time [
13]. Failure to sustain HPP can give rise to three “serious problems”:
1.The issue that a program was established to address remains or recurs.
2.Programs see their funding withdrawn before activities have been fully realized and outcomes reached, despite significant human, fiscal and technical start-up costs.
3.Diminished community support and lack of trust in communities with a history of programs that were abruptly/inappropriately terminated [
11].
Several key challenges affect HPP sustainability. First, there is a lack of consistency regarding what ‘sustainability’ means and how it is defined [
11,
14‐
17]. Second, many sustainability frameworks and tools exist, but few are validated [
14]. Third, although it is often acknowledged that planning for sustainability should begin early in a program’s life cycle, [
10,
11,
15,
16,
18‐
20] it tends to be a ‘latent’ concern often thought of at the end of the program when remaining staff, time and resources are limited [
11].
In order to promote the sustainability of HPP, the factors that promote long term sustainability need to be understood [
6,
14,
21]. This would allow for a clearer conceptual understanding of how a sustainable program is built from the outset.
This systematic review aims to identify the barriers and facilitators that influence the sustainability of HPP. Particular attention is given to those factors that can be considered during the program planning processes. Our research question is “what factors facilitate or influence the sustainability of health promotion programs?” In conducting this review, we assessed the body of literature on this topic to determine how sustainability is being defined, which frameworks are used to plan for sustainability and the overall implications for the health promotion and public health fields.
Methods
We conducted a systematic review that focused on identifying barriers and facilitators for HPP sustainability. For this study, we used Grant and Booth’s 2009 definition of a systematic review: an evidence synthesis that adheres to guidelines on conduct of a review/method for integrating or comparing the findings from qualitative studies, looking for ‘themes’ or ‘constructs’ that lie in or across qualitative studies [
22].
The authors (AB and SH) jointly developed a protocol for completing this review in advance, which included the following steps outlined in PRISMA guidance: 1) identifying a search strategy; 2) determining inclusion and exclusion criteria; 3) screening papers; 4) assessing methodological quality; 5) extracting data; and, 6) synthesizing results [
23].
Search strategy
Two search approaches were used: 1) electronic database searching; and 2) grey literature searching. A three concept database systematic search strategy was developed by a librarian from Public Health Ontario’s (PHO) Library Services team, which included controlled vocabulary and natural language keywords related to the concepts of health promotion (e.g., “health promotion”, “public health”, “health education”), programs (e.g., “program”, “campaign”, “initiative”) and sustainability (e.g., “sustainable”, “routinization”, “long term implementation”). Searches were limited to articles published in English after 1998. The initial search strategy was developed using the MEDLINE database. The MEDLINE search strategy was peer-reviewed by other members of PHO’s Library Services team. The search syntax and controlled vocabulary terms in the MEDLINE search strategy were then translated for execution in secondary databases. The final search strategies were run in the following five databases on June 18, 2018: 1) Ovid MEDLINE; 2) Ovid PsycINFO; 3) EBSCOhost CINAHL plus with Full Text; 4) EBSCOhost SocINDEX with Full Text; and 5) Scopus.
A grey literature search was conducted on July 7, 2018. It included grey literature repositories, custom web search engines, and a general web search (see Additional file
1). Natural language keywords from the MEDLINE search were streamlined and adapted to develop search strings for identifying grey literature. Grey literature searches were run using the following search tools: New York Academy of Medicine Grey Literature Report; two custom search engines (one which searched the websites of Canadian provincial and regional health authorities and public health units; and one which searched the websites of international public health agencies and health authorities), and a web search engine (Google Canada). The grey literature strategy expanded the search beyond academic publishing, allowing for greater scope.
The database and grey literature searches were updated on June 26, 2019 in order to capture new papers that were released following our initial search. No changes to the search strategy were made when search results were updated. See Additional file
1 for full search strategy, including MeSH terms.
Inclusion and exclusion criteria
Published and grey literature papers were included if they were published in English during the last 21 years; were about the sustainability of a HPP such as chronic disease health interventions in public health or a related setting (e.g., community-based, non-governmental or governmental organizations); explicitly used or referenced a sustainability framework or model to ground their HPP or guide their work; specifically reported on facilitators and/or barriers (or equivalent terms such as promote/inhibit) for the sustainability of a program; offered synthesis level research or primary level studies that included assessments of program sustainability and outcome data related to facilitators and barriers of program sustainability. The 21 year time frame was based on preliminary searches conducted to inform the strategy, and a decision made to coincide with the publication of Shediac-Rizkallah and Bone’s (1998) study, which continues to be cited today. For our review, we used Shediac-Rizkallah and Bone’s definition of sustainability, which defines it in three ways: 1) maintenance of health benefits from a program; 2) continuation of program activities within an organization; and 3) capacity building in the recipient community [
11]. We focused our review on the second part of the definition, “continuation of program activities within an organizational structure”. Papers that addressed sustainability as it relates to parts 1 or 3 of the definition were excluded. We chose the second part of the definition because it focuses on the sustainability of program activities. A program was considered a HPP if it met the definition from the Ontario Public Health Standards, which states that HPP are public health programs designed to achieve program outcomes that improve “population health outcomes by reducing preventable disease, injury or death, and taking action on health inequities” [
2].
Exclusion criteria included non-English language papers published before 1998, and those from countries outside of the Organisation for Economic Co-operation and Development (OECD). Papers were excluded if they were about sustainable development, sustainable food systems, or environmental sustainability; focused on sustainable organizations or sustaining systems or sectors (e.g., health care system, nursing); the program took place in a clinical or hospital setting; focused on sustainability of a partnership or coalition with no focus on program delivery; focused on sustainability of program impacts (e.g., continued health benefits for individuals after the initial program ends) and continued capacity of a community to develop and deliver HPP; no outcome data were provided; and commentaries and editorials.
The two authors (AB and SH) developed the inclusion and exclusion criteria at the outset of the project.
Screening and selection of papers
Electronic data base
Titles and abstracts of the identified papers from the search were screened by two authors (AB and SH), who first independently screened 20% of search results for inclusion and had an agreement of 86.6%. The remaining 80% was split in half with each half screened independently by one of the two authors (AB and SH). For full text screening, two authors (AB and SH) first independently screened 20% of the articles and had agreement of 90%. The remaining 80% was split by 60% (AB) and 40% (SH) and screened independently. Any discrepancies were resolved by discussion until consensus was reached. For two articles, a third reviewer was consulted to apply the criteria and discuss ratings with both authors until consensus was reached.
Grey literature
Titles and abstracts of all grey literature search results were screened for inclusion independently by the two authors (AB and SH). All included papers were full-text screened by independently by both authors (AB and SH) and consensus was reached on all discrepancies through discussion.
Quality assessment
Electronic data base
Methodological quality was assessed independently by a research coordinator (TO) who reviewed all included papers, and by two authors (AB and SH) who each reviewed half of the papers as a second reviewer, using appropriate criteria in regard to the design of the included papers. The Health Evidence Quality Assessment Tool for systematic reviews and meta-analyses was used for systematic reviews [
24]. Papers that used qualitative methodology were appraised using the Critical Appraisal Skills Programme for qualitative methodology checklist [
25]. A single cohort study was assessed using the Newcastle-Ottawa Scale for Cohort Studies [
26]. All discrepancies on individual quality ratings were discussed until consensus was reached. The selected tools allowed the researchers to rate papers as strong (8–10), moderate (5–7) or weak (0–4) based on their methodological rigour, transparency and biases. All papers rated weak were excluded from this review, with the exception of one, due to its seminal contribution to the topic area and the fact that it remains highly cited across a 21 year time frame [
11]. The ratings displayed are the final ratings agreed via consensus discussion by both authors (AB and SH) with the independent appraiser (TO) who critiqued all papers. Additional file
2 shows the ratings of the included studies listed as weak, moderate, and strong.
Grey literature
To assess the quality of the grey literature, a quality assessment tool was developed by the authors (AB and SH) using combined criteria from Caldwell [
27] and Bergeron [
28] for guidance. It included the following questions:
1)Is the methodology identified? If not, could the methodology be found within two internet clicks to additional resources?
2)Are the authors credible? (e.g., document is published by a university, government agency, author has other published work)
3)Is the rationale for the resource clearly identified? (e.g., a purpose statement or research question)
4)Is the methodology clear and transparent?
5)Did the authors use a sustainability framework? If they created their own, is it clear, and is it clear how the various components interact?
6)Are the results transferable? (e.g., to other sectors or settings)
7)Is it clear how the content could be used by practitioners?
Questions were answered with a scoring of ‘yes’ or ‘no’. If ‘no’ was answered for either of the first two questions, the paper was excluded. Remaining papers were screened with the remainder of the questions for a final score out of 6. Papers scored below 1–2 were considered weak, 3–4 moderate, and 5–6 strong. Moderate and strong papers were included and weak ones were excluded. Methodological quality was independently assessed by two authors (AB and SH) who each reviewed all papers. Any discrepancies were discussed until consensus was reached. Quality appraisal results for included grey literature are found in Additional file
2.
A data extraction table was drafted to meet the research focus of this project, and refined by discussion between the authors. Two authors (AB and SH) tested and discussed the table, which created a guide for data extraction and instructions for completing each item to ensure consistency. Two authors (AB and SH) independently conducted data extraction for all included papers to ensure that all appropriate data were extracted from the included papers. Information extracted from each paper included: author and year of publication; study design; methods; purpose; objective or research aim; definition of sustainability; sustainability framework; barriers to sustainability; facilitators to sustainability; implications for the program planning process; conclusion; implications for practice; and future directions or research. Study limitations were also extracted and recorded as part of the conclusion column. Extraction tables from both authors were reviewed line by line and a harmonized data table was created (see Additional file
3). All discrepancies were discussed and consensus was reached. Authors of the included papers were not contacted to validate data.
Data analysis
We conducted a deductive thematic analysis of our data, as per the definition provided by Vaismoradi [
29]: “a method for identifying, analysing and reporting patterns (themes) within data.” A deductive thematic analysis involves the search for and identification of common threads, and comparing them across included studies [
29]. After completion of the thematic analysis, we used Carroll’s [
30] ‘best fit’ framework synthesis methodology to code the data. This method involves using an
a priori framework and coding data from the review’s included papers against that thematic or conceptual framework [
30]. Data that cannot be accommodated within the framework require interpretation using thematic analysis techniques, with new themes generated from data not captured by the
a priori framework.
Schell’s [
5] nine domain framework was used to code barriers and facilitators extracted from the included papers. This framework was the most recent and comprehensive framework identified within the health promotion sustainability literature. Barriers and facilitators that could not be coded within one of the nine domains were discussed by two authors (AB and SH) and additional codes were developed for them via consensus. The new codes allowed for the identification of new factors based on frequency. For this review, we define a factor as a collection of related barriers and facilitators. Frequency relates to the number of times the barrier/facilitator appears in the extraction results.
Once coding of all barriers and facilitators identified in the data extraction table was complete, they were examined for relevance and importance. Relevance was determined by analysing to what extent the factors were present in PHO’s 6-step approach for planning HPP [
31], an evidence-based process used to inform and guide health promoters and public health practitioners. Step descriptions from PHO’s Program Planning Workbook were examined to determine commonalities between them and the identified factors. Factors that were similar were rated as more relevant. One author (AB) re-read all included articles and coded elements of each step with the factors.
Importance was coded by one author (SH) by re-reading all included articles and searching for instances where the authors of the included papers directly reported that a barrier or facilitator or multiple ones, were particularly important to the sustainability of the HPP. For example, authors stated that the barrier or facilitator was “the most important facilitator” [
32], “integral to sustaining practice” [
19], or “key to sustaining the HPP” [
33]. Each of these were then coded as important.
All data were compiled into one table (See Additional file
4) and ranked into one of three tiers, set out according to frequency of mention, mentions of importance and mentions of relevance assessed for each barrier or facilitator.
Discussion
This review provides health promotion and public health practitioners with an updated and practical inventory of factors that impact HPP sustainability based on a comprehensive and systematic review of published and grey literature. The results of this review support the findings of key papers in the health promotion program sustainability literature in regard to the factors for sustainability [
5,
11,
12], while adding new information to the body of knowledge on this topic. The literature search for this review covered the last 21 years, allowing for a comprehensive assessment of the sustainability literature relevant to health promotion. More than half of the included papers were rated as ‘strong’ methodological quality with the remainder being ‘moderate,’ suggesting that the body of literature on this topic is comprehensive and methodologically sound, in addition to being relevant and applicable to the current context.
We identified 14 key factors for the sustainability of HPP that could be embedded into the program planning process. Nine of these factors were from Schell’s framework [
5], and five were identified as new factors. Although we have reported on factors for sustainability as separate and distinct from one another, it should be noted that they do not always exist in isolation, and can be complex and interrelated. In this section, we discuss the factors as they related to one another, rather than in order of frequency.
Organizational capacity was the most highly coded factor in terms of frequency and importance, meaning that this factor heavily influences program sustainability. Organizations must ensure that the capacity required to fully implement and sustain their HPP is available prior to implementation of the program. Staffing, leadership and management support could be considered during the program planning stage. Factors related to organizational capacity include program implementation, as it lays the foundation for sustainability. Additionally, the factor of funding stability must be prioritized. Dedicating resources and staffing to this is one way to achieve this [
10,
15]. Having multiple strategies to obtain funds [
12] is another.
Given the strong relationship between partnerships and sustainability, partnerships with communities, organizations and stakeholders from multiple sectors should be embedded into program planning. Strategies for effective partnerships include engaging with partners who have a shared vision [
10] and similar missions [
32], dividing tasks between partners [
32], using participatory approaches [
15,
19], and building sustainability into partnership agreements [
10]. This factor had the highest mentions in relevance, reinforcing its importance in program planning. A specific strategy to involve partners is to engage them as program champions. Practitioners should give particular consideration to strategically and purposefully identify and secure the support of a champion during the planning stages of a HPP, prior to program implementation.
Strategic planning was an important factor for program sustainability and considerable time and attention should be given to it. Planning the HPP and its associated activities around an established theory or framework, for example the trans-theoretical model, helps structure the HPP activities around validated approaches [
31], thus contributing to sustainability. Assessing the environment and local context in which the program will be implemented prior to it beginning is an important component of the planning process [
10,
31]. Practically, this can be done by conducting a situational assessment or a Strengths, Weakness, Opportunities and Threats (SWOT) analysis. Practitioners should give consideration to socio-economic and political climate when conducting a situational assessment as it will likely impact the success of their HPP. As part of the situational assessment, consider the fit/alignment of the program. Lack of fit/alignment with the organization, partners and community is as likely to inhibit sustainability as fit/alignment would promote sustainability. In addition, potential adaptability of the program should be considered. Practitioners should ensure that their HPP is modifiable to meet the evolving needs of their local context, community, and health system.
Program evaluation contributes to program sustainability in several ways. Identifying and selecting specific outcomes and indicators to measure sustainability is important for overall sustainment, as this clarifies for practitioners what needs to be sustained, how elements of the HPP can be sustained, and indicates when sustainability has been achieved. As demonstrating the public health impacts of the HPP was also identified as a factor for sustainability, particular focus should be given to data collection and analysis, and routinely planning for evaluations during the program lifecycle. Having the appropriate resources to ensure sound data collection and analysis is important because it allows for objectively demonstrating to stakeholders and funders whether the program is being implemented as intended, is having its intended impact, and if it is worth sustaining. As such, practitioners should give ample attention to how they will evaluate the sustainability of their HPP, as well as to securing the appropriate tools to collect and analyse data. It may be necessary to prioritize data collection, if there is limited time or resources.
Another key consideration for sustainability is the role of capacity building services for program staff. Capacity building is an important factor because it provides program staff with the necessary skills and knowledge to properly implement the HPP. Attention should be given to providing staff with ongoing technical assistance to support them as they implement the HPP. As well, providing educational opportunities that further their understanding of the issue being addressed by the HPP improves buy-in and quality of service delivery. Providing ongoing skills and professional development opportunities improves staff retention which allows for program continuity and improved organizational memory. When implemented strategically capacity building can help during periods of uncertainty by providing the necessary support system for staff and participants. It also helps improve staff retention, which is valuable for sustainability because it improves organizational memory.
This review provides several practice recommendations in addition to the 14 factors. Particular attention should be given to identifying which specific component of the HPP should be sustained. Shediac-Rizkallah and Bone’s definition of sustainability suggests that three components could be sustained: health impacts of the program; the program itself; or community capacity to sustain the program. Second, the sustainability literature commonly cites that there is variation in how sustainability is defined, when it is defined at all. This creates confusion and suggests a lack of conceptual understanding of what sustainability is and what is required to sustain programs. Adding to this, a number of terms related to sustainability are used in the literature [
7]. For example, ‘institutionalization’ refers to the continuation of a program or program activities within an organization beyond the funding period [
37], while ‘routinization’ refers to the program (or its components) becoming established on a durable basis (i.e., routine) [
7]. This issue has plagued the health promotion field for decades as the lack of a clear definition makes it difficult to plan, implement, and evaluate whether sustainability has been achieved. Therefore, it is recommended that practitioners define sustainability as it relates to their HPP prior to attempting to sustain the program. Practitioners could use or adapt a definition commonly-available in the literature, such as Shediac-Rizkallah and Bone [
11] and Scheirer [
12], or develop a definition that is specific to the program. The critical point is that sustainability needs to be clearly defined in some way. Once a definition of sustainability has been selected or developed, a sustainability framework or model can provide direction on how sustainability can be achieved and evaluated. This review identified 13 sustainability frameworks. Two grey literature workbooks [
34,
35] provide step by step models and practical worksheets to assist practitioners in planning sustainable programs. Program planners should, within their local context, consider factors that are key to implementation and likely influence outcomes [
47], and select a framework that best fits to use or adapt.
Further research could include additional work to validate the 14 factors identified in this review to assess which factors are most important and their applicability in different practice contexts. It may be useful to consolidate objective indicators for sustainability to help practitioners assess whether their sustainability efforts have been achieved. Also, given the number of frameworks and models that exist in the field, a concerted effort to organize health promotion and public health practitioners around a single set of measures to assess sustainability would be beneficial.
Review limitations
There may be other relevant documents beyond the published articles and grey literature searches, not available in the public domain or published in other languages on this topic, which could not be included in this review due to time and resource restrictions. Given this review’s focus on health promotion and public health, there is the possibility that our search strategy missed literature from related fields such as implementation science. However given the focus on health promotion and public health, this review assessed the state of literature closest to that context based on the most relevant and current papers. A potential limitation is the manner in which frequency, importance and relevance of facilitators and barriers was assessed. Although it served the purpose of our review, the assessment of relevance involved a subjective interpretation of the literature in some cases. However, including importance and relevance added to the depth and strength of our findings by allowing us to go beyond simply reporting results based on frequency of codes.
Conclusions
This systematic review identifies 14 factors that influence the sustainability of HPP. Nine factors were previously identified in the sustainability literature: organizational capacity; partnerships; strategic planning; funding stability; program evaluation; communications; political support; program adaptation; public health impacts. Five new factors were identified: fit/alignment; capacity building; program implementation; program champions; and socio-economic and political climate. Specific barriers and facilitators as they relate to each factor were highlighted. This paper expands on established and persistent sustainability factors, adding new dimensions to for a more complete picture of program sustainability. A unique aspect to these findings is that they build on the existing literature by providing new considerations for the factors that impact HPP sustainability which can be used by health promotion and public health practitioners to promote the sustainability of their HPP. A strength of this review is the inclusion of grey literature, and the development of quality assessment criteria in order to provide transparency around the quality of the included papers.
The findings also highlight the need for practitioners to clearly define sustainability and identify what needs to be sustained early on during the planning process: the health benefits of the program, program itself (or its components), or capacity of the community to continue the program. In addition, practitioners should use a sustainability framework to guide their sustainability planning efforts; several were identified in this review. Given the importance of HPP in improving population health, reducing health inequities, and reducing health care costs, health promoters and public health practitioners should strive to prioritize the sustainability of their programs by actively planning for sustainability throughout program implementation and the program lifecycle. The factors identified in this systematic review could be incorporated into the design of HPP to increase their likelihood of sustainment.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.