Background
In recent years, in response to the increasing awareness of the complexity of many public health problems, there has been growing interest in the role of systems-based approaches in public health. In 2007, the UK Foresight map [
1] presented a pioneering portrayal of the complex web of obesity causation. In the same year the American Journal of Community Psychology devoted an edition to systems-thinking. In 2008 Mabry et al. [
2] outlined the strategic vision of the Office of Behavioural and Social Sciences Research at the National Institutes of Health, listing systems science as one of four key programmatic directions, and the importance of systems thinking was also noted in the 2011 and 2015 Lancet Series on Obesity [
3,
4].
In 2010 a number of evidence reviews were undertaken for the National Institute for Health and Care Excellence (NICE) [
5‐
7], intended to inform the development of NICE guidelines on the prevention of obesity using a whole system approach (WSA) (note that the scope of the work was changed and instead resulted in the development of NICE guidelines on whole of community approaches) [
5,
8].
The Garside et al. [
5] review - one of those commissioned by NICE – which aimed to identify key elements of a WSA to obesity, reported that an “authentic” WSA draws on complexity science and complex adaptive systems. Although there is no consensus on a formal definition of “complex adaptive systems”, there is broad acceptance they contain: heterogeneous interacting elements; an emergent effect that is different from the effects of the individual elements; and persisting effects over time that adapt to changing circumstances [
9]. The NICE reviews [
5‐
7] did not find any “authentic” WSAs, and the definition was therefore widened to include those programmes that were designed to work at multiple levels among multiple agencies in a locality. Using this definition, they identified ten features of a WSA to tackle obesity [
5] (Table
1).
Table 1
10 features of a systems approach to tackle public health problems, adapted from NICE [
9] and Garside et al. [
5]
Identifying a system.
| Explicit recognition of the public health system with the interacting, self-regulating and evolving elements of a complex adaptive system. Recognition given that a wide range of bodies with no overt interest or objectives referring to public health may have a role in the system and therefore that the boundaries of the system may be broad. |
Capacity building
| An explicit goal to support communities and organisations within the system. |
Creativity and innovation
| Mechanisms to support and encourage local creativity and/ or innovation to address public health and social problems. |
Relationships
| Methods of working and specific activities to develop and maintain effective relationships within and between organisations. |
Engagement
| Clear methods to enhance the ability of people, organisations and sectors to engage community members in programme development and delivery. |
Communication
| Mechanisms to support communication between actors and organisations within the system. |
Embedded action and policies
| Practices explicitly set out for public health and social improvement within organisations within the system. |
Robust and sustainable
| Clear strategies to resource existing and new projects and staff. |
Facilitative leadership
| Strong strategic support and appropriate resourcing developed at all levels. |
Monitoring and evaluation
| Well-articulated methods to provide ongoing feedback into the system, to drive change to enhance effectiveness and acceptability. |
Aims & objectives
The aim was to undertake a systematic review of national and international published evidence on WSAs targeting obesity, other public health areas and areas outside public health (such as social care, crime and justice), to understand what is known about WSAs and how they can be implemented in practice.
Review questions
1.
What has been done in terms of a WSA to obesity, and other complex public health problems, and how effective have these been?
2.
What elements of a WSA are effective or not effective in (a) obesity (b) other areas of public health (c) areas other than public health?
3.
What are the barriers and facilitators to implementing a WSA in (a) obesity (b) other areas of public health (c) areas other than public health?
4.
What is the evidence on cost-effectiveness of WSAs in (a) obesity (b) other areas of public health (c) areas other than public health?
Methods
This systematic review follows standard methodological guidelines [
10,
11].
Search strategy
A broad, sensitive search strategy was designed. The following databases were searched from January 1995 to September 2015 using a combination of text and Medical Subject Headings (MeSH terms): MEDLINE, CINAHL, Social Science Citation Index, The Cochrane Library (includes CENTRAL, DARE, NHSEED, HTA and INAHTA databases), PsycLIT/ PsycINFO, DoPHER, TRoPHI and IDOX.
In February 2017 an additional update search was run in MEDLINE, using the same search strategy applied from January 2015 to February 2018.
Key search terms included:
(i)
“whole systems approach” and related terms such as holistic; cross-sector; systems-based approach; multi-strategy approaches etc.
OR
(ii)
Terms related to relevant initiatives such as: Healthy Cities; Healthy Towns; Together Let’s Prevent Childhood Obesity (EPODE); Change4Life etc.
We also manually searched the websites of relevant organisations such as: Department of Health; Public Health England (PHE); Local Government Association (LGA); World Health Organisation (WHO); National Institute for Health and Care Excellence (NICE); Association for the Study of Obesity (ASO); National Obesity Forum etc.
The full search strategy is available as Additional file
1.
Study selection
Titles and abstracts from electronic database searches were transferred to EPPI-Reviewer 4 [
12], and screened against the inclusion criteria (Table
2). A random 20% of titles and abstracts were screened by all the review team, and once good agreement (80% or more) was reached, the remaining 80% were allocated to a single reviewer. Any queries were discussed within the review team and if agreement could not be reached, were referred to the local steering group for decisions. Records which potentially met the inclusion criteria, including those found on organisational websites, were retrieved in full and assessed for inclusion.
Table 2
Inclusion criteria
Population | Any population where a WSA has been used, at local, regional, national and international level | |
Intervention | WSAs, defined as those that: • Consider, in concert, the multifactorial drivers of overweight and obesity, as outlined by Foresight [ 1], public health or the social determinants of health [ 13]; • Involve transformative co-ordinated action (including policies, strategies, practices) across a broad range of disciplines and stakeholders, including partners outside traditional health sectors; • Operate across all levels of governance, including the local level so that such approaches are reinforced and sustained, and • Identify and target opportunities throughout the life course (from infancy to old age) | • Multiagency partnership working across sectors e.g. health & social care, but not at more than one level; case management initiatives focused on individuals or individual families; • Studies which looked at only one part of a WSA (i.e. one specific intervention delivered as part of a wider approach). |
Comparator interventions | Any or none | |
Outcomes | Review questions 1 and 2: • Health outcomes, e.g. weight, Body Mass Index (BMI), type 2 diabetes, diet and nutrition, physical activity, psychological well-being & quality of life; co-morbidities related to obesity, reductions in health inequalities, reductions in premature morbidity and mortality, cardiovascular disease and obesity-related cancers. • Organisational outcomes e.g. cross-sector collaboration; new partnerships; environmental changes; resource allocation; leadership etc. • Process outcomes, e.g. what each project aimed to achieve and barriers and facilitating factors associated with achieving or not achieving those aims. Outcomes may be at individual, local, regional or national/ federal/ principality level. |
Review question 3
Process and implementation outcomes e.g. training, recruitment, sustainability, people’s views on barriers and facilitators to implementation of WSAs. |
Review question 4: Cost, cost-effectiveness, cost-benefit or cost-utility. |
Study designs | Review questions 1 and 2: primary research or evaluation studies. These may be randomised controlled trials (RCTs) or non-RCTs, natural experiments, before and after studies, or mixed methods evaluations (including case study approaches). Review question 3: process evaluations (qualitative or mixed method studies). Review question 4: cost-effectiveness, cost-benefit or cost-utility studies. |
Data were extracted from included articles by one reviewer into a piloted electronic form, and checked by the lead reviewer. Queries were resolved as above. We extracted data into the following fields: study details; study design; setting; population (including PROGRESS-Plus indicators [
14]); public health or other issue; intervention; comparator (if appropriate); outcomes; findings; reviewer comments.
We also assessed all included studies against the ten features for WSAs for obesity, in the working definition prepared by Garside et al. for the NICE guidance [
5].
Validity assessment
Two reviewers carried out validity assessment of included articles using checklists developed for quantitative and qualitative study designs of public health interventions [
15]. These were adapted from the National Institute for Health and Care Excellence Public Health methods guidance, and the Critical Skills Appraisal Programme [
16,
17]. Studies were given a quality rating based on how many criteria they met on the appropriate checklist.
Synthesis
Due to the substantial clinical and methodological heterogeneity of included studies, a narrative approach to synthesis was chosen [
18]. Evidence on health and non-health outcomes is presented as a descriptive thematic summary, grouped within each review question according to whether it relates to obesity, other public health or non-health issues, with the most methodologically robust evidence presented first.
We also looked at whether there was any association or pattern between the direction of reported health effects and the number of WSA features [
5] that a study met, using the cross-tabulation function in EPPI-reviewer and carrying out a Fisher’s exact test in IBM SPSS 24 statistical software.
Qualitative evidence on barriers and facilitators to implementation and delivery of WSAs was summarised using a framework synthesis approach, allowing themes to emerge inductively from the included studies, within the framework of ‘barriers’ and ‘facilitators’. The framework was agreed within the review team, and data were aggregated according to the major themes.
Discussion
Although 65 studies met the broad inclusion criteria, the heterogeneity of studies in terms of different outcomes, research designs, populations and interventions, prevented the data from being analysed statistically. It is also worth noting that most of the included studies did not set out to implement or evaluate a WSA. Furthermore, reporting of intervention and approaches in published articles was usually brief and lacked detail. This underlines the lack of robust evidence in this area and the need for further research to expand and support the suggested associations, in order to increase our understanding of how WSAs can be implemented. Nevertheless, it does seem reasonable that programmes in a community setting that adopt the principles of the ten features identified by Garside et al. [
5] such as developing relationships and engaging stakeholders, ensuring the approach is robust and sustainable and having supportive leadership, are more likely to be successful than programmes that do not adopt these principles. Similarly, it is also feasible that all ten of the features [
5] are associated with positive health effects, however in relation to current thinking around systems approaches, these ten features do not comprehensively describe a WSA.
The review also found consistent evidence from process evaluations that ownership and commitment, strong relationships between stakeholders, and allowing sufficient time to build relationships, trust and community capacity are all key to building a successful WSA.
There is recognition in the literature that several public health problems, including obesity, are complex issues requiring system-based approaches [
84,
85]. However, although the concepts and terminology have existed for some decades the degree to which the field has progressed is debatable, partly due to the multitude of ways in which the language surrounding systems approaches is used.
Systems science “
refers to a range of methods, composed largely of mathematical or computational modelling and simulation, that enable the user to explore complex problems by addressing both interactions between components of a system and the behaviour of the system over time” [
2]. On the one hand, there have been significant advances in the application of a range of systems science approaches to a variety of obesity-related public health issues using techniques such as micro-simulation, social network analysis, agent-based modelling and system dynamics modelling. On the other hand, Garside et al. (2010) reported that the term “whole system approach” was found to represent approaches informed by theory about complex systems which propose new ways of organising, managing and evaluating activities, and also as terminology within a long list of approaches which referred to cross-disciplinary, multi-agency, multi-level community activities aimed at addressing health concerns affected by complex socio-economic conditions [
5] and which rarely, if ever, encompass the use of the system science methods as described above.
Similarly to Garside et al. [
5], this review found little evidence of systems science or systems thinking in included studies. Few programmes had been explicitly designed and delivered with an a priori recognition of the public health issue as a system and thus rarely approached implementation from a perspective encapsulating a systems approach – the implications of which are significant for the reporting and evaluation of interventions. Furthermore, because interventions to date have not been undertaken with a systems-thinking lens or set out to take a WSA at the outset, there has been little recognition of properties inherent in a complex system (e.g. nonlinear relationships, feedback loops, dynamic interacting elements) and little attention afforded to the reporting of the central underlying operational mechanisms (e.g. improving networks, developing a common agenda, developing relationships), as suggested by Allender et al. [
86]. Hawe et al. [
85] noted that although the majority of health promotion programmes claim to take an ecological approach, in reality this is realised as multiple interventions at multiple levels with “
little theory put forward about how these levels impact the unfolding of the intervention or how they affect intervention outcomes”. Moreover, several authors have noted that implementation of a suite of activities across multiple-settings or multiple-levels is not necessarily the same as taking a “systems approach” [
5,
85,
87]. Systems approaches focus on the context into which the intervention is introduced [
85] and relate to intervening directly on the feedbacks, structures and goals of a system [
87]. Considered in this light the evidence highlights the limited progress that has been made in the practical implementation and evaluation of WSAs to public health issues to date. What is needed a framework to incorporate the complexity of systems approaches into public health research, policy and practice [
88].
Garside and colleagues in 2010 also looked for examples of a whole system in action, finding only eight articles on the effectiveness of community wide programmes displaying features of a WSA to prevent obesity, none of which were undertaken in the UK and all of which targeted children below 14 years of age [
6]. Most findings favoured the interventions but improvements were found to be small and not always statistically significant. The inconclusive evidence relating to the 10 NICE guidance features in the present review may be because these emerged from a systematic review with the aim of providing a working definition of a WSA to obesity prevention, which did not find any “authentic” WSAs, and the definition was therefore widened to include those programmes that were designed to work at multiple levels among multiple agencies in a locality [
5]. So, meeting all ten of these criteria still does not indicate an “authentic” WSA. In addition, although studies may have met many or all of the 10 NICE guidance features in their description of their proposed intervention, in studies which presented effectiveness outcomes there was little evidence of whether these interventions had been implemented with fidelity to the WSA framework.
Limitations
Our search strategy was designed to look as widely as possible to minimise the risks of missing valuable material. Although 20% of titles and abstracts were double screened, screening of the remaining 80% was limited to a single reviewer, an “acceleration strategy” recommended for rapid reviews [
89]. While this was a pragmatic necessity, it does potentially introduce bias and human error, which may have resulted in some relevant studies being missed.
Methodological details of included studies were, on the whole, poorly reported, which limits our confidence that the findings are not at significant risk of bias.
Only 11 of the included studies were UK-based, however their findings might be expected to be generalisable to the UK context and three of these [
32,
53,
65] met all ten of the NICE criteria for a WSA.
Few studies targeted population groups known to be at higher risk of obesity and other public health issues, such as black and minority ethnic groups and people with low levels of education or low socioeconomic status. This limits the usefulness of the findings.
Thirteen of the 65 included studies were judged to meet all ten of the criteria for a WSA proposed by Garside et al. [
5] in an earlier review. However, the heterogeneity of studies in terms of interventions, outcomes, research designs, and populations prevented the data from being analysed statistically.
Conclusions
Using a broad lens, this systematic review aimed to obtain a greater insight on the effectiveness of WSAs and how they can be implemented in practice. Evidence exists to demonstrate promise with interventions working towards systems approaches. This was most clearly demonstrated through a suite of WHO Healthy Cities process evaluations and evidence from whole of community approaches. A range of positive health outcomes were reported, but there was little evidence of an association between specific WSA features and health impacts. Evidence of systems science and systems thinking was less clear, even in the most “joined up” approaches, similar to the findings of the series of reviews carried out for NICE in 2010.
Recommendations
It is important to note that most of the included studies did not report that they set out to implement or evaluate a WSA, and reporting of interventions and approaches in published articles was usually brief and lacked detail. This underlines the lack of evidence in this area and the need for further research. Whilst several learnings on multi-level, community wide interventions have been obtained which are likely to be relevant to the implementation of a true WSA (e.g. evidence about barriers and facilitators to implementing such approaches), it is also evident that evidence of how to operationalise a whole systems approach to address public health problems is still in its infancy. We recommend that future researchers and policy makers develop consistency in language and an agreed definition of what a WSA should be in relation to obesity. Future research studies into the effectiveness of WSAs should look across sectors and should include detailed descriptions of interventions including approaches, and embedded process and economic evaluations, as recommended by existing guidance on developing and evaluating complex interventions [
90,
91].
Acknowledgements
This report was commissioned from Leeds Beckett University by Public Health England as part of a larger project looking at Whole System Approaches to Obesity. We would like to thank Ben Mitchell for designing the search strategy and carrying out the electronic database searches, and Leonie O’Dwyer and Robin Kaye for their help with data extraction and validity assessment of included studies. We would also like to thank the Interlibrary loans team (Susan Ellis and Karl Derbyshire) in the Sheila Silver Library at Leeds Beckett University, for their excellent help in locating articles. We would also like to acknowledge the support given by the PHE Whole Systems Advisory Group, chaired by Jim McManus, and thank the following individuals for their helpful input: Adrienne Cullum, Diet, Obesity and Physical Activity Health Improvement Directorate PHE; Jim McManus, Director of Public Health, Hertfordshire County Council and Chair, PHE Whole Systems Advisory Group; Verity Hawkes, Higher Nutrition Scientific Officer, Diet & Obesity, Public Health England.