Background
Implementation is ‘the Achilles heel of innovation’ ([
1], p. 10) and is often defined as an evidence-to-practice gap [
2‐
5] in which successful implementation of evidence-based interventions is fraught with challenges [
4,
5]. If public health program and policy interventions are not implemented effectively, they will not have their intended effects on improving population health or reducing health inequities. Furthermore, the cost to the system will be considerable in a time of scarce public health resources [
6,
7]. Given opportunity costs, a poorly implemented intervention can quickly erode policy and practice support, creating more challenges to ‘getting it right’ over the longer term [
8].
Implementation research, often equated with knowledge translation [
9,
10], has been conducted across many disciplines [
1,
11] to document the frequency of unsuccessful implementation of policies and programs, identify factors influencing successful implementation, evaluate the effectiveness of implementation strategies, and develop theoretical frameworks to analyze or guide the implementation process. Systematic reviews of implementation studies have been conducted in several fields [
12‐
15], but when we began this project, we had not located any comprehensive systematic reviews of implementation specific to public health. Because of public health’s population focus and location in the community, there may be unique features of public health systems and interventions that raise questions about the applicability of the broader health care implementation literature to public health interventions [
16,
17].
The UK’s National Institute for Clinical Excellence, in their work to develop public health guidance [
17], identified three problems with Cochrane-style systematic reviews for public health: (1) the breadth of the public health evidence base is vast, encompassing social, political, economic, and cultural factors; (2) explanations of effects in public health are multi-level; and (3) the length of the causal chain in public health interventions is extended, not proximal or direct as in clinical interventions. These factors make it very difficult to utilize randomized controlled experimental designs in the study of public health interventions.
To elaborate, public health has an expansive interdisciplinary evidence base that draws on diverse data types as well as on social science theories and methodologies that do not make it to the top of most evidence hierarchies (e.g., Cochrane), despite their legitimacy in many other disciplines. This proposition was supported by Kelly et al. [
17] who found that search strategies and inclusion criteria in systematic reviews immediately eliminated significant bodies of evidence. For example, one review on knowledge translation strategies in public health located 346 potentially relevant publications but only five met the inclusion criteria [
18] thereby excluding many articles that could provide relevant guidance on implementation to public health decision makers. Others have argued similarly that conventional systematic reviews are often not relevant in public health because there may be ethical constraints in randomly selecting or assigning people to experimental conditions [
19].
An emerging methodological alternative is realist review or synthesis [
20‐
23], which may be more helpful to knowledge users [
21,
22] in public health for synthesizing evidence on effective implementation [
23‐
26]. Realist reviews are now widely accepted in the field of research synthesis and are increasingly being published in journals like Systematic Reviews and Implementation Science.
Realist review allows for inclusion of a broad range of study designs with both qualitative and quantitative data. It is distinguished from other reviews by its focus on causal mechanisms in interaction with context to produce outcomes. In contrast to conventional reviews that focus on intervention effectiveness, realist synthesis is a theory-driven approach that aims to explain how and why observed outcomes occur. They focus explicitly on what works, for whom, and in what contexts. As a theory-driven approach, there are at least three levels of theory involved. A realist synthesis begins with an initial or ‘rough program theory’ [
26] which is a general theory of the intervention(s) or program that lays out what is being investigated and how it is expected to work. It is not specified in realist terms—that is, with respect to contexts, mechanisms, and outcomes. The initial program theory guides the search, selection, and synthesis process but is continually refined throughout the review to create a realist program theory that specifies the relevant contexts, mechanisms, and outcomes and their configurations. Ultimately, the refined realist program theory is finalized as a middle-range theory. In realist synthesis, this is defined as a theory that is ‘detailed enough and close enough to the data that testable hypotheses can be derived from it but abstracted enough to apply to other situations’ ([
26], p. 12).
Realism is the philosophy of science underlying realist synthesis. Pawson states that realism is: ‘…a methodological orientation, or a broad logic of inquiry that is grounded in the philosophy of science and social science’ [
22]. Realists acknowledge the existence of an external reality that has an influence on human action. The notion of ‘mechanism’ is thus central in realism for explaining the relationship between the social world (context) and human actions or behaviour (outcomes) [
27]. Realists argue that mechanisms have generative causation, or causal force. In realism, mechanisms can be defined as
… underlying entities, processes, or [social] structures which operate in particular contexts to generate outcomes of interest. Here ‘entities’ may refer to things such as norms or belief systems, ‘processes’ are sequences where later events depend on earlier ones, and social structures may refer to things such as gender, class, or cultural patterns of relationships ([
26], p. 5).
The assumption behind realist synthesis is that an intervention will trigger mechanisms differently in different contexts (e.g., in different health authority or health unit organizational structures) to produce different outcomes (e.g., variable degrees of success in implementation [
27]). In synthesizing the evidence, we seek to explain the interrelationships among context (C), mechanism (M), and outcomes (O) (i.e., CMO configurations or CMOCs). The locus of comparison across interventions is the mechanism(s), which may or may not be activated in particular contexts, and may or may not lead to the projected outcomes. The central question in realist synthesis is, What are the mechanisms that cause desired outcomes to occur and in what contexts are they triggered [
26]?
The need for a realist synthesis of the research on implementation of public health interventions was identified by our research team comprising researchers and knowledge users (i.e., public health decision makers and practitioners) across two provinces. This team came together in 2007 to develop a program of research focussed on studying the implementation and impact of public health renewal policies in both provinces. Specifically, these policy interventions were the British Columbia Core Public Health Functions Framework [
28] and the Ontario Public Health Standards [
29]. Our findings in one study [
1] suggested variable implementation within and between provinces influenced by unique contextual factors. As such, our team identified the need to undertake a realist synthesis that would provide knowledge users with timely, relevant, and usable information to guide implementation of subsequent public health policy and program interventions. This paper presents the protocol for our realist review.
Discussion
Evidence-based public health requires knowledge, not only about effective interventions but also about strategies for successful implementation. Public health interventions are often complex and context-sensitive making knowledge about effective implementation critical to achieve the public health goals of improving population health and promoting health equity. Although there is some literature on the implementation of public health interventions, there is no comprehensive synthesis that encompasses the full range of research outputs. Thus, there is limited information in a useable synthesized form for public health decision makers, program planners, and practitioners.
Ineffective implementation wastes scarce resources and is neither affordable nor sustainable. Travis and colleagues [
45] have identified that developing tools to support and ensure effective implementation is one of six essential health stewardship sub-functions of Ministries of Health. This synthesis will provide evidence needed for governments and local public health agencies to identify and support effective strategies for implementing public health policy and program interventions while taking into account the variable context of public health structure and organization.
The need for this realist synthesis was identified and initiated by our knowledge user partners based on challenges they have experienced, or are anticipating, in implementing public health interventions. This entire project reflects an integrated knowledge translation approach [
34] using a collaborative process among knowledge users and academic researchers. Conventional approaches to knowledge synthesis such as systematic reviews are often not well suited to more complex public health interventions [
17,
19]. Thus, we believe that a realist synthesis is the most appropriate approach to synthesize knowledge about what is necessary to ensure successful implementation of public health interventions. We anticipate that in this synthesis we will generate useable and relevant information for policy makers, program planners, and practitioners that will contribute to better understanding the context and process by which effective implementation of public health interventions can be achieved.
Acknowledgements
This study is funded by the Canadian Institutes of Health Research (CIHR), Knowledge Translation Unit, through a Knowledge Synthesis grant (FRN# KRS 138213) to M. MacDonald, B. Pauly, and A. Paton (Principal Investigators). It builds on previous work done in a CIHR Emerging Team Grant (FRN #92255) to M.MacDonald, T. Hancock, B. Pauly, and R. Valaitis (Principal Investigators) and on concurrent work being conducted in a CIHR Pathways to Health Equity Grant (FRN#116688) to B. Pauly, M. MacDonald, T. Hancock, and W. O’Briain (Principal Investigators). During the period when much of this background work was completed, including the preparation of the CIHR proposal for this realist synthesis, M. MacDonald was supported by a CIHR/Public Health Agency of Canada Applied Public Health Chair Research Award (FRN # 92365). A. Kothari was partially supported by a CIHR new investigator award (MSH 95370). K. Schick-Makaroff was partially supported by a postdoctoral fellowship from the Kidney Research Scientist Core Education and National Training Program (KRESCENT; KRES110011R1). R. Valaitis was supported by the Dorothy C. Hall Primary Health Care Nursing Chair at McMaster University.
We would like to acknowledge the exemplary work of our library scientist Carol Gordon for her substantial contribution to developing, piloting, and revising the search strategy and conducting the search. We are grateful to PhD students Renee O’Leary (University of Victoria) and Adinet Lock (University of South Africa) for their involvement in piloting the inclusion criteria and the screening process. We would particularly like to thank Trevor Hancock for his leadership in developing the larger research program of which this study is a part. Without his contributions, this study would not have taken place. Thanks also to Beth Jackson, Cheryl Martin, Chris Buchner, and Trevor Hancock for their important contributions to and support of the study. Finally, we are grateful for the support and contributions of Arlene Paton, Assistant Deputy Minister, BC Ministry of Health for her role as the Principal Knowledge User on the project. Her support and that of the BC Ministry of Health allowed Cheryl Martin and Warren O’Briain to participate in this study.
Competing interests
All authors except Geoff Wong declare that they have no competing interests. Dr. Wong is an Associate Editor for the Systematic Reviews Journal, to which this manuscript is submitted. The funding body had no role in the design or conduct of this review.
Authors’ contributions
MM is principal investigator on the project, BP is Co-Principal Investigator, GW is methodological consultant, HS is Research Coordinator, and they contributed to the conceptualization of the study, writing of the grant proposal, revision of the original protocol and development of the initial program theory, first draft of the paper, provision of feedback on the first draft of the paper, and revision of the draft for substantive content. KSM is a co-investigator and contributed to the conceptualization of the study, writing of the grant proposal, revision of the original protocol and development of the initial program theory, provision of feedback on the first draft of the paper, and revision of the draft for substantive content. TvR is Project Coordinator, AK and RV are co-investigators, HM, VL, and ST are knowledge users (KUs), and they contributed to the revision of the original protocol and development of the initial program theory, provision of feedback on the first draft of the paper, and revision of the draft for substantive content. WO is a KU and contributed to the conceptualization of the study, provision of feedback on the first draft of the paper, and revision of the draft for substantive content. SC is a co-investigator and contributed to the conceptualization of the study, writing of the grant proposal, revision of the original protocol and development of the initial program theory, provision of feedback on the first draft of the paper, and revision of the draft for substantive content. KDS and MW are KUs and contributed to the provision of feedback on the first draft of the paper and revision of the draft for substantive content. All authors read and approved the final manuscript.