Background
It is widely accepted that the transfer of new knowledge from public health research into policy and practice is far from optimal. Government agencies, including the National Institute of Health in the USA emphasise the need for widespread dissemination of evidence-based interventions to help bridge the gap between research and practice [
1]. There is a growing body of literature investigating the translation of public health research into practice [
2‐
4]. Five main stages of building evidence in public health have been proposed: Stages one and two,
Problem definition and
Solution generation relate to program development; stage three,
intervention testing, represents process and impact evaluation to determine program efficacy or effectiveness; and stage four,
intervention replication, refers to subsequent studies in which effective programs are adapted for other settings to determine if and how similar outcomes can be achieved in different places and populations [
5]. Finally stage five,
intervention dissemination, focuses on the scaling up of an effective program to population level to maximise public health impact [
5].
Translational research in public health has been defined as studies that focus on stages four and five, that is replication and scaling up of effective interventions [
5]. Scaling up is the process by which health promotion interventions shown to be effective in controlled conditions or on a small scale are expanded into real world practice [
6]. There is growing interest in the concept of ‘scaling up’; however existing literature to date has been limited in focus, for example, investigations of conceptual frameworks [
6‐
8] or case studies of scaled up programs in low income countries [
8‐
10]. There are relatively few examples of published studies reporting on the scaling up of effective public health interventions into practice [
11‐
14].
In the emerging field of obesity prevention in young children much of the research conducted to date is in the early stages of intervention development and efficacy testing (stages one to three), with a lack of effective population wide programs, particularly in young children 0–5 years [
15‐
18]. Further, reporting of external validity information such as selection and representativeness of settings, intervention characteristics and delivery costs and program sustainability is poor in existing intervention studies [
17,
19] and in systematic reviews on the topic [
20]. Given the extent of child obesity as a public health problem, there is currently limited practice-relevant information for policy makers and practitioners to inform decisions about how effective programs can be disseminated ‘at scale’.
This paper provides a case study of the scaling up of the Melbourne InFANT Program (herein referred to as the InFANT Program), a group-based obesity prevention program that was effective in improving child and maternal diet, parental feeding behaviours and sedentary behavior in children [
21]. The aim of the study was to explore the key factors influencing the scaling up and translation of this program into routine practice from the perspective of the main players involved, including researchers, policy makers and implementers. This will provide much needed insights into the ‘scaling up’ process and key lessons for public health researchers, policy makers and practitioners to inform the dissemination of other obesity prevention programs targeting young children into practice.
Study context – The InFANT Program
The InFANT Program was a cluster randomised controlled trial (RCT) targeting first time mothers through existing universal care services which trialled the efficacy of a low dose program (six sessions delivered by dietitians quarterly over 15 months, commencing when Infants were 3 months of age) to improve parent and child diet and physical activity and to reduce sedentary behaviours [
22]. This study was informed by qualitative research with the target group [
23] and by two systematic reviews [
15,
24]. At follow up when children were 18 months old, the Program did not change children’s growth, but did improve aspects of child’s diet and sedentary behaviours [
21]; improved child diet quality [
25]; improved water and vegetable intakes in sub-groups [
26]; increased maternal knowledge and improved preferred feeding behaviours [
27]; and improved mother’s dietary patterns [
28].
Dissemination context
The Program was delivered as part of The Victorian Department of Health and Humans Services (referred to throughout as ‘the Department’) prevention platform taking a complex whole of systems approach to reducing chronic disease risk. This approach involves delivering multiple strategies, policies and initiatives at both the state and local levels to target individuals in places where they spend their time, including childcare centres, schools, workplaces, food outlets, sporting clubs, businesses, local governments, health professionals and more to create healthier environments for all. This has shifted action from projects, small in scale, to prevention at scale delivered by multiple stakeholders, organisations, and sectors together with a community led placed based approach in 12 prevention areas across the state. Prevention areas were generally defined by local government area boundaries, however in some cases prevention area incorporated more than one local government area.
High quality health promotion programs have long played a significant role in health promotion, and with an opportunity to implement a complex system approach to prevention in Victoria, an opportunity emerged to re-think how programs can be delivered to contribute to prevention system change. The Department provided the opportunity for the 12 prevention areas to select quality health promotion programs using a selection criteria developed by the Department (Additional file
1) to support this decision making process. A list of recommended programs was created, which included the InFANT Program to fast track and support this process.
Dissemination process
The InFANT Program researchers were funded by The Department to prepare program materials for dissemination. This included a facilitator manual, a parent handbook, a program website (
www.infantprogram.org) and a guide for program implementation. A one day training program was developed and delivered by InFANT Program research staff to facilitators. Facilitators included dietitians, Maternal and Child Health (MCH) nurses and parent support workers. MCH nurses in Victoria provide a universal free service to parents of children 0-6 years and parent support workers typically work alongside MCH nurses to facilitate group programs such as first time parent groups. Prevention area staff were responsible for coordinating the implementation of the Program in each locality in partnership with key stakeholders and delivery agents.
Methods
A case study approach was used to explore factors influencing the translation of the InFANT Program into routine practice. Case study methods are appropriate for answering ‘how’ and ‘why’ questions and when the phenomenon of interest (research translation) is embedded within a real-world policy and practice context [
29]. The study consisted of a case series of five areas implementing the InFANT Program along with interviews with research staff and policy makers involved in the translation efforts across sites.
The study methods were informed by a constructionist epistemology. This assumes that knowledge is constructed and shaped by people’s perception, that phenomenon can only be understood in the context in which they are studied, and that truth is a matter of consensus amongst informed constructions, not a correspondence with an objective reality [
30]. This was applied to the study by collecting detailed information on contextual factors influencing participants’ perceptions and recognising that the findings are co-created by participants and researchers and not an objective truth to be discovered.
Recruitment
The sampling frame for the study sites were prevention areas that had been implementing the Program for at least 6 months at the commencement of data collection. The 6 month timeframe for implementation was selected to allow areas to have had some experience of key implementation issues. As of January 2015, eight out of the 12 prevention areas choose to deliver the Program, five were eligible to participate in this study having implemented the Program for at least 6 months.
The sampling strategy was purposeful with the aim to obtain insight from a range of stakeholders in each site as well as researchers and policy makers (Table
1). Coordinators (
n = 5) from these five areas were emailed by the Department personnel to invite them to participate in an individual telephone interview, and all agreed. Following each interview, coordinators were asked to pass on the interview invitation to key local stakeholders and program facilitators who they considered might offer additional insights into program implementation in their area. Local key stakeholders included MCH Nurse Managers, those working in roles within Child and Family Services, staff involved in hosting or organising program venues, administrative or evaluation staff. A total of 11 staff (seven key stakeholders and four facilitators) were invited and agreed to participate using this snowballing method (Table
1). In one area, this took the form of a focus group (as this was already planned as part of local level evaluation), in the remaining areas individual telephone interviews were conducted. An online survey of program facilitators was also used as a concurrent recruitment strategy. A total of eight program facilitators from the participating areas completed the survey, and all agreed to be interviewed including three who had not already been identified through the snowballing strategy. Research staff and policy personnel who were actively involved in the translation effort were also invited to participate by email invitation from the study lead (RL), and all except one (a policy maker who had moved on to a different role) agreed. Research staff included the lead investigator involved in the design and testing of the Program in the RCT, those involved in training program facilitators and liasing with with policy personnel and local implementers.
Table 1
Participant position, area, recruitment and data collection methods
Research staff |
1 | Research Staff | n/a | Direct invitation | Individual interview |
2a | Research Staff | n/a | Direct invitation | Individual interview |
2b | Research Staff | n/a | n/a | Meeting 1 |
2c | Research Staff | n/a | n/a | Meeting 2 |
3 | Research Staff | n/a | Direct invitation | Individual interview |
4 | Research Staff | n/a | Direct invitation | Individual interview |
Implementers |
5a | Coordinator | 1 | Direct invitation | Individual interview |
5b | Coordinator | 1 | n/a | Meeting 1 |
5c | Coordinator | 1 | n/a | Meeting 2 |
5d | Coordinator | 1 | Direct invitation | Focus group |
6 | Program Facilitator | 1 | Snowballing | Focus group |
7 | Program Facilitator | 1 | Snowballing | Focus group |
8 | Program Facilitator | 1 | Snowballing | Focus group |
9 | Key Stakeholder | 1 | Snowballing | Focus group |
10 | Key Stakeholder | 1 | Snowballing | Focus group |
11 | Key Stakeholder | 1 | Snowballing | Focus group |
12 | Key Stakeholder | 1 | Snowballing | Focus group |
13 | Key Stakeholder | 1 | n/a | Meeting 2 |
14a | Coordinator | 2 | Direct invitation | Individual interview |
14b | Coordinator | 2 | n/a | Meeting 1 |
15 | Program Facilitator | 2 | Facilitator survey | Individual interview |
16 | Program Facilitator | 2 | Facilitator survey | Individual interview |
17a | Coordinator | 3 | Direct invitation | Individual interview |
17b | Coordinator | 3 | n/a | Meeting 2 |
18 | Key Stakeholder | 3 | Snowballing | Individual interview |
19 | Key Stakeholder | 3 | Snowballing | Individual interview |
20 | Program Facilitator | 3 | Snowballing | Individual interview |
21 | Key Stakeholder | 3 | n/a | Meeting 1 |
22a | Coordinator | 4 | Direct invitation | Individual interview |
22b | Coordinator | 4 | n/a | Meeting 1 |
23 | Program Facilitator | 4 | Facilitator survey | Individual interview |
24 | Key Stakeholder | 4 | Snowballing | Individual interview |
25 | Program Facilitator | 5 | Direct invitation | Individual interview |
26 | Coordinator | 5 | n/a | Meeting 1 |
Policy makers |
27a | Senior Project Officer | n/a | Direct invitation | Individual interview |
27b | Senior Project Officer | n/a | n/a | Meeting 1 |
27c | Senior Project Officer | n/a | n/a | Meeting 2 |
28 | Senior Project Officer | n/a | Direct invitation | Individual interview |
Data collection
Data were collected from January to June 2015 and included 18 individual semi-structured interviews, a focus group in one area and observation of two meetings involving researchers, policy makers and implementers (Table
1). In total 28 individuals contributed to the data collected consisting of research staff (
n = 4), policy makers (
n = 2) and implementers (
n = 22) from across five prevention areas. Implementers included program coordinators (
n = 5) responsible for overall implementation of the Program in each area, program facilitators (
n = 8) who delivered the Program to parents, and local stakeholders (
n = 9).
The interviews and focus group guides were informed by the consolidated framework for advancing implementation science [
31] which integrates existing implementation theories into five key domains shown to be critical to implementation success. Key topics covered included:
-
Role in implementing the InFANT Program
-
Planning for program implementation - who, how and why the InFANT Program was selected?
-
Models of implementation, including any adaptions to the Program and degree of program fit with existing services/programs and policies.
-
Support for implementation, including on the ground logistical support, management support, researcher and policy support
-
Implementation challenges
-
Perceived outcomes including program strengths and weaknesses
-
Perceived sustainability of program and factors influencing sustainability
-
Key lessons for researchers, implementers and policy makers
Individual interviews were conducted over the phone by RL and lasted 40 min on average (range:17 to 65 min). The focus group was conducted face to face, facilitated by RL and lasted one hour and five minutes.
Opportunistic data collection also occurred at two meetings of local implementers, research staff and policy personnel. These meetings provided an opportunity to gather data on key issues relating to the implementation of the Program. The first meeting lasting 1 h and 5 min involved seven individuals including representatives (n = 5) from all the prevention areas as well as research staff (n = 1) and policy personnel (n = 1). This meeting focused on models for implementation where participants discussed implementation progress, challenges and various program adaptions that had been made. The second meeting, lasting 1 hour and 35 min included representatives (n = 3) from two prevention areas, research staff (n = 2) and policy personnel (n = 1). The focus of this meeting was to share the preliminary findings from the interviews and focus group to firstly verify key themes arising and secondly, to use this to generate further discussion and insights about key implementation issues.
Data analysis
The interviews, meetings and the focus group were audio recorded with participants’ permission and transcribed verbatim. Transcripts were imported into Nvivo 10 which was used for coding, sorting and retrieval of data. The study used thematic analysis informed by the methods of Braun and Clarke [
32] and involved the following steps undertaken by RL:
1.
familiarisation with the data by checking the accuracy of transcripts against audio recording.
2.
line by line coding of the data using an inductive approach guided by the research aims, resulting in the development of an initial coding framework. The coding framework was iteratively refined based on new concepts identified in the data.
3.
review of codes to identify broader conceptual themes. At this stage the researchers’ knowledge of empirical literature and existing frameworks for implementation science helped in conceptualising codes into broader categories.
4.
Review of all data within a given theme to identify common and divergent views using constant comparison technique [
30].
5.
First draft of results section which involved re-reading data coded at each theme, along with memo’s about sub-themes to succinctly summarise the theme using illustrative quotes.
6.
Member checking which involved 1) presentation of key findings at a meeting of study participants (see above), which contributed to further data collection and elaboration and refinement of themes; 2) emailing coordinators (n = 3), research staff (n = 2) and policy personnel (n = 1) a copy of the initial draft results section of the manuscript. Participants were asked to comment on whether the themes sufficiently captured their views on the key factors influencing translation process and whether any key issues were omitted in the representation of the results. There was strong agreement with the themes presented and only minor modifications were made to the results based on feedback from participants.
Reflexivity and the role of the researcher
As the study methods were informed by a constructivist approach, it is important to make explicit the role and background of the researcher which may influence how one understands and interprets the data [
30]. RL undertook all data collection and analysis with member checking and input into final results. RL is a postdoctoral researcher with experience in research translation having worked on previous empirical studies exploring the process of research translation [
33‐
37] and conceptual studies [
5,
38]. Over the past 18 months, RL had been engaged with the researchers, implementers and policy makers involved in the translation of the Program. This has provided in-depth understanding of the translation issues where RL has essentially been a participant observer. We believe this adds strength, heightening RL’s sensitivity to translation and implementation issues in the data. RL was not involved in the original InFANT Program trial and had no input into the development of the Program.
Discussion
This study provides new insights into the respective role that policy makers, researchers and implementers play in the translation of a health promotion program into practice and factors influencing this process. As discussed below, a number of key themes were identified by these stakeholders as being important in the translation of the InFANT Program from research to community level implementation.
In line with the findings from this study, the evidence of program efficacy or effectiveness has been found to be only one of many intervention characteristics influencing research uptake [
6,
13,
31,
33,
40]. Other important factors identified in previous implementation research include, the credibility of the Program source, its feasibility, the quality of program materials, program adaptability, trialability, relative complexity and cost [
31]. Our findings concur with recent research highlighting the importance of end users having information on program reach and costs, key service delivery issues such as acceptability and fit of the interventions with existing delivery models, to help inform decisions about the scaling up of public health interventions [
40]. It also suggests the importance of researchers designing scalable and feasible interventions from the outset that align with the policy context. This is likely to require co-development of programs with practitioners on the ground or extensive formative work, as was the case with the InFANT Program [
15,
23,
24]. This underscores the importance of researchers conducting rigorous process evaluations as part of efficacy /effectiveness trials to inform external validity of public health interventions, an area generally poorly reported by researchers [
17,
19,
35].
The tension of balancing program adaption against maintaining program fidelity identified in this study, has been a commonly reported theme in the dissemination of research interventions into practice [
13,
41‐
44]. While greater program fidelity has been shown to be associated with better outcomes [
45], adapting intervention programs to better suit the needs and circumstances of local communities has also been shown to be essential for successful sustained implementation [
8,
41,
42]. To help find this balance between fidelity and adaption, implementation researchers have proposed various guidelines [
41,
46]. At the heart of these recommendations are the need for researchers to clearly articulate the Program’s core components based on a logic model of how the intervention is proposed to work, ideally supported with mediation analysis to identify the ‘active ingredients’ of the intervention. This is then married with consultation with community implementers to refine and modify non-core components to ensure program fit with local circumstances in an iterative fashion [
46]. It is also recommended that adapted programs are evaluated to assess effectiveness and that these steps ideally involve consultation between program developers and implementers [
46]. Systems such as licensing agreements or program guidelines may be helpful in monitoring program fidelity and adaptions and encouraging consultation between researchers and implementers. Service delivery organisations (not for profit or commercial) may have a role to play in ‘rolling out’ evidence based public health programs and monitoring program quality and fidelity. An example of this is the DECIPHer Impact, a not for profit organisation set up to license the 'ASSIST' peer-smoking intervention to schools to ensure fidelity [
47].
For researchers, avenues need to be explored to fund research translation activities, including initial consultation with communities around program adaptions and ongoing program support, such as the provision training and updating of program materials. This remains a challenge, with translation activities not traditionally part of research grant proposals when the study outcomes are not yet known. More recently, specific research translation grants have become more common and this may provide an avenue for researchers to work with community partners to fund such activities. In the absence of external funding, researchers are likely to need to consider program licensing fees to cover costs associated with supporting program delivery, as has been done with other widely implemented programs such as the Standford Chronic Disease Self Management Programs [
48]. At a system level, there is likely to be more incentive for researchers to focus on translation in the future with a growing focus on measuring research ‘impact’ as part of the assessment of universities research outputs. In some countries, research ‘impact’ is being link to university funding, for example, in the UK Research Excellence Framework.
Strong partnerships between researchers, policy makers and implementers, as well as local partnerships were identified as critical to the translation of the InFANT Program into practice. Partnerships between researchers and end users of research, such as policy makers and practitioners, has been consistently shown to be a facilitator of research use and scaling up in previous empirical studies [
8,
11,
12,
33,
37]. As with this study, engaging end users from the inception of a project and forging ongoing relationships with policy makers and practitioners has been shown to be important in promoting the uptake of health promotion programs [
33,
37,
49]. In this study, engaging practitioners in program modification ensured that the Program was designed to fit existing service delivery structures (MCH services), was relevant to the policy focus on obesity prevention in the early years, all of which were important facilitators of program uptake. The implementation of the Program was facilitated by local implementers having direct access to researcher staff who designed or delivered the Program to provide training and guidance for implementation. Mentoring programs where researchers and practitioners can learn from each other during the translation process have been shown to be useful for both parties [
50]. However, challenges remain in funding researcher time, with research translation activities often considered outside of the traditional academic role. Dedicated funding for research translation and the establishment of mentoring or secondment opportunities for researchers, policy makers and practitioners to work together may be a useful step forward in supporting partnerships between researchers and end users of research.
The findings highlight the importance of having key individuals responsible for driving and coordinating research translation across the domains of research, policy, and practice. This case study is unique in that funding was provided to support the translation efforts. At the local level, the prevention workforce as part of the local prevention infrastructure was critical in engaging and working with key partners to deliver the Program. These coordinators undertook critical research ‘translation activities’ including exploring how the Program fitted with and enhanced existing services and how it could be best adapted to ensure sustained program delivery. In the scaling up of public health programs, consideration needs to be given to who will undertake these research translation activities at the local level. From a researcher perspective, research staff need funding to support the translation of research interventions into pre-packaged programs ready for community wide implementation. This study also demonstrates the important role that policy personnel can play in supporting research translation, highlighting the value of incorporating this component into existing policy positions. Previous case studies of research interventions with positive practice and policy impacts [
33] as well as a recent literature review of facilitators of scaling up [
8], have demonstrated the importance of leaders, champions and coordinators in advocating for and supporting adoption of public health interventions into practice.
The study findings point to important lessons regarding the scaling up and sustainability of program implementation. It is yet to be seen whether the InFANT Program can be scaled up and delivered on an ongoing basis in the absence of funding from the Department for the Program. Given that ongoing funding for any program at a state or local level was unlikely at the time of this study, it appears that embedding program delivery into existing service infrastructure will be critical for both scaling up and sustainability of the Program. With the strong competing demands on practitioner time and resources, it will be important that ongoing program evaluation be conducted to support a business case for continuation of the Program locally within existing services. This points to the importance of cost effectiveness analysis to be conducted as part of intervention trials to help make the case for investing in particular intervention programs. This is in line with a recent narrative review, which identified establishing monitoring and evaluation systems and costing and economic modeling of intervention approaches as important success factors for scaling up public health interventions [
8]. Alignment of the Program to both state and local policy context will also be important in harnessing ongoing support for the Program.
This study has a number of strengths and limitations. The strengths include the use of multiple varied case studies and cross case comparisons to explore the important influence of local context on implementation. We did however, only have one rural site participate in the study and the inclusion of additional rural sites may have resulted in different implementation issues emerging. We also did not include sites that chose not to implement the Program, which may have provided interesting additional insights into factors influencing initial uptake. Within cases, we interviewed a range of stakeholders including coordinators, program facilitators and those involved in supporting program delivery. There was variation however in the number of people interviewed across sites ranging from nine participants (site one) to two participants (site five) and this may have limited insights gained from some sites. The inclusion of researchers and policy makers involved in supporting program delivery as participants in the study was also a strength as it enabled a comparison of views across roles, providing important new insights from these various perspectives. While the study used a range of data collection methods including individual interviews, focus groups and recording of meetings involving key players, additional methods such as observation of program sessions, analysis of key documents and interviews with parents may have yielded additional insights. While all data was collected and analysed by a single researcher (RL), the trustworthiness of the findings was verified by participants on two separate occasions and the role and background of the research was made explicit.
Acknowledgements
The authors would like to acknowledge Rowland Watson and Maya Rivas (The Victorian Department of Health and Human Service) for their contribution and support in the delivery of the InFANT Program. Finally, the authors would like to acknowledge all participants for their time and valuable insights.