SIPsmartER delivered through rural, local health districts: adoption and implementation outcomes

Understanding the implementation of effective interventions is critical to promoting their sustained translation into practice-based settings and enhancing their potential impacts on population health. Successful implementation impacts both service and client outcomes [1, 2]. In combination with understanding the generalizability of an intervention’s reach into the intended population and its adoption and maintenance at the organizational level, implementation is key factor in replicating effective interventions in typical community or clinical settings [1]. Further, additional implementation outcomes including acceptability, appropriateness, costs, and feasibility can inform the external validity of evidence-based interventions when delivered outside of the research context [2].

SIPsmartER is a six-month, multi-component, community-based, behavioral intervention designed to reduce the intake of sugar-sweetened beverages (SSBs) among rural, Appalachian adults [3, 4]. SSBs are non-alcoholic beverages that contain sugar and few other nutrients, such as soda/pop, energy drinks, sweet tea, and fruit drinks. SIPsmartER is grounded in the Theory of Planned Behavior [5] and health literacy principles [6]. The intervention design is described in more detail elsewhere [3]. It is one of only two known interventions targeting SSB intake among adults that have demonstrated significant improvement in adults’ SSB consumption [3, 4, 7‐9]. SIPsmartER also is the only SSB-focused intervention included in the National Cancer Institute’s repository of Research-Tested Intervention Programs (RTIPs) [10]. Translating SIPsmartER into practice-based settings is important given the numerous preventable health conditions associated with excessive SSB intake (e.g., obesity, diabetes, heart disease, cancer, dental caries) [11‐15], particularly among populations with low socio-economic status and/or living in rural areas [16‐19]. Given SIPsmartER’s demonstrated effectiveness [4], it is important to explore how it could be disseminated, implemented, and integrated within a system that reaches health disparate communities in Appalachia.

A pilot dissemination and implementation (D&I) trial was collaboratively developed with medical directors and leadership staff from the four local health districts within the Virginia Department of Health (VDH) that service the rural Appalachian counties [20]. The trial design was grounded in the RE-AIM framework [1] and the Interactive Systems Framework [21, 22], and the evaluation was guided by RE-AIM. Specifically, this pilot trial was designed to measure the reach, effectiveness, adoption, and implementation of SIPsmartER when delivered through local health districts. To support the delivery of SIPsmartER, the research team and health department stakeholders developed and applied an implementation strategy [23] that would build both the general and innovation specific capacity necessary to deliver SIPsmartER. This strategy utilized the key elements of consultation identified by Edmunds (e.g., on-going instruction, self-evaluation, and feedback) [24].

The purpose of this paper is to describe SIPsmartER’s adoption and implementation when delivered through rural, local health districts [1, 2]. In addition, determinants of adoption, implementation, and organizational maintenance that align with the Interactive Systems Framework were assessed from the perspective of the delivery agents from the local health districts: (i) acceptability (satisfaction with aspects of the innovation), (ii) appropriateness (perceived fit, relevance, and suitability), and (iii) feasibility (actual fit). This paper specifically focuses on outcomes related to both delivery expectations and the implementation strategy.

This study is a mixed-methods process evaluation of a pilot type 2 hybrid effectiveness-implementation trial of SIPsmartER [20]. A type 2 hybrid effectiveness-implementation trial allows for the simultaneous testing of an intervention and an implementation strategy to supports its delivery [20]. It specifically reports on SIPsmartER’s adoption and implementation at the organizational-level. A complete description of effectiveness outcomes are outside the scope of this manuscript. However, during this trial, significant improvements from baseline to 6-months in SSB intake (− 403(CI = − 528, − 278) kcals/day, (p < 0.001)), which were comparable to findings from the effectiveness trial [4]. Also, significant changes (all p < 0.05) in SSB-related attitudes, perceived behavioral control, behavioral intentions, and media literacy were observed.

Study procedures were approved by the Institutional Review Boards of Virginia Tech, the University of Virginia, and Virginia Department of Health. Written informed consent was obtained from health department staff.

This hybrid effectiveness-implementation trial examined adoption and implementation factors related to SIPsmartER, an evidence-based intervention to reduce sugary beverage consumption in a region experiencing health disparities. Through the use of a multi-component implementation strategy that included a consultee-centered approach, it was demonstrated that SIPsmartER could be adopted and implemented with high fidelity across four rural public health districts and by delivery agents with different roles within the health districts. This study contributes to the ISF literature in that findings demonstrate that the packaging an evidence-based intervention to reduce complexity and use of a facilitation process including consultee-centered training can result in the adoption and high-fidelity implementation of an intervention to address sugary beverage consumption in underserved communities. However, findings also found that the potential for sustainability and broader adoption could be jeopardized by intervention features that were less feasible and required research facilitation due to a lack of compatibility with the health department context and a lack of perceived relative advantage. Specifically, delivery agent-initiated telephone contacts were difficult to consistently implement while the in-person small group sessions were more compatible with health department practices. Similarly, qualitative feedback indicated that agent delivered telephone calls and the recruitment processes may not be as feasible.

While public health systems are beginning to play an increasing role in the implementation of evidenced-based interventions targeting chronic health conditions, little is known about how evidence-based programs can be best implemented and sustained in these systems [39‐42]. Related to recruitment, fidelity to the protocol was high in terms of delivery agents being able to recruit 89% of the cohorts. However, findings from post-training and post-cohort interviews indicate low perceptions of feasibility, with most delivery agents clearly expressing their frustration with the recruitment process. Delivery agents employed two broad strategies to recruit participants: (1) surveying of the community through canvassing health department customers or the general community through local events and (2) targeting established groups, including housing and work sites [43]. The former was the strategy most districts started with and it was much less efficient. Recruiting for adult participants within the community for a program consisting of group classes and having to reach a threshold of approximately ten participants to start a class was not a common practice for any of the districts. The lack of experience and protocols within the districts may have weakened the potential reach of the program. This finding highlights the potential usefulness of systematically collecting and recording key patient health behaviors as a means to efficiently identify patients who would be good candidates for interventions [44]. It also highlights the need to consider efficient recruitment strategies that could vary based upon community resources [32].

Delivery agents’ engagement with and perceptions of the consultee-centered implementation strategies demonstrated high fidelity to these activities and reported high perceptions of acceptability, appropriateness, and feasibility. This finding is notable as it provides support for the use of consultation as an implementation strategy in community-based interventions when using professional health district staff as delivery agents. Although consultation is regularly used as an implementation strategy and has potential for use across contexts [45], its use is most commonly reported within the context of supporting community and clinic-based mental health professionals to implement evidence-based programs [33]. The strong implementation fidelity may be due to the design of the consultee-centered approach [24]. Particularly, allowing for a non-hierarchical relationship between the researchers (consultant) and delivery agent (consultee) acknowledged the past training and professional experiences of the delivery agents. Also, being able to adjust the intensity of instruction and feedback activities to reflect the growing skill and content mastery of the delivery agents with the intervention allowed them to still feel well-supported without overburdening them. However, it is important to note, that clear explanations of the purpose of a consultation approach and the specific reasons activities were chosen is needed, as the relative intensity of the strategy made one delivery agent uncomfortable initially. She felt like she was being judged and that the level of support was not needed; yet, after engaging with the implementation strategy, she recognized its purpose was not to judge her but to support the program delivery.

Our findings about the implementation of SIPsmartER in these rural, local health districts reflect previously identified benefits, facilitators, and barriers of implementing evidence-based programs in public health agencies. Related to benefits, findings suggest that by implementing SIPsmartER, their districts added programming that better addressed common risk factors for disease [46]. Delivery agents demonstrated contrasting views of some agents on the relative advantage of the approach from three districts. Three districts mentioned SIPsmartER was similar to another program they implement (a statewide intervention to reduce sugar-sweetened beverages that includes social marketing and single workshops for children, adolescents, and adults). However, delivery agents from two districts reflected that SIPsmartER was better designed to foster behavior change and could serve as a next step to that program. Future training approaches may want to address how SIPsmartER compares to existing programs to underscore the uniqueness and benefit in helping participants change behavior.

Our findings reflect barriers previously identified related to staff turnover, leadership, and agency structure [46‐48]. Staff turnover was particularly noticeable in this D&I pilot as 75% of the districts had changes in medical directors early in the study. The medical directors were key in the promotion and support of SIPsmartER within their districts. This occurrence is important as past research has demonstrated that leadership is a necessary factor to further implement evidence-based programs within health departments [48‐50]. Allen and colleagues’ findings suggest that effective leaders within health departments did not just talk about valuing and supporting evidence-based programs but also created an environment that fostered consistent conversation about the evidence-based programs and provided a supportive organizational environment [48]. Hu and colleagues identified that public health agencies with “high agency leadership” and “supportive workplace” were 2.08 times and 1.74 times more likely to use research evidence in the workplace compared to unsupportive environments [50]. In this trial, these changes in medical directors in the three districts may have impacted specific leadership actions related to this pilot study, which could have impacted implementation outcomes. However, researchers cannot control staff turnover, so these finding stresses the need to cultivate multiple formal and informal leaders within the health districts (e.g., opinion leaders, internal implementation leaders, and champions) from early on. In doing so, if one leader leaves the organization or the project, others remain to maintain the support and legitimacy of the program and to drive forward the implementation of the intervention [51].

Agency structures and processes influence the ability and motivation of delivery agents to complete delivery expectations and implementation strategy activities. The impact of agency structure and processes on implementation outcomes was noticeable in this trial. Notably, from interviews and capacity surveys, it was evident that in three of the four districts, delivery agents adhered to work schedules that were within normal business hours. While there was flexibility to adjust schedules and there was specific funding for the delivery agent time during the work day, they did not feel that it was appropriate, acceptable, or feasible both in terms of time and resources for them to make calls to participants during normal work hours and/or to adjust their schedules to accommodate the calls. Therefore, those agents who attempted teach-back and missed class calls did so during normal business hours. Also, as this type and scale of participant recruitment was not common practice in any of the districts, the health districts were lacking both the structure and capacity to efficiently recruit for the program.

The approach to this trial was pragmatic. Planning was guided by the RE-AIM framework with a goal to design an implementation approach that would allow SIPsmartER be able to significantly impact SSB intake, have a broad reach, and be readily adopted, implemented, and sustained in a typical community delivery setting, i.e., local health departments. Additional pragmatic decisions were made to allow for health departments to test out the program (i.e., trialability) [22, 28]. For example, it was decided from the outset that delivery agents would not play a role in the administration of the IVR call system and later decided they would not manage participant retention activities. These decisions were purposeful in order to allow delivery agents the ability to gain experience with recruitment, lesson delivery, and execution of teach-back and missed class calls.

Findings suggest SIPsmartER’s group classes and implementation strategy are appropriate and acceptable to health district delivery agents and can be feasibly implemented with high fidelity within the structure of rural, local public health districts in rural Virginia. However, the execution of teach-back and missed class calls as well as recruitment efforts are perceived as less appropriate and acceptable and may, therefore, be less feasible to faithfully implement. These findings, in conjunction with those related to the effectiveness of the intervention when implemented in this system, will be used to inform the further translation of SIPsmartER into this system.