Influences of post-implementation factors on the sustainability, sustainment, and intra-organizational spread of complex interventions

In understanding how to implement to sustain, most work addresses influences of peri-implementation factors (salient during the implementation phase) on sustainability. These factors include supports from implementers and features of pre-implementation context [2, 9‐11]. Our premise is that influences of post-implementation factors may be just as, or more, important to longer-term sustainability – particularly with well-established challenges to sustaining behavior change in notoriously intractable social settings such as healthcare [12]. We argue that post-implementation factors are likely to prevail and gain salience after implementation supports are removed. While some post-implementation factors may not directly affect continued application of new practices, others may have profound influence. Factors may support new processes and practices, erode changes during implementation, or reinforce reversion to old ways of doing [12]. Withdrawal or redirection of management support, or loss of staff with knowledge of a practice change, may lead to knowledge loss and organizational forgetting.

Our study was further motivated by two other major literature gaps. First, the related phenomena of sustainability, sustainment, and spread (defined below) have been studied separately, or sustainment and sustainability are conflated. Little is known about whether factors that influence one phenomenon influence others. Second, how sustainability, sustainment, and spread are related is understudied and important to advancing sustainability science [5, 13].

The terms sustainment, sustainability, and spread lack universal definitions [13]. In the implementation science literature, sustainment and sustainability are considered post-implementation phenomena of concern once (peri)implementation supports are withdrawn or exhausted. Common supports are research funding for coaching and facilitation, access to clinical and other expertise, training, and financial incentives. Chambers et al. (2013) [14] usefully distinguish sustainment and sustainability as related but distinct aspects of continued knowledge use. Sustainment occurs when stakeholders continue to enact core processes and related behaviors originally conveyed through an intervention. Sustainability is continuation of key benefits or outcomes from the intervention. We refer to spread as uptake or diffusion of an intervention (or core components) from its original context to intra- or inter-organizational contexts [15]. While spread is often considered conceptually separate from sustainment and sustainability, we contend that it is a logically linked related phenomenon. If change processes are not sustained, or benefits cease to be realized, a change initiative is unlikely to spread [16]. We refer to sustainment, sustainability, and spread collectively as sustainability phenomena.

Stirman and colleagues’ (2012) [2] critical review of the literature on sustainability of evidence-based programs and practices in healthcare settings highlighted four categories of factors that influence continuation of procedures and outcomes: (1) organizational contexts for practice change interventions (e.g., culture, leadership, structure); (2) internal and external capacity (e.g., resources, support, champions); (3) processes (e.g., training, engagement); and (4) aspects of new programs or practices themselves (e.g., fit, modifiability). These factors are influential peri-implementation and influence sustainability phenomena post-implementation. Stirman et al. (2012) [2] noted lack of consideration of sustainability dynamics. Most studies they reviewed considered sustainability at one time point post-implementation, for static presence/absence/extent of peri-implementation factors. Stirman et al. (2012) [2] concluded that sustainability research needs careful critical development, including enhanced conceptual clarity on sustainability and concerted examination of dynamics of contexts where sustainability is a concern.

More recent findings align with the categories of Stirman et al. (2012) [2]. Important peri-implementation factors that influence sustainability are intervention design and delivery, intervention process, people involved, resources, organizational setting, and external environment [13]. Proctor and colleagues (2015) [17] highlighted the need for definitional and conceptual consistency within the sustainability literature. They sought perspectives of sustainability researchers and stakeholders to identify important issues for sustainability research. They highlighted two priority areas: testing how intervention properties and adaptations affect sustainability, and identifying contextual factors that influence sustainability. They noted the importance of distinguishing between influences of factors on implementation and post-implementation.

Lennox and colleagues (2020; 2018) [8, 13] reviewed approaches to planning, assessing, or evaluating continued intervention activities and benefits. Their consolidated summary of sustainability determinants in these approaches is highly consistent with Stirman et al. (2012) [2]. They usefully identified recent empirical exploration of sustainability dynamism, noting a detectable “shift in [researchers’] perspectives from sustainability as an outcome to sustainability as an ongoing process” ([13], p. 12). About one third of identified approaches draw on theories that view sustainability as an ongoing process [8, 13]. Those relying on complexity theory emphasize interactions among intervention, organizational context, and broader context. Those relying on ecological theory propose ongoing intervention adaptation as important to maintaining fit with constantly changing inner and outer contexts. Those referring to open systems theory posit that broader environment influences organizational context of the intervention, requiring constant organizational adaptations to sustain the intervention. In all theoretical perspectives, the dynamic view of sustainability calls for continuous intervention improvement and adaptation and (where feasible) its inner and outer contexts [18]. These perspectives depart markedly from traditional linear views that treat sustainability as a static outcome.

While emerging dynamic perspectives on sustainability are promising and much more likely to reflect the complex reality of sustaining practice change over time, none of this work adequately considers changing intervention contexts. Assuming that factors active during implementation are equally active and effective post-implementation is counter-intuitive (and possibly naïve). The literature is silent on specific influences of post-implementation factors on sustainability.

Our study is part of the Sustainment, Sustainability, and Spread Study (SSaSSy) [16] of a complex quality improvement (QI) initiative, SCOPE (ClinicalTrials.gov NCT03426072) [19]. Supplementary Fig. 1 shows the different stages of SCOPE, data collection for the current study, and SSaSSy. Our focus is sustainability phenomena in CHs in the province of Manitoba, Canada, where SCOPE was implemented during 2016–2017. We report more details of the SCOPE implementation in Manitoba elsewhere [20].

In Canadian CHs, care aides (unregulated direct care workers) account for the largest workforce and provide up to 90% of direct care for older adults [21]. SCOPE is a complex intervention to empower care aides to lead a QI initiative on their CH unit, with facilitated support. SCOPE has four components (Supplementary Fig. 2) [20, 22]. (1) Learning Congresses led by experienced quality advisors convey key QI skills and processes identified in the Institute for Health Improvement’s collaborative model (Institute for Healthcare Improvement, 2003) [23]. Interim action periods of Plan-Do-Study-Act (PDSA) cycles provide experiential QI learning. Small PDSA change cycles, enacted by collaborative teams and emphasizing pre-post measurement, gauge effects of small-scale practice change/QI initiatives. (2) SCOPE introduces a team QI approach. Care aides lead teams of 3–4 CH staff. (3) SCOPE formalizes internal team support, with Team Sponsors (unit managers) and Senior Sponsors (CH managers) mentoring and supporting QI activities of care aides. (4) Quality advisors provide external team supports, with ongoing coaching and guidance to Sponsors and QI teams over PDSA cycles between Learning Congresses.

SCOPE’s key components convey its core elements (processes, behaviors) designed to deliver key benefits (Table 1). Core elements are essential to sustain, as “most closely associated with desired health benefits” ([2], p. 10). Each care aide-led QI team selected one of three clinical areas for their project focus: responsive behaviors, pain, or mobility. SCOPE’s intended key benefits (Table 1) are care aides’ QI capacity and leadership, use of best evidence in their practices, improved quality of work life, and enhanced work engagement. Research in work psychology [12] suggests that SCOPE’s influence on outcomes for work autonomy, empowerment, and engagement is likely to influence downstream work performance, including improved quality of resident care.

A SCOPE pilot was conducted in Alberta and British Columbia CHs (2010–2011) [24]. We examined sustainability and spread of SCOPE-related QI activities in 7 participating homes at 6 and 12 months post-implementation [25]. We did not differentiate peri- and post-implementation factors or sustainment and sustainability, but we identified factors influencing continued processes (sustainment) and sustained benefits (sustainability) up to 12 months post-implementation: turnover of key team members (care aides), facilitative team dynamics, observable improvements from SCOPE, strong management support of QI teams, buy-in from other CH staff, and (in)adequate resources [25]. Factors contributed to absence of continued processes (sustainment), sustained benefits (sustainability), or spread: lack of resources (financial, human) in one CH, leadership change in another, and competing government priorities in the third. These findings on short-term sustainability suggested factors warranting further exploration in this study.

Our overarching goal is to understand post-implementation factors that influence dynamic longer-term sustainment, sustainability, and spread of complex care interventions. In this exploratory study, we aimed to explore post-implementation factors that influenced the sustainment, sustainability, and spread of the SCOPE intervention in the complex environments of Canadian care homes (CHs; long-term care homes). Our research question was: What were unit and care home managers’ perspectives of multi-level post-implementation factors that influenced the continuation of core SCOPE elements and key benefits/impacts? The SCOPE intervention has been evolving as this research team developed [22], piloted [24], and implemented [20] the intervention. Therefore, our research team has the unique opportunity to contribute to knowledge about the dynamism of sustainability phenomena.

We employed a qualitative, descriptive research design [26] to explore factors influencing SCOPE sustainability phenomena two years post-implementation. This design allowed us to stay close to the data with low levels of researcher interpretation or inference [26]. We collected data through small group interviews by telephone (May to July 2019), two years after SCOPE’s implementation in Manitoba (2016–2017).

SCOPE was implemented in 7 CHs in Winnipeg (Manitoba’s largest urban center). While multiple groups of staff (care aides, regulated staff, managers) participated in the SCOPE implementation, in this study we focused on managers’ perspectives of sustainability phenomena post-implementation. We invited all unit and CH managers to participate via email; 11 managers from 6 of 7 CHs agreed. We undertook 6 small group interviews (1 to 3 managers per interview), one for each CH participating.

We based our semi-structured interview guide (Supplementary File 1) on literature on QI program sustainability in healthcare settings [2, 8, 13, 17] and our prior work [25]. We elicited participants’ descriptions of their experiences during SCOPE, continued QI activities post-implementation, post-implementation factors influencing sustainment (dis/continuation of behaviors) and sustainability (dis/continuation of outcomes) in the QI unit, and inter-unit spread.

LM sought written informed consent from potential participants prior to each interview. LM and WB conducted telephone interviews of up to 45 min, each with manager participants from the same care home. Interviews were audio-recorded, professionally transcribed, and de-identified before analysis.

We used inductive thematic analysis to identify data patterns and themes [27, 28] and Microsoft Word to support the analysis. Table 2 details the analysis steps, including familiarizing with the data, generating, and applying coding scheme, generating themes, and refining themes. When developing and refining the coding scheme, we organized observed factors into three levels: (1) micro-level factors relating to individuals, teams, and their interactions; (2) meso-level factors relating to unit and CH; and (3) macro-level factors relating to CH operating environment or systems.

For qualitative rigor [29], multiple researchers worked closely in analysis and recorded major decisions and findings in extensive memos. A three-member team (YS, LM, WB) led the data analysis; this team included a health services research PhD student, a nursing postdoctoral fellow, and a health services researcher. Several other authors (LG, MD, AW, LC) contributed to the identification and differentiation of themes and to the refinement of them.

For the post-implementation factors at micro-, meso-, and macro-levels, many relate to sustainment of SCOPE core elements. Some are differentially associated with sustainability and inter-unit spread. Figure 1 summarizes the relationships among post-implementation factors and sustainability phenomena.

Factors influencing sustainability phenomena are not well understood [2, 10], for reasons including persistent lack of conceptual consistency, under-specification of complex interventions’ core elements, reductionism leading to limited insight on interrelations among broad influence categories (context, intervention characteristics, processes relating to sustainability, capacity to sustain), and failure to consider factors acting dynamically on sustainability phenomena at different levels within and outside the post-implementation context [2, 8, 18, 30].

Distinctions between core elements, processes, and associated behaviors that are sustained (sustainment) and ensuing benefits (sustainability) are helpful to conceptualize and study sustainability phenomena. Our deep history in developing [22], piloting [24], and implementing [20] SCOPE helped us make distinctions and consider influences of factors active at different levels on sustainment of core elements and sustainability of key benefits. Distinguishing sustainment and sustainability had clear value to our study and may be helpful in other settings and studies of sustainability phenomena.

While we differentiate sustainment and sustainability in analysis levels, the identified post-implementation factors align well with the four broad categories of influence for peri-implementation factors [2]: organizational context (culture, leadership), internal and external capacity, knowledge-conveying and supportive processes, and intervention adaptability. In the broadest sense (discussed further below), our findings contribute understanding of the particular importance of factors. Several micro-level factors and the macro-level factor of system alignment contribute to sustainment of core process elements. Meso-level factors contribute to sustainability of key benefits/outcomes and sustainment of some core processes. Critically, the meso-level factors of observability and practice change institutionalization related exclusively to inter-unit spread.

Our findings suggest multiple influential factors that merit further research. As observed across studies in Stirman and colleagues’ review (2012) [2], and as described by our study participants, changes to outer context (e.g., policies, legislation) can alter internal priorities and shift limited resources away from QI initiatives. Capacity to sustain, including funding, resources, and workforce stability, is similarly susceptible to external pressures [2]. Even smaller changes to local or regional contexts that influenced management turnover or care aide workforce instability profoundly influenced sustainment of core processes and sustainability of QI capacity and leadership.

Birken and colleagues (2020) [31] note that research rarely distinguishes between determinants of sustainment and implementation. Our findings suggest that these differ. Specifically, we found that ability to sustain core processes is susceptible to several micro-level factors and vulnerable to vagaries of extra-organizational change. Sponsors perceived competing system priorities as barriers to sustaining and spreading SCOPE-related activities. This was not identified as a factor influencing SCOPE’s implementation in our earlier work [19]. Unlike post-implementation, peri-implementation supports (e.g., additional resources, expertise, and focused attention) likely protect the implementation micro-context temporarily from external pressures that discourage internal change and innovation.

Adaptation is widely acknowledged as an important, but understudied, factor influencing sustainment and sustainability [17, 32]. Here, two aspects of adaptation emerged as important: adaptability of the intervention itself and staff’s adaptive capacity to change the intervention. Sponsors described care aides adapting features of SCOPE elements post-implementation, making features easier to sustain. One unit adapted application of the PDSA cycle, sharing cause-and-effect relationships orally. This viable, reliable alternative to formalized pen-and-paper repeated measures aligned the intervention with context. Reliance on oral communications is characteristic of long-term care and care settings with non-standardized work, complex work environments, and learning by doing [33]. Our findings corroborate observations by other sustainability scholars that “interplay between contextual factors and the innovation itself is to be expected given the dynamic nature of the complex systems into which innovations are introduced” ([2], p. 10).

Such observations also relate to fidelity. Partial sustainment is more common than complete sustainment [2]. Adaptation sustains intervention elements that fit with context and discards others. Elements most closely related to desired benefits are more feasibly executed. Prevalence of partial sustainment suggests that fidelity is rarely absolute and raises questions about whether departures from fidelity that achieve partial sustainment – and improve fit – jeopardize intervention integrity. We must distinguish core intervention elements and less critical peripheral ones. Fidelity sufficiency is essential for pragmatic implementation [2].

Lack of knowledge about causal pathways to sustainment and sustainability is a major acknowledged gap in sustainment and sustainability research [31]. Our study offers insights into these pathways, including evidence of differentiable influences of some factors on particular phenomena and on interrelationships among factors. Research that extends examination of interrelationships among sustainability phenomena and influence factors strikes us and others as promising [7, 25, 32]. Stirman et al. (2012) [2] points to potential for this research to illuminate compensatory factors if elements like funding are missing. We observed the importance of strong social dynamics to sustainment and sustainability, even with diminished dedicated QI support.

Only two factors affecting sustainment and sustainability influenced intra-CH spread of SCOPE core processes: institutionalized practice changes and observability of benefits from QI initiatives. Spread occurred in only half of study CHs, suggesting that these factors have more nuanced differences in units where spread occurred (e.g., how practices were embedded, how observable benefits were captured or communicated) that warrant further study.

With this study and earlier work on factors influencing sustainability phenomena [25], we can compare and contrast factors influencing those phenomena at one year and two years post-implementation. This addresses a critique that sustainability is generally examined at a single time point, overlooking its dynamism [2, 18]. All factors influencing sustainability phenomena that were noted at one year post-implementation [22] were also noted in this study two years post-implementation: turnover of key team members, facilitative team dynamics, observable improvements from SCOPE, strong management support of QI teams, buy-in from other CH staff, and (in)adequate resources. However, several additional factors emerged at two years: adaptive capacity of SCOPE teams, adaptability of SCOPE original elements, structural mechanisms for routinization, and institutionalized practice changes. Adaptive capacity of team and intervention adaptability sustained SCOPE core elements post-implementation. Lack of influence of these factors at one year suggests that peri-implementation research supports (e.g., coaching) may compensate. With lingering effects of external research support removed by two years, intervention fit with organizational context became more important to continuing SCOPE processes.

Structural mechanism for routinization and institutionalized practice changes both sustained momentum two years post-implementation. During peri-implementation, several SCOPE elements boosted momentum (e.g., exchanging experiences during Learning Congresses, regular coaching) and may have remained high one year post-intervention [19]. Other factors sustaining momentum at two years may have been deemed unimportant at one year.

Our findings corroborate relationships suggested in extant sustainability theories, models, and frameworks. Many factors identified as influencing sustainment and sustainability related to social interactions among care workers, aligning with well-established implementation science theories that highlight the crucial role of social dynamics among individuals and organizations in implementing and sustaining change interventions [34‐37]. Our findings also align with learning theories that underscore importance of institutionalizing changes to practice routines [35]. We identify several unit- and CH-level factors that facilitated integration of SCOPE elements into institutional routines. We observed system-level imperatives that compete with sustaining core elements and key benefits, aligning with open systems theories of organizations as systems profoundly influenced by micro- and macro-environments [37].

Our empirical evidence supports the dynamic view of sustainability [8, 11, 18]. Our findings support the importance for sustainability of constant adaptations and improvement of intervention and organizational context. Facilitative factors for sustainability at two years post-implementation emerged in response to decreasing organization resources for intervention delivery. This corroborates the dynamic sustainability framework, which “embraces change as a central influence on sustainability” ([14], p. 7), including intervention–organization fit. That framework also posits the critical role of an organization’s ability to continuously learn and adapt as “a core value of the implementation setting” ([14], p. 6). Facilitative factors for sustainability that emerged only by two years post-implementation indicate that CHs can adapt to external environment changes by spontaneously improving organizational context and intervention.