Background
There is growing interest in examining how and what aspects of organizational context impact whether service providers use evidence-based interventions (EBI), treatments found to produce positive outcomes in rigorous research studies [
1], with their clients. A number of implementation frameworks (such as the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework [
2] and the Consolidated Framework for Implementation Research (CFIR) [
3]) highlight organizational characteristics and processes comprising the inner organizational context of EBI implementation. Factors include organizational culture, climate, structure, size, and readiness for change. In line with these frameworks, research across a number of health and human service settings (e.g., hospitals, mental health, substance use disorder treatment, child welfare) points to the critical role of the inner context for implementation (e.g., [
4‐
8]). For example, the extent to which leadership and organizational environments foster team learning in surgical settings is associated with implementation success [
9,
10].
A long history of research on organizational culture and climate has identified leadership as a critical facilitator of the context that develops in organizational settings (see review by Ehrhart, Schneider, and Macey [
4]). Although there is little consensus in the literature on the exact distinction between leadership and management [
11], we use the term leadership to be consistent with related literature in this area and to capture the influence those in a leadership role have on the units they lead. Thus, for this study, we adopt Yukl’s [
11] definition of leadership as “the process of influencing others to understand and agree about what needs to be done and how to do it, and the process of facilitating individual and collective efforts to accomplish shared objectives” (p. 7).
With regard to EBI implementation, the role of leaders across different levels has been emphasized in multiple implementation frameworks (e.g., EPIS [
2] and CFIR [
3]). Despite the need for more empirical studies of leadership in implementation [
12,
13], recent research has linked a variety of leadership behaviors with implementation outcomes (e.g., [
14‐
19]). Although much of this research has focused on general leadership behaviors, Aarons, Ehrhart, and Farahnak [
20] have proposed that leadership behaviors focused on implementation may be more proximal predictors of implementation-related outcomes. Their development of the Implementation Leadership Scale (ILS) forms the foundation for the current paper’s focus on leadership for sustainment.
The development of the ILS was based, in part, on research literature from business/management and industrial/organizational psychology on leaders’ use of “embedding mechanisms” (i.e., actions that leaders can take to communicate their core values) to develop or shape organizational culture and climate [
21] and strategically focused leadership as a predictor of strategic climates and subsequent outcomes [
22‐
24]. As supported by theory and empirical research on service climate [
23] as well safety climate [
25], strategically focused leadership is proposed to be a primary predictor of strategic climates, which are strong predictors of employees’ motivation and behavior in support of the achievement of the unit’s goals related to the specific strategic imperative. These behaviors are then the proximal predictors of the unit’s strategic outcomes, such as accidents in safety research [
25] or customer satisfaction in the service literature [
23].
To develop implementation leadership items, Aarons et al. [
20] drew from other measures of strategically focused leadership [
23,
25], Schein’s [
21] embedding mechanisms (e.g., what leaders pay attention to and how they react to organizational crises), and implementation subject matter experts with the goal of identifying those aspects of the leadership of first-level supervisors that are most relevant for influencing their subordinates during implementation efforts. After a process of narrowing the item pool and dimensions through exploratory factor analysis, they identified four dimensions of implementation leadership: proactive leadership (i.e., the degree to which the leader establishes clear goals and plans, and removes obstacles during implementation), knowledgeable leadership (i.e., the degree to which the leader is knowledgeable about EBIs), supportive leadership (i.e., the degree to which a leader supports staff efforts to use and learn EBIs, and recognizes their efforts in doing so), and perseverant leadership (i.e., the degree to which the leader persists and moves forward in the implementation process despite problems and challenges).
Despite progress in understanding the organizational context for implementation and specifically the role of leadership in implementation, its role in sustainment has received almost no attention. Sustainment has been defined as the continued use of an EBI over time with fidelity and ongoing support such that the desired benefits of the EBI are maintained [
13,
26]. In general, factors affecting the long-term sustainment of EBIs are understudied relative to those influencing implementation [
13,
27]. Measures asking about leadership for implementation may be less relevant in organizations that have moved to the sustainment phase. In addition, there may be similarities and differences between implementation and sustainment that create some ambiguity about whether the specific dimensions of leadership that are critical during implementation are also critical during sustainment. For instance, whereas implementation involves establishing new practices in an organization, sustainment is focused on continuing those practices over time. Both implementation and sustainment involve ongoing organizational learning to be successful [
28], but sustainment is, by definition, dependent on implementation since implementation processes come first. Thus, if implementation is managed poorly, then sustainment will likely have little chance of success [
29]. Although many of the same organizational characteristics that lead to successful implementation are also likely to contribute to successful sustainment [
29], the two processes may have both common and unique aspects requiring distinct leadership approaches.
The purpose of this study is to contribute to our understanding of EBI sustainment by employing mixed-method research to examine a measure of sustainment leadership, the Sustainment Leadership Scale (SLS). We define sustainment leadership as the attributes and behaviors of leaders that support the effective sustainment of EBI implementation. In light of the arguments to be made for both the overlap and distinctiveness of implementation leadership and sustainment leadership, we used a research design that addresses both. Specifically, we first used a quantitative approach to test whether the dimensions of the ILS hold in the context of sustainment. We next used a qualitative design to connect descriptions of leadership behaviors during sustainment to that same dimensional structure, and then to address the possibility of differences in critical leadership behaviors in implementation and sustainment phases, we identified any new themes/dimensions in the qualitative study that did not align with the original proposed SLS dimensions. In addition, although the ILS was originally developed with a focus on the leadership behaviors of first-level supervisors (which we follow in our quantitative analyses), qualitative methods allowed for frontline providers to comment on leadership of both their first-level supervisors and upper-level leaders in their organization.
This research addresses multiple gaps in the implementation literature. First, we contribute to the general knowledge on sustainment of EBIs, which has been understudied in the implementation literature [
13,
27,
30]. Second, we specifically address the role of leadership in sustainment and clarify whether the behaviors that are critical to implementation are also viewed as crucial to sustainment. Finally, we contribute to the limited amount of validated measures assessing various components of sustainment [
13] by developing a short, practical measure of sustainment leadership. Such a measure should be useful for researchers with a specific interest in the role of leadership and the organizational context on EBI post-implementation, and for practitioners interested in what leaders can do to contribute to the sustainment of EBI use over time.
Discussion
The goal of this research was to extend the concept of implementation leadership to sustainment leadership using both quantitative and qualitative methods, thus expanding our understanding of how leaders “lead for the long haul.” We adapted a measure of implementation leadership (the ILS) to represent sustainment leadership (i.e., the Sustainment Leadership Scale). We also analyzed qualitative data on sustainment leadership to provide further support for the generalizability of the ILS dimensions to sustainment and to explore the possibility that additional dimensions of leadership may manifest during sustainment. Analyses of the quantitative data indicated strong support for the adapted measure. These results suggest that the aspects of leadership that research has shown to be critical during implementation (being proactive, knowledgeable, supportive, and perseverant) are also highly relevant to sustainment. This finding is consistent with the suggestion by Scheirer [
29] that the organizational issues factoring into successful implementation likely resemble those needed for successful sustainment. Thus, the SLS can be used by both researchers and practitioners interested in studying and/or assessing the role of leadership in sustainment. One particular avenue for future research is to consider change in sustainment leadership over time, particularly in light of the dynamic nature of the sustainment process [
28]. Such research could clarify whether leaders’ behaviors stabilize over time, as well as address whether consistency versus variability in leadership predicts future EBI sustainment.
Qualitative analyses also provided evidence to support the SLS. The convergence between the quantitative and qualitative results was high in that there was evidence for congruence in all four of the SLS dimensions in both methods. In addition, expansion was demonstrated in that the qualitative analysis revealed a possible fifth emergent theme: available leadership. Comments related to this theme addressed the benefits of having the leader in close proximity, being able to ask questions of the leader when needed, and leaders making time to assist with issues even when busy. This is consistent with independent work in other public sector service settings that identifies leadership availability as an implicit, but important aspect of support for EBI implementation [
16]. It is important to note that available leadership has connections to other dimensions of the SLS. For instance, being available is one way in which leaders show their support for sustainment. It is also related to perseverant leadership in that the availability comments by providers were typically related to the leader being accessible and engaged when problems or issues came up, much like perseverant leadership involves the leader persevering through the challenges of sustaining an EBI. Thus, we consider the finding of this dimension as preliminary and recommend that future quantitative research evaluate whether survey items related to availability emerge as a distinct factor or dimension and the degree of overlap with items related to either supportive leadership or perseverant leadership.
Another issue identified in the qualitative analyses that deserves additional research attention concerns level of leadership. In particular, when commenting on the leader’s knowledge of the EBI, providers suggested that there were differences between the lower-level leaders’ knowledge and the knowledge of leaders further removed from points of care. The quantitative research on the ILS by Aarons et al. [
20] and on the SLS in this study have focused on lower-level leaders, but more research is needed on middle- and upper-level leaders and their cross-level alignment as well [
42]. It may be that different or additional behaviors are necessary at the executive level during sustainment, such as pursuing funding resources. As such, research should evaluate whether the factor structure of the SLS is consistent across leadership levels, and whether additional items are warranted depending on leadership status. In addition, it may be that different dimensions of leadership are more or less relevant at different levels. For instance, it may be the case that showing general support for the EBI is most critical for upper-level or system leaders, but that being knowledgeable about the EBI is most critical for lower-level leaders who must manage the day-to-day issues that arise with direct service providers during both implementation and sustainment.
It is important to note that the approach used here and in the development of the ILS relied on provider reports of what makes for effective leadership during implementation and sustainment, rather than a criterion-related approach. Thus, more research is needed to show the relationship between the SLS and implementation-related outcomes, such as the climate for implementation and sustainment [
4,
7], employee attitudes toward EBIs [
43], and ultimately, fidelity [
44]. Although we did account for the nesting of the subordinates within teams and showed initial support for the aggregation of the measures at the team level, our sample size of 31 teams was not large enough to conduct multilevel confirmatory factor analysis [
45]. Thus, we cannot be certain that the factor structure of the scale is the same at the group level, and thus researchers using the scale to study aggregate perceptions should do so with caution. In addition, the ICC(1) values for the measure were not as strong as we would have expected. Given the relatively high within-group agreement levels, the ICC(1) values were likely attenuated by low between-group variability in the scores. Future research with a larger number of units and higher levels of between-group variability should address both the factor structure at the unit level and the ICC(1) values for the measure. Our focus in this study was subordinate perceptions of leadership, and we acknowledge recent streams of research addressing factors associated with levels of agreement or differentiation among subordinates in their perceptions of leadership [
46‐
48]. Future research should ensure that the level of theory for a particular study is aligned with the level of analysis for that study [
49] and should specifically address possible factors that may shed light on when there is agreement about sustainment leadership within the unit and when there is not. Another avenue for future research would be to examine the nature of sustainment leadership across settings. This study was performed in child welfare service settings, but it is possible that the other dimensions could emerge in settings such as mental health, substance abuse treatment, or nursing. Finally, the congruence or incongruence between leader self-perceptions and supervisee perceptions of the leader’s sustainment leadership could also provide insights into sustainment-related outcomes [
50,
51].
There are two potential limitations relating to the qualitative methods that are worth noting. The first is that the coding for this study focused on the set of questions that explicitly asked about leadership rather than the full transcript. This could have been a limitation as we did not examine whether additional responses were made about the leadership constructs, or whether we excluded additional themes that may have emerged in other parts of the focus groups. A second potential limitation is the use of focus groups rather than individual interviews. Although the majority of participants were open to discussing their leaders in a group setting, it is possible that individual interviews would have resulted in a more nuanced analysis [
52].
It is useful to consider the practical uses of the SLS, particularly in light of the ILS as a related tool. We do not view the ILS and the SLS as competing measures, but instead created the SLS so that researchers and agencies would have a tool available to address perceptions of leaders and their support of the ongoing delivery of an EBI after active implementation has ended. Although we would expect that strong implementation leadership would increase the likelihood for more positive sustainment, it is also the case that with changing organizational priorities, sustainment may be less emphasized by leaders than implementation, which could counteract initial positive sustainment as time passes. Thus, interventions to improve implementation leadership should also address possible approaches for how leaders can continue to support an EBI and its sustainment over time.