Incorporators, Early Investors, and Learners: a longitudinal study of organizational adaptation during EBP implementation and sustainment

Dominant definitions emphasize adaptation as the systematic modification of an evidence-based practice (EBP) to fit provider characteristics, organizational contexts, and service settings [1‐3]. It is widely accepted that such adaptation is an unavoidable and even beneficial aspect of EBP implementation [1, 4]. Baumann and colleagues provide a framework for understanding EBP adaptation that includes three types of adaptation: changes to content, context, and training and evaluation [1]. Context-level changes involve changing the EBP format (group or individual), setting (outpatient or community-based), personnel (mental health professional or peer specialist), and population (tailoring intervention to a new client group, e.g., ethnic minority groups, veterans) [1, 5].

Researchers have relied on this conceptualization of EBP adaptation to expand the literature in several important ways, including systematic reviews that explore methods and reasons for, outcomes of, and guidance around intervention adaptation [2, 6, 7]. Scholars have also proposed a framework for reporting EBP adaptations and modifications and a taxonomy for practice adaptations [3, 8]. Additionally, new research suggests that EBP adaptation should be considered as a value equation in which the needs, preferences, and contingencies of stakeholders at system, organization, provider, and patient levels are taken into account [9]. To date, however, researchers often treat organizations as a “structural factor” that can influence how EBPs are adapted [1]. In the present study, we examine organizational adaptation as an ongoing process and explicitly address the internal changes that organizations make (i.e., their adaptations) to accommodate implementation of new EBPs.

An implementation strategy is “a systematic intervention process to adopt and integrate evidence-based health innovations into usual care” [10]. The Expert Recommendations for Implementing Change (ERIC) compilation includes a number of strategies that encourage changes or adaptations to structure and/or process in an implementing organization [11]. Examples include creating new clinical teams, organizing clinician implementation meetings, changing physical structure, enhancing or procuring new equipment, developing quality-monitoring tools, and conducting ongoing training [11]. We conceptualize organizational adaptation as an implementation process that includes both planned and unplanned organizational changes and use of implementation strategies.

Strategic management is a subfield of organizational research that focuses on “the analysis of internal and external environments of [an organization] to maximize the utilization of resources in relation to objectives” [12]. This type of research is motivated by the notion that organizations must adapt to and influence the external environment to survive, and they do so by pursuing a number of different strategies [13, 14]. Strategic management research explores how organizations achieve competitive advantage and manage both predictable and unpredictable environments [13, 14]. Major literature streams describe how organizations develop and implement strategies and assess the relationship between different strategies and organizational performance [13]. Practically speaking, “The major importance of strategic management is that it gives organizations a framework for developing abilities for anticipating and coping with change” [12]. This is directly relevant to understanding how organizations might adapt (to and with a dynamic external environment) during implementation to achieve EBP sustainment.

Typologies focus on “patterns or profiles rather than individual independent variables” [15]. Since the late 1970s, they have served as influential tools for organizational theory-building in the management literature [15‐20]. One of the most widely cited and empirically tested typologies is the four organizational types proposed by Miles and Snow: Prospector, Analyzer, Defender, and Reactor [19]. Prospector organizations embrace change and opportunity, actively seek out opportunities, and create a range of new products [19]. Defender organizations prioritize stability and focus on a limited number of products in a small market sector [19]. Analyzer organizations combine the strengths of Prospectors and Defenders in that they balance the exploration of new market opportunities with the maintenance of a core base of traditional products and customers [19]. Reactor organizations are defined by constant instability that causes them to respond ineffectively to environmental change and uncertainty [19].

It is important to note that this is not just a conceptual exercise, as evident in the wide empirical application of and development of measures for Miles and Snow’s profiles [21‐24]. In an example that may be more familiar to implementation researchers, researchers used a multi-item measure of the Miles and Snow typology to identify distinct strategy types that hospitals use [25]. In addition to the original Miles and Snow groupings, they identified new typologies that were particular to hospitals (e.g., “Defensive Analyzer” and “Prospective Analyzer”) [25]. We hope that the development of organizational adaptation profiles can prompt similar measurement development and that profiles can be a new way to understand relationships between organizational patterns and implementation outcomes.

Typologies require a configurational view of organizations in which organizations are viewed holistically and a variety of activities and characteristics can occur within each type [15]. When assessing organizational adaptation that occurs during EBP implementation, it may be tempting to focus on a variable such as organizational readiness. Organizational readiness has been defined as “shared resolve to implement a change (change commitment) and shared belief in their collective capability to do so (change efficacy)” [26]. This can be a useful construct in pre- and early implementation phases for assessing the degree to which organizations are prepared to adopt a new innovation.

A core assumption that underlies organizational typologies is that there may be a set of different activities, approaches, and characteristics that define types. Therefore, measuring the same pre-determined list of independent variables across organizations and across time (e.g., readiness) may not capture the unique dynamic or static structures and/or processes that explain a given profile or type of organization. To our knowledge, this way of categorizing organizational adaptation has not yet been applied to implementation science.

Perhaps the closest example in existing implementation literature is the organizational culture and climate profiles that derive from Glisson et al.’s organizational social context (OSC) measure [27]. For example, Williams et al. identified four profiles (comprehensive, supportive, indifferent, and constrained) that describe the organizational cultures and climates of schools implementing EBPs for autism [28]. The OSC profiles have been associated with implementation outcomes, clinician behavior, and service quality, and are the basis of the Availability, Responsiveness, and Continuity model of organizational effectiveness [29]. Additionally, the presence of national OSC profile norms allows organizations to compare themselves to other similar organizations [29]. The OSC profiles inform targeted efforts to improve culture and climate. We aim to use profiles to describe the adaptation approaches that organizations take when implementing and sustaining an EBP.

SafeCare® is a well-established, behaviorally prescriptive, and flexible home visitation EBP designed to prevent child neglect and improve outcomes for children at risk of abuse and neglect [30, 31]. SafeCare home visitors work in families’ homes to improve parent skills in parent-infant and child interactions, home safety, and health [32]. Home visitors, coaches, and trainers achieve and maintain certification, and coaches work closely with home visitors to ensure ongoing adherence to the SafeCare model [32]. Current SafeCare adaptation research focuses on changes to the content and delivery of the intervention or cultural adaptations [33‐39]. Aarons and colleagues recently published a taxonomy of SafeCare model adaptations [8]. Adaptation types included changes to process, presentation, dosage/intensity, ordering, and content, as well as forestalling, selective integration, exclusion, and supplementation of material [8]. The present study shifts our focus and extends existing EBP implementation research by examining ways that organizations adapt. We hope that drawing attention to organizational adaptation can provoke a more nuanced and layered conversation about adaptation in implementation research.

The goal of this study is to identify and compare different organizational-level profiles of adaptation documented qualitatively during SafeCare implementation and sustainment. The research question is: How do community-based organizations (CBOs) adapt internally over the course of SafeCare implementation? Our longitudinal qualitative dataset is based on interviews and focus groups conducted among administrators and staff at 17 CBOs in one statewide and seven county-based child welfare systems. Rather than focus our analysis on discreet instances of organizational change, we create composite, comprehensive organizational adaptation profiles based on our prospective data.

We use the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework as a conceptual lens. The EPIS framework draws attention to the inner (organizational) and outer (system) contexts, factors that bridge inner and outer contexts, innovation factors, as well as determinants and mechanisms specific to, and across, implementation stages [40, 41]. EPIS framework determinants consider how organizations may adapt in relation to specific factors (e.g., organizational staffing processes, the system-level outer contexts in which organizations operate, e.g. funding/contracting), and how this context may help explain organizational adaptation. Additionally, we used the EPIS phases to attend to the ways that organizational adaptation develops and changes over time. Lastly, the concepts of structural and ideological fit featured in the EPIS framework may explain why an organization might employ a particular adaptation approach. Structural fit describes the degree to which an EBP (or the process of implementing the EBP) aligns with an organization’s existing profile including roles, responsibilities, resources, and capabilities [40]. Ideological fit describes the ways that an EBP aligns with an organization’s mission, and the values, goals, and priorities of the individuals within it [40]. Lack of structural or ideological fit may prompt organizational adaptation to assimilate (with good fit) or accommodate (requiring organizational change).

Organizational adaptation profiles were constructed using a cross-case pattern analysis [42]. The empirical unit, or case, was the CBO [42]. We constructed the case summaries using several sources of data: transcript material, SafeCare manuals, and contracting documents from a previous study [43]. Using the case summaries, we assessed each organization’s adaptation process and then compared patterns of organizational adaptation across the cases. These adaptation patterns became our adaptation profiles. Each organization’s profile label was checked and validated using a visual elicitation tool.

This secondary analysis draws from qualitative data collected from CBOs for three prospective mixed-methods parent studies. The parent studies were conducted 2008–2013, 2011–2015, and 2012–2017 and built upon a program of SafeCare effectiveness and implementation research that began in 2005. Inclusion criteria were that CBOs were in systems that achieved SafeCare sustainment status and had at least two time points for the qualitative data collection. Sustainment status was determined by Aarons and colleagues’ criteria based on Stirman et al.’s systematic review of sustainment [44, 45]. “Full sustainment” referred to sites that continued to deliver SafeCare while meeting key fidelity requirements [44]. These sustaining sites had certified SafeCare providers, ongoing coaching and fidelity monitoring, and SafeCare team meetings that adhered to model developer standards [44]. The sample for this study included 17 CBOs in one state (State) and seven county-based systems (County A, County B, etc.).

We drew from individual and small group semi-structured interviews and focus groups with personnel involved in SafeCare implementation at each of the 17 CBOs. The personnel include frontline staff, team leaders and supervisors, and agency executives, administrators and/or leaders. We supplemented these data with interviews conducted among statewide and county administrators in the participating service systems. Interview and focus group questions assessed various aspects of SafeCare implementation and sustainment. Previous studies detailed data collection methods and the foci of interview guides [46‐49]. For the 2016 data collection cycle, the first author (RL) developed stem questions and prompts specific to organizational adaptation processes (Additional File 1).

Data collectors in the parent studies digitally recorded, professionally transcribed, de-identified, and checked all transcripts for accuracy. For this analysis, 177 transcripts were analyzed, including 119 individual interviews, 20 small group interviews, and 38 focus groups. Table 1 describes the data types and years collected for each of the eight service systems. We used the analysis of 113 SafeCare contracting documents from Lengnick-Hall et al. to contextualize the transcripts and verify information that participants provided [43].

The number of qualitative sources and data collection time periods depended on when each CBO took part in the parent studies. Given the specific aims of the parent studies, some CBOs have a greater number of transcripts represented in the dataset than others. Several steps were taken to mitigate this. First, of the 17 CBOs in the sample, two had only one round of qualitative data. RL conducted follow-up interviews for these two CBOs in August and September 2018 to get a second time point. RL asked participants to describe implementation challenges, organizational changes that were made, and plans for sustaining SafeCare. Second, peer debriefing and member-checking activities (described below) provided additional opportunities to learn in more depth about each organization and supplemented the transcripts. Third, participants frequently discussed other organizational and service system actors in their interviews and focus groups. Information about each CBO not only featured the perspectives of individuals in that organization, but also those of other participants familiar with the service system and informed by knowledge of interorganizational networks (as described in the EPIS framework) [40, 41].

A second data source was the most recent SafeCare manuals for home visitors, coaches, and trainers, issued by the National SafeCare Training and Research Center (NSTRC). The home visitor manual included an overview of SafeCare, how to engage clients, and detailed steps for implementing each module. The coach and trainer manuals provided an overview of the key responsibilities and processes for each respective role. The coaching manual directly addressed adaptation. In our study sample, minor practice-level adaptations to the SafeCare model may have occurred, and if made, were vetted by NSTRC.

We employed several strategies to ensure analytic rigor. Peer debriefing meetings among research team members occurred early in the process. Peer debriefing “is a type of group reflexivity that gives the researcher fresh perspectives and guards against bias” [50]. The purpose of the peer debriefing meetings was to identify any gaps or biases in the initial coding and review coding decisions for each case. There were five peer debriefing meetings (November 2017 to June 2018) that included a qualitative data expert from the parent studies and lead investigators, all of whom had extensive experience with the CBOs and SafeCare implementation issues.

Member checking was used at the end of the process as a way to validate the final organizational profile labels for each case. Member checking also helped to guard against researcher bias and allowed RL to verify the adaptation profiles with a broader group of individuals involved in the parent studies [50]. RL led six member-checking meetings (July 2018 to September 2018) with seven researchers who were highly involved in the parent studies (including CW, MH, GAA) and three practitioners. One of the practitioners served as a SafeCare coordinator and assisted with implementation in five of the eight service systems. The other two were from organizations that only participated in the most recent iteration of the parent study. Meetings occurred in-person, via phone call, and by video conference. Case summaries and profile labels were revised after each meeting.

Detailed written notes were taken at every peer debriefing and member-checking meeting. We also created a comprehensive audit trail that included samples of raw data, memos for each transcript, iterative versions of a codebook, case summary reflections, and initial figures that we made as we began to categorize different organizational adaptation patterns [50].

We created case summaries for each organization, which involved assembling and summarizing types of adaptation, contexts for adaptation, and when changes were made to characterize each CBO holistically across the implementation stages [50]. First, RL completed line-by-line initial coding of the 177 qualitative transcripts for content that was potentially adaptation related. Initial coding is more inductive, “provisional, comparative, and grounded in the data” [51]. Coded material was sorted by CBO so that each organization’s codes could be reviewed in peer debriefing meetings [50]. In response to peer debriefing, all transcripts were reviewed a second time using focused coding, a more deductive approach [52]. The qualitative data expert and lead parent study investigators highlighted academic partnerships, referral processes, job roles, and client age as particularly important to understanding this study context. These were the sensitizing topics for the focused coding [51]. During focused coding, we began to move across cases and identify and compare organizational adaptation patterns that could help distinguish the profiles [51].

Transcript coding was supplemented and contextualized with a document review. We reviewed the most recent home visitor, coach, and trainer manuals to see how program developers discussed adaptation. SafeCare manuals helped us determine the boundaries of the intervention, and we structured our review around several questions provided in Table 2. The last row of the table includes potential organizational adaptation, which was our focus. We also used the extensive assessment of contracting arrangements completed in Lengnick-Hall et al. [43] to contextualize how contracts supported and/or constrained SafeCare implementation, if organizations were adapting in response to contracting arrangements, and if changes to contracting arrangements represented a type of adaptation. Each case summary described organizational adaptation in terms of what, when, and how organizational-level changes were made.

We used a cross-case pattern analysis to develop the organizational adaptation profile labels and sort each case into a profile [42]. When looking across the case summaries, the major patterns that we observed pertained to the scope of organizational changes that were made and at what implementation stage these changes were happening. During the cross-case pattern analysis, we realized that the initial categories of minimal, moderate, and extensive adaptation (interview questions in Additional File 1) did not account for the change in organizational adaptation over time that we were observing in some cases. Our initial profile labels were “Minimal adaptation,” “Adaptation decreases over time,” and “Active adaptation over time.” The next iteration of profile labels was “Steady level of minimal or moderate engagement in adaptation,” “Active adaptation at first, then drops off,” and “Proactive adaptation over time.” Our final profile labels, after extensive conversations with the broader research team, were “Incorporators,” “Early Investors,” and “Learners”. Each case was assigned to one of these profile categories for the member checking described next.

We created and used a visual elicitation tool to confirm the adaptation profiles through member checking [50, 53]. During the meetings, specific examples and each organization’s overall approach to adaptation were discussed using a visual elicitation tool “adaptation snapshots” [50, 53]. We created “adaptation snapshots” (Fig. 1) for each CBO. The snapshot had case summary information (e.g., specific changes that they were made and during what implementation stage) as well as an organizational adaptation profile designation. The snapshots were not meant to count every instance of adaptation. Instead, we used the snapshots during member checking to visually describe how the organization adapted, check in about our interpretations of these adaptation processes and examples, review the broader service system context for these adaptations, and verify analytic decisions (including the profile label we applied) [50].

There were seven incorporators in five of the service systems. Incorporators were CBOs that “incorporated” SafeCare into existing structures, processes, and capabilities. A key feature of Incorporators was that they primarily integrated the EBP into what they already did as an organization, rather than making substantial changes or creating new ways of operating. This was akin to assimilation. Over time, Incorporators made relatively small-scale organizational adaptations in response to contract and intervention requirements. Small-scale adaptation included adding screening assessments or supplemental material required by the service system to the home visit, adjusting caseloads, adding SafeCare to job descriptions and titles, modifying supervision formats to accommodate SafeCare coaching requirements, and adding the SafeCare program to existing agency forms and documents. As one Incorporator agency leader commented about SafeCare implementation, “…you have to do some things but it’s not overwhelming or hard to do.”

Five of the Incorporators were in a service system that had a dominant lead agency (three within the same service system). One Incorporator agency leader explained the dynamics with the lead agency as, “Over time especially I have really learned how to allow the coaches and to allow [lead agency name] to be the people who they go to for implementation issues.” The major responsibilities associated with initiating, building, and maintaining SafeCare in the system fell on these lead agencies, not the Incorporators.

Six of the Incorporators had contracting arrangements that covered the full costs of implementing SafeCare. An Incorporator supervisor explained, “…I cannot think of anything that needs to be changed for us to better do SafeCare…We have our supplies, we have our transportation, we have our supervision, we have everything...” Another Incorporator agency leader said, “I don’t think there was a need to make extraordinary changes with the resources that were put in place.” Changes related to insufficient funding or lack of resources were not an issue for Incorporators.

Finally, for two of the Incorporators, SafeCare was described as a small part of the agency’s service array. One Incorporator agency leader observed, “It’s a small piece of kind of the tapestry of who we are…our program [SafeCare] has literally one person.” This leader further explained the organization’s adaptation as, “… minimal. Again just the scale of it really wouldn’t drive a lot of heavy lifting changes.” When asked about the percentage of total revenue derived from SafeCare, an agency leader at the other Incorporator organization responded, “A small percentage, very small percentage.”

Six Early Investors were present in six service systems. Early Investors were CBOs that made substantial adaptations during the initial phases of implementation. Agency leaders at two different Early Investors described this initial effort as “blood, sweat, and tears getting this up and running,” and they “set the stage” for what SafeCare would look like in the entire system. The substantial adaptations that Early Investors made in early implementation were related to the critical system-wide implementation role that these CBOs played. Five of the six Early Investors acted as lead SafeCare agencies in their respective service systems. The sixth Early Investor was initially a lead agency but then became a subcontractor after contracts were renegotiated.

Lead agencies bore significant responsibilities in initiating and maintaining SafeCare in their systems and may have been formally designated as leads in the contracts. Early Investors made strategic adaptations to fulfill a variety of lead agency responsibilities. These responsibilities centered on: (1) coordinating and conducting SafeCare training, coaching, and fidelity monitoring across multiple CBOs; (2) acting as liaisons with service systems, other CBOs, and the NSTRC; (3) managing system-wide SafeCare data; and (4) housing, supporting, and finding ways to pay for at least one SafeCare coordinator. Terms used across the transcripts to describe SafeCare coordinators included “main contact person,” “a resource,” “SafeCare subject matter expert,” “a buffer,” “de facto champion,” and “honorary administrator.”

All of these lead agency responsibilities required substantial adaptation related to developing EBP expertise, creating new processes and procedures, hiring staff or modifying staff roles (especially in the context of training and coaching), and building leadership and data management capacity. These types of changes required greater organizational effort and resources than the way that Incorporators engaged in adaptation.

Additionally, Early Investors changed their approaches to adaptation over time. While Early Investors maintained the organizational changes that they made during early implementation, overall engagement in adaptation tapered off as SafeCare was embedded in the CBO itself and the broader service system. When reflecting on adaptation longitudinally, an agency leader at an Early Investor stated, “I’d say we made extensive changes in the beginning and then minimal changes to sustain it over time.” In this case, the extensive changes had to do with the “entire cultural shift” that had to occur to support SafeCare (described below). An Early Investor team leader at a different agency noted, “Since we’ve been doing this for a few years now, it’s the expected practice…We have a well-oiled machine.” Regarding later implementation stages, an Early Investor supervisor said, “Creativity had been performed already.”

For four Early Investors, SafeCare implementation represented a major cultural shift in the organization. Overcoming ideological misfit and resistance to EBPs generally or SafeCare specifically was part of the early investment that these organizations had to make. Factors that helped to resolve ideological misfit included hiring new staff who were more amenable to the model (one CBO had “massive turnover” as a result of SafeCare implementation), having leaders who made a commitment to “find out what’s good about it and move forward,” creating SafeCare guidelines that could be used when training new staff, and generating buy-in from social workers in the service system who needed to refer clients. Another Early Investor supervisor had to figure out how to integrate providers who had not worked together prior to SafeCare implementation. This supervisor explained, “it was very important for us to get that as one of those elephants in the room, address it, and really have worked strategically to keep our team blended.”

Four Learners were present in three service systems. Learners were characterized by steady and continuous attention to new ways that the CBOs could support SafeCare. Learners did not necessarily have more instances of organizational-level changes. Instead, these organizations appeared to have an intangible quality related to a culture and/or leader attitudes that reflected and encouraged curiosity, openness, and a focus on continuous improvement. One Learner agency leader described adaptation in response to SafeCare implementation as, “I want to say we’ve made a lot of changes just to maintain or support the program…we are constantly trying to adapt...” Another Learner leader noted, “I’m committed to knowing or finding out if we’re being effective and wanting to evaluate. I think we have to. We can’t just stumble along in the dark…”

When talking about SafeCare implementation experiences, individuals in Learner organizations used phrases such as: “continually update,” “took the initiative,” “able to quickly adapt or make changes as needed,” “constantly trying to assess and reassess,” “keeps us from being stale,” “constantly tweaking,” “ongoing conversation,” “is there another way we can deliver the service that we haven’t thought about,” and “we’re just ever evolving.” A Learner agency leader explained, “I think there’s just always something to look at through the eyes of how do we sustain and make it part of the overall program versus it’s just something that we do?” Another agency leader noted, “…we’ve fostered or trained ourselves to really value that ongoing learning.” For Learners, SafeCare implementation represented a positive educational experience for the organization’s staff.

Two of the Learners were in stable, well-funded service systems, while two Learners did not have SafeCare contracts that covered full costs of implementing the EBP. One of these Learner agency leaders explained, “…We have to be quite vigilant about keeping up the caseloads and keeping up the presence and the visibility so that we don’t get forgotten about.” The other Learner agency leader with the underfunded SafeCare contract discussed multiple financially related adaptations that the CBO had made, including cross-training staff (beyond the full-time employee covered by the SafeCare contract) and pulling from discretionary funds to cover the costs of SafeCare implementation. Similar to Incorporators, Learners maintained their overall approach to adaptation across the implementation stages outlined in EPIS.

This study examined organizational adaptation profiles in the context of SafeCare, one particular EBP. Intervention complexity varies widely, and different organizational approaches to adaptation may emerge with other EBPs. A second limitation is that this study examined organizational adaptation in the context of systems that utilized contractual arrangements to promote and implement SafeCare. Organizations in systems that do not have contractual obligations to implement a particular EBP may engage in other adaptation approaches.

This study used secondary qualitative data to understand adaptation. Social desirability biases and the tenure of participants within a CBO may have influenced the quality of responses. Participants were also recalling and reflecting upon past events. A document review and member checking activities facilitated triangulation of the findings. However, these limitations echo the broader issue that adaptation can be difficult for practitioners to explicitly recognize and report, and accurately assessing adaptation is a persistent methodological issue in implementation science [1].

This study lays the groundwork for multiple avenues of future research. First is the creation of adaptation profiles for other contexts including organizations that do not reach sustainment, are de-implementing a non-evidence based practice, or are implementing a novel EBP. Future research can also explore which implementation activities and strategies are most effective for Incorporators, Early Investors, and Learners. Quantitative methods can be used to further explore characteristics that define the profiles and the relationship between profiles and specific implementation (e.g., fidelity) and client outcomes (e.g., engagement and satisfaction).

Additionally, future research can explore adaptation in the context of the constellation of EBPs that an organization implements. Organizations may use one adaptation approach for one EBP and a different approach for a different practice. For example, some interventions may be a more integral piece of the agency’s total service array (and therefore require a more intense organizational adaptation process). Organizational adaptation profiles may also change in response to environmental threats such as drastic changes in funding, changes in the organization’s mission, or the entrance or exit of key organizational players in the broader systemic network. Or perhaps, an overarching adaptation philosophy may influence organizational adaptation approaches across EBPs.

Future work may also consider the relationship between concurrent organizational adaptation processes, e.g., simultaneous practice, organizational, and system level adaptation that occurs during EBP implementation. Finally, while the importance of cultural and leadership factors for the Learner profile emerged in the qualitative analysis, understanding more precisely how this profile differs from the others requires targeted data collection around published learning organization characteristics.