The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions

We employed a pragmatic, multifaceted approach, detailed in Table 1, to develop a more comprehensive strategy for characterizing adaptation design and process. This approach included multiple sources of data that better aligned with our goals than approaches such as scoping or systematic reviews [32]. We chose this pragmatic approach because several systematic reviews of adaptation have been published in recent years, and we recognized that such reviews might not capture aspects of modification and adaptation that had not been adequately documented.

We first searched the literature and identified existing frameworks, systematic reviews, and discussions of adaptations of public health and behavioral health interventions that had been published since 2013 [1, 6, 29, 33]. We also drew on systematic reviews of intervention modifications and adaptations, existing adaptation frameworks, and reviews on cultural adaptation [3‐5, 15, 17, 20, 21, 33, 34]. We employed a snowballing approach to examine a total of 170 individual articles in the literature that described adaptations to interventions. After reviewing all articles from three systematic reviews [1, 17, 29], we implemented a “stop rule” such that if no new aspects of adaptation were identified after examining 10 original sources from each additional systematic review, we ceased reviewing individual articles. Additionally, we examined two widely cited frameworks of implementation that listed potential determinants [30, 35] and a framework of social determinants of health [36, 37] to further refine the reasons for adaptation. After steps 1–8 (Table 1) were complete, we piloted the resulting framework by coding a sample of articles and interview responses to ensure that no additional elements of adaptation were identified, once again implementing the 10-article “stop rule”.

To complement our literature review, we conducted a rapid coding process with a set of responses to questions about adaptation. We generated the responses from interviews with 55 mental health providers and administrators that detailed processes and reasons for adaptation of a psychosocial intervention (see Table 1). These interviews were conducted through studies on sustainability and adaptation [2, 38].

Using the information generated through the two data sources, we categorized both reasons for adaptation and aspects of the adaptation process not captured in the previous framework. We then collapsed similar subcategories and organized them by consensus among the three authors.

Finally, to increase the likelihood that the updated framework would document aspects of modification and adaptation that were important to stakeholders, we presented a draft of the framework to several groups of stakeholders and solicited suggestions regarding additions, refinements, and clarifications.

Our approach yielded several additions and refinements to the original framework, which are indicated in bold in Fig. 1. Just as Proctor and colleagues [39] advocate for a multifaceted approach to reporting implementation strategies, our framework is intended to facilitate comprehensive documentation of modifications. Our Framework for Reporting Adaptations and Modifications-Enhanced (FRAME) includes the following eight aspects: (1) when and how in the implementation process the modification was made, (2) whether the modification was planned/proactive or unplanned/reactive, (3) who determined that the modification should be made, (4) what is modified, (5) at what level of delivery the modification is made, (6) type or nature of context or content-level modifications, (7) the extent to which the modification is fidelity-consistent, and (8) the reasons for the modification, including (a) the intent or goal of the modification (e.g., improve fit, adapt to a different culture, reduce costs, etc.) and (b) contextual factors that influenced the decision. Below, we describe in further detail aspects of the FRAME, with attention to aspects not included in the original framework.

Timing, not included in the original framework, is important to measure as modification can occur in any phase of the long-term implementation process: pre-implementation activities, an implementation phase, and scale-up and sustainment [40‐42]. During the planning or pre-implementation phase, there are opportunities to anticipate changes and discover adaptations that need to be made through a pilot phase. Despite these efforts, new challenges, constraints, or potential enhancements may come to light during the implementation phase. During scale-up or scale-out, other changes may be necessary, particularly for contexts where the intervention is likely to reach populations that may differ from the population that received the intervention during implementation [43]. Additionally, during sustainment, changes in the system or population served by that system may also necessitate changes [7, 44, 45].

Reporting on when and how decisions to modify interventions are made will allow investigation of whether planned adaptations are different in nature or in outcomes than those that are improvised during implementation. To avoid unplanned or reactive modifications that are inconsistent with an intervention’s goals, research base, or theory, investigators have advocated a process of planned adaptation [46], ideally as early as possible in the implementation process. However, in practice, many modifications are made less systematically [4]. Reactive modifications have been defined as those that “occur during the course of program implementation, often due to unanticipated obstacles” [47]. These modifications often occur in an impromptu manner, in reaction to constraints or challenges that are encountered [3], and may or may not be aligned with the elements of the intervention that make it effective [2, 4].

Adaptations are typically made proactively through a planning process that identifies ways to maximize fit and implementation success while minimizing disruption of the intervention [10]. However, due to the reactive nature of other modifications, we added a specifier to the framework to capture whether modifications were planned prior to delivery (i.e., an adaptation), or unplanned and made in response to an unanticipated challenge. It is important to note that iterative changes are not necessarily reactive—iteration can accommodate unanticipated challenges. For example, a modification made during the “Act” portion of a “Plan-do-study-act” cycle would not be considered reactive, because it was determined through a systematic process rather than through improvisation.

The driver of change, and how participatory the decision was, may be important predictors of whether the changes have the desired impact. The FRAME can be used to list all who play a role in the decision, but in some contexts, it may be important to specify who makes the ultimate decision, as this detail might affect whether and how widespread the modification may occur. It may also be closely linked to reasons for modification. For example, administrators’ decisions to modify an EBP may be related to restrictions in sourcing or contracting while policymakers’ decisions may be in response to political factors or funding availability. Modifications made by individual providers may respond to recipient-level needs or local constraints that may not be visible to policymakers.

The original 2013 framework [16] focused largely on characterizing what types of changes were made to facilitate understanding of which changes are associated with implementation success and recipient-level outcomes. Much of this component of the framework remains unchanged, although we added implementation and scale-up activities to reflect that these processes may differ across contexts. If little about the intervention were changed, but the implementation strategies differed significantly across otherwise similar contexts, then differences in outcomes may be attributable to differences in how the intervention was implemented.

Codes for the level of delivery also remain mostly unchanged from the 2013 framework. Reporting the level at which modifications occur has implications for understanding whether and under what circumstances implication success or effectiveness are associated with making individual-level modifications, and when modifications may need to be applied more broadly. The FRAME now differentiates the entire target group (e.g., women who are at risk of developing diabetes) and individual sub-groups (e.g., new mothers who are at risk of developing diabetes), as modifications may be made for broad or specific groups depending on the circumstances.

Although many items in this section are unchanged from the original framework, items were added to reflect a larger variety of modifications identified through the literature, ongoing observational work (c.f., [1, 3, 20, 21, 23, 48]), and stakeholder interviews. For example, it may be important to understand whether drift occurs for a relatively brief period of time before returning to a protocol or whether it occurs for the duration of a meeting or session, without a return to planned content. Additionally, spreading out psychosocial or educational content intended for a single meeting or session over multiple sessions was added, as this may occur when an individual requires more time to understand content or when unforeseen or emergent issues need to be addressed during a given session.

In previous work, we have made a distinction between fidelity-consistent and fidelity-inconsistent modifications. Fidelity-consistent modifications are defined as those that preserve core elements of a treatment that are needed for the intervention to be effective [49]. In contrast, fidelity-inconsistent modifications are those that alter the intervention in a manner that fails to preserve its core elements. The identification of fidelity-consistent and fidelity-inconsistent modifications can be made in consultation with the existing literature, input from the treatment developer, and any available evaluation data. Others have suggested that the function of intervention elements be prioritized over the form, such that what is core to the intervention is conceptualized as one of several possible activities or materials that accomplish a core intervention function (e.g., education, skill building, connecting to supports or resources) [50].

When implementing an intervention with a population that differs in important ways from the populations with which the intervention has been tested and implemented, new understanding of which elements are core vs. peripheral may emerge. For example, a component intended to improve basic health literacy may be essential for one population, but less necessary for a population that has a solid foundation of specific health information. Moreover, it may not be known whether some modifications, such as preserving the original spacing of sessions or activities, would have an impact on outcomes, and there may not be theory to inform decisions about such modifications either way. We have therefore added an “unknown” code that can be used when there is no theory or evidence to inform a decision about whether an element is core vs. peripheral.

The additions related to the rationale for modifications represent the most significant enhancements to the original framework. Capturing the rationale for a given modification may be crucial in determining links to key implementation or health outcomes [20]. For example, modifications made to cut costs may have a very different impact than those made to improve fit or engagement. Thus, we specify the goal for modification—to improve feasibility or acceptability, to increase reach or engagement, to improve fit (note that cultural modifications intended to improve fit are assigned a subcategory under this goal), to reduce costs, to improve clinical outcomes, or to align the intervention with cultural values, norms, or priorities.

To identify reasons to modify EBPs for FRAME, we referred to existing reviews and original literature that enumerated reasons for specific projects and implementation frameworks that specified potential determinants at different levels. We specifically added cultural modifications within the FRAME due to the importance of cultural modifications in implementation science. Attention to cultural factors is important in investigating modifications to EBPs because of its potential consequences in terms of healthcare disparities [15]. Cultural adaptation can be defined as the systematic modification of an intervention to “consider language, culture, and context in such a way that it is compatible with the client’s cultural patterns, meanings, and values” [35]. Focusing on culture allows us to expand the characterization of content modifications (e.g., whether content is added, removed, or tailored) and context-level modifications (e.g., whether personnel, training, or education are modified) to explicitly address cultural patterns or values at the client, provider, or sociopolitical level. Designation of cultural or religious norms at the organizational level distinguish adaptations made to distinguish this form of culture from the construct of organizational culture used in the implementation and organization literature to signify “the way things are done in an organization” [51]. At times, cultural or religious beliefs may contribute to organizational culture or policies, but the constructs do not fully overlap.

By explicitly defining cultural aspects and determinants that affect inequities in care delivery, we hope to identify the types of modifications made to address cultural aspects of the populations included in our studies. Thus, we also examine, more broadly, factors at the recipient, provider, organizational, and socio-political levels that are examples of important components that affect the modifications of the EBPs.

At the organizational level, we found that factors that may lead to modifications overlap somewhat with those found in existing determinant frameworks (e.g., [41, 52]). For example, staffing shortages may suggest a variety of context modifications. This may include delivery by different personnel (e.g., providers from a different discipline) or changes to the format or timing of delivery. Such staffing shortages may also affect training or evaluation of the intervention. For example, implementation may require streamlining training sessions, adapting them to fit with providers’ previous training, or spreading them over several weeks to accommodate busy clinic schedules.

Other practical constraints play a role in decisions to adapt or modify interventions [2, 15, 19, 23]. Space shortages may indicate the need for context modifications (e.g., changing from group to individual delivery). In contexts without easily accessed health centers, community-, home-, or telehealth-based delivery formats, there may be contextual adaptations to address demand or need. Limitations to available technology can have far-reaching implications as well as it may require removal or adaptation of aspects of the original intervention that can be delivered in under-resourced settings. Time constraints may lead to removing elements or compressing the intervention.

Aspects of organizational/setting (including local community if not delivered in a healthcare setting) culture may also impact how interventions are delivered. Competing demands, de-prioritization of an intervention, or high rates of turnover may lead to changes in who delivers the intervention, how many sessions or elements are provided, and whether and how training is provided. Regulatory or compliance issues or legal concerns may lead to certain aspects of an intervention not being delivered (e.g., limits to the types of physical activity or activities that may occur off-premises). An organizational culture that has long espoused a different theory or intervention may leave providers wary of new practices and lead to integration of elements of interventions into existing practices rather than de-implementation of preferred practices [53]. In sum, organizational/setting factors—including organizational culture and available resources—may necessitate a variety of modifications even to interventions with a strong evidence base in specific contexts.

At the provider level, there has been discussion around the positive or negative consequences of modifying interventions to fit with provider preferences or to improve the interaction between providers and their clients [1]. Providers of psychosocial interventions frequently modify interventions for a variety of reasons, including perceived client preferences, providers’ preferences or self-efficacy [54], and efforts to maintain a good therapeutic alliance [55‐58]. Factors such as provider gender and cultural beliefs may also impact decisions about delivery of the intervention. Additionally, some provider factors, such as previous training and experience, may lead to changes to training and evaluation.

Recipient level factors are also identified in Fig. 1. Each identified factor may contribute to a need for modification to promote optimal levels of engagement and outcomes at different levels. For example, limited transportation might make face to face meetings in a clinic difficult, which could lead to modifications in the format (e.g., telephone or internet-based) or setting (home-based). On the other hand, low literacy might make full engagement in an intervention that relies on written materials difficult and require tailoring to deliver content to clients in other formats. Other aspects, such as cultural norms, legal status, or physical capacity, may necessitate the removal or alteration of some elements of the EBP.

Although our approach to refining the framework is unique in its use of multiple data sources and stakeholder input, some limitations are important to describe. We did not employ a systematic review or a traditional thematic analysis because these approaches did not fully align with the current project goals. While it is possible that additional items would have been identified through these processes, our use of a “stop rule”, stakeholder feedback, and coding of subsequent articles using the FRAME after consolidating information from all of our data sources suggested that the framework was sufficiently comprehensive.

Furthermore, stakeholders identified a need to balance comprehensiveness with feasibility and pragmatism in documentation and reporting. The feasibility of using a comprehensive framework is likely to differ across research and applied settings. We must also acknowledge that comprehensively cataloging modifications to EBPs may be difficult in some contexts even with a well-developed framework like the FRAME. For example, if an intervention has not been exhaustively described and tested—as is frequently the case in healthcare—then it may be impossible to reliably detect adaptations and their associated consequences. In these cases, the FRAME will only be as useful as the data informing its application. Evaluation of different approaches to using the FRAME for documentation is needed to inform efforts to achieve the appropriate balance.

Several strategies for reporting adaptations and modifications that may be applicable to the FRAME have been developed and described in the literature. In this section, we discuss the advantages and drawbacks to these strategies.