Criteria for selecting implementation science theories and frameworks: results from an international survey

Theories and frameworks offer an efficient way of generalizing findings across diverse settings within implementation science [1]. Theories and frameworks (see Department of Veterans Health Administration’s Quality Enhancement Research Initiative [2013] for a taxonomy of theories, frameworks, and models, hereafter “theories”) generalize findings by providing a synthesizing architecture—that is, an explicit summary of explanations of implementation-related phenomena to promote progress and facilitate shared understanding [2]. Furthermore, theories guide implementation, facilitate the identification of determinants of implementation, guide the selection of implementation strategies, and inform all phases of research by helping to frame study questions and motivate hypotheses, anchor background literature, clarify constructs to be measured, depict relationships to be tested, and contextualize results [3].

Given their potential benefits, the underuse, superficial use, and misuse of theories represent a substantial scientific challenge for implementation science [4‐8]. In one review, Tinkle et al. [4] highlighted pervasive underuse of theory (i.e., not using a theory at all); most of the large National Institutes of Health-funded projects that they reviewed did not use a theory. Likewise, a review of evaluations of guideline dissemination and implementation strategies from 1966 to 1998 showed that only a minority (23%) used any theory [5]. A scoping review of guideline dissemination strategies to physicians covering 2006 to 2016 showed that, although theory use had increased over time, fewer than half (47%) of included studies used a theory [8]. While theory use appears to be on the rise, it remains underused. Kirk et al. [9] reviewed studies citing the Consolidated Framework for Implementation Research and found that few applied the framework in a meaningful way (i.e., superficial use). For example, many articles cited the Consolidated Framework for Implementation Research (CFIR) in the “Background” or “Discussion” sections to acknowledge the complexity of implementation but did not apply the CFIR to data collection, analysis, or reporting findings. Similar results were found for studies conducted through 2009 that cited the use of the Promoting Action on Research Implementation in Health Services (PARIHS) framework [10]. Gaglio et al. [11] found that the most frequently studied dimension of the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework (reach) was often used incorrectly (i.e., misuse) [12]. Reach compares intervention participants (numerator) to non-participants (denominator). Examples of misuse include comparisons of participants to each other rather than to non-participants (e.g., [13]). The underuse, superficial use, and misuse of implementation theories may limit both the field’s advancement and its capacity for changing healthcare practice and outcomes.

The underuse, superficial use, and misuse of theories may relate, in part, to the challenge of selecting from among the many that exist in the field [14, 15], each with its own language and syntax, and varying levels of operationalized definitions [16] and validity [17]. Implementation researchers and practitioners (i.e., implementation scientists) have at their disposal myriad theories developed within traditional disciplines (e.g., sociology, health services research, psychology, management science) and increasingly within implementation science itself [18]. A move toward synthesizing theories may address the overlap; however, the question of which theory to select remains [19]. Therefore, implementation scientists would benefit from guidance for selecting a theory for a specific project. Guidance for theory selection may encourage implementation scientists to use theories, discouraging underuse; to use theories meaningfully, discouraging superficial use; and to be mindful of the strengths, weaknesses, and appropriateness of the theories that they select, discouraging misuse. Guidance for theory selection will promote theory testing and identification of needs around theory development, contributing to the advancement of the science. Indeed, applying theory meaningfully provides an opportunity to test, report, and enhance its utility and validity and provides evidence to support adaptation or replacement. As a first step toward the development of guidance for theory selection, this study aimed to identify which theories implementation scientists use, how they use theories, and the criteria used to select theories.

An international sample of implementation scientists collectively reported using more than 100 theories to inform implementation planning and evaluation to guide data collection and analysis, characterize features of the project environment and relationships between key constructs, and guide interpretation and dissemination of project outcomes. The theories that were most commonly used, including the CFIR, TDF, and Diffusion of Innovations, were used by respondents from nine different countries across four continents. Some respondents developed theories “in-house,” adapted existing theories, or combined components of multiple theories to meet the needs of their project.

Findings indicate that implementation scientists use a large number of criteria to select theories. It is possible that this large number reflects the varying sets of criteria that implementation scientists must consider depending on a theory’s intended use. (We describe this possibility in more detail below.) It may also be possible that the large number of criteria that implementation scientists consider when selecting theories reflects a lack of clarity regarding how to select theory. Indeed, our findings suggest that there is little consensus on which criteria are the most important. This may contribute to the persistent underuse, superficial use, and misuse of theories [4‐8].

Our qualitative results suggest that the process for selecting implementation theories is often haphazard or driven by convenience or prior exposure. Selecting theories based on convenience or prior exposure may deepen knowledge about a given theory with repeated use; however, doing so also has the potential to limit theories’ benefits, particularly if theories are poorly suited to users’ objectives (e.g., selecting implementation strategies, framing study questions, motivating hypotheses). Convenient or familiar theories may contribute to silos in the field, limiting our ability to generalize findings, promote progress, and promote shared understanding.

This study had several limitations. We opted not to conduct a systematic literature review to develop the survey because the literature on this topic is likely diffuse and difficult to identify; thus, we began with key contributions of which we were aware. We included open-ended items to address the bias associated with this approach; however, we acknowledge that the criteria listed in the survey may have influenced responses. Though we tried to capture variation across respondent characteristics, our survey sample might not have been fully representative of perspectives in the field. For example, we did not have as many practitioner respondents or as many respondents from outside North America and Europe as we hoped; nevertheless, this appears to be the first attempt to assess the criteria that implementation scientists use to select theory, and future efforts to understand and streamline theory use should further consider the perspectives of these groups. Additionally, since we recruited a convenience sample, those who completed the survey may be systematically different from those who opted not to complete the survey. We did not ask respondents about the extent to which they value theory in their work. Indeed, we acknowledge that some implementation scientists do not view theory as critical to their work (e.g., [24, 25]).

Our results suggest that implementation scientists may benefit from guidance for theory selection. Developing such guidance is challenging given potential variation in implementation scientists’ roles, priorities, and objectives, limiting the benefit of prescriptive guidance. Indeed, there may not be one “best” theory for a given project. Instead, implementation scientists will benefit from considering the broad range of criteria that we propose in this paper.

The field of implementation science will benefit from transparent reporting of the criteria that implementation scientists use to select theories. Transparent reporting may encourage implementation scientists to carefully consider the relevance of a selected theory instead of defaulting to theories that are convenient or familiar but are poorly suited to implementation scientists’ objectives. In turn, transparent reporting may diminish silos in the field by making explicit scientists’ thinking in selecting a particular theory, thus promoting progress through generalizable findings and shared understanding.

Examples of transparent reporting of the criteria that we identified in this study (or any others) exist. Birken et al. [26] justified the use of a taxonomy of top manager behavior [27] to explore of the relationship between top managers’ support and middle managers’ commitment to implementation by describing three categories of behavior in which top managers might engage to promote middle managers’ commitment (i.e., logical consistency, description of a change process). Alexander et al. [28] described using Klein and Sorra’s [29] theory of innovation implementation to assess the influence of implementation on the effectiveness of patient-centered medical homes [30] because the theory explains how the proficient and consistent use of an innovation influences its effectiveness (i.e., outcome of interest, specificity of causal relationships among construct). Yet transparent reporting of criteria used to justify theory selection is limited. Requiring manuscripts to include a section describing the criteria used to justify theory selection may promote more consistent reporting.

Several areas of future research would extend our initial attempt in this paper to explore criteria for selecting implementation theories. Specifically, the criteria that implementation scientists use to select theory may relate to how they intend to use theory. Understanding this relationship would help to refine the criteria presented here. We also recognize that substantive differences between theories and frameworks likely have implications for the criteria used to select them. For example, specificity of causal relationships among constructs is likely to be of greater importance for selecting theories than frameworks. We are currently refining the criteria and working to develop a useful, practical, and generalizable checklist of criteria based on concept mapping by implementation scientists [31]. The exercise will categorize the criteria and rate their clarity and importance. Our goal is for the checklist to take into account how and what kind of theory implementation scientists intend to use. This represents a first step toward what we hope will be a continued effort to refine the criteria, thus promoting more consistent and appropriate use of theory in implementation science and more effectively building the range of knowledge necessary to help ensure successful implementation across diverse settings.