Theoretical approaches to process evaluations of complex interventions in health care: a systematic scoping review protocol

Efforts to improve healthcare delivery and outcomes often require complex interventions. Complex interventions were originally defined as interventions with multiple interacting components [1, 2], indicating that complexity is particularly linked to the intervention itself. Ongoing debates about which characteristics constitute complex interventions [3‐5] have complemented this original definition with the awareness that complexity is given “…primarily due to the social systems within which these actions occur, the contextually contingent nature of impacts, and the agency of the groups and individuals whose behaviours they aim to influence” [3].

Due to this changing definition, it is increasingly recommended to integrate process evaluations in the development and evaluation of complex evidence-based interventions to assess implementation aspects and to gain a better understanding of causal mechanisms and contextual factors associated with variation in outcomes [1, 6‐8]. Process evaluations can especially help to distinguish between those interventions that are inherently faulty (failure of intervention) and those interventions that are delivered inaccurately (implementation failure) [9]. Thus, they can prevent a so-called type III error, which refers to the question of whether the intervention actually has been delivered or if there is in fact something measured that does not exist [10].

The MRC (Medical Research Council) framework for developing and evaluating complex interventions provides a well-established guide for the development of complex interventions in healthcare [1, 2]. The revised version from 2008 [1, 11] for the first time took into account that process evaluations are highly recommendable to understand processes and to obtain explanations if interventions fail or have unanticipated consequences [1]. Nevertheless, the 2008 framework provided only sparse information on how to integrate and conduct process evaluations. Process evaluation is described as being “nested within a trial” that in the first place focuses on outcome evaluation [11].

In recent years, methodological aspects of process evaluations have been increasingly discussed; thus, the current MRC provides comprehensive guidance for process evaluations of complex interventions [6, 12]. This guidance refers to [1] implementation, [2] mechanisms of impact, and [3] context as key aspects to be examined [6, 12]. Furthermore, it “provides researchers, practitioners, funders, journal editors and policy-makers with guidance in planning, designing, conducting and appraising process evaluations of complex interventions” ([12] , p. 9}. It comprises process evaluation theory as well as practical information [12].

Regardless of the scientific discourse in the context of the MRC framework, the concept of process evaluation is not new to the field of health care research; it had already emerged by the 1960s [13]. Suchmann [14], as one of the first to emphasize the need to evaluate health interventions/programmes in relation to their practical setting, identified [1] the attributes of the programme, [2] the population exposed to the programme, [3] the situational context in which the programme takes place, and [4] the different kinds of effects produced by the programme as obligatory domains for “making sense” of the evaluative findings. Although these basic assumptions of Suchmann [14] remain relevant for the context of process evaluation, a wide range of developments have produced a diverse research field with many underlying concepts.

Commonly quoted concepts are recruitment (of cluster/of individuals), maintenance, context, resources, implementation, reach (of individuals), barriers, exposure, initial use, continued use, contamination, dose delivered, dose received, fidelity (degree to which an intervention is implemented in accordance with the original protocol), delivery (to clusters/to individuals), response (of clusters/of individuals), effectiveness, unintended consequences, theory, efficacy, and adoption [13, 15‐17]. Many of these concepts are also closely linked to the broad field of implementation research [18]. Thus, the underlying theoretical allocation and a clear distinction of these concepts is challenging.

The existing theoretical approaches relevant for process evaluations set varying priorities and organize their concepts in different ways (e.g. [19]). In addition, concepts termed identically in different theoretical approaches are not always defined in the same way, and they are not always selective between the different theoretical approaches. The MRC guidance for process evaluation of complex interventions [12] presents “an overview of a number of core frameworks” ([12], p. 18) relevant to process evaluation, but the authors explicitly state that they “make no claims to exhaustiveness” ([12], p. 18). While the theoretical approaches considered in the MRC guidance for process evaluation include many of the concepts mentioned above, the guidance does not provide a thorough analysis or comparison of the concepts the theoretical approaches refer to. Therefore, it remains unclear why the proposed theoretical approaches were selected and if there may be other theoretical approaches of relevance. This complicates the suggestion of the MRC guidance to combine concepts from different theoretical approaches for the development and conduct of process evaluations [12]. However, given the variety of theoretical approaches that are of relevance for process evaluations, it remains challenging to select and combine theoretical approaches or single concepts that fit the requirements and aims of a specific process evaluation approach.

Therefore, we will perform a comprehensive and systematic scoping review and analysis of the theoretical approaches suitable for process evaluations of complex interventions in health care. For this purpose, we understand theoretical approaches as interrelated propositions that intend to explain and predict general sets of phenomena [20]. Accordingly theoretical approaches help to structure our knowledge, understanding, explanation, and prediction of the world and they allow to generate new research hypotheses as well [20‐22]. In the context of process evaluation, a theoretical foundation is essential to better understand and systematically investigate the black box of implementing complex interventions in complex social settings [22]. At the same time, theoretical approaches are not uniform but might be e.g. implicit or explicit, descriptive, explanatory or predictive [22] or address high, middle or low abstraction levels [21]. Moreover, a range of heterogenous terms is being used in order to consider theoretical aspects in research projects and real life, e.g. theoretical/conceptual model, (theoretical/conceptual) framework, theoretical concepts [21, 22] These terms are often not clearly distinguished and used in an interchangeable way [20, 21]. Based on the above-noted explanations, we will therefore consider any kind of theoretical approach suitable for process evaluation in order to achieve a comprehensive overview.

The review results will further support researchers in considering, selecting and combining the available theoretical approaches (or just single concepts) and thus promote tailored and theory-informed process evaluation approaches.

We will perform a review of theoretical approaches. This kind of review differs substantially from a review of empirical data. Methodological issues related to such approaches have been rarely discussed in the literature [23], and they are not taken into account in overviews of review types [24, 25]. We will therefore consider the methodological issues related to this review carefully and ensure its transparency and comprehensiveness. The development and reporting of the review will follow the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for Scoping Reviews (PRISMA-ScR). The PRISMA extension, PRISMA-P [26] (see Additional file 1) has been used for the preparation of this review protocol.

Important protocol amendments will be reported with the review results.

The relevance of process evaluations is being increasingly discussed in the scientific discourse on the development and evaluation of complex interventions and is now widely recognized [1, 6, 8, 15, 16]. Although an increasing number of process evaluations are being conducted, these approaches are often not well founded on theory since there is little guidance available on how to derive concrete process evaluation procedures from theoretical allocation [16]. Frequently, this leads to arbitrary methodological procedures and subsequently to low-quality results. Furthermore, the phenomenon investigated is very demanding, since the focus is not only on the intervention to be developed but also on the implementation, the mechanisms of impact, and above all on the context [1, 6], which makes it necessary to consider complex social systems [3]. A theoretical foundation is therefore essential for the planning and realization of process evaluations. The results of this review will provide a systematic and critical overview and comparison of existing theories for process evaluations and thus facilitate the targeted selection of suitable theoretical approaches for the respective process evaluation approach. The review will build upon and further contribute to the scientific discourse on process evaluations that has been inspired by the MRC and its guidance on process evaluation.

From a methodological point of view, there will be some challenges related to this review since little literature exists with regard to literature reviews aiming at identifying theory. However, we expect that we will achieve high-quality results with the methodological approach described in this protocol, which is based on the guidelines for conducting systematic scoping reviews and the demonstrated iterative, inductive and reflexive approaches of qualitative research.