Background
The range of approaches to research review and synthesis has been growing over recent years [
1]. One approach that has been growing in popularity is realist synthesis. A realist review focuses on understanding and unpacking the mechanisms by which an intervention works (or fails to work), thereby providing an explanation, as opposed to a judgment about how it works [
2]. The realist approach is fundamentally concerned with theory development and refinement [
2‐
4], accounting for context as well as outcomes in the process of systematically and transparently synthesizing relevant literature [
3,
4]. Given the complex, multifaceted nature of strategies and interventions used to promote evidence-informed healthcare, and the current limited understanding of their mechanisms of action, the realist approach is particularly suited to the synthesis of evidence about complex implementation interventions. Whilst the use of this method is increasing and the evidence base growing [
5‐
8], few published studies provide a detailed account of how it has been used. This paper therefore adds to the methodological evidence base about realist synthesis by describing application of the approach of an international project team (Realist Synthesis of Implementation Strategies (ReS-IS) to synthesise evidence about knowledge translation interventions for enabling evidence-informed healthcare.
Realist synthesis
Traditional systematic review approaches have been criticised for being too specific and inflexible [
2,
9,
10]. This is an important consideration when examining the complexity of implementing health and social care interventions. The context of service delivery is complex, multi-faceted and dynamic, which arguably means that rarely would the same intervention work in the same way in different contexts. Consequently, conventional systematic review approaches to evaluating the evidence of whether interventions work (or not) often result in limited answers such as ‘to some extent’ and ‘sometimes’ [
4,
9].
Realist review has emerged as a strategy for synthesising evidence and focuses on providing explanations for why interventions may or may not work, in what contexts, how and in what circumstances [
11]. For example, Greenhalgh
et al.[
6] undertook a realist review to supplement a Cochrane review of school feeding programmes. Whilst the Cochrane review provided evidence that feeding programmes work, it did not provide information about how they work and in what contexts. The findings from their realist review resulted in evidence regarding situations in which programmes may be more likely to be effective.
The realist approach is philosophically rooted in realism, which combines three social science principles: causal explanations are achievable; social reality is mainly an interpretative reality of social actors; and social actors evaluate their social reality [
12]. Realism involves identifying underlying causal mechanisms and exploring how they work under what conditions [
13,
14]. This contextually bound approach to causality is represented as context + mechanism = outcome [
15]. Therefore, it is an intuitively appealing approach to those trying to expose and unpack the complexities of contexts and interrelated mechanisms underlying implementation activity.
The aim of a realist synthesis is ‘…to articulate underlying programme theories and then to interrogate the existing evidence to find out whether and where these theories are pertinent and productive. Primary research is examined for its contribution to the developing theory…’ [
4]. In the context of implementation interventions, which are usually multi-faceted and complex, when setting out to develop and implement an intervention there is always an underlying theory about how it should work: if we do X in this way, then it will bring about an improved outcome [
2,
4,
9]. The logic underpinning the aim of uncovering underlying theories about interventions is that no deterministic theories can always explain or predict outcomes in every context [
16]. Focussing on what it is about an intervention that makes it work (or not) in a given context should enable implementation researchers to work at the level of mechanisms of action. The premise is that in certain contexts individuals are likely (although not always certain) to make similar choices, and therefore particular contexts influence our choices such that reoccurring patterns emerge,
i.e., demi-regularities [
4]. Realist review provides an approach to uncover the underlying theories that explain these demi-regularities by examining the interactions between mechanism, context, and outcome.
A realist synthesis follows similar stages to a traditional systematic review (Table
1), but with some notable differences:
1.
The focus of the synthesis is derived from a negotiation between stakeholders and reviewers and therefore the extent of stakeholder involvement throughout the process is high.
2.
The search and appraisal of evidence is purposive and theoretically driven with the aim of refining theory.
3.
Multiple types of information and evidence can be included.
4.
The process is iterative.
5.
The findings from the synthesis focus on explaining to the reader why (or not) the intervention works and in what ways, to enable informed choices about further use and/or research [
3].
Table 1
Approach to realist review (adapted from Pawson[
9]
)
Define the scope of the review | Identify the question | What is the nature and content of the intervention? What are the circumstances or context of its use? What are the policy intentions or objectives? What are the nature and form of its outcomes or impacts? Undertake exploratory searches to inform discussion with review stakeholders. |
Clarify the purpose(s) of the review | Theory integrity – does the intervention work as predicted? Theory adjudication – which theories around the intervention seem to fit best? Comparison – how does the intervention work in different settings, for different groups? Reality testing – how does the policy intent of the intervention translate into practice? |
Find and articulate the programme theories | Search for relevant ‘theories’ in the literature. Draw up list of programme theories. Group, categorise or synthesise theories. Design a theoretically based evaluative framework to be ‘populated’ with evidence. Develop bespoke data extraction forms. |
Search for and appraise the evidence | Search for the evidence | Decide and define purposive sampling strategy. Define search sources, terms and methods to be used (including cited reference searching). Set the thresholds for stopping searching at saturation. |
Test of relevance | Test relevance – does the research address the theory under test? Test rigour – does the research support the conclusions drawn from it by the researchers or the reviewers? |
Extract and synthesise findings | Extract the results | Extract data to populate the evaluative framework with evidence. |
Synthesise findings | Compare and contrast findings from different studies. Use findings from studies to address purposes(s) of review. Seek both confirmatory and contradictory findings. Refine programme theories in the light of evidence including findings from analysis of study data. |
Develop narrative | | Involve commissioners/decision makers in review of findings. Disseminate review with findings, conclusions and recommendations. |
The rest of this paper describes in detail the ReS-IS team’s approach to applying the realist synthesis method to a review of interventions that enable evidence-informed healthcare, including the strengths and challenges encountered in its use. A separate publication will describe the findings of the review and synthesis in detail.
Discussion
There are few published examples of realist syntheses and those that exist do not include a detailed account of the approach used because authors tend to focus on the dissemination of findings within publications [
6,
7,
24‐
27]. This lack of information about application of the realist approach is unhelpful to a novice realist reviewer. To fill this gap, this paper presents in some detail the approach we took to conduct a realist synthesis of evidence about the effect of change agency on evidence-informed healthcare. Change agency is a complex implementation intervention, which made realist synthesis an appropriate approach for unpacking its effects within different contexts and groups. However, undertaking this review was not without its challenges, not least because of the practicalities of working as an unfunded and geographically dispersed group.
Our approach deviated in some respects from the approach recommended by Pawson et al. For example, Pawson et al. do not advocate a comprehensive literature search, or double reviews and data extraction. In this sense we developed a hybrid approach that was fundamentally rooted in realist synthesis philosophy and principles (i.e., theory led, purposive, iterative, stakeholder involvement), but which also drew on some of the practices of traditional systematic reviewing.
One of the strengths of realist review is the approach’s firm roots in philosophy and social sciences [
2,
9]. Rather than being a method or formula, it is a ‘logic of enquiry’ [
9], which enables a flexible, all-embracing approach to explanation of what works for whom in what circumstances and in what respects. Rather than controlling for real life events, realist synthesis provides a framework for working with and untangling the complexity of real-life implementation. This allows for an equal focus on what works, as much as what does not work, in an attempt to learn from failures and maximise learning across policy, disciplinary and organisational boundaries. Furthermore, realist synthesis is inherently stakeholder driven, which facilitates engagement and the inclusion of multiple perspectives.
The strengths of realist synthesis underpin its limitations. Realist synthesis is premised on a set of principles rather than a formula, and whilst this allows for flexibility and inclusivity, it means that the findings from a review are theoretically transferable. For example, it follows that if the appraisal and data extraction needs to be bespoke to the particular review questions that arise from the theoretical framework, these will be different for each review. Furthermore, given that the fundamental interest in realist synthesis is about finding out what works in what contexts, the recommendations one can make will not be generalisable. Rather a realist review results in findings that are theoretically transferable; ideas (‘theories’) that can be tested in different contexts, with different stakeholders.
Pawson
et al. [
2] suggest that realist syntheses are not for novices. Unlike a Cochrane review, for example, which relies on standardised protocols and tools, the demands on a realist synthesiser are different. For example, quality assurance within realist synthesis is dependent on reviewers’ explicitness and reflexivity. During this review, we kept a log of the process and decisions made throughout the process, which we developed into a technical report. In addition, we undertook a more formal reflective process during the review because members of the group had varying experiences of realist review. This involved reflecting on questions about what was going well, what was going less well, as well as engaging in group learning activities. Throughout the review process, we had large and small group discussions that provided the opportunity for building in checks and balances, and for explicating processes. In turn, this requires a high level of expertise in reasoning, research methods and quality appraisal, and expertise in the subject area. The complexity of the realist synthesis approach means that it is time-consuming and human resource intensive, and for those reasons a potentially expensive endeavour.
Conclusions
Realist synthesis is a new but emerging approach to evidence review. In this paper, we have described our use and development of the approach. It is particularly appropriate for unpacking the impact of complex interventions because it works on the premise that one needs to understand how interventions work in different contexts, and why. It is not an easy option. Realist review demands much of the reviewer, including an ability to think flexibly and deal with complexity. There is not one prescribed approach to doing a realist synthesis; rather, there is a set of principles that the reviewer must particularise to the issue being explored whilst being sympathetic to the philosophy of realism. This presents unique challenges, but with it, the opportunity to develop more pragmatically insightful conclusions than those produced by some other approaches to systematic reviewing.
Competing interests
Bridie Kent is an Associate Editor for Implementation Science; all decisions on this manuscript were made by other editors.
Authors’ contributions
BM and JRM led the project. All authors participated in defining the scope of the review. KD and ESC executed the search. The appraisal of evidence and data extraction were undertaken by all authors. All authors were involved in the analysis process. AMH led the documentation of the study process. KD led the development of the narrative. JRM wrote the first draft of the paper; BM, AH, and KD commented on it. All authors provided feedback on various drafts, and read and approved the final manuscript.