Background
Synthesising multiple qualitative primary research studies, referred to as ‘qualitative evidence synthesis’ by the Cochrane Qualitative Methods Implementation Group, is increasingly gaining acceptance as a valid and rigorous way to distil qualitative evidence to inform health and social care decision making [
1‐
8]. Noblit and Hare’s [
9] meta-ethnography, originally developed for synthesising education ethnographies, is one of the most frequently used and influential methodologies for qualitative evidence synthesis in health and social care research [
10‐
12] with a rapidly increasing volume of published meta-ethnographies [
10,
12,
13].
Meta-ethnography is theoretically-based drawing on Geertz’s concept of thick description [
14] and Turner’s [
15] theory of sociological understanding as ‘translation.’ It is unique among qualitative evidence synthesis methodologies in synthesising conceptual data from primary studies and was designed specifically to take into account the unique research contexts in primary studies. Meta-ethnography is suited to producing a new interpretation, model or theory, which goes beyond the findings of the individual studies synthesised, and does not simply aggregate findings [
9]. In our view, theory development is one of meta-ethnography’s key strengths. If adequately conducted and reported, meta-ethnography has the potential to generate new evidence on how patients experience their own health condition, disease, or treatments and how this may influence their adherence to treatments [
1]. It may also help us to understand why interventions or services work in certain settings but not in others [
16]. For instance, meta-ethnographies have been included in clinical guidelines for asthma management [
17,
18], medication adherence [
4,
5] and head and neck cancer care [
7,
8].
Meta-ethnography has seven iterative and overlapping phases [
9], which we now describe with emphasis on the complex analytic synthesis Phases 4 to 6.
Phase 1 Getting started
This involves deciding the focus of the synthesis. Noblit and Hare [
9] described this phase as ‘identifying an intellectual interest that qualitative research might inform’ ([
9], pp. 26–27).
Phase 2 Deciding what is relevant to the initial interest
This comprises identifying and selecting study accounts to synthesise. Noblit and Hare did not advise in detail how to do this. Unlike many recent health-related meta-ethnographies [
10,
19] they selected studies purposefully; they did not employ systematic review methods, which had not been developed in the 1980s.
Phase 3 Reading the studies
Noblit and Hare [
9] described this phase as ‘the repeated reading of the accounts and the noting of interpretative metaphors [...] this requires extensive attention to the details in the accounts.’ ([
9], p. 28).
Phase 4 Determining how the studies are related
Noblit and Hare [
9] described the process of Phase 4 as:
‘the various studies must be “put together.” This requires determining the relationships between the studies to be synthesized. We think it makes sense to create a list of the key metaphors, phrases, ideas, and/ or concepts (and their relations) used in each account and to juxtapose them.’ ([
9], p. 28).
Noblit and Hare [
9] stated that when deciding how studies relate reviewers should consider what the studies are about, the theoretical approach of studies, and the meaning of their concepts, themes or metaphors. They explained three different ways in which studies might be related:
(1)
the accounts are directly comparable as ‘reciprocal translations’
(2)
the accounts stand in relative opposition to each other and are essentially ‘refutational,’ or
(3)
the ‘dissimilar but related studies’ ([
9], p. 64) taken together represent a ‘line of argument,’ also described as a process of discovering ‘a “whole” among a set of parts’. ([
9], p. 63).
Noblit and Hare called Phase 4 a ‘key judgment call’ ([
9], p. 81) because reviewers must determine the relationship between studies in order to decide how to synthesise them.
Phase 5 Translating the studies into one another
This is one level of synthesis involving systematically comparing the meaning of metaphors, concepts or themes and their relations across study accounts to identify the range of metaphors, concepts, and themes. Translation is underpinned by the theory of social explanation [
15] and also draws on Brown’s (1977 in [
9]) idea that all knowledge is metaphoric. Noblit and Hare said:
‘we have adapted Turner’s (1980) notion that all explanation is essentially comparative and takes the form of translation. A meta-ethnography based in Turner’s conceptualization simply extends his argument by constructing syntheses by translating multiple qualitative studies into one another’s terms’ ([
9], p. 25).
Interpreting
meaning is key to translation, i.e. translation is idiomatic rather than literal, and should take account of each study’s context (e.g. where and when it was done and with whom) [
9]. There are two types of translation described by Noblit and Hare [
9]: reciprocal translation and refutational translation. They specified that reciprocal translation.
‘requires the assumption that the studies can be “added” together. That is, they are clearly studies about some similar things’ ([
9], p. 40).
They described the conduct of reciprocal translation as follows:
‘we conceive of meta-ethnographic syntheses as translations (one case is like another, except that . . .). […] in an iterative fashion, each study is translated into terms (metaphors) of the others and vice versa.’ ([
9], p. 38).
When the studies are not similar enough to be added together then it may be appropriate to conduct refutational translation, which Noblit and Hare [
9] described as a particular type of interpretation:
‘Ethnographies that are implicitly or explicitly refutations of each other […] require a more elaborate set of translations - translations of both the ethnographic accounts and the refutations […]. Our approach treats the refutation itself as part of the interpretation to be synthesized.’ ([
9], pp. 47-48).
Attention should be paid to the assumptions, motivations and ideology behind a study. A benefit of conducting a refutational translation is that it allows reviewers to identify if the theories or ideologies underlying two or more studies differ [
9].
Phase 6 Synthesising translations
Noblit and Hare [
9] defined phase 6 as follows:
‘Synthesis refers to making a whole into something more than the parts alone imply. [..] when the number of studies is large and the resultant translations numerous, the various translations can be compared with one another to determine if there are types of translations or if some metaphors and/or concepts are able to encompass those of other accounts. ([
9], p. 29).’
This is a process of going beyond the findings of any individual study. It is ‘a second level of synthesis’ ([
9], p. 28) in which the translations from phase 5 are compared to identify common or overarching concepts and to develop new interpretations from these.
Phase 7 Expressing the synthesis
Communicating the synthesis to your audience in a suitable format [
9].
Translation, synthesising translations and line of argument synthesis are particularly poorly understood, as evidenced by many published meta-ethnographies which state that they have used meta-ethnography when they have not adhered to the principles of the methdolology [
10,
12]. These complex synthesis processes form the heart of the methodology, but were not described in detail by Noblit and Hare [
9] whose seminal publication was not intended to be a step-by-step, procedural guide. Indeed, Noblit (in Thorne et al. [
20]) has stated that the 1988 book was not a definitive work on meta-ethnography, hence there are gaps. Meta-ethnography has evolved since 1988 (e.g. [
1,
21]), mainly as a result of its application and adaption in health research [
13], but there is little guidance on its conduct which incorporates recent methodological developments. Poor meta-ethnography conduct and reporting may limit the effective use of meta-ethnography findings [
3,
10] and indicate the need for more detailed and current guidance on conduct.
The findings reported in this article come from a methodological systematic review to identify recommendations and guidance for conducting and reporting a meta-ethnography carried out as part of the eMERGe project [
22] which created the first methodology-specific reporting guidance for meta-ethnography [
23‐
27]. The aim of this article is to give guidance on the conduct of the complex analytical stages – phases 4 to 6 – of meta-ethnography through analysis of the latest methodological evidence identified from publications included in our systematic review. Specifically, we describe, contrast and critique different methods for conducting phases 4 to 6.
Methods
Data for this article were drawn from the systematic review (PROSPERO CRD42015024709), for which we now describe the methods [
22,
23]. The review question was: what are the existing recommendations and guidance for conducting and reporting each process in a meta-ethnography, and why?
Literature search methods
A comprehensive search for published and unpublished texts in any language was performed in multiple information sources. Fourteen bibliographic databases and five other electronic resources covering a spectrum of academic disciplines were searched between June and August 2015. The search strategy was first designed in Medline following testing and refining against a set of key papers and then adapted to the remaining databases (listed in Additional file
1). An example of the search terms used in the review, based on those used for Medline, is:
1.
(“qualitative synthes#s” or Qualitative systematic review*).ti,ab.
2.
(“meta-ethnograph*” or “metaethnograph*” or “meta ethnograph*” or “meta-synth*” or “meta synth*” or “metasynth*” or “line* of argument”).ti,ab.
3.
(“critical synth*” or “textual synth*” or “framework synth*” or “thematic synth*” or “grounded synth*” or textual narrative synthe#s) adj2 review*).ti,ab.
4.
(“metasynthes#s” or “meta synthes#s” or “metasynthes#s” or “meta-stud*” or metastud*).ti,ab.
5.
((“qualitative” adj2 “synth*”) or (“third order” adj2 “construct*”) or (qualitative adj2 review)).ti,ab.
6.
knowledge synthesis.ti,ab.
8.
((“method*” or steps) adj2 (“insight*” or lessons or learnt or “explor*” or learned or conduct* or “approach*”)).ti,ab.
10.
((good or best or recommend* or quality or publishing or reporting) adj3 (guid* or design* or standard* or practi#e* or report* or method* or steps)).ti,ab.
12.
((challenges or steps) adj5 (synthesis* or qualitative or conduct* or report* or design* or method* or present* or practical*)).ti,ab.
13.
(practical adj5 (guid* or design* or standard* or approach* or framework*)).ti,ab.
14.
((methods or methodological) adj5 (guid* or design* or standard* or approach* or framework*)).ti,ab.
16.
qualitative research/ and “meta-analysis as topic”/
We also employed expansive search techniques which involved gathering relevant publications known to our expert panel and the project team; forward and backward citation tracking of all included publications (i.e. checking if there were any further relevant texts that either cited or were cited by included publications), and citation alerts. Any new relevant published or in press publications identified through these expansive methods were included up to March 2016.
Screening and selection
Literature search results were downloaded into Endnote® bibliographic software and screened against eligibility criteria presented in Table
1. Titles and abstracts were first screened independently by two reviewers for references published from 2006 onwards (6271 records) and by one reviewer for references published before 2006 (1251 records), owing to time and resource constraints. Based on our familiarity with the literature, we were confident that references prior to 2006 were known to the project team and its expert advisors already, or they would be identified through expansive searches. Any publications identified as potentially relevant were then retrieved in full-text and screened by two independent reviewers, with any disagreements resolved through discussion or by a third reviewer.
Table 1
Systematic review inclusion and exclusion criteria
1. Does not report on methodological issuesa in conducting meta-ethnography AND 2. is not a reporting guideline/ providing guidance on reporting meta-ethnography | 1. Reports on methodological issuesa in conducting meta-ethnography OR 2. Is a reporting guideline for or provides guidance on reporting qualitative syntheses including meta-ethnography |
3. Published before 1988 (date of the publication of the original meta-ethnography text by Noblit and Hare) | 3. Published after 1988 |
4. Theses below PhD level | 4. Book, book chapter, journal article/ editorial, report or PhD thesis |
5. Any language |
6. Any discipline or topic (not just health related) |
Data coding and analysis
Fifty-seven full texts were coded using qualitative analysis software NVivo 10.0 by four reviewers who used a bespoke coders’ guidance document developed, piloted, and refined by the team. One reviewer coded each publication; a second reviewer checked completeness of coding for 13 (23%) publications, judged as rich and/or seminal by the team, which confirmed that overall the coding guidance had been applied consistently and coding was accurate. ‘Nodes’ or coding categories were primarily based on the seven phases of meta-ethnography [
9], with additional nodes for other relevant data (e.g. ‘definition or nature of meta-ethnography,’ ‘selection of a qualitative evidence synthesis approach’). Findings presented in this article focus on the conduct, not reporting, of meta-ethnography.
Full publications and coded data were read repeatedly and compared using constant comparison, mainly by two team members who recorded their analysis in memos in NVivo for each node. For nodes concerning the complex Phases 4, 5 and 6, each researcher independently identified the key themes and issues, then compared them and wrote a joint analytic memo. Each researcher maintained an analysis journal in NVivo to record development of ideas, and analysis decisions made at wider project team meetings were documented. Each researcher noted which publications they considered “rich in detail” about meta-ethnography overall and for phases 4 to 6, i.e. a detailed account with in-depth explanation and rationales that went beyond description. We wrote a detailed definition for each phase of a meta-ethnography, analysed and summarised advice and recommendations, and documented pitfalls in the conduct and reporting of meta-ethnography, noting any contradictions or uncertainties.
From our inclusion criteria, we developed a system to classify the publications according to the type of evidence they contributed to the review; where possible we differentiated between those based on the authors’ opinion and those which were supported by ‘evidence.’ Evidence could be empirical data from published literature or experience conducting a meta-ethnography, or reasoned argument. We developed seven main categories:
-
A meta-ethnography with methodological detail on the application of methods (referred to as ‘worked examples’)
-
Other methodological text (i.e. not a meta-ethnography) exploring particular aspects of meta-ethnography conduct in-depth (e.g. conduct of reciprocal translation)
-
Critique of meta-ethnography
-
Descriptive overview of the methodology (some of which compared qualitative evidence synthesis methodologies)
-
Guidance on meta-ethnography conduct
-
Reporting of meta-ethnography methods
-
Generic reporting guideline for qualitative evidence syntheses that could potentially be applied to meta-ethnography.
To add rigour to the process and enhance interpretation, the preliminary review findings were presented to academic experts and other key stakeholders at various fora including:
-
a project team meeting
-
an online workshop in May 2016 with 12 academic experts in meta-ethnography, 3 professional end users of evidence syntheses, 11 lay people, and 5 project team members. A further six academics and three lay people commented on the workshop materials and notes after the workshop;
-
a project advisory group meeting in November 2016 attended by 9 project team members, 1 independent chairperson, 7 lay advisors and 10 academic experts; and
-
two formal and several informal meetings with one of the two originators of meta-ethnography, Professor George W. Noblit in June 2016.
These meetings added to our understanding of meta-ethnography conduct and have influenced the review findings; where a direct link can be traced from our findings to our discussions with stakeholders we state this. We describe the literature search and screening results, the characteristics of included publications, highlight the key findings and then focus in detail on the complex analytic synthesis Phases 4 to 6 which are often poorly understood and reported in published meta-ethnographies [
10].
Discussion
This systematic review provides an in-depth analysis and critique of methodological publications on meta-ethnography conduct since 1988 when Noblit and Hare [
9] published their seminal meta-ethnography monograph. It provides guidance on the conduct of meta-ethnography Phases 4 to 6 which involve relating, translating, and synthesising studies. We undertook comprehensive and expansive literature searches. We conducted a rigorous analysis of 57 publications involving a multi-disciplinary team including social scientists, academic health professionals, lay people and other users of research evidence.
Our findings indicate that there is no ‘one size fits all’ recipe for reviewers to follow when conducting reciprocal translation. Each research team conducting a meta-ethnography will need to select methods which suit: the review aim; the nature, e.g. heterogeneity, and volume of the data to be synthesised; and their resources, such as team size and expertise and the time available [
23,
24]. Large amounts of data have been synthesised by grouping studies into smaller sets then synthesising within and then across the groups of studies [
1,
37] or by using analysis software to manage analysis [
19]. Alternatively reviewers could manage the volume and nature of the data by, for instance, purposefully sampling studies [
70] to reduce the volume and to ensure studies are similar enough to synthesise while still providing opportunities for inclusion of refutational data.
A key consideration in meta-ethnography conduct is which studies to include. The nature of the primary study data available to synthesise is an important factor. Incorporating predominantly superficial or ‘thin’ descriptive data in a meta-ethnography is potentially problematic: further interpreting data which lack depth and detail is difficult. We define conceptual data as explanatory, i.e. they explain a phenomenon. Rich descriptive data are those which provide sufficient detail that they can be further interpreted to develop conceptual insights. Rather than including ever-increasing volumes of studies based on topic relevance alone, selecting studies containing data suitable for a meta-ethnography is potentially more conducive to producing an interpretive synthesis.
The process of translation we, and others [
1], believe is what distinguishes meta-ethnography from other qualitative evidence synthesis methodologies, therefore we propose it should be done using the theoretical principles laid out by Noblit and Hare [
9]. Less labour-intensive methods of translation, such as grouping concepts without an account by account comparison (e.g. used by Toye et al. [
19]), diverge more from Noblit and Hare’s original methodology. Nonetheless, such methods are likely to be popular with reviewers in light of the trend in health sciences towards synthesising high numbers of journal articles into a single meta-ethnography, e.g. over 100 in some published examples [
71]. This is not a development we would advocate because the sheer volume of data might interfere with the ability to produce a useful, interpretive output and could result in an aggregative synthesis. There is a need to empirically compare alternative methods of synthesis to deal with large amounts of data. The order in which studies are synthesised could also influence the overall interpretation [
10,
19,
21,
34,
49,
60], this too requires empirical investigation. Reviewers choosing methods for phases 4 to 6 should consider their potential impact on not only the efficiency of conduct, but also the outputs of a meta-ethnography.
We maintain that different kinds of syntheses (reciprocal, refutational and line of argument) can, and should, co-exist in one meta-ethnography [
1], rather than it containing only one of these. Indeed in his new book, which credits his discussions with the eMERGe team, Noblit [
72] accedes that these are not mutually exclusive types of syntheses. Refutational data are important for developing new understandings. We believe that theory development is of key importance to meta-ethnography conduct and that capitalising on the ability of meta-ethnography to move beyond the development of new concepts to theory development should enhance the evidence base for decision making. The methodology is suited to complex data and complex questions. If reviewers do not intend to develop theory, then an alternative qualitative evidence synthesis methodology could be better suited to their purposes.
Since conducting our systematic review in 2015 to 2016 further relevant publications have been published. For instance, they include adapting meta-ethnography for synthesising qualitative evidence syntheses (‘mega-ethnography’) [
73]; for analysing multiple primary qualitative datasets [
74], and for synthesising ethnographies while they were still being conducted [
75]. Urrieta and Noblit’s new edited book [
72] focuses on the relation of meta-ethnography and theory with identity theory. It also explores how meta-ethnography has been adapted in health and in education and clarifies some ambiguities in Noblit and Hare’s 1988 book [
9]. After our systematic review was completed Cahill et al. [
76] produced a guide entitled ‘A guide to conducting a meta-ethnography’ in 2018. Their article gave a concise overview of meta-ethnography conduct based on only 10 publications, all included in our systematic review, but did not provide in-depth analysis or guidance on conducting phases 4 to 6, which is the main focus of our article.
Meta-ethnographies conducted in education versus health and social care disciplines may evolve distinct versions of the methodology to suit their different needs and philosophical approaches. A special issue in the journal
Ethnography and Education in 2017 [
77] reflects a new interest in meta-ethnography in the field of education; in this, Borgnakke [
78] challenges the transferability of the ‘evidence movement’ basis of meta-ethnography in healthcare research to education and social fields. Education tends towards synthesis of a small number of rich ethnographies and the identification of metaphors (e.g. [
59,
75]), whereas health science tends to synthesise concepts and themes from large numbers of journal articles reporting interview studies [
10]. Meta-ethnography is still evolving, in health and other disciplines, and future research by our team will seek to incorporate these newer publications, not all of which can be covered here, into future guidance on meta-ethnography conduct and reporting.
We identified a lack of empirical methodological research comparing the different methods of relating studies, translation and synthesis meaning that there remain unanswered questions. Future methodological research should focus on establishing the consequences of different methods for the quality (e.g. credibility and trustworthiness) of meta-ethnography outputs, such as, the impact of grouping concepts thematically compared to translating them one by one. In addition, research should explore the impact of the order in which accounts are translated and synthesised, including the effect of using an index study. A further issue to examine is the relationship between volume of data and quality of output.
Limitations
We originally conducted our systematic review to inform development of reporting guidance [
22‐
27]. In order to discern what should be reported in a meta-ethnography, we had to establish how a meta-ethnography should be conducted and hence many publications contained rich data on meta-ethnography conduct. A possible limitation is the lack of formal methodological evidence, however we critiqued the methods through comparing and contrasting them and reflected through a process of logical reasoning. Not all publications in the review contributed to the findings, especially for phases 5 and 6; some, such as worked examples of meta-ethnographies containing methodological detail, did contribute rich data. However, other kinds of texts in our review, such as overviews of the methodology, also contributed to our understanding and analysis. Our review included publications up to 2016. We have since updated our systematic searches in five databases (CINAHL, Web of Science, PubMed, SCOPUS and PsychInfo) during June and July 2018 and also identified publications through citation alerts and our networks. Newer publications have been incorporated into the discussion section.
We chose not to critically appraise texts in order to exclude any on the basis of (low) quality. No tool exists to judge the quality of a meta-ethnography, nor the quality of such a wide range of methodological publications which ranged from worked examples to critiques and overviews of the methodology. We did however record which ones we considered to be rich in detail. A publication’s richness is reflected in how much it contributed to our review findings. It is worth mentioning that one worked example, which contributed to our findings, was conducted by a lone reviewer [
50]; good practice is for multiple reviewers in order to enhance interpretation of data [
24].
A further limitation is the lack of clarity around conduct of Phase 6 and line of argument synthesis in the review publications. This reflects its complexity and unclear guidance on its conduct. There was lots of variation in definitions and no clear consensus on methods, although we could discern some commonalities. There is scope for future research to further develop methods for conducting phase 6 and line of argument synthesis.
Acknowledgements
We warmly thank Derek Jones and Rachel Russell for their involvement in the systematic literature searches; Margaret Cunningham, for her involvement in co-ordinating the later stages of the project; and the members of our project advisory group: Geoff Allan, Marjory Barton, Ian Gallagher, Anne Ward, Lorna Neill, Gordon Snedden, Veronica Saunders (lay members), Andrew Booth, Nicky Britten, Rona Campbell, Margaret Cargo, Kate Flemming, Ruth Garside, Claire Glenton, Karin Hannes, Angela Harden, Gina Higginbottom, Christine Johnstone, Simon Lewin, George W. Noblit, Sandy Oliver, Cathy Pope, Karen Ritchie, James Thomas, Meredith Vanstone, and Geoff Wong (academics, other experts and professionals); and Sheena Blair, independent chair of the advisory group; and our funder the National Institute of Health Research (NIHR).