Building blocks for meta-synthesis: data integration tables for summarising, mapping, and synthesising evidence on interventions for communicating with health consumers

The Cochrane Consumers and Communication Review Group coordinates the production of Cochrane systematic reviews of interventions which affect consumers' interactions with healthcare professionals and the health system. In describing the work of the review group, we use the term 'consumers' to include patients, their family carers, or members of the public with an interest in health. For brevity, we describe the wide range of interventions which affect people's interactions as interventions to communicate with and promote the participation of consumers in health care. Many of the interventions covered by the scope of the review group would be classified as complex according to the UK's Medical Research Council definition mentioned above [3]. A few examples illustrate both the complexity and diversity: interventions to improve consumers' adherence to medicines could include instruction, counselling or psychological therapy, simplified dosing, self-monitoring, reminders, lay health mentoring, crisis or family interventions, and many others, and these can each be delivered alone or as complex multifaceted interventions [4].

Broadly, interventions for improving communication and increasing participation aim to promote consumers who:

• are more knowledgeable and competent;

• are able to express and exchange their views and beliefs,

• make treatment and healthy behaviour choices alone or with health professionals;

• seek or provide support if required;

• minimise risks and harms;

• are able to access high quality information and quality health services; and

• participate in policy making, planning, service improvement and research.

These aims will therefore relate to the wide range of outcomes being sought, and which might affect individuals (consumers or professionals), health services and systems, as well as contribute to societal goals, for example, healthier populations.

As indicated, interventions for communication and participation are characterised by the diversity and multiplicity of their purposes and the complexity of the situations surrounding their use. They may operate at the level of individuals, groups, systems, organisations or society, they may be applied through many different media in addition to human interaction, and their delivery may involve many players. For example, contract-based interventions to improve adherence to treatment, screening or health promotion may be directed at patients, family carers and/or professionals providing these services [5]. Some interventions may be developed to be disease-specific, although many have general application across populations (or could be adapted to be so). Consequently, many reviews coordinated by the Consumers and Communication Review Group are broad in focus, rather than narrow, thus making them relevant and important for the treatment and management of many diseases, and to the formation of health policies for more responsive health systems.

The World Health Organization, through its Regional Office for the Western Pacific, has developed a policy framework for a more responsive and people-centred health system [6] and during this process commissioned the Consumers and Communication Review Group to provide summaries of systematic reviews to inform the development of the framework. At the time the project commenced (2004), methodology for the preparation of overviews or reviews of systematic reviews was at an earlier stage. Our view in 2004 was that overviews could play an important role in informing policy, health professionals and consumers about the state of evidence, enabling summary and comparison across a wide range of intervention types and settings. The issue we debated was whether an overview should be a straightforward summary of the results of included systematic reviews, that is, it would be not much more than a listing of summaries of data largely derived from the abstracts of reviews, or alternatively it could provide more information and be in a format that enabled a 'higher' or meta level of analysis.

Decisions on which data to collect and analyse are framed by the question that is posed. In the case of a systematic review, the question might be what are the effects of interventions to improve adherence to medicines [4], whereas an overview might ask the broader question of what interventions directed to consumers improve the use of medicines – including, for example, procurement, consumers' understanding, and medical outcomes as well as adherence [7]. However, in seeking to contribute to the development of policy, and in recognition of some of the challenges of summarising review-level evidence for complex interventions, we opted to provide more than the information that is currently provided in a typical abstract of a review. This choice led us to develop a method that would enable standardised messages on the evidence, its uncertainties and evidence gaps, and which would be concise but sufficiently informative for policy deliberation [8].

The process of summarising systematic reviews of interventions for communication and participation was more difficult than we anticipated and in the course of the work for WHO and other bodies subsequently, we have developed a method of summarising and assessing data from systematic reviews which is also an output. We call the output 'data integration tables'. The term data integration table was chosen because it indicates key inputs to this process: data that are selected, a process of integrating data, and data that are tabulated (either for stand-alone purposes, such as dissemination of the review results, or for inclusion in the next (higher) stage of analysis). The output is therefore both a series of steps and a table: the tables themselves can then be used (entire or modified) in different ways. This article describes the developmental process of the data integration tables. In relation to other key terms, and in common with others [2], we call a review of systematic reviews an overview and we term the method of analysis in an overview meta-synthesis. This article does not describe the analytical process of meta-synthesis but it does consider the implications of the data integration tables in the context of conducting a meta-synthesis.

As mentioned above, the production and use of systematic reviews has risen at pace with the increasing amount of primary literature available. As the science of research synthesis has developed interest has also grown in ways to combine and present review-level evidence in accessible, consistent formats [8‐13].

Overviews have been used to respond to clinical questions such as the effectiveness of a range of interventions for a particular condition (Ch. 22 [2]); or the effects of the same intervention across conditions. Cochrane Collaboration review groups, including the Cochrane Musculoskeletal, Cochrane Menstrual Disorders and Subfertility, and the Cochrane HIV/AIDS Groups, have developed methods and prepared overviews to address these types of questions [14‐16]. Others have focussed on addressing the needs of policy makers in relation to evidence-based decision making in health priority areas [17]. This work has involved the development of rating schemes to standardise and report outcomes. Using these tools, the strength of evidence derived from reviews can be rated and summarised in relation to key outcomes.

Methodological approaches to preparing overviews vary according to their aims (for instance, to be an input to policy deliberation, to develop specific recommendations, or to inform the field about the effects of different strategies) and key audiences (this may be dependent on who would implement the interventions). They are also determined by the populations, interventions and outcomes evaluated in systematic reviews. For example, the interventions assessed in clinically-focussed reviews are, in general, less complex and diverse than those seen in areas incorporating social complexity or social process, such as communication and participation in health. Reviews of clinical interventions also typically focus on a relatively narrow range of outcomes that are readily measurable in a clearly defined population. This contrasts with the hugely diverse outcomes, assessed in a multitude of ways, that typify systematic reviews of public health and socially complex interventions [10, 18, 19]. In these latter categories, a narrative report and synthesis of results rather than statistical meta-analysis is more common, due to the heterogeneity of the included studies. Such differences and others mean that meta-synthesis methods must be developed to accommodate methodological complexity but remain able to be tailored to assist the analytic process. In other words, meta-synthesis must be more than just a listing of findings from individual systematic reviews.

We were unable to find a model of synthesising review-level evidence with direct applicability to the area of communicating with and promoting the participation of consumers in health. Models with a clinical or public health focus provided a basis for approaching assessments and rating of the strength of evidence. However, these approaches did not provide a clear basis for standardising the reporting of diverse outcomes spanning clinical or disease areas, one of the most difficult features of the evidence on communication and participation. Our aim was to develop a model of synthesis that might standardise the evidence and so enable broad questions about the effectiveness of interventions for communication and participation to be answered, such as what helps people to make informed decisions about screening? We therefore sought to develop methods that would deal with the particular features of the evidence on communication and participation and provide us with a basis for the preparation of overviews, or meta-synthesis.

This article describes the developmental process, methodological issues, in association with its output or result, so for ease of understanding we have combined the traditional Methods and Results sections into one. The Discussion section describes the uses of the tables, their strengths and limitations, and future work.

Four steps were undertaken in designing the data integration tables: first, relevant information for tabulating key characteristics of a review was identified, extracted and summarised. Second, results for the outcomes of the review were assessed and translated to standardised evidence statements. Third, the outcomes and evidence statements were mapped into an outcome taxonomy that we developed, using language specific to the field of interventions for communication and participation. Fourth, the implications of the review were assessed after the mapping step clarified the level of evidence available for each intervention.

For each of these Steps 1–4, we outline the methods we adopted, methodological issues debated, modifications required and the output or result of each step. Tables 1, 2, 3 illustrate key inputs: see Additional file 1 for a worked example of a data integration table for one entire systematic review.

The first step involved identifying information from a review that would provide a context for interpreting it and aid an assessment of applicability by policy makers. This step also included two quality assessments – of the review, and of its included trials. Combined, this information summarised the key characteristics of the systematic review.

The tables aim to improve the relevance and consistency of reporting on the effects of complex interventions; to allow clear messages about the evidence to be drawn from reviews, particularly around issues of certainty; and to enable systematic identification of gaps in the evidence base. The data integration tables are a building block with which to undertake meta-synthesis and construct overview of reviews. A sharp rise in the number of available systematic reviews has seen demand grow for methods to summarise and synthesise review-level evidence [2, 8‐13, 29]. Overviews are one way of meeting these challenges. In addition, the discrete tables can serve as a valuable tool for accurate dissemination of data in individual reviews to professionals as well as to members of the public.

The data integration tables can be employed for various uses, in addition to the development of overviews. We have utilised and evaluated the format to disseminate new review-level evidence via evidence bulletins: electronic newsletters with wide distribution among professionals and lay people that must maximise content in a limited space [30, 31]. Another avenue of data dissemination and improving accessibility to evidence on communication and participation is via publicly available internet-based databases, an example of which is Rx for Change Database [32]. Rx for Change is administered by the Canadian Agency for Drugs and Technologies in Health and includes evidence derived from systematic reviews that deal with prescribing for and use of medicines by professionals and consumers. We utilised selected components of data integration tables to generate review summaries and to populate the consumer section of this database. As part of this work, a Cochrane overview of systematic reviews is in preparation, addressing the question: what improves evidence-based prescribing for, and medicine use by, consumers [7].

The development of the tables provided us with a relatively succinct format, with well-demarcated sections, allowing for location of key information as well as a more easily digestible presentation. Some Cochrane systematic reviews, particularly those with narrative presentation of results, are substantial documents, and trade-offs had to be made between detail and key messages. With this goal in mind, the data integration table format maintains a comprehensive representation of the original review article, in that key descriptive data is maintained. Mapping outcomes specific to the field of communication and participation, and also performing an assessment of the level of evidence available for each intervention, engenders an end product that is more than the addition of characteristics of the review plus results for review-level data. Rather, the table represents an integration of summary and synthesis. The transparency of process that makes it easily replicable is also a valuable component.

The WHO Patient at the Centre project, which involved gathering and synthesising a broad range of literature, also benefited from the use of data integration tables in the policy formulation process [33]. In conducting this project, time and resource constraints made it impossible to utilise fully the step where outcomes are mapped to the Group's taxonomic categories (Step 3) and at that time, there were insufficient published reviews coordinated by the Group to prepare a stand alone overview addressing a specific question. We have applied Step 3 only to reviews coordinated by the Review Group. We believe that this step would aid meta-synthesis but we remain unsure of its value beyond our Group's reviews and would welcome application by others.

One weakness of our method is that the summary evidence statements still require interpretation, and may not always be easy to apply, particularly for the process of evidence assessment in the context of narrative reporting of results in a review. However, we have used the approach to summarise and assess 19 Cochrane reviews in the Rx for Change Database of the Canadian Agency for Drugs and Technologies in Health [32]. User testing of this database is anticipated in the future and may help to clarify areas of confusion or ambiguity.

The benefit of the comprehensive nature of the data integration tables is that it allows some extrapolation and application of the results to local contexts – although this is not yet an integrated component. Concurrent with the methodological work described here has been the SUPPORT Project [34], with the development of review summaries and an overview of interventions relevant to primary care [35]. This will influence our ongoing work. We may consider adding a step for 'evidence to practice/policy' or translational issues. This could be derived from the review, in conjunction with consideration of local factors, or could be further developed through focus group research with key decision makers. Other concurrent developments that may influence future research and testing of our approach include the GRADE approach, as mentioned, for rating the quality of included studies [36‐38], and the development of lay summaries of Cochrane reviews using the Summary of Findings tables [2, 39].

We developed our approach in response to the particular challenges we faced in relation to socially complex interventions: various methods of analysis in reviews, complex interventions that are poorly described and understood, and lack of awareness of the range of health outcomes in the field of health communication and participation. Mapping the evidence using the assessment scheme and taxonomy clearly highlights relevant outcomes for interventions promoting communication and participation, differentiates between outcomes where there is sufficient evidence upon which to base a decision and less conclusive bases. The latter would then presuppose a greater role for preferences, value judgements or expert opinion subject to local circumstances [40]. Systematic gap analysis – both for evidence and for outcomes – is also facilitated, leading to easier identification of whether less rigorous research or other study designs will be required to address knowledge gaps. Given this context, we hope that they will also enable evaluation of bodies of evidence as a theoretically-related whole, to assist decision makers by producing a complete and concise representation of the available evidence.

Deficiencies in the patient-centredness of care and poor communication with consumers are persistent and pervasive problems in health care and a large range of interventions now exist to promote active participation and patient-centred care [10, 41]. These interventions have the potential to prevent and address the problems associated with communication and participation, and the evidence base from which to make decisions about the effectiveness of these interventions continues to grow [42]. We share with others the challenges of summarising reviews of complex interventions and hope that the methods and outputs described may offer a method for developing coherent overviews for policy makers and other users.