Background
It is estimated that around 22 new systematic reviews are published every day [
1]. In order to keep pace with the increasing volume of reviews, new methodological approaches have been developed for synthesising this evidence including overviews (systematic reviews of systematic reviews). Overviews are most frequently employed where multiple systematic reviews already exist on similar or related topics, and aim to systematically bring together, appraise and synthesise the results of related systematic reviews. Overviews have evolved to address a growing need to filter the information overload, improve access to targeted information and inform healthcare decision-making [
2,
3]. Overviews can be useful tools to support decision-making by clinicians, policy makers and developers of clinical guidelines [
2,
4]. There are a range of factors to reflect upon prior to deciding whether to conduct an overview, including consideration of the methodological challenges and uncertainties. These challenges are discussed in depth in our accompanying paper on this topic [
5].
Overviews are known by a variety of different names, all potentially reflecting different aspects and aims of the syntheses. Terms used include: overview; umbrella review; meta-review; (systematic) review of (systematic) reviews; synthesis of systematic reviews; and summary of systematic reviews. The common feature of the methods associated with all of these terms is the fundamental process of synthesising evidence which is derived, often exclusively, from systematic reviews. The systematic review forms the primary ‘unit of analysis’ and is the basis upon which an overview is built [
6].
The term ‘overview of systematic reviews’ (often shortened to ‘overview’) has gained widespread acceptance, and is the term used by Cochrane to describe a review of systematic reviews published in the Cochrane Library [
7]. We use the term ‘overview’ within this paper to describe systematic summaries of systematic review evidence, in line with the most commonly used terminology.
Overviews can play a role in signposting the reader to evidence, summarising existing research or highlighting the absence of evidence [
7]. For this reason, overviews can provide an ‘entry point’ for policy makers and other consumers by summarising broad issues and current knowledge around a topic, and directing the reader to more detailed, fine-grained material contained in component systematic reviews and primary research [
8‐
10].
Likewise, the involvement of stakeholders at an early point in planning and conducting an overview may help in shaping these aims for maximum overview impact [
2,
11,
12].
Overviews arguably have a valuable role where evidence relating to a specific topic exists but is conflicting, bringing together reviews in a transparent and systematic way and aiding informed decision making by gathering, appraising and systematically analysing this evidence. While the evidence synthesised within an overview may be used to generate new insights and understanding, it is important to note that overviews are fundamentally a method of bringing together, summarising and enhancing accessibility of existing evidence.
Methods
Overviews are a relatively new and emerging method of summarising evidence, and consequently universally-accepted guidance for good practice relating to the conduct of overviews is currently lacking [
5,
13‐
17]. During a 2016 UK Cochrane Symposium workshop [
18] focused on the methods and challenges associated with overviews, it became apparent that there was a need to clarify, and distinguish between, different types of overviews and the objectives which these overviews addressed. Within this paper, we therefore describe types of overview and the common research questions and objectives they address. Within a second, linked paper [
5], we build on this description of overview types, objectives and research questions, illustrating this through the use of five exemplar overviews, and exploring the impact and implications of different methodological approaches.
In presenting and discussing common research questions addressed by overviews with different objectives, and relating this to real examples in the second paper [
5], we aim to help readers understand important issues to consider during the first steps to planning an overview.
Research questions and objectives addressed by overviews
In common with all research, overviews are carried out to address a clearly-stated research question. When planning an overview, determining the nature of the initial research question, and identifying who is asking the question, will dictate the scope of the overview objective(s). The objectives of an overview may include summarising existing evidence on a range of different topics, including: interventions; diagnostic accuracy of medical tests or procedures; prognosis or risk prediction; health equity [
19]; or more qualitative aspects associated with any of the above, such as patient preference or device acceptability. In addition to summarising the results of multiple systematic reviews on related topics, overviews may also be used to investigate different aspects of questions already tackled by existing systematic reviews, such as variations in population, condition or intervention [
10‐
12]. One example of this latter approach is provided in an overview which aimed to synthesise current evidence of the relationship between sedentary behaviour and health outcomes [
20], reporting variation in results across populations and condition studied.
The principles which guide development of focussed clinical questions for systematic reviews remain valid for the development of research questions for overviews. Clearly defining the target population and setting, context, intervention, index test or phenomenon of interest, comparator or reference standard and outcome or treatment decisions are all essential parts of any overview protocol. The research question and overall overview objective will dictate the ‘type’ of overview that is required. This may be an overview of specific types of systematic review, or of systematic reviews which contain specific types of primary research study.
These defining elements of research questions and objectives are illustrated in Table
1, and we consider the objectives of each overview type in more detail below.
Table 1
Forming an overview question and defining the type and objective of an overview
Population | Is the focus on one discrete population, or bringing together evedence from different populations? |
Intervention/phenomenon of interest | Is the focus on one specific intervention or a range of different possible interventions? | Is the focus on one single medical test or a range of different medical tests? | Is the focus on overall prognosis, prognostic factors or risk prediction (prognostic models)? | Is the focus on associations between disease indices and risk factors, or the impact of risk factors on single/multiple outcomes? | Is the focus on qualitative aspects of an intervention, such as views or experiences of a particular group of people? |
Outcome | Is the focus on one specific outcome (including adverse events) or a range of different outcomes? |
Context/setting | Describe the context of setting of interest |
Type of review | Review of RCTs | Review of intervention studies | Reviews of diagnostic test accuracy studies | Review of prognosis/prevalence | Reviews of associated risk factors | Review of qualitative studies |
Type of overview | Overview of intervention reviews | Overview of diagnostic test accuracy reviews | Overview of reviews of prognostic studies | Overview of reviews of risk factors | Overview of qualitative reviews |
Overview objective | To summarise evidence from more than one systematic review of different interventions for the same condition or problem; OR | To summarise evidence from more than one systematic review of diagnostic test accuracy assessing the same medical test to address the same condition of problem; OR | To summarise evidence about prognosis/prevalence from more than one systematic reviews. | To summarise systematic review evidence about risk factors not directly aligned to predictive values. | To summarise systematic review evidence relating to qualitative views or experiences. |
| To summarise evidence from more than one systematic review of the same intervention for the same condition of problem where different outcomes are addressed in different systematic reviews; OR | To summarise evidence from more than one systematic review of diagnostic test accuracy assessing different medical tests to address the same condition or problem. | | | |
| To summarise evidence from more than one systematic review of the same intervention for different conditions, problems or populations; OR | | | | |
| To summarise evidence about adverse effects of an intervention from more than one systematic review of use of the intervention for one or more conditions. | | | | |
Overviews of qualitative reviews
Overviews of reviews of qualitative reviews should be considered when the objective is to summarise systematic review evidence relating to qualitative views or experiences. There is clear guidance available on the good conduct of an overview of qualitative syntheses [
6], with commonalities across all types of overviews. Common features include employing an a priori peer-reviewed protocol formed around a clearly pre-specified research question with detailed inclusion and exclusion criteria, search strategies and methods for data extraction and appraisal, followed by clear and replicable methods for synthesis and summary of included data [
6]. An example overview using qualitative data as well as quantitative information is provided by an overview exploring improving quality of care for persons with diabetes looking at a broad range of interventions, including patient education and support, telemedicine, organisational changes and outcomes relating to the process of care [
10]. In combining these approaches, overview authors had potential to synthesise data on patient experiences of quality of care alongside quantitative evaluation of effectiveness, which could result in a richer set of evidence for informing practice and policy.
Whilst many overviews tacitly assess quantitative outcomes reported in systematic reviews [
6], often the nature of overviews results in narrative synthesis which can draw on either quantitative or qualitative data within included systematic reviews. In this sense, many overviews include elements of qualitative data identified within the source systematic reviews.
Results and Discussion
There are many similarities between overviews and systematic reviews, and the principles which guide the planning of a systematic review (including production of a clinically-relevant research question and a pre-specified peer reviewed protocol) are relevant in conducting an overview [
2]. Within this paper, we have described a brief classification to organise common research questions and objectives, using overviews based on frameworks developed within the Cochrane Handbooks for Systematic Reviews of Interventions [
41] and Diagnostic Test Accuracy [
42]. These descriptions cover overviews of intervention reviews, overviews of diagnostic test accuracy reviews, overviews of reviews of prognosis/prevalence, overviews of reviews of risk factors and overviews of reviews of qualitative studies.
Overviews aim to summarise evidence and to signpost readers to relevant sources to support decision making; this paper has highlighted that there are a wide range of potential reasons for selecting to do an overview, and that these varied reasons lead to overviews which may have a number of different methodological features.
Overviews of reviews of different interventions for the same condition, or of the same intervention but looking at different outcomes, will have high clinical relevance where clinical decisions are made between different treatments. Overviews of intervention reviews, bringing together evidence relating to the effectiveness of a specific treatment applied in alternative populations or settings will be of interest to healthcare providers delivering that treatment, or to consumers seeking information about the effective interventions. Overviews of risk factors will have similar clinical interest and potential relevance for policymakers and regulators. Overviews relating to the adverse effects of an intervention in the same or different conditions may allow commonalities to be drawn across a broader range of evidence than in a more focussed systematic review, with the potential to highlight equivalence or patterns not previously identified. Similarly, overviews of systematic reviews of diagnostic test accuracy provide an opportunity to gain greater insights into test accuracy data summarised across different populations, settings or other variables, with potential to reduce the impact of data heterogeneity by drawing on a broader evidence base. Overviews of prognosis are also increasing in number and scope, offering potential to provide useful insights by summarising evidence of the likely course of a condition, factors associated with health outcomes or identifying risk groups associated with different health outcomes [
38] . When applied within systematic frameworks, overviews of qualitative evidence provide scope for creating theoretically-defined conceptions of complex topics [
43].
Often, the scope of systematic reviews can be described as either ‘lumping’ or ‘splitting’ information [
44,
45]. Lumping refers to finding commonalities across different approaches, whereas splitting creates a more narrowly-refined focus within a broader research field. Systematic reviews of primary research often split data by addressing a focussed and specific research question which may not be very useful for informing broader clinical and policy decision making. Conversely, overviews commonly adopt a ‘lumping’ approach, allowing greater leeway for generality in research findings [
45], and arguably having greater applicability for policy makers. There are clearly challenges in ‘lumping’ large volumes of information, and presenting this in an accessible format, which is relevant and useful to the end user. Another significant challenge in lumping information is how to consistently synthesise such information in the face of inevitable heterogeneity.
The classification we have employed here suggests a range of common objectives and research questions which may be addressed by an overview, where the primary objective is to summarise the existing body of systematic review evidence on a topic. The scope of this summary of evidence is defined by previously stated inclusion and exclusion criteria [
6,
13]. This summary of evidence should not simply duplicate the reporting of individual systematic review summaries, but instead should aim to synthesise across included systematic review evidence in order to bring new insights to existing evidence. The suitability of reanalysis of existing data within an overview is debated, and it has been argued that, where novel analyses are the aim, conducting a review of trials may be more appropriate than an overview of reviews [
14]. Methodological guidance on the reporting of systematic reviews using individual participant data has been published by the PRISMA-IPD Group [
46] and may prove relevant to reporting within overviews which aim to incorporate novel analyses. It is clearly important for the stated overview research questions and objectives to specify any plans for data analysis, and for this to be planned with reference to the available methodological guidance, and with appropriate justification of the use of any overview of reviews, rather than a review of trials.
At its broadest sense, the common purpose of an overview is to provide an accessible summary of evidence, in order to support decision making by clinicians, policy makers and developers of clinical guidelines [
2]. It is now widely accepted that in order to ensure relevance and impact of health research, key stakeholders (including but not restricted to people with a healthcare condition, their families, friends and caregivers, health professionals and decision makers) should be involved in the process [
47,
48]. Central to the conduct of an overview are the people involved in its production. From formulating the question to conducting the overview and disseminating findings, the specific purpose of an overview may change depending on who is asking the research question and clearly stakeholders should be actively involved throughout the process. The involvement of key stakeholders, including patients and their families or carers, should occur at the earliest opportunity in order to ensure that the planned overview is relevant and meaningful to the potential end users of the overview.
Conclusions
Overviews are a relatively new methodological approach and consequently a number of aspects of overview methodology remain uncertain. It is the responsibility of a research team to decide on their approach before conducting an overview; central to this is determining what type of overview is to be conducted. Clear decisions relating to the research questions and objectives to be addressed by the overview are a fundamental first step during the initial planning stages for an overview, and should be developed with the involvement of key stakeholders. Following best practice, these aspects should be covered within a published overview protocol as a mechanism for ensuring transparency and reducing opportunities for introduction of bias in the conduct of the overview. Our second paper [
5] outlines a number of key methodological decisions which we consider important to address when planning an overview, and which will be important to incorporate within an overview protocol.
Despite a need for improved guidance for the conduct of overviews [
2], there are a number of resources available which support the conduct of overviews [
2,
6,
7,
13], and updates to the relevant chapter of the Cochrane Handbook are currently in production [
7]. Further guidance on the less common types of overview (such as those addressing reviews of diagnostic tests accuracy and prognosis) and more challenging aspects of overview production, such as methods for narratively synthesising findings, dealing with missing data, poor reporting and dealing with complexity versus granularity [
10], would be a great benefit to those tackling overviews. In the absence of empirical evidence to support the selection and implementation of overview methods, we believe that the use of illustrated examples of real-life overviews will be helpful to authors planning new overviews, and to those seeking to establish evidence relating to optimal overview methods. This is therefore the focus of our second paper on this topic [
5].