What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences

For the purpose of this typology a systematic review is defined as a robust, reproducible, structured critical synthesis of existing research. While other approaches to the synthesis of evidence exist (including but not limited to literature reviews, evidence maps, rapid reviews, integrative reviews, scoping and umbrella reviews), this paper seeks only to include approaches that subscribe to the above definition. As such, ten different types of systematic review foci are listed below and in Table 1. In this proposed typology, we provide the key elements for formulating a question for each of the 10 review types.

Systematic reviews assessing the effectiveness of an intervention or therapy are by far the most common. Essentially effectiveness is the extent to which an intervention, when used appropriately, achieves the intended effect [11]. The PICO approach (see Table 1) to question development is well known [23] and comprehensive guidance for these types of reviews is available [24]. Characteristics regarding the population (e.g. demographic and socioeconomic factors and setting), intervention (e.g. variations in dosage/intensity, delivery mode, and frequency/duration/timing of delivery), comparator (active or passive) and outcomes (primary and secondary including benefits and harms, how outcomes will be measured including the timing of measurement) need to be carefully considered and appropriately justified.

Experiential (qualitative) reviews focus on analyzing human experiences and cultural and social phenomena. Reviews including qualitative evidence may focus on the engagement between the participant and the intervention, as such a qualitative review may describe an intervention, but its question focuses on the perspective of the individuals experiencing it as part of a larger phenomenon. They can be important in exploring and explaining why interventions are or are not effective from a person-centered perspective. Similarly, this type of review can explain and explore why an intervention is not adopted in spite of evidence of its effectiveness [4, 13, 25]. They are important in providing information on the patient’s experience, which can enable the health professional to better understand and interact with patients. The mnemonic PICo can be used to guide question development (see Table 1). With qualitative evidence there is no outcome or comparator to be considered. A phenomenon of interest is the experience, event or process occurring that is under study, such as response to pain or coping with breast cancer; it differs from an intervention in its focus. Context will vary depending on the objective of the review; it may include consideration of cultural factors such as geographic location, specific racial or gender based interests, and details about the setting such as acute care, primary healthcare, or the community [4, 13, 25]. Reviews assessing the experience of a phenomenon may opt to use a mixed methods approach and also include quantitative data, such as that from surveys. There are reporting guidelines available for qualitative reviews, including the ‘Enhancing transparency in reporting the synthesis of qualitative research’ (ENTREQ) statement [26] and the newly proposed meta-ethnography reporting guidelines (eMERGe) [27].

Costs/Economics reviews assess the costs of a certain intervention, process, or procedure. In any society, resources available (including dollars) have alternative uses. In order to make the best decisions about alternative courses of action evidence is needed on the health benefits and also on the types and amount of resources needed for these courses of action. Health economic evaluations are particularly useful to inform health policy decisions attempting to achieve equality in healthcare provision to all members of society and are commonly used to justify the existence and development of health services, new health technologies and also, clinical guideline development [14]. Issues of cost and resource use may be standalone reviews or components of effectiveness reviews [28]. Cost/Economic evaluations are examples of a quantitative review and as such can follow the PICO mnemonic (see Table 1). Consideration should be given to whether the entire world/international population is to be considered or only a population (or sub-population) of a particular country. Details of the intervention and comparator should include the nature of services/care delivered, time period of delivery, dosage/intensity, co-interventions, and personnel undertaking delivery. Consider if outcomes will only focus on resource usage and costs of the intervention and its comparator(s) or additionally on cost-effectiveness. Context (including perspective) can also be considered in these types of questions e.g. health setting(s).

Essentially prevalence or incidence reviews measure disease burden (whether at a local, national or global level). Prevalence refers to the proportion of a population who have a certain disease whereas incidence relates to how often a disease occurs. These types of reviews enable governments, policy makers, health professionals and the general population to inform the development and delivery of health services and evaluate changes and trends in diseases over time [15, 29]. Prevalence or incidence reviews are important in the description of geographical distribution of a variable and the variation between subgroups (such as gender or socioeconomic status), and for informing health care planning and resource allocation. The CoCoPop framework can be used for reviews addressing a question relevant to prevalence or incidence (see Table 1). Condition refers to the variable of interest and can be a health condition, disease, symptom, event of factor. Information regarding how the condition will be measured, diagnosed or confirmed should be provided. Environmental factors can have a substantial impact on the prevalence or incidence of a condition so it is important that authors define the context or specific setting relevant to their review question [15, 29]. The population or study subjects should be clearly defined and described in detail.

Systematic reviews assessing diagnostic test accuracy provide a summary of test performance and are important for clinicians and other healthcare practitioners in order to determine the accuracy of the diagnostic tests they use or are considering using [16]. Diagnostic tests are used by clinicians to identify the presence or absence of a condition in a patient for the purpose of developing an appropriate treatment plan. Often there are several tests available for diagnosis. The mnemonic PIRD is recommended for question development for these types of systematic reviews (see Table 1). The population is all participants who will undergo the diagnostic test while the index test(s) is the diagnostic test whose accuracy is being investigated in the review. Consider if multiple iterations of a test exist and who carries out or interprets the test, the conditions the test is conducted under and specific details regarding how the test will be conducted. The reference test is the ‘gold standard’ test to which the results of the index test will be compared. It should be the best test currently available for the diagnosis of the condition of interest. Diagnosis of interest relates to what diagnosis is being investigated in the systematic review. This may be a disease, injury, disability or any other pathological condition [16].

Systematic reviews of etiology and risk are important for informing healthcare planning and resource allocation, and are particularly valuable for decision makers when making decisions regarding health policy and prevention of adverse health outcomes. The common objective of many of these types of reviews is to determine whether and to what degree a relationship exists between an exposure and a health outcome. Use of the PEO mnemonic is recommended (see Table 1). The review question should outline the exposure, disease, symptom or health condition of interest, the population or groups at risk, as well as the context/location, the time period and the length of time where relevant [17]. The exposure of interest refers to a particular risk factor or several risk factors associated with a disease/condition of interest in a population, group or cohort who have been exposed to them. It should be clearly reported what the exposure or risk factor is, and how it may be measured/identified including the dose and nature of exposure and the duration of exposure, if relevant. Important outcomes of interest relevant to the health issue and important to key stakeholders (e.g. knowledge users, consumers, policy makers, payers etc.) must be specified. Guidance now exists for conducting these types of reviews [17]. As these reviews rely heavily on observational studies, the Meta-analysis Of Observational Studies in Epidemiology (MOOSE) [30] reporting guidelines should be referred to in addition to the PRISMA guidelines.

Expert opinion and policy analysis systematic reviews focus on the synthesis of narrative text and/or policy. Expert opinion has a role to play in evidence-based healthcare, as it can be used to either complement empirical evidence or, in the absence of research studies, stand alone as the best available evidence. The synthesis of findings from expert opinion within the systematic review process is not well recognized in mainstream evidence-based practice. However, in the absence of research studies, the use of a transparent systematic process to identify the best available evidence drawn from text and opinion can provide practical guidance to practitioners and policy makers [18]. While a number of mnemonics have been discussed previously that can be used for opinion and text, not all elements necessarily apply to every text or opinion-based review, and use of mnemonics should be considered a guide rather than a policy. Broadly PICo can be used where I can refer to either the intervention or a phenomena of interest (see Table 1). Reviewers will need to describe the population, giving attention to whether specific characteristics of interest, such as age, gender, level of education or professional qualification are important to the question. As with other types of reviews, interventions may be broad areas of practice management, or specific, singular interventions. However, reviews of text or opinion may also reflect an interest in opinions around power, politics or other aspects of health care other than direct interventions, in which case, these should be described in detail. The use of a comparator and specific outcome statement is not necessarily required for a review of text and opinion based literature. In circumstances where they are considered appropriate, the nature and characteristics of the comparator and outcomes should be described [18].

Psychometric systematic reviews (or systematic reviews of measurement properties) are conducted to assess the quality/characteristics of health measurement instruments to determine the best tool for use (in terms of its validity, reliability, responsiveness etc.) in practice for a certain condition or factor [31‐33]. A psychometric systematic review may be undertaken on a) the measurement properties of one measurement instrument, b) the measurement properties of the most commonly utilized measurement instruments measuring a specific construct, c) the measurement properties of all available measurement instruments to measure a specific construct in a specific population or d) the measurement properties of all available measurement instruments in a specific population that does not specify the construct to be measured. The COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) group have developed guidance for conducting these types of reviews [19, 31]. They recommend firstly defining the type of review to be conducted as well as the construct or the name(s) of the outcome measurement instrument(s) of interest, the target population, the type of measurement instrument of interest (e.g. questionnaires, imaging tests) and the measurement properties on which the review investigates (see Table 1).

Prognostic research is of high value as it provides clinicians and patients with information regarding the course of a disease and potential outcomes, in addition to potentially providing useful information to deliver targeted therapy relating to specific prognostic factors [20, 34, 35]. Prognostic reviews are complex and methodology for these types of reviews is still under development, although a Cochrane methods group exists to support this approach [20]. Potential systematic reviewers wishing to conduct a prognostic review may be interested in determining the overall prognosis for a condition, the link between specific prognostic factors and an outcome and/or prognostic/prediction models and prognostic tests [20, 34‐37]. Currently there is little information available to guide the development of a well-defined review question however the Quality in Prognosis Studies (QUIPS) tool [34] and the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS Checklist) [38] have been developed to assist in this process (see Table 1).

Systematic reviews can be conducted for methodological purposes [39], and examples of these reviews are available in the Cochrane Database [40, 41] and elsewhere [21]. These reviews can be performed to examine any methodological issues relating to the design, conduct and review of research studies and also evidence syntheses. There is limited guidance for conducting these reviews, although there does exist an appendix in the Cochrane Handbook focusing specifically on methodological reviews [39]. They suggest following the SDMO approach where the types of studies should define all eligible study designs as well as any thresholds for inclusion (e.g. RCTS and quasi-RCTs). Types of data should detail the raw material for the methodology studies (e.g. original research submitted to biomedical journals) and the comparisons of interest should be described under types of methods (e.g. blinded peer review versus unblinded peer review) (see Table 1). Lastly both primary and secondary outcome measures should be listed (e.g. quality of published report) [39].