Cochrane primarily aims to systematically review trials of effectiveness that are important to inform clinical decisions. Editorial groups support authors to achieve high-quality reviews and prioritise review proposals in their clinical domain that are submitted or elicited. Prioritising proposals requires two approaches, identifying (1) clinical practises for which the evidence of effectiveness is uncertain and (2) interventions in which there are trials of effectiveness (especially randomised controlled trials (RCTs)) not systematically reviewed. This study addresses this second approach for the Cochrane Acute Respiratory Infections Group (CARIG) in order to identify RCTs of acute respiratory infections that have not been systematically reviewed.
Methods
We exported, on the 9th of September 2014, and then compared the group’s trials register of RCTs against a list of current Cochrane ARI (systematic) Reviews to identify gaps in topics (the same intervention and health condition) where completed trials have not been systematically reviewed. We assigned a principle intervention and health condition to each of 157 Cochrane reviews (CRs) and 5393 RCTs.
Results
A majority of topics had been systematically reviewed; however, a substantial number (2174 or 41%) of RCTs were not included in any review. The topic that had been systematically reviewed the most was antibiotic vs placebo for pneumonia with 11 CRs and 205 RCTs. The topic that was the subject of most RCTs was vaccination for influenza with 525 RCTs and 6 CRs. Also, 6 CRs had no RCTs (‘empty reviews’).
Conclusions
We identified many RCT topics that have not been systematically reviewed. They need to be addressed in a separate process to establish their priority to clinicians.
Abkürzungen
ARI
Acute respiratory infections
CARIG
Cochrane Acute Respiratory Infections Group
CDSR
Cochrane Database of Systematic Reviews
CR
Cochrane review
RCT
Randomised controlled trial
SR
Systematic review
Background
Systematic reviews (SRs) summarise and synthesise randomised controlled trials (RCTs), the best method for testing interventions, to produce high levels of evidence. Cochrane is an organisation committed to generating the highest level of evidence by systematically reviewing the medical literature [1]. It comprises 53 editorial groups. The Cochrane Acute Respiratory Infections Group (CARIG) focuses on reviewing and summarising the evidence of treatments for acute respiratory infections (ARIs) [2]. ARIs carry a large burden of disease [3].
Cochrane reviews are among the most rigorous forms of systematic review [4] and, accordingly, require the greatest editorial support provided by the Cochrane ARI Group. They typically take 6 months to 2 years to complete [5, 6]. Not all potential titles submitted can be supported, and so, a priority-setting process is necessary. [7‐10]. As part of the CARIG’s priority-setting process, we resolved to determine interventions and health conditions for which there are RCTs not systematically reviewed by Cochrane.
Anzeige
Methods
Study selection and categorisation
We exported Cochrane reviews specific to the CARIG by interrogating Cochrane’s management software (‘Archie’), equivalent to searching the Cochrane Database of Systematic Reviews (CDSR) Issue 9 of 12, September 2014. We also exported a list of ARI-specific RCTs from the CARIG trials register (the date range of trials was 1930 to 2014). Both exports were done on the 9th of September 2014. Both lists were imported via a reference manager (EndNote) into a spreadsheet where two authors (JA and SV) independently examined the titles (and if necessary, abstracts) to classify the main interventions and health conditions (together forming a ‘topic’). Disagreements were settled by consensus or resolved by a third author (CDM).
We sorted CRs and RCTs by topic (the same intervention and health condition) and then matched CRs and RCTs with the same topic (or paired intervention and health condition). This process enabled us to identify the CRs and RCTS where a match was made on the same topic (intervention and health condition matched) and where there were existing RCTs on particular topics (intervention and health condition) but no CRs. We also identified where there were CRs on particular topics but no RCTs (empty reviews).
Studies that did not consistently cover a single disease or intervention
For each CR or RCT, the intervention was classified followed by the health condition. Where more than one health condition and/or intervention was represented in a single CR or RCT, all health conditions and interventions were classified.
Resolving categorisation discrepancies
Once all studies had been assigned an intervention and health condition, the categories were checked for consistency. Where appropriate, categories were merged together (e.g. the physiotherapy and exercise categories were merged into a single category called ‘physiotherapy/exercise’). Medical and common terms (e.g. ‘pharyngitis’ and ‘sore throat’) were also combined into a single category.
Anzeige
Results
Out of 162 Cochrane reviews screened, 5 were excluded as they were either withdrawn or out of date, leaving 157 for inclusion. Out of 5393 RCT titles screened, 108 were excluded due to not addressing an ARI, had no intervention or were not an RCT, leaving 5285. Of these 409, required reading the abstract and, or, full text.
We initially listed 54 Cochrane review intervention categories, which we merged into 45, and 35 health condition categories were merged into 27. Similarly, 377 RCT intervention categories were merged to 182, and 168 health condition categories to 101.
The most common topics systematically reviewed by the CARIG were antibiotics for pneumonia (11 CRs, 6.4% of the total); vaccination for influenza (n = 6, 3.5%); vaccination for pertussis (n = 4, 2.3%); antiviral drugs for influenza (n = 4, 2.3%); antibiotics for otitis media (n = 4, 2.3%); and antibiotics for sore throat (n = 4, 2.3%) (Table 1). The most common interventions reviewed by the CARIG is antibiotic vs placebo (5 out of the top 10 most common topics). The topics which had been the focus of the most RCTS were vaccinations for influenza (525, 7.7%); vaccination for pertussis (303, 4.4%); and antibiotic vs antibiotic for pneumonia (269, 4%) (Table 2). The most commonly occurring intervention in the ARI trials register was vaccination (6 out of the top 10 most common topics). However, antibiotic vs antibiotic in general was the least common intervention Cochrane reviewed (only 1, 0.6%).
Table 1
Number of CRs and RCTs ranked by number of CRs
Intervention
Health condition
No. of RCTs
No. of CRs
Antibiotic vs placebo
Pneumonia
205
11
Vaccination
Influenza
525
6
Vaccination
Pertussis
303
4
Antiviral
Influenza
147
4
Antibiotic vs placebo
Otitis media
114
4
Antibiotic vs placebo
Pharyngitis/sore throat
90
4
Antibiotic vs placebo
ARI non-specific
129
3
CAM
Common cold
44
3
Antibiotic vs placebo
Meningitis
36
3
Vaccination
Diphtheria
246
2
Vaccination
Tetanus
236
2
Vaccination
Measles
161
2
Vaccination
Pneumococcus
143
2
Antibiotic vs placebo
Bronchitis, acute
60
2
Vaccination
Hepatitis
58
2
CAM
ARI non-specific
49
2
Antiviral
Herpes zoster
43
2
Antibiotic vs placebo
Bronchiolitis
42
2
Antihistamine
Common cold
41
2
Corticosteroid
Meningitis
39
2
Antitussive/decongestant/expectorant
Cough
36
2
Vaccination
Otitis media
35
2
Vaccination
Herpes zoster
30
2
Antibiotic vs placebo
Common cold
21
2
CAM
Influenza
18
2
Table 2
Number of CRs and RCTs ranked by number of RCTs
Intervention
Health condition
No. of RCTs
No. of CRs
Vaccination
Influenza
525
6
Vaccination
Pertussis
303
4
Antibiotic vs antibiotic
Pneumonia
269
0
Vaccination
Diphtheria
246
2
Vaccination
Tetanus
236
2
Antibiotic vs placebo
Pneumonia
205
11
Vaccination
Croup
188
1
Vaccination
Measles
161
2
Antiviral
Influenza
147
4
Antibiotic vs antibiotic
Bronchitis, acute
146
0
Vaccination
Pneumococcus
143
2
Antibiotic vs antibiotic
Pharyngitis/sore throat
140
0
Antibiotic vs antibiotic
Otitis media
135
0
Antibiotic vs placebo
ARI non-specific
129
3
Vaccination
Meningitis
116
1
Antibiotic vs placebo
Otitis media
114
4
Antibiotic vs antibiotic
ARI non-specific
106
0
Antibiotic vs placebo
Pharyngitis/sore throat
90
4
Vaccination
Mumps
86
1
Antibiotic vs antibiotic
Sinusitis
82
0
Vaccination
Rubella
80
1
Vaccination
Polio
67
0
Immunotherapy
ARI non-specific
65
1
Antibiotic vs placebo
Bronchitis, acute
60
2
Antibiotic vs placebo
Sinusitis
60
1
There were many RCTs with no corresponding CRs (2174 or 41%) (Table 3). Most used the intervention of antibiotics, which accounts for 878 RCTs (12.8%). Similarly, there were (only) 6 CRs which reviewed no RCTs (that is they were ‘empty reviews’) (Table 4).
Table 3
RCTs with no CR ranked by number of RCTs
Intervention
Health condition
No. of RCTs
No. of CRs
Antibiotic vs antibiotic
Pneumonia
269
0
Antibiotic vs antibiotic
Bronchitis, acute
146
0
Antibiotic vs antibiotic
Pharyngitis/sore throat
140
0
Antibiotic vs antibiotic
Otitis media
135
0
Antibiotic vs antibiotic
ARI non-specific
106
0
Antibiotic vs antibiotic
Sinusitis
82
0
Vaccination
Polio
67
0
Antibiotic vs antibiotic
Bronchiolitis
53
0
NSAID
ARI non-specific
36
0
Immunotherapy
Common cold
34
0
NSAID
Pharyngitis/sore throat
31
0
Antiviral
Common cold
27
0
Vaccination
Herpes simplex
25
0
Vitamin A
ARI non-specific
23
0
Antitussive/decongestant
Otitis media
23
0
Vitamins and supplements
ARI non-specific
19
0
Antibiotic vs placebo
Staphylococcus
19
0
Vaccination
ARI non-specific
18
0
CAM
Bronchiolitis
17
0
Vaccination reminder
Influenza
17
0
Humidification/steam
Pneumonia
16
0
Antiviral
Respiratory syncytial virus
16
0
Vaccination
Respiratory syncytial virus
15
0
Antibiotic vs antibiotic
Staphylococcus
15
0
Antibiotic vs antibiotic
Streptococcus
14
0
Table 4
CRs with no RCTs (empty reviews)
Intervention
Health condition
No. of RCTs
No. of CRs
Zinc
Otitis media
0
1
Acupuncture
Mumps
0
1
CAM
Bronchitis, acute
0
1
CAM
Mumps
0
1
Fluid therapy
ARI non-specific
0
1
Nasal irrigation
ARI non-specific
0
1
We devised a novel method of representing the extensive relationship between CRs and RCTs, which conveys the information in Tables 1, 2 and 3 that allows the user to interact with the data (Figs. 1, 2 and 3). This is an interactive online graph available from our website [11].
Fig. 1
Graphical representation of CRs mapped to RCTs overview. Colours are used to identify the same interventions, for instance blue, as seen here, represents vaccination. In the live version, users can select the topic of interest by passing the mouse over it, which will expand the number of RCTs and CRs associated with that topic. The figure shows a static sub-section of the graph at http://crebp.github.io/CREBP-Disease-Treatment/
Fig. 2
Graphical representation of Cochrane review mapped to RCTs (antibiotic vs antibiotic). This static screenshot shows the effects of a reader’s mouse over an intervention which is covered by a Cochrane review (here, antibiotic vs antibiotic): the right side of the figure expands to show the diseases for which antibiotic vs antibiotic has been studied
Fig. 3
Graphical representation of Cochrane reviews mapped to RCTs (no CR, vaccination reminder). In this example, there is an intervention (vaccination reminders) with RCTs but no corresponding Cochrane review (a ‘review gap’). The RCT section has expanded to show the topics with available evidence that could be incorporated into a Cochrane review. The Cochrane review section is empty
×
×
×
Discussion
We found many topics which have been trialled but not reviewed, consistent with the previous findings [12]. These data should allow the CARIG to identify clinical questions in need of review.
The topic with the most Cochrane reviews was antibiotics for pneumonia. Pneumonia makes an important contribution to the burden of disease worldwide, especially in the developing world [3], and so, this over-representation seems appropriate. The interventions trialled with least representation with CRs, antibiotic vs antibiotic, are often driven by pharmaceutical companies (interested in demonstrating that a new member of an existing antibiotic class has equivalent efficacy), something perhaps less interesting to clinicians.
Strengths of our methods were the exploitation of the set of trials already collected by Cochrane and the collection of CRs and our matching methods and online visualisation techniques. Weaknesses include the limitation to CRs (there are undoubtedly other systematic reviews outside Cochrane), the potentially arbitrary over-simplification of the topics to one or two interventions for each health condition category and our limitation to treatment questions (Cochrane has a minority of diagnostic reviews as well)—nor did we account for ‘stabilised reviews’ (those in which the intervention is no longer current, e.g. amantadine and rimantadine for influenza [13] or where there is sufficient evidence to settle for a clinical question), e.g. vaccination to prevent polio [14].
Anzeige
Conclusions
These data will inform our forthcoming priority-setting exercise during which they will be presented to stakeholders (health consumers and clinicians) to allow judgement to be made about which topics should be given higher priority.
Acknowledgements
The authors would like to thank Dr Anna Mae Scott from The Centre for Research Excellence in Minimising Antibiotic Resistance from Acute Respiratory Infections (CREMARA) for providing valuable input on the draft of the manuscript.
Funding
At the time of the study, JA and SV were medical students and not funded. CDM, ST, MC and JC were funded by the National Health and Medical Research Council (NHMRC).
CDM, ST, JA, SV and MC contributed to the conception and design of the work. JA, SV and JC conducted the data analysis. MC devised the graphical analysis of the data. All authors drafted the work or revised it critically for important intellectual content, gave the final approval for the version to be published and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated or resolved.
Competing interests
The authors declare that they have no competing interests.
Consent for publication
Not applicable
Ethics approval and consent to participate
Not applicable
Anzeige
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
