The situation in 1992/1993
The Cochrane Collaboration was established in 1993, with roots in the opening of the UK Cochrane Centre in October 1992. Other than the Information Specialist employed at the UK Cochrane Centre (CL), there were no Trials Search Co-ordinators (TSCs) employed within The Cochrane Collaboration in the early 1990s. The term TSC, now generally used across The Cochrane Collaboration for the information specialists who identify relevant studies for inclusion in Cochrane Reviews, did not exist. Authors relied on their own skills and expertise in searching to identify reports of studies or on the skills of medical librarians, who often had limited awareness of systematic reviews and the specific searching approaches required. There was little training for, or awareness amongst, medical librarians regarding the role, importance and complexity of systematic reviews and little focus on searching for adverse events, economic evaluations, non-randomized designs or qualitative studies. There was very limited guidance for the authors of Cochrane Reviews or Cochrane Review Group staff with respect to the conduct or reporting of searches.
When work began at the UK Cochrane Centre in 1992 to identify reports of randomized controlled trials (RCTs) for inclusion in Cochrane Reviews, approximately 20,000 reports were indexed as RCTs in MEDLINE. Randomized Controlled Trial had only been introduced into MEDLINE as a Publication Type term in 1991 and at that time there was no indexing term at all for quasi-randomized studies. In Embase, there were no indexing terms whatsoever for RCTs or quasi-randomized studies. No filters had been designed specifically to identify all RCTs in MEDLINE or any of the other major databases (irrespective of other study characteristics). This presented a tremendous challenge for a newly-established organization such as The Cochrane Collaboration in terms of efficient identification of relevant studies [
1,
2].
In December 1992, a meeting was held with Elsevier which led to the introduction of RCT as an indexing term into Embase in 1993 [
3] and a commitment by Elsevier to improve the indexing of clinical trials [
4]. This was followed in December 1993 by a conference hosted by the US National Library of Medicine (NLM) which led to agreement to 're-tag’ RCTs in MEDLINE (the MEDLINE re-tagging project). This project resulted in more than 125,000 reports of randomized and quasi-randomized trials, not already indexed as such in MEDLINE, being identified, re-tagged in MEDLINE and included in the Cochrane Central Register of Controlled Trials (CENTRAL).
In 1995, a new indexing term for quasi-randomized trials (Controlled Clinical Trial) was introduced into MeSH [
5]. The Cochrane Central Register of Controlled Trials (CENTRAL) was launched in 1996 (under its original name of The Cochrane Controlled Trials Register), as part of the newly published
Cochrane Library[
6]. CENTRAL was referred to in its early days as 'likely to be the best single source of published trials for inclusion in systematic reviews and meta-analyses’ [
7]. In 1996, Elsevier agreed that reports of trials identified from Embase could be included in CENTRAL (the Embase project). CENTRAL developed rapidly over the next few years [
6]. The contributions of the MEDLINE re-tagging project [
6,
8] and the Embase project [
3] now form the basis of CENTRAL, which is the single largest source of reports of trials, with 700,000 records drawn from MEDLINE, Embase, Cochrane groups and other sources [
9].
Considerable progress has been made in searching across a range of areas important to the systematic review process. Some examples include the identification of information on adverse effects of interventions [
10‐
14]; diagnostic test studies [
15‐
17]; economic evaluation [
18‐
21]; studies of prognosis [
22‐
24] and causation [
24‐
27]; non-randomized studies related to interventions [
28] and qualitative studies [
29‐
33]. Progress has also been made in the identification of systematic reviews, as sources of potential studies [
34‐
37].
With respect to RCTs, filters aimed at identifying all RCTs in MEDLINE, irrespective of other study characteristics, began to be developed by members of The Cochrane Collaboration in the early 1990s [
2]. They were revised using objective methods of search strategy design (textual analysis) in 2006 [
38] and subsequently kept up to date in the 'Searching for Studies’ chapter of
The Cochrane Handbook for Systematic Reviews of Interventions[
39].
The proliferation of search filters across a range of methodological areas, and their ease of access through incorporation in services such as Ovid and PubMed, has led to the development of the InterTASC Information Specialists’ Sub-Group (ISSG) Search Filter Resource. This offers critical appraisals and summaries of search filters together with references to comparative testing data [
40,
41]. Given the growing interest in identifying information beyond particular types of study, such as age groups, geographic areas and ethnic groups, the ISSG Search Filter Resource has recently been expanded to incorporate these topics. Examination of the conduct and reporting of searches for Cochrane Reviews [
42,
43] has led to structured approaches to peer review of search strategies (for example, the Peer Review of Electronic Search Strategies (PRESS) checklist) [
44‐
47].
In addition to the advances in the identification of studies from bibliographic databases outlined above, methodological work has been undertaken on the value of searching the 'grey literature’ which has been defined as 'information produced and distributed at all levels by government, academics, business and industry in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body’ [
48,
49]. Related research has assessed the value of handsearching for trials [
50‐
52] and examined the characteristics associated with full publication of meeting abstracts [
53]. Alternative search techniques such as 'pearl-growing’/'snowballing’ from known key references and checking reference lists have also been explored [
54,
55].
Keeping up-to-date in methodological advances has been facilitated by the advent and development of the Cochrane Methodology Register, published in
The Cochrane Library. Updating of this resource is currently on hold pending decisions regarding its future within The Cochrane Collaboration. The recently-launched Summarized Research in Information Retrieval for Health Technology Assessment (SuRe Info) provides research-based information regarding the latest developments in the information retrieval aspects of producing systematic reviews and health technology assessments [
56]. It provides critical appraisals and summaries of current methods papers and general overviews of the state of the evidence across a range of topics relevant to information retrieval for systematic reviews.
The guidance in Chapter 6 of
The Cochrane Handbook for Systematic Reviews of Interventions entitled 'Searching for Studies’ started life in 1994 as an internal Cochrane document produced by Kay Dickersin and Carol Lefebvre entitled 'Establishing and Maintaining Registers of RCTs’. The document provided limited guidance with respect to the conduct and reporting of searches. This subsequently became incorporated into the
Handbook and now provides detailed guidance for authors of Cochrane Reviews and Cochrane Review Group staff, including TSCs, Managing Editors, Co-ordinating Editors and Editors [
39]. It is also used by other evidence synthesis organizations and provided a model for the Campbell Collaboration’s Guide to Information Retrieval for Campbell Systematic Reviews [
57]. The
Handbook is revised and updated in consultation with the information retrieval community of The Cochrane Collaboration, that is, the Cochrane Information Retrieval Methods Group and TSCs. Standard Training Materials have been produced and have been updated in the light of the standards recently introduced under the Cochrane Methodological Expectations of Cochrane Intervention Reviews programme (MECIR) [
58].
In 2013, almost all Cochrane Review Groups and some Cochrane Centres and Fields have a dedicated TSC – usually a qualified librarian/information specialist with experience of searching the medical literature. These TSCs carry out a vital role in study identification within their respective groups, although the nature of their contributions varies considerably according to resources and other factors. There is also far greater awareness amongst medical librarians and other information specialists regarding the role of systematic reviews and how to search for studies for inclusion in systematic reviews.
Focus on the future: 2033 and beyond
In making any assessments as to the possible situation in twenty years’ time, we should be mindful of the words attributed to Niels Bohr, the Danish physicist (1885–1962): 'Prediction is very difficult, especially about the future’ [
92]. In twenty years’ time, we envisage that universal social networking (or its successors) as well as national and international legislation will mean that all trials involving humans will be registered at inception. In addition, details of ongoing and completed trials will be accessible to all, irrespective of whether or not they have been published in the scientific literature, in a manner suitable for synthesis in systematic reviews and for other purposes. Registration of trials will become universal, in part, because information about trials will be broadcast by active trial participants who will publicize their experiences. Some compromise between commercial interests and public interests will have been reached so that far more detailed trial results will be available than we see at present, perhaps held in a common format in a single international clinical trials results register. This would build on the progress already made in creating and developing ClinicalTrials.gov [
93] and the ICTRP [
94]. The ClinicalTrials.gov dataset is already being used by The Cochrane Collaboration in the Cochrane Register of Studies and systems are being developed to add value to ClinicalTrials.gov, such as by downloading study results into a spreadsheet format ready for analysis [
95]. All data from clinical trials required for systematic reviews and meta-analyses will be available in a single international clinical trials data repository, building on the progress already made in creating and developing the Systematic Review Data Repository (SRDR) [
96]. Challenges will remain, however, in ensuring the discoverability of relevant information in these diverse and often complex sources and in developing the metadata needed to provide the most efficient access to information to answer specific questions reliably.
Conclusions
Considerable progress has been made in the field of information retrieval within the context of systematic reviews over the last twenty years, as outlined above. There will, however, be many challenges as well as opportunities in the years ahead. We envisage that, in twenty years’ time, there will still be a role for experts in identifying new resources, researching efficient ways to link or mine them for relevant data and managing their content for the efficient production of systematic reviews. Whether these experts will be referred to as Trials Search Co-ordinators, Information Specialists or something else entirely in 2033, remains to be seen.
CL is an independent information specialist. She was a founding member of the UK Cochrane Centre (UKCC), where she was employed as the Senior Information Specialist from its inception, in 1992, until June 2012. She has an MSc in Library and Information Studies. She is a founding Co-Convenor of the Cochrane Information Retrieval Methods Group and a member of the Cochrane Methods Executive. She was a member of the Trials Search Co-ordinators Executive until 2012. JG is an Associate Director of York Health Economics Consortium at the University of York, York, UK. She has worked on systematic reviews since 1993 when she was one of the founding members of the Centre for Reviews and Dissemination, the sister organization to the UK Cochrane Centre. She has a Postgraduate Diploma in Librarianship and Information Studies and an MSc in Information Processing. She is a Co-Convenor of the Cochrane Information Retrieval Methods Group, has previously chaired the Cochrane Library Users’ Group and has provided extensive training in the use of The Cochrane Library over many years. She is a co-author of Cochrane Reviews and a peer reviewer for Cochrane Diagnostic Test Accuracy reviews. LSW is a Research Associate at the Center for Evidence-Based Medicine, Brown University, RI, US. She has a PhD in Epidemiology. She worked at the New England Cochrane Center, Providence Office in 2001–2002 and the US Cochrane Center from 2003–2005. She is the TSC for the Cochrane Complementary Medicine Field. BC has worked for Cardiff University since 1987 and is Site Librarian at the Cancer Research Wales Library in Cardiff, UK. She has an MSc in Health Information Management and became involved in systematic reviewing when the Cochrane Prostatic Diseases and Urological Cancers Group was established in 1998. She has recently been appointed as an Assistant TSC for this Group. She has been the Co-ordinator of the Cochrane Information Retrieval Methods Group since 2006. She is a co-author of 10 Cochrane Reviews and teaches a wide variety of courses for National Health Service staff in the UK. AW is the Associate Director for Research & Academic Engagement, Information Services and Director of the Support Unit for Research Evidence (SURE) at Cardiff University, UK. She has a Postgraduate Diploma in Librarianship, an MA in Library and Information Science and a PhD in Microbiology. She is a Co-Convenor of the Cochrane Information Retrieval Methods Group and a Cochrane Review author. She has more than 15 years' experience as a systematic reviewer and a particular interest and expertise in reviewing and developing systematic review techniques for complex public health topics.
Acknowledgements
With respect to CENTRAL, we are grateful to: Update Software, the original designers, developers and publishers of CENTRAL; John Wiley and Sons, the current publishers of CENTRAL; staff, past and present, at the UK Cochrane Centre and the US Cochrane Center who contributed records to CENTRAL through the MEDLINE re-tagging project; staff, past and present, at the UK Cochrane Centre who contributed records to CENTRAL through the Embase project; all TSCs and others who have submitted records for publication in CENTRAL; staff, past and present, at the US Cochrane Center for processing handsearch records and Specialized Register records for publication in CENTRAL; handsearchers for reading the records to identify reports of trials; all those who contributed in other ways to the building and development of CENTRAL; the funders of the organizations involved in this work; staff, past and present, of the US National Library of Medicine and Elsevier who collaborated with us with respect to the inclusion of MEDLINE and Embase records, respectively, in CENTRAL. We should like to acknowledge members of the Cochrane Information Retrieval Methods Group and others for the various achievements described in this paper. We should also like to thank Ruth Foxlee (TSC, Cochrane Wounds Group and Information Specialist, Cochrane Editorial Unit) and Becky Skidmore (Medical Research Analyst, the Society of Obstetricians and Gynaecologists of Canada) as peer reviewers of this manuscript and Mike Clarke (All Ireland Hub for Trials Methodology Research, Queen’s University Belfast, Northern Ireland) as Guest Editor for their helpful comments and suggestions regarding this manuscript.
Competing interests
CL declares that, within the past five years, she has received reimbursements from the publishers of The Cochrane Library (John Wiley and Sons) towards travel expenses to attend conferences and deliver presentations about The Cochrane Library. She has not received any fees, other funding, or salary from John Wiley and Sons. She declares that, as an independent information specialist, she derives income from consultancy and teaching in information retrieval for evidence synthesis, including some of the topics addressed in this paper. Whilst employed at the UK Cochrane Centre, she was responsible for projects that contributed a significant proportion of the records to the Cochrane Central Register of Controlled Trials (CENTRAL). JG declares that she has received funding from The Cochrane Collaboration as a member of two consortia. As part of one consortium she managed the production of an annotated bibliography of research on identifying unpublished study data and in the second consortium she is managing the production of a search filter to identify reports of RCTs in Embase. She has not received any fees, other funding, or salary from The Cochrane Collaboration or their publishers, John Wiley and Sons. She derives income from consultancy and teaching in systematic reviewing and information retrieval for evidence synthesis, including some of the topics addressed in this paper. LSW declares that, whilst she was employed at the US Cochrane Center, she co-ordinated the project for re-tagging MEDLINE records and processed Cochrane handsearch records and Specialized Register records, which constituted a significant proportion of the records included in the Cochrane Central Register of Controlled Trials (CENTRAL). BC and AW declare that they have no competing interests.
Authors’ contributions
CL was responsible for the conception and design of the paper; drafted the original outline of the paper; drafted the sections on the past and the present and revised the paper critically for important intellectual content. JG approved the original outline of the paper; drafted the sections on the future and revised the paper critically for important intellectual content. LSW approved the original outline of the paper; revised the paper critically for important intellectual content and identified relevant references. BC approved the original outline of the paper and created the EndNote Library and the Bibliography. AW approved the original outline of the paper and revised the paper critically for important intellectual content. All authors read and approved the final manuscript.