Bibliometric and content analysis of the Cochrane Complementary Medicine Field specialized register of controlled trials

All citations in the CAM Field specialized register are required to meet the following two inclusion criteria: (1) they must be reports of controlled trials, and (2) they must be CAM-related.

We considered controlled trials to be studies meeting the inclusion criteria for CENTRAL that were formulated and agreed upon in November 1992 and are published in Chapter 6.3 of the Cochrane Handbook [6]. We considered trials to be CAM-related if they described interventions that are outside the practices and theories of disease and healing that are intrinsic to the conventional Western medical model [7]. To retrieve citations of CAM trials from MEDLINE and CENTRAL, we relied upon the CAM on PubMed search strategy, which was jointly developed by the US National Library of Medicine and the US National Institutes of Health, National Center for Complementary and Alternative Medicine, and introduced in PubMed in 2001 [8, 9]. In cases where some uses of the intervention are accepted within conventional Western medicine and others are not (e.g., vitamin supplementation), the CAM on PubMed search strategy generally does not distinguish between conventional and unconventional uses of an intervention. Therefore, for classifying interventions as conventional or CAM, we followed the same operational criteria we had previously developed for classifying Cochrane systematic reviews as conventional or CAM [10]. Some of the major decisions about the scope of CAM were as follows: we excluded vitamins and other supplements that are administered parenterally in hospital settings, we excluded dietary supplementation for treatment or prevention of medically diagnosed deficiency states (e.g., iron supplementation for preventing or treating iron deficiency), and we excluded vitamin supplements for preventing or treating disease in countries where vitamin deficiency is widespread (e.g., vitamin A for treating measles in children in Niger). We included vitamins for other conditions, even vitamins that are accepted for the prevention or treatment of specific disorders (e.g., folic acid for preventing neural tube defects). In general, we decided that we should be over-inclusive rather than under-inclusive with vitamin therapies, aside from the three major exclusions detailed above, and therefore some of the vitamin trials in the database would not be accepted as CAM by most people. Finally, we excluded exercise interventions with the exception of mind-body exercise (e.g., yoga), and we excluded conventional psychotherapies. A full description of the CAM Field operational definition of CAM has been published previously (see Additional file 2) [10].

We began the expansion of the CAM Field specialized register of trials by building upon the reference management database of nearly 5,000 CAM controlled trial citations developed during the 1990s and early 2000s by Vickers and colleagues [5]. In 2006, we began regular searches of MEDLINE in PubMed using the CAM on PubMed search strategy. In 2008, an information specialist translated the CAM on PubMed search strategy into a format for use in CENTRAL, and we replaced searches of PubMed with regular searches of CENTRAL. The rationale for replacing searches of PubMed with searches of CENTRAL is that CENTRAL includes not only controlled trial citations from MEDLINE, but also controlled trial citations from multiple other sources. These other citations, which may be from other databases or from difficult to locate sources such as trial proceedings, are identified by Cochrane contributors around the world and contributed to CENTRAL. All Cochrane groups then search CENTRAL in order to identify relevant citations that others have contributed. We began by searching CENTRAL from inception and then searched newly added citations in each subsequent issue of CENTRAL.

An important subset of CAM is traditional medicine, defined by the World Health Organization as “the sum total of knowledge, skills and practices based on the theories, beliefs and experiences indigenous to different cultures that are used to maintain health, as well as to prevent, diagnose, improve or treat physical and mental illnesses. Traditional medicine that has been adopted by other populations (outside its indigenous culture) is often termed alternative or complementary medicine” [11]. Different countries often have their own forms of traditional medicine (e.g., traditional Chinese medicine, traditional Korean medicine). Because we did not have access to the traditional medicine trial reports that are published in regional or national databases and journals, in 2008 we began contacting Cochrane colleagues and contributors to solicit the contributions of citations of traditional medicine trials published in their regions. These efforts are described below. Searches of PubMed, and then of CENTRAL, were thus complemented with searches of bibliographic databases and journals conducted by several international groups who contributed citations of trial reports to the CAM Field for inclusion in the CAM Field register. Citations provided by contributing organizations were not restricted by publication year. As described below, two of these contributing groups also provided PDFs of the full text publications for all identified citations. Collaborators are listed below in order of numbers of citations submitted to the CAM Field for the specialized register.

Beginning in 2008, staff at the Center for Evidence-Based Medicine of the Beijing University of Chinese Medicine, under the direction of Jianping Liu, searched both electronic databases and Chinese journals to identify citations of controlled trials of TCM interventions. The journal titles, article titles, and abstracts (if available) of all identified citations were translated into English, entered into a reference management database with added topic keywords, and submitted to the CAM Field for inclusion in the specialized register. The full text report was also submitted to the CAM Field for each citation in a PDF format.

In 2008, information specialists under the direction of David Moher of the Ottawa Hospital Research Institute undertook a project to search several specialized databases for difficult to locate controlled trials of CAM interventions, and these searches were replicated in 2010 [12]. All identified citations were imported into a reference management database, information about the source database and the type of CAM intervention was included for each citation, and the database was submitted to the CAM Field for inclusion in the CAM Field specialized register.

In 2010, researchers at the Korea Institute of Oriental Medicine, under the direction of Myeong Soo Lee, searched both electronic databases and journals to identify citations of controlled trials of traditional medicine interventions conducted in Korea and primarily published in non-MEDLINE journals. Initial searches were focused on identifying trials of acupuncture [13] and ginseng, and were then expanded to include all other CAM interventions. The journal titles, article titles, and abstracts (if available) of all identified citations were translated into English and entered into a reference management database with topic keywords, and the citations were submitted to the CAM Field. For each citation, the full text publication was also submitted to the CAM Field in a PDF format.

Kampo is the Japanese adaptation of traditional Chinese medicine. While Kampo uses most of the interventions of Chinese medicine, including acupuncture and moxibustion, its primary focus is on the study and evaluation of traditional herbal medicines. In 2001, the Japan Society for Oriental Medicine undertook a project to collect controlled trial evidence on Kampo interventions through searches of both electronic databases and journals [14]. As randomized controlled trials of Kampo interventions are identified, structured abstracts are prepared for each trial, and the citations and structured abstracts are published online in English. In March 2011, one of the leaders of this initiative, Kiichiro Tsutani of the University of Tokyo, provided permission for the CAM Field staff to incorporate the citations associated with these Kampo trials into the CAM Field specialized register together with links to the online structured abstracts.

The South Asian Database of Controlled Clinical Trials (SADCCT) is an online database of citations of controlled trials that have been conducted in countries for which the South Asian Cochrane Network & Centre is the reference Cochrane Centre, including Afghanistan, Bangladesh, Bhutan, India, the Maldives, Nepal, Pakistan, and Sri Lanka. The SADCCT was developed by searching South Asian journals and conference proceedings for all controlled trials [15]. In 2011, staff at the South Asian Cochrane Network & Centre, under the direction of Prathap Tharyan, identified and forwarded citations of South Asian trials of Ayurveda, Unani, Siddha, and other CAM interventions included in the SADCCT, and in 2012 staff at the CAM Field identified additional trial citations from the online SADCCT.

The African Trials Register is a database of citations of controlled trials conducted in Africa. It has been developed at the South African Cochrane Centre by the Cochrane Centre staff searching African journals and electronic bibliographic databases [16, 17]. In 2011, citations of CAM-related trials included in the African Trials Register were identified by Elizabeth Pienaar, information specialist at the South African Cochrane Centre, and a database of citations was forwarded to the CAM Field.

In August 2011, we suspended additions of new search results to the CAM Field specialized register and began an intensive program of cleaning and updating the register in preparation for analysis, focusing on detection and removal of any non-CAM or non-controlled trial citations, deduplication of register citations, identification of whether each register citation was present or absent in MEDLINE, and standardization of journal names (see Additional file 1 for details of procedures used). Because EMBASE is a second major database from which controlled trials are automatically downloaded to CENTRAL, we also wished to characterize the EMBASE coverage of CAM Field register citations. However, register citations do not contain EMBASE identifiers, and we did not have the resources to comprehensively check all citations for presence in EMBASE. We therefore estimated the proportion of register citations present in EMBASE by taking a random sample of 200 register records and searching EMBASE for each citation in the sample. We used the same random sample to estimate the overlap in coverage between MEDLINE and EMBASE, and the proportion of register citations not present in either MEDLINE or EMBASE.

We sorted journal titles by frequency and the 100 journals associated with the greatest number of trial citations were classified as either CAM or conventional in focus, using the classification method described in the protocol (see Additional file 1). For each of the 25 CAM and 25 conventional journals with the greatest number of trial citations, we determined the language of publication and whether the journal was indexed in MEDLINE. In addition to characterizing the journals with the greatest number of citations, we also examined the distribution of citations across all journals in the register to determine to what extent citations are scattered across journals overall.

To characterize register citations by CAM intervention, we chose 21 different types of CAM interventions within five broad categories. We based the 5 broad categories and the 21 intervention types within categories upon the CAM Field topics list for Cochrane reviews of CAM interventions [10] as well as other classifications of CAM interventions (e.g., the classifications of CAM interventions used in the 2007 NHIS survey of use of CAM in the United States [18]). We then developed subject searches for each of the 21 types of CAM interventions by parsing the 2006 translation of the CAM on PubMed search strategy into the CAM intervention topic areas. We also consulted additional sources to identify any supplementary terms and to help understand and delineate between the CAM intervention topic areas. Searches were run and tested in MEDLINE, using each relevant term. An information specialist developed the MEDLINE version of each search strategy, which was then peer reviewed by an independent information specialist using the PRESS standard [19]. Searches were then adapted so that they could be used to search the reference management database containing the register. MEDLINE Medical Subject Heading (MeSH) terms and free text terms were sought in all database fields. The search strategies for identifying each of the 21 different types of CAM interventions are included as an additional file so that these strategies will be publicly available to CAM systematic reviewers developing searches for any of these classes of therapies (see Additional file 3).

To characterize register citations by medical conditions, we used the 25 categories of medical conditions listed in the browse list on the home page of The Cochrane Library[20]. We developed subject searches for each medical condition category by consulting Cochrane reviews grouped under each of these 25 categories to ascertain relevant search terms and concepts, and additional sources to identify any supplementary terms. Searches were run and tested in MEDLINE in the same manner as described above for the CAM intervention searches, although these searches on medical conditions were not peer reviewed. Search strategies were then adapted so that they could be used to search the CAM register and augmented with additional free text synonyms for relevant medical conditions. Both MeSH and free text terms were sought in all reference management database fields.