Randomized controlled trials in pediatric complementary and alternative medicine: Where can they be found?

Complementary and alternative medicine (CAM) is defined as a broad domain of healing resources that encompasses all health systems, modalities and practices and their accompanying theories and beliefs, other than those intrinsic to the politically dominant health system of a particular society or culture in a given historical period [1].

The safety and effectiveness of CAM interventions are of great relevance to pediatric health care providers. Two recent surveys indicate a use of pediatrics CAM in the United States, finding a 12% and 21% overall use, respectively [2, 3]. Factors such as ongoing medical problems and parents' use of CAM are associated with pediatric use [3, 4]. Ideally, evidence-based practitioners can approach children and their guardians regarding their use of CAM and on a case-by-case basis to help identify relevant evidence regarding therapies that patients use or wish to consider.

Such discussions are likely to be more rewarding when practitioners and families can advocate for the use of CAM therapies for which there is stronger evidence, such as from the results of randomized trials of effective herbal therapies [5]. Similarly, interventions for which there is little of no evidence might be best avoided [6].

We undertook a program of research to discover whether such evidence, in the form of published systematic reviews and randomized controlled trials involving children, existed [7, 8]. Based on the large number of randomized controlled trials we did find, we identified a need for a guide for evidence-based practitioners who wish to determine whether sound evidence exists.

The objective of the research presented here was to identify where health care practitioners can locate reported RCTs of pediatric CAM; specifically, in which journals these reports are published and through which databases those reports may be identified.

A total of 908 reports of RCTs met the criteria for inclusion in the study.

We identified over 900 reports of RCTs using a CAM intervention in children. We found a steady growth in the literature. We found fewer articles in most recent years, likely due to a lag in indexing. There was an apparent decline in the frequency of reporting from 1998 to 2001. We expected that the indexing of reports of RCTs published during this period would not have been completed by the time we conducted our research in 2001, but post hoc examination of the year by year number of items retrieved from Medline by search strategy 1 shows that some articles in this area can take more than one year to be indexed in bibliographic databases. Those wishing to obtain all journal articles describing randomized controlled trials of pediatric complementary and alternative medicine may wish to consider hand-searching the most recent years of key journals unless it can be determined that recent issues have been indexed by the bibliographic databases.

Our results indicate that most reports (93%) of pediatric CAM RCTs are published in English. This finding has been reported elsewhere [14], although not specifically for pediatrics. Although we had to exclude relatively few trials (n = 41) on the basis of language, and the databases we used provide content for journals published in many languages besides English, it is possible that we missed important clusters of non-English reports. In particular, Tang et al. [15] describe a large number of RCTs of traditional Chinese medicine conducted in China, (approximately 7500 published prior to 1997) as being for the most part inaccessible to Western doctors. However, the journal cited by Tang as having published ten times as many RCTs as any other they examined (Zhongguo Zhong Xi Yi Jie He Za/ Chinese Journal of Integrated Traditional and Western Medicine) is indexed in Medline with entries dating back to 1992. Of 1568 articles from that journal indexed, 284 are tagged as randomized controlled trials in Medline. Of these, only 87 relate to childhood age groups, and all of these have abstracts, thus would be have been picked up by our search and eligible for inclusion in our study. Still, Tang's review underscores the risks of over-reliance on database searching to identify all trials in a given area. For those wishing to search exhaustively, there is potentially great value to involving subject experts who can identify the locations of specialty literatures, such as the literature reporting trials of traditional Chinese medicine described by Tang et al.

The 908 reports were identified in more than 300 journals, yet almost a quarter (24%) of them were found in just four journals (American Journal of Clinical Nutrition, Pediatrics, Journal of Pediatrics, and The Lancet). By expanding the core journal list to 28 journals, it is possible to account for slightly more than half of all reports of pediatric CAM RCTs we found. These results suggest that most reports of randomized controlled trials of CAM are 'mainstream' medical journals. This finding is similar to that reported by Pittler and colleagues [16] who investigated the relationship between the statistical directions of CAM RCT results and where they were likely to be published. They found that mainstream medical journals with an impact factor greater or equal to 1 published an equal number of CAM trials with positive and negative results, while lower impact factor journals published more CAM trials with positive findings.

In terms of database coverage, Medline provided the best coverage of the Zone 1 and 2 journals, indexing all 28. Embase and CAB Health indexed 85.7% and 92.3% respectively. CINAHL and AMED indexed relatively few of the journals in Zones 1 and 2, (25% and 14% respectively) but they did provide unique material not available from the other databases. Together, these five databases gave complete coverage of all journals and reports of RCTs identified in this study. Two other databases, CCTR and IBIDS, provided relatively good coverage of these first two zones (65.3% and 57.1%), but because of overlapping coverage with other databases, provided no unique coverage.

In terms of database selection, a search of Medline alone could potentially identify 97.7% of these reports and a search combining Medline, Embase and CAB Health could achieve 99.4% retrieval, assuming a comprehensive search strategy is used. Given the recent development of the CCTR we were impressed with its coverage although its unique contribution is only marginal in terms of the cumulative increase of identified reports of RCTs resulting from searching this source. In an effort to further enhance the CCTR's coverage we have forwarded the database from this study to the Cochrane Collaboration's CAM field so that the information can be added to the CCTR.

Searching Medline, along with any two of Embase, CCTR, or CAB Health, would appear to constitute a reasonable approach for those wishing to search this topic comprehensively. More exhaustive searching, of the type undertaken for a systematic review, would ideally be supplemented by CINAHL and AMED, as well as selective hand searching and other traditional bibliographic techniques [17].

Those wanting to keep abreast of the latest pediatric CAM reports of RCTs by scanning journals as they are published, may be able to maximize their time by focusing their attention on four journals. Information specialists are likely to be able to track most new evidence by searching three databases.

Shortly after we completed the retrieval work for this project, the National Library of Medicine introduced a complementary medicine subset to its Medline database [18]. Searches can be limited to this subset through PubMed (available at http://​www.​pubmed.​org, and this feature is likely to become available through database vendors. This may prove to be a useful tool for clinicians wishing quick access to some of the best literature in this area. PubMed includes Medline, is available at no cost to those with Internet access and can be searched without complex syntax. As a case study, we entered the word random* in the search box, applied the limits of age "All Child: 0–18 years" and subset "Complementary Medicine" and publication type of "Randomized Controlled Trial" retrieved 1858 citations. A clinician replacing the search string random* with a subject term massage would find 56 citations as a starting point, the term chiropract* with the same limits would yield 29, st johns wort or st john's wort would yield 8 reports and herbal would yield 140 citations.

A limitation of this study is that our figures on the cumulative percent of trials accounted for (Table 4) assume that all trials published in that journal were indexed in the database. The actual indexing rate may be less than 100%, and the indexing status of each report of an RCT in each database was not verified in this study. In addition, researchers should be aware that indexing practices and search capabilities available with particular databases vary, thus there is some advantage to building redundancy in coverage into a search strategy that has high recall as an objective.

It should be noted that the Cochrane Controlled Trials Registry is a special case among the databases we searched. As its name suggests, it is a database of controlled trials, including randomized controlled trials, and is constructed from trials found in other databases and identified through hand searching of journals [19, 20]. At the time our searches were conducted, it was searchable only through its own proprietary interfaces. It has subsequently become available through at least one database aggregator, Ovid. Had such an interface been available at the time of our searching, we would have searched it using search strategy 1. This could potentially have increased its unique contribution beyond what is seen in Table 4. In a recent study in which known item searching techniques were used to determine if reports of randomized controlled were indexed in various databases, the CCTR performed extremely well, and was identified by the authors as a valuable but underused source for meta-analysts [21].

We used a strict definition of CAM, following the Registry Guidelines adopted by the Cochrane Complementary Medicine Field Registry of Randomized Controlled Trials, thereby excluding some psychological interventions that might be considered CAM by some (such as cognitive therapy) while including others (such as biofeedback). Our view is that some psychological interventions, such as cognitive behavior therapy, might well be considered a 'standard of care' in many Canadian and American settings.

We did not specifically follow a mega-dose definition for classifying nutritional therapies as CAM or conventional medicine. This may have overstated the contribution of the American Journal of Clinical Nutrition. We attempted to follow the Registry Guidelines for the Cochrane Complementary Medicine Field Registry of Randomized Controlled Trials but erred on the side of over-inclusion of therapies as CAM if the author's self-classification was not clear from the title, abstract and indexing information.

It is clear from this study that there is a large body of evidence from randomized controlled trials in pediatric complementary and alternative medicine. The quality of that literature is also relevant [22‐24], although a recent paper suggests that quality measures may be less important than previously reported in obtaining unbiased estimates of intervention effectiveness in meta-analysis [25]. Schulz and colleagues reviewed 250 reports of RCTs and found that those with inadequate allocation concealment, compared to reports in which this information was adequately reported, exaggerated the estimates of an intervention's effectiveness by 30%, on average. Recently, Moss and colleagues [26] reported that in a review of 134 pediatric surgical RCTs, less than half of the trials (46%) reported any details of the method of randomization used. Tang et al. notes that although improving, methodological problems remain in the large body of trials of traditional Chinese medicine [15]. We previously reported on the quality characteristics of a subset of the trials found in this project [8].

RCT evidence in pediatric CAM is being published in a substantial number of journals. This study identifies the primary sources of reports of RCTs in the field of pediatric CAM-those journals found in Zone 1. These journals are a necessary core for any collection in this field. The journals in Zone 2 are also important sources of evidence, and are also recommended resources to be included in a pediatric CAM collection. Journals in Zone 3 would be a useful addition for those with larger collection budgets, and are good resources for those needing a comprehensive collection. The journals within Zone 4 contain only one report of an RCT each identified during the time period studied. While not prime candidates for a collection focusing on complementary and alternative medicine, they should be considered by systematic reviewers, who may identify potentially relevant studies from them through the relevant bibliographic databases. This study also indicates that those seeking pediatric CAM evidence require access to at least three indexing databases. These are Medline, Embase, and CAB Health.