Inclusion criteria were as follows: (1) Randomized, positive-controlled and parallel trials, irrespective of whether blinding was adopted; (2) Subjects with chronic gastritis, peptic ulcers, remnant gastritis or gastro-esophageal reflux disease who were H pylori positive. Subjects with bleeding ulcer or gastric cancer were excluded. (3) Determination of H pylori positive status was in accordance with the criteria of the 2007 3rd National Consensus Report on H pylori Infection (Consensus Report)[7]; (4) The treatment groups received TCM including single herb, formulae or Chinese medicine products; (5) The control groups received the first line therapy regimens recommended in the Consensus Report; (6) Outcome measurements included H pylori eradication rate and the incidence of adverse effects. The definition of H pylori eradication must meet the requirements of the Consensus Report.

The electronic databases of MEDLINE (1982-2008), Cochrane Controlled Trials Register (1982-2008), Wei-Pu database (1989-2008) and Wan-Fang database (1998-2008) were searched by using a combination of MESH subject headings of “H pylori”, “Medicine”, “Traditional Chinese” and “clinical trial” without language limitation. Reference lists from trials selected by electronic searching and conference compilations were hand searched.

Two independent reviewers screened all retrieved trials according to the inclusion criteria. If the two reviewers disagreed, the difference was discussed. If a consensus could not be reached, a third reviewer was consulted. The quality of all the eligible trials was scored using the Jadad criteria[8]. Allocation concealment was assessed. Central randomization, pharmacy controlled randomization or random numbers sealed in envelopes were regarded as appropriate concealment. If the method of allocation concealment was not mentioned or the subjects were allocated by the sequence produced by an open random number table, it was regarded as inappropriate concealment[9].

H pylori eradication rate and the incidence of adverse effects were used to evaluate the efficacy and safety of the regimens. The relative risk of H pylori eradication rate and the incidence of adverse effects were calculated using the original data and presented with 95% confidence interval. If there was no heterogeneity among the trials, a fixed effects model was applied in the meta-analysis. If there was heterogeneity among the trials and I² < 50%, a random effects model was used instead in the meta-analysis. If I² > 50%, the meta-analysis was aborted. Potential publication bias was analyzed by the funnel plot. All statistics were conducted using RevMan4.2 software.

Three hundred and nine papers were obtained after initial searching. 293 papers were excluded after reading titles, abstracts or texts due to the non-clinical nature of the studies, repetitious publication or failure to meet the inclusion criteria. Sixteen papers[10-25] were finally included with a total number of 1983 subjects (124/per trial). All the clinical trials were conducted in China and published in Chinese language. The characteristics of the included trials are presented in Table 1.

The Jadad scores of all the included trials were not more than two. One trial presented details of the randomization methods (random number table)[22] while the rest did not present any such detail. All the trials were single-centered and non-blinded without reporting allocation concealment and sample size calculation. Three trials[10,17,23] reported dropout cases and one of these trials[23] conducted intention-to-treatment analysis.

Due to the wide variety of TCM treatment applied in each trial and the statistical heterogeneity of the trials (χ² = 36.97, P = 0.001, I² = 59.4%), the data could not been pooled to conduct a meta-analysis. The H pylori eradication rates of the TCM treatment in each trial are presented in Table 2. The average H pylori eradication rates following TCM treatment and triple therapy were 72% (36%-90%) and 78% (44%-93%), respectively. The eradication rates following Chang Wei Qing oral liquid and Jian Pi Qing Hua formula were lower than those of CBS-based triple therapies. The eradication rates following Jian Wei Yu Yang granule and Wei Kang formula were lower than those of PPI-based triple therapies. There were no differences in the eradication rates between the remaining TCM treatments and triple therapy. Worst-case scenario was performed in two trials[10,17] which reported dropout cases and did not conduct intention-to-treat analysis. There was no significant difference between the new result and the original result.

Eleven trials[10,11,14,16-20,23-25] reported adverse effects without presenting the methods of acquiring the information on adverse effects. Eight of 11 reported that TCM treatment had adverse effects with an average incidence of 2% (21/903), while 10 of 11 reported that triple therapy resulted in adverse effects with an average incidence of 29% (204/697). The adverse effects associated with TCM included dizziness, loss of taste, nausea/vomiting, poor appetite, belching and loose stool/diarrhea. There were no abnormalities in routine blood tests, liver or kidney function tests in the TCM groups. Twenty subjects withdrew from the trials due to adverse effects and all were in the triple therapy groups[10,11,17,23].

Funnel plot analysis was conducted using the eradication rate as the index. The figure was obviously asymmetrical (Figure 1).

All 16 trials were of poor quality and did not meet the requirements of the CONSORT statement[26]. Although these trials claimed to adopt randomization, only one trial presented details of the randomization method and the remaining trials did not give any details of the randomization method. None of the trials described the methods of allocation concealment. Thus, whether the randomization was truly or effectively conducted in these trials is doubtful. Inappropriate randomization or allocation concealment can lead to selection bias. All trials were single-centered and non-blinded, and the treatment courses of the study groups were longer than those of the control groups in some trials, which may have resulted in performance bias. Dropout cases were reported in only three of 16 trials and were not mentioned in the other trials. It is not clear whether these trials had any dropout cases or whether they were simply not reported. Having dropout cases without reporting can result in attrition bias. Selection bias, performance bias and attrition bias may overestimate the efficacy of treatment groups. Additionally, none of the trials showed the calculation of sample size, thus, whether the sample size met the statistical requirement or the reliability of the statistical results cannot be judged. Therefore, confidence in the results obtained from these poor quality trials is questionable.

The asymmetry of the funnel plot may have been caused by the poor quality of the trials and publication bias. All 16 trials were conducted in China and published in Chinese language. Fourteen trials were published in TCM journals, one in a journal of integrated Chinese and Western medicine and one in a college journal. TCM journals are inclined to publish only positive results showing efficacy of TCM treatment. Therefore, when we searched the literature, most of the papers we found reported positive results, with only a few presenting negative results.

The TCM formulae used in the 16 trials were composed of various herbs which can clear heat, strengthen the spleen, regulate Qi or promote blood circulation. The composition of the formula and the dosage of each herb were different in each trial. The treatment courses also varied in each trial, from 1 wk to 2 mo. This resulted in clinical heterogeneity. Heterogeneity analysis also showed that there was substantial statistical heterogeneity. Due to the limited number of trials and the variety of composition and treatment courses, these trials could not be classified into subgroups for meta-analysis. An overall comparison between TCM and triple therapy could not be conducted.

Eight of 11 trials showed that the incidence of adverse effects following TCM treatment was lower than those following triple therapy, while three trials showed that there was no difference between them. Although non-blinded and lacking in detail on how the information on adverse effects was obtained, due to the obvious difference in the incidence of adverse effects between the two groups (2% vs 29%), we speculate that TCM treatment might be safer than triple therapy.

On the basis of the above, we cannot conclude that TCM treatment has the same efficacy as triple therapy in the treatment of H pylori infection. TCM should not be recommended as monotherapy in the treatment of H pylori infection until stronger positive evidence is obtained. However, neither can we completely ignore the efficacy of TCM in the treatment of H pylori infection because of the poor quality of the trials. We suggest that a scientific proposal should be designed and clinical trials performed strictly in accordance with the requirements of evidence-based medicine, in order to truly show and objectively assess the efficacy of TCM in H pylori treatment. Additionally, based on the tenet that medical study originates from clinical practice and serves clinical practice, studies on TCM in the treatment of H pylori infection should focus on the problems faced in western medicine, such as antibiotic resistance, the high incidence of adverse effects and high cost of the treatment, in addition to comparing the efficacy between TCM and Western medical treatment.

As antibiotic resistance to Helicobacter pylori (H pylori) has increased over the past few years, the eradication rates following triple therapies have decreased. New approaches to eradicate H pylori are under development.

Some herbs show obvious inhibitory effects on H pylori in vitro. Traditional Chinese Medicine (TCM) alone or integrated with western medicine have been used to treat H pylori infection in many clinical trials. However, the results of these trials are controversial and the efficacy of TCM in H pylori treatment is indefinite.

This is the first systematic review to assess the efficacy of TCM in the treatment of H pylori infection. It truly presents the quality of the clinical trials using TCM in H pylori treatment. The authors objectively concluded that the validity of TCM in the treatment of H pylori infection could not be determined due to the poor quality of the trials.

This study suggested that TCM should not be recommended as monotherapy to treat H pylori infection and the quality of TCM clinical trials need to be improved in the future.

A systematic review is a summary of research that uses explicit methods to perform a thorough literature search and critical appraisal of individual studies to identify valid and applicable evidence.

This is a good paper.