Quality of reporting of trial abstracts needs to be improved: using the CONSORT for abstracts to assess the four leading Chinese medical journals of traditional Chinese medicine

Traditional Chinese medicine (TCM) plays an important role in maintaining the health of the Chinese population (especially before the introduction of western medicine into China) and its benefits are gradually being recognized worldwide. A number of randomized controlled trials (RCT) are conducted assessing the efficacy and safety of TCM, the majority of which are published in Chinese in Chinese medical journals with only the abstract being translated into English.

Health professionals often rely on the abstract of a journal article to decide whether to retrieve the full-text. In countries with limited resources to access the full-text, sometimes health care decisions are made based only on the information reported in the abstracts [1, 2]. In TCM this problem is exemplified as the English abstract may be the only way for non-Chinese speaking countries to know about RCTs of TCM.

It is therefore important that abstracts of RCTs provide details of the trial participants, interventions, methodology (such as details of randomization, blinding, and intention-to-treat analysis) and the importance of the trial results (including estimates of benefits and harms, and their precision). To date, little is known about the quality of reporting of RCT abstracts in TCM. This study aims to assess and compare the reporting quality of both Chinese and English abstracts of RCTs published in four leading TCM Chinese medical journals using the recent CONSORT for reporting randomised trials in journal and conference abstracts (CONSORT for abstracts) [3].

We identified 495 RCTs from hand searching the full-text and abstracts of the four leading TCM journals from 2006 to 2007, of which 345 (70%) included both the Chinese and English version of the same abstracts and all were structured abstracts as it is a formal requirement for the four journals. The remaining 150 RCTs were published as short reports with no abstract. Seven RCT reports mentioned "randomized" in the abstract but not in the full text, while 18 mentioned "randomized" in the full text but not in the abstract. Eighty-five papers mentioned "randomized" in the abstract but were judged as "quasi-randomization" after reading the full text. Table 1 provides information on the impact factor for the four journals, the number of RCTs identified and the number of reports with both the Chinese and English version of the same abstracts for each journal. Agreement between the two reviewers was good with a kappa score of greater than 0.75 for each item.

Table 2 presents the criteria used to judge whether the abstract reported the item, and the number and proportion (%) of items reported for both the Chinese and English version of the same abstracts.

Our study identified 345 of RCTs published in four leading Chinese TCM journals with both the Chinese and English version of the same abstracts between 2006 and 2007. Based on the CONSORT for Abstract checklist criteria, the quality of reporting of trial abstracts in these journals was far from satisfactory. Except for details of the trial authors, participants, the intervention for control group, and the number of participants randomized in the trial, all other items were reported in less than half of the abstract reports. Reporting of methodological quality or key features of trial design and trial results was even worse. Less than 5% of the included abstracts fully or correctly reported relevant items including details of the trial design, clearly defining the primary outcome, the method of random sequence generation and allocation concealment, specifying details on who was blinded, the number of participant analyzed in each arm of the trial, and the estimated effect size and its precision for the primary outcome.

Our study has several strengths. We hand searched four high impact TCM medical journals to ensure that all RCTs were identified. In addition, all items in modified version of the CONSORT for Abstracts checklist were carefully discussed and understood within the research team. The identification of RCTs, eligibility decision and data extraction were conducted in duplicate. However, our study has limitations. We did not take a random sample of all RCT published in TCM medical journals and therefore our findings may not reflect the whole picture of reporting quality of trial abstracts in the TCM field.

Our findings are consistent with or worse than previous studies that have assessed the quality of reporting of journals RCT abstracts and conference RCT abstracts. Previous work has shown that abstracts frequently under report key features of study design [1, 2, 5] and omit important results. In the mid-1990s, Li and Hu reported that the quality of abstracts in Chinese medical journals was poor [6, 7]. The problem is particularly apparently when trying to assess details of methodological quality. A recent study evaluating 227 RCT abstracts published in four high impact general medical journals in 2006 found that only one abstract reported details of allocation concealment, 23% reported the use of intention-to-treat analysis, 14% reported loss of follow-up, and 9% reported details of blinding [8]. A further study evaluating RCT abstracts presented at the American Society of Clinical Oncology Conference and found no abstracts reported allocation concealment, 16% reported on blinding, and only 14% reported intention-to-analysis [9].

Possible explanations for our findings of poor reporting include poor reporting of RCT full-text articles in China, when an item isn't reported in the full publication, it may also be missing from the abstract. A study evaluating the reporting quality of 142 RCT full-text articles published in five leading Chinese medical journals found that only 27% described the method to generate the randomized sequence, 4% had adequate allocation concealment, and only 17% mentioned blinding [10]; these findings are supported by others in the field of TCM [11, 12].

Lack of detailed requirements for structured abstracts in journal's "Instruction for Authors" in China is also a problem since none of the journals included in our study mentioned CONSORT for Abstracts in their Instructions to Authors. Some studies have proved that journal adoption of structured abstracts can improve the reporting of abstracts [13‐17]. Although most of the Chinese medical journals have adopted four-heading structured abstracts (objectives, methods, results and conclusion), the structured format alone is insufficient to guide authors regarding their content for reporting RCTs.

Finally, the word limitation of an abstract restricts the amount of detail an author is able to provide. Most Chinese medical journals have abstract word restrictions, usually within 150-500 Chinese words, making it difficult for authors to comprehensively report trial methods and results, especially when authors are not aware the importance of research methodology and the way to correctly present the clinical relevance of research results. The word limit for the journals we assessed is 150-250 Chinese words and 150-400 English words.

In summary, we found that the quality of reporting of trial abstracts in these four leading TCM medical journals is far from satisfactory, especially with respect to details of the trial methodology. Many studies have proved that adoption of the CONSORT Statement has demonstrated benefits in improving the reporting quality of RCT of full-text articles [18], and we hope that it will do the same with the introduction of CONSORT for Abstracts in Chinese medical journals of TCM field.