Introduction

Since its publication in 1996, the Consolidated Standards of Reporting Trials (CONSORT) statement has been endorsed by over 400 journals worldwide, is available in seven different languages, and is viewed through its website more than 100,000 times each year. [1], [2].

Despite this high level of endorsement and dissemination, studies have suggested that inadequate reporting is still highly prevalent, even among journals that have endorsed the CONSORT statement calling for the evaluation of the impact of such reporting guidelines over a long period of time. [3], [4] Research also suggests that reporting quality varies according to the type of journal, with RCTs published in general journals having higher reported quality than those in specialized journals. [5] This study was designed to assess the quality of reports of RCTs evaluating interventions for chronic diseases that were published from 1995 to 2010 in high-impact journals that claim to have adopted the CONSORT statement.

The study only focuses on the quality of the reports of RCTs, not the quality of the RCTs as a whole, because the latter is likely an impossible undertaking. Quality, as beauty, anxiety, happiness, or love, is in the eye of the beholder, and means different things to different people, at different times, with different needs. [6] Therefore, this article provides data on key variables that have been empirically associated with an increased likelihood of bias in the original research effort, which could be easily reported by authors, and which could be easily identified as present or absent by readers.

Methods

Results

The search yielded 3854 potentially eligible articles. After screening their titles and abstracts, 284 reports were selected for inclusion in the analysis (Figure 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Flow Diagram of articles identified, screened, and included in analysis.

https://doi.org/10.1371/journal.pone.0084779.g001

Of those included, 165 of the trials were published in specialized journals (58.1%). Most of the studies used parallel study design (88.4%) and were conducted or coordinated in North America (48.2%) or Europe (46.5%). The majority of the reports (56.3%) received quality scores compatible with low methodological quality. Trials published in general medical journals had higher quality scores than those in specialized journals (55.5% vs. 35.2% had high quality scores, respectively, with p = 0.001), reported adequate concealment of allocation more often (50.4% vs. 35.8%, p = 0.013), and included a trial participant flow diagram more often (57.1% vs. 32.1%, p<0.001) (Table 1). There was a significantly higher proportion of reports with high quality scores over time (29% in 1995, 42% in 2000, 48.7% in 2005, 51.9% in 2010, p = 0.038) with the largest improvement occurring between 1995 and 2000. There were significant improvements in reporting of allocation concealment (32.3% in 1995, 31.9% in 2000, 42.1% in 2005, and 58.4% in 2010, p = 0.003) and presentation of a trial flow diagram (6.5% in 1995, 29.0% in 2000, 42.1% in 2005, and 84.4% in 2010, p<0.001) with the largest improvements occurring between 2005 and 2010 (Figure 2). Upon further analysis, there was a significantly higher proportion of reports published in general medical journals with high quality scores over time (21.9% in 1995, 56.0% in 2000, 67.9% in 2005, and 76.5% in 2010, p<0.001) while no significant differences in the proportion of reports published in specialized journals with high quality scores were observed over time (36.7% in 1995, 34.1% in 2000, 37.5% in 2005, and 32.6% in 2010, p = 0.961) (Table 2). Although reporting of allocation concealment in general medical journals improved over time, these differences were not significant (34.4% in 1995, 48.0% in 2000, 53.6% in 2005, and 64.7% in 2010, p = 0.100). Reporting of allocation concealment in specialized journals significantly improved over time (30.0% in 1995, 22.7% in 2000, 35.4% in 2005, 53.5% in 2010, p = 0.023). The inclusion of trial flow diagrams improved significantly over time for general medical journals (6.3% in 1995, 64.0% in 2000, 75.0% in 2005, 85.3% in 2010, p<0.001) and specialized journals (6.7% in 1995, 9.1% in 2000, 22.9% in 2005, 83.7% in 2010, p<0.001).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Quality scores, reporting of allocation concealment, and inclusion of trial flow diagram significantly improved over time.

https://doi.org/10.1371/journal.pone.0084779.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Report Characteristics by Type of Journal.

https://doi.org/10.1371/journal.pone.0084779.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Report Characteristics by Type of Journal Over Time.

https://doi.org/10.1371/journal.pone.0084779.t002

Discussion

In this review, we present the results of one of the largest efforts to date to assess the reported quality of RCTs since the publication of the CONSORT statement. Our findings suggest that reporting quality of RCTs in the highest impact journals has improved significantly over the last fifteen years. The largest improvement was seen in the 13-fold jump in inclusion of a trial participant flow diagram (from 6.5% in 1995 to 84.4% in 2010) which could be attributed to the endorsement of the CONSORT diagram by most of the high impact journals included in the study and the development of international initiatives to improve reporting quality such as EQUATOR. [9] Although significant improvements in the number of trials with description of allocation concealment and with high quality scores were also observed over time, these two variables displayed different trends. Adequate description of allocation concealment improved considerably over the fifteen years with steady improvements at each successive time point, exceeding in 2010 what has been previously reported. [10], [11] Although an increasing trend of high-quality trials was observed over time, this increase appears to be leveling off with time, as there was only a 3% increase in the proportion of high-quality reports published between 2005 and 2010.

Although this study only examined selected items from reports of RCTs on interventions for chronic diseases published in high-impact journals, the findings appear consistent with previous research, suggesting that there have been improvements in reporting quality of RCTs in the biomedical literature as a whole, [12], [13] with RCTs published in CONSORT endorsing journals having relatively higher reported quality than reports published in non-CONSORT endorsing journals. [14] However, despite this encouraging trend, significant differences remain between general and specialized journals, with ample room for improvement, even among the former. Our findings are consistent with previous research showing that over half of the articles published in high-impact general medical journals reported adequate allocation concealment, [15] or that reports published in the specialist literature have shown little or no improvement since the development of the CONSORT statement. [16]–[19] Indeed, even those specialized journals that claim to support the CONSORT statement tend to comply less often than their general medical journal counterparts [5].

As this study was limited to assessment of several selected reporting items, additional analyses of various components of reported quality would yield more insights into the shortcomings of reporting of trials.

Poor compliance with reporting standards by authors and journal editors, just as poor adherence to therapies by patients or to clinical practice guidelines by clinicians, is becoming a chronic challenge for which effective interventions are urgently needed. Reporting guidelines such as the updated CONSORT statement, [20], [21] and the inclusion of clearer and stronger instructions to authors by journals [22] may not be sufficient.

It is important to underscore, however, that even if the reports of RCTs were perfect, such reports should not be equated with the quality of the trials themselves. There is no such thing as a perfect trial. As stated by Jadad and Enkin, [6] “To be perfect, among other things, trials would be designed in a way that would balance out all possible competing interests, and help to answer clear, relevant, previously unanswered questions. They would be conducted, and reported by researchers who did not have conflicts of interest, and who follow strict ethical principles.

They would have to evaluate all possible interventions, for all possible variations of the conditions of interest, in all possible types of patients, in all settings, using all relevant outcome measures. Moreover, they would include all available patients, and make full use of strategies to eliminate bias during the administration of the interventions, the evaluation of the outcomes, and reporting of the results, thus reflecting the true effect of the interventions.

The data would be properly analyzed statistically, and would include individual patient data, and an accurate description of the patients who were included, excluded, withdrawn, and who dropped out. The reports, which would give an exact account of all the events that occurred during the design and course of the trial, would be written in clear and unambiguous language.

Unfortunately, a perfect trial can only exist in our imagination. In real life, researchers can only do the best that they can, and report it as clearly as they can.”

Conclusion

We found significant improvements in reporting quality of RCTs published in high-impact factor general and specialized journals over the last fifteen years. These improvements are likely attributed to concerted international efforts to improve reporting quality such as the CONSORT Statement. However, there is still much room for improvement, especially among specialized journals.

Supporting Information