Reporting quality of abstracts in phase III clinical trials of systemic therapy in metastatic solid malignancies

An abstract is a snapshot of the overall design and results of a clinical trial and in many circumstances may be the only portion of a manuscript read by a clinician due to time constraints and barriers to accessing full-text publications. Therefore, sufficiently detailed and transparent reporting of oncology clinical trial abstracts is critical. Several guidelines have been developed, including those from the CONSORT group, in an effort to standardize the quality of abstract reporting [1, 2, 4, 15]. However, despite the 2008 CONSORT extension statement for reporting randomized controlled trials in journal and conference abstracts, there is significant heterogeneity in reporting of individual elements as demonstrated by our analysis and that of others [16‐23].

Variability in reporting randomized controlled trials in journal abstracts may be secondary to several factors. Authors may be unaware of the CONSORT extension recommendations and journals may not require or enforce compliance to these guidelines. The CONSORT extension guidelines are not oncology specific and emphasis on certain reporting elements may be different for various disciplines. Limitations on word count for abstracts may also impact the ability of authors to comply with the CONSORT recommendations.

Notably, our unadjusted analyses demonstrated that most analyzed variables, including journal endorsement of CONSORT recommendations, placebo controlled trials, treatment type, or funding source, were not associated with completeness of abstract reporting. This suggests that suboptimal reporting is not limited to subsets of journals but rather a generalized issue. Interestingly, we reported that completeness scores have been linearly increasing over time as publications from 2011 had higher completeness scores. Our inclusion of publications spanning only a 3 year time interval (2009–2011), however, limits the interpretation of this finding.

There are several prior analyses addressing the quality of abstract reporting in randomized clinical trials but the majority of these have not evaluated oncology trials. However, during the preparation of this manuscript, Ghimire et al. published an analysis of oncology randomized controlled trials over a three-year period both prior to and after adoption of the CONSORT extension statement for abstract reporting. Substantial heterogeneity and selectivity in the quality of abstract reporting was demonstrated with only half of the CONSORT recommendations consistently being reported after the adoption of the CONSORT extension statement [24]. Our analysis extends these findings in several ways. We investigated the potential association between several study characteristics and abstract reporting completeness. For example, we demonstrated that journal endorsement of the CONSORT statement was not associated with abstract reporting completely, nor was public availability of the full-text manuscript. These findings suggest that other factors, including authors’ lack of awareness of the guidelines or journals lack of enforcement of guidelines, may potentially be contributing to the poor quality of abstract reporting. Another study done prior to adoption of the CONSORT extension statement utilized a checklist derived from the original CONSORT statement and applied to American Society of Clinical Oncology Annual Meeting abstracts and also demonstrated the generally poor quality of abstract reporting.

Prior studies have investigated the use of a more stringent application of the CONSORT guidelines during the peer review process in improving the quality of abstracts. Hopewell et al. conducted an interrupted time series analysis of the quality of abstracts describing medical randomized controlled trials both before and after the implementation of the CONSORT extension statement for abstracts in five leading medical journals. Journals were divided into three categories - those that did not endorse the CONSORT guidelines, those that endorsed the CONSORT guidelines but did not have an active policy to enforce them, and those that endorsed the CONSORT guidelines and had an active editorial process to enforce them. Interestingly, there was only a significant change in the mean number of items reported pre and post implementation of the CONSORT extension statement in the journals that had an editorial process to enforce the guidelines during the peer review process [25]. This highlights the importance of not only awareness of the guidelines by authors but the need for implementation and enforcement of these guidelines during the editorial review. Cobo et al. investigated the effect of adding an additional review based on reporting guidelines to the conventional peer review process for all original research manuscripts submitted to one journal. The overall quality of the manuscripts was improved with the addition of a second guideline-based review. Interestingly, during the editing process, authors were more likely to make manuscript revisions based on the conventional reviews rather than the guideline-based reviews. The authors postulated several potential reasons for this including a lack of awareness of these guidelines when designing a research study, making it more difficult to adhere to reporting certain elements in the publication process [26].

Accurate and transparent reporting of clinical trial information in abstracts may be particularly important in situations in which there is no access to full-text articles. Community oncologists, who provide 85 % of the cancer care in the USA, as estimated by the National Cancer Institute, are less likely to have comprehensive journal access and may be less likely to read multiple full-text articles when payment is required. The cost of purchasing these articles may be prohibitive to oncology physicians in the developing world where cost and access to information technology is potentially prohibitive, and who may rely only on the abstract content to make clinical decisions. Fortunately, we did find that open access to full-text articles was available in 80 % of our dataset. It is arguably more imperative for conference abstracts, in which a full text of a research study is not yet available, to stringently follow CONSORT guidelines as this is the only reflection of the details of a study.

There are some limitations to our study. This analysis was conducted specifically in randomized phase III metastatic solid tumor trials and further standardization of reporting may be necessary for other types of trials and disease states. The analysis included only English language studies. The importance of certain reporting elements may not be agreed upon universally by all authors or sub-specialities. Some may argue that several of the most poorly reported elements such as the setting in which the trial took place, reporting of allocation concealment, recruitment status, registration number, and funding source are important elements to include in the full text but are not necessarily informative in the abstract. Removal of these may raise completeness scores as well as allow researchers more word space to address other key trial details in the abstract. Future studies are needed to address this.

Importantly, CONSORT guidelines and extension statements represent a minimum standard in reporting. Adoption and expansion of these guidelines by the oncology community, and implementation and enforcement of these guidelines by journal editors, will potentially lead to greater adherence and improved quality of publications and improved clinical decision making.