Skip to main content
SpringerLink
Log in

Prevention of selective outcome reporting: let us start from the beginning

  • Point of View
  • Published:
European Journal of Clinical Pharmacology Aims and scope Submit manuscript

Abstract

Background

Healthcare professionals and patients could be negatively influenced in their judgments by articles and meta-analyses presenting selective outcome reporting. Clinical trials should be transparent from inception to the publication of results. To this end, trial prospective registration is an ethical and scientific requirement that have shown to be effective in preventing selective reporting of outcomes. However, even journals with a clear pre-registration policy publish trial results that were retrospectively registered.

Situation

Analyses of registration of randomized clinical trials recently published in top specialty journals and of meta-analyses with suspicion of including trials with outcome reporting bias have shown that retrospective registration is in the range from 56 to 76 %. This translates into publication of primary endpoints that differ from those included in the registry: some 30 % of trials showed discrepancies between the primary endpoint in the trial registry and the article. Furthermore, it has been shown that 8 % of all clinical trials published by 6 high-impact ICMJE-member journals was retrospectively registered after primary endpoint ascertainment could have had taken place, raising concerns that endpoints may not have been pre-specified, or were changed. With regards to meta-analyses, 34 % of Cochrane systematic reviews included one or more trials with a high suspicion of selective reporting bias for the primary outcome.

Proposal

Retrospective registration of trials may foster selective outcome reporting unless journal editors implement specific quality control processes aiming to prevent or minimize this type of bias. Prospective registration of trials—and protocol public disclosure if proven effective in future studies—prevents outcome reporting bias, a must to ensure clinicians and patients have access to reliable clinical trial results. Journal editors should enforce, rather than encourage, appropriate measures to ensure publication of trials free of outcome reporting bias.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. WMA Declaration of Helsinki—Ethical Principles for Medical Research Involving Human Subjects. http://www.wma.net/en/30publications/10policies/b3/index.html

  2. World Health Organization. International clinical trials registry platform (ICTRP). http://www.who.int/ictrp/trial_reg/en/

  3. CONSORT (2010) Transparent reporting of trials. http://www.consort-statement.org/consort-2010

  4. ICMJE. Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals. Updated December 2014. http://www.icmje.org/icmje-recommendations.pdf

  5. Emdin C, Odutayo A, Hsiao A, et al. (2015) Association of cardiovascular trial registration with positive study findings: epidemiological study of randomized trials (ESORT). JAMA Int Med 175:304–307

    Article  Google Scholar 

  6. Kaplan RM, Irvin VL (2015) Likelihood of null effects of large NHLBI clinical trials has increased over time. PLoS One 10:e0132382

    Article  PubMed  PubMed Central  Google Scholar 

  7. Reveiz L, Cortés-Jofré M, Asenjo Lobos C, et al. (2010) Influence of trial registration on reporting quality of randomized trials: study from highest ranked journals. J Clin Epidemiol 63:1216–1222

    Article  CAS  PubMed  Google Scholar 

  8. Viergever RF, Li K (2015) Trends in global clinical trial registration: an analysis of numbers of registered clinical trials in different parts of the world from 2004 to 2013. BMJ Open 5:e008932

    PubMed  PubMed Central  Google Scholar 

  9. Wager E, Williams P, Project Overcome failure to Publish nEgative fiNdings Consortium (2013) "Hardly worth the effort"? Medical journals' policies and their editors' and publishers' views on trial registration and publication bias: quantitative and qualitative study. BMJ 347:f5248

    Article  PubMed  PubMed Central  Google Scholar 

  10. Dal-Re R, Ross JS, Marušić A (2016) Compliance with prospective trial registration guidance remained low in high-impact journals and has implications for primary endpoint reporting. J Clin Epidemiol 75:100–107

  11. Killeen S, Sourallous P, Hunter IA, Hartley JE, Grady HL (2014) Registration rates, adequacy of registration, and a comparison of registered and published primary outcomes in randomized controlled trials published in surgery journals. Ann Surg 259:193–196

    Article  PubMed  Google Scholar 

  12. Mann E, Nguyen N, Fleischer S, Meyer G (2014) Compliance with trial registration in five core journals of clinical geriatrics: a survey of original publications on randomised controlled trials from 2008 to 2012. Age Ageing 43:872–876

    Article  PubMed  Google Scholar 

  13. Scott A, Rucklidge JJ, Mulder RT (2015) Is mandatory prospective trial registration working to prevent publication of unregistered trials and selective outcome reporting? An observational study of five psychiatry journals that mandate prospective clinical trial registration. PLoS One 10:e0133718

    Article  PubMed  PubMed Central  Google Scholar 

  14. De Oliveira GS Jr, Jung MJ, McCarthy RJ (2015) Discrepancies between randomized controlled trial registry entries and content of corresponding manuscripts reported in anesthesiology journals. Anesth Analg 121:1030–1033

    Article  PubMed  Google Scholar 

  15. Rayhill ML, Sharon R, Burch R, Loder E (2015) Registration status and outcome reporting of trials published in core headache medicine journals. Neurology 85:1789–1794

    Article  PubMed  Google Scholar 

  16. van Lent M, IntHout J, Out HJ (2015) Differences between information in registries and articles did not influence publication acceptance. J Clin Epidemiol 68:1059–1067

    Article  PubMed  Google Scholar 

  17. PLoS Medicine http://journals.plos.org/plosmedicine/s/submission-guidelines; PLoS ONE http://journals.plos.org/plosone/s/human-subjects-research; PLoS Neglected Tropical Diseases http://journals.plos.org/plosntds/s/submission-guidelines

  18. BioMed Central. Editorial policies. http://www.biomedcentral.com/about/editorialpolicies

  19. All Trials initiative. All trials registered; all trials reported. September 2013. http://www.alltrials.net/wp-content/uploads/2013/09/What-does-all-trials-registered-and-reported-mean.pdf

  20. Walker KF, Stevenson G, Thornton JG (2014) Discrepancies between registration and publication of randomised controlled trials: an observational study. JRSM Open 5:2042533313517688

    Article  PubMed  PubMed Central  Google Scholar 

  21. Goldacre B, Drysdale H, Dale A, et al. The COPMare project. http://compare-trials.org/

  22. Hartung DM, Zarin DA, Guise JM, McDonagh M, Paynter R, Helfand M (2014) Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications. Ann Intern Med 160:477–483

    Article  PubMed  PubMed Central  Google Scholar 

  23. Becker JE, Krumholz HM, Ben-Josef G, Ross JS (2014) Reporting of results in ClinicalTrials.gov and high-impact journals. JAMA 311:1063–1065

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Huić M, Marušić M, Marušić A (2011) Completeness and changes in registered data and reporting bias of randomized controlled trials in ICMJE journals after trial registration policy. PLoS One 6:e25258

    Article  PubMed  PubMed Central  Google Scholar 

  25. Scherer RW, Huynh L, Ervin A-M, Dickersin K (2015) Using ClinicalTrials.gov to supplement information in ophthalmology conference abstracts about trial outcomes: a comparison study. PLoS One 10:e0130619

    Article  PubMed  PubMed Central  Google Scholar 

  26. Chan AW, Tetzlaff JM, Altman DG, et al. (2013) SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med 158:200–207

    Article  PubMed  Google Scholar 

  27. Dal-Ré R, Bracken MB, Ioannidis JP (2015) Call to improve transparency of trials of non-regulated interventions. BMJ 350:h1323

    Article  PubMed  Google Scholar 

  28. Kirkham JJ, Dwan KM, Altman DG, et al. (2010) The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ 340:c365

    Article  PubMed  Google Scholar 

  29. Reid EK, Tejani AM, Huan LN, et al. (2015) Managing the incidence of selective reporting bias: a survey of Cochrane review groups. Syst Rev 4:85

    Article  PubMed  PubMed Central  Google Scholar 

  30. Viergever RF, Karam G, Reis A, Ghersi D (2014) The quality of registration of clinical trials: still a problem. PLoS One 9:e84727

    Article  PubMed  PubMed Central  Google Scholar 

  31. Thaler K, Kien C, Nussbaumer B, et al. (2015) Inadequate use and regulation of interventions against publication bias decreases their effectiveness: a systematic review. J Clin Epidemiol 68:792–802

    Article  PubMed  PubMed Central  Google Scholar 

  32. Smith V, Clarke M, Williamson P, Gargon E (2015) Survey of new 2007 and 2011 Cochrane reviews found 37% of prespecified outcomes not reported. J Clin Epidemiol 68:237–245

    Article  PubMed  Google Scholar 

  33. Jamtvedt G, Klemp M, Mørland B, Nylenna M (2015) Responsibility and accountability for well informed health-care decisions: a global challenge. Lancet 386:826–828

    Article  PubMed  Google Scholar 

  34. Hudson KL, Collins FS (2015) Sharing and reporting the results of clinical trials. JAMA 313:355–356

    Article  CAS  PubMed  Google Scholar 

  35. Hooft L, Korevaar DA, Molenaar N, Bossuyt PM, Scholten RJ (2014) Endorsement of ICMJE's clinical trial registration policy: a survey among journal editors. Neth J Med 72:349–355

    CAS  PubMed  Google Scholar 

  36. Dal-Ré R, Caplan AL (2015) Journal editors impasse with outcome reporting bias. Eur J Clin Investig 45:895–898

    Article  Google Scholar 

  37. UK NHS Health Research Authority. HRA Clinical Trial Registration Deferral Policy and Procedure. 2014. http://www.hra.nhs.uk/documents/2015/08/hra-clinical-trial-registration-deferral-policy-procedure-v-1-6-19-08-15.pdf

  38. Odutayo A, Altman DG, Hopewell S, Shakir M, Hsiao AJ, Emdin CA (2015) Reporting of a publicly accessible protocol and its association with positive study findings in cardiovascular trials (from the epidemiological study of randomized trials [ESORT]). Am J Cardiol 116:1280–1283

    Article  PubMed  Google Scholar 

  39. Moher D, Glasziou P, Chalmers I, et al. (2016) Increasing value and reducing waste in biomedical research: who's listening? Lancet 387:1573–1586

    Article  PubMed  Google Scholar 

  40. Goldacre B (2015) How to get all trials reported: audit, better data, and individual accountability. PLoS Med 12:e1001821

    Article  PubMed  PubMed Central  Google Scholar 

  41. Dal-Re R, Ioannidis JP, Bracken MB, et al. (2014) Making prospective registration of observational research a reality. Sci Transl Med 6:224cm1

    Article  PubMed  Google Scholar 

  42. André F, McShane LM, Michiels S, et al. (2011) Biomarker studies: a call for a comprehensive biomarker study registry. Nat Rev Clin Oncol 8:171–176

    Article  PubMed  Google Scholar 

  43. Jansen of Lorkeers SJ, Doevendans PA, Chamuleau SA (2014) All preclinical trials should be registered in advance in an online registry. Eur J Clin Investig 44:891–892

    Article  Google Scholar 

  44. Miguel E, Camerer C, Casey K, et al. (2014) Social science. Promoting transparency in social science research. Science 343:30–31

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Booth A, Clarke M, Dooley G, et al. (2013) PROSPERO at one year: an evaluation of its utility. Syst Rev 2:4

    Article  PubMed  PubMed Central  Google Scholar 

  46. American Economic Association. The American Economic Association’s registry for randomized clinical trials. https://www.socialscienceregistry.org/site/instructions

  47. Weber WE, Merino JG, Loder E (2015) Trial registration 10 years on. (edit). BMJ 351:h357

    Google Scholar 

  48. Academy of Medical Sciences. Reproducibility and reliability of biomedical research: improving research practice. Symposium Report. October 2015. http://www.acmedsci.ac.uk/viewFile/56314e40aac61.pdf

  49. Kien C, Nußbaumer B, Thaler KJ, et al. (2014) Barriers to and facilitators of interventions to counter publication bias: thematic analysis of scholarly articles and stakeholder interviews. BMC Health Serv Res 14:551

    Article  PubMed  PubMed Central  Google Scholar 

  50. Meerpohl JJ, Schell LK, Bassler D, et al. (2015) Evidence-informed recommendations to reduce dissemination bias in clinical research: conclusions from the OPEN (overcome failure to publish nEgative fiNdings) project based on an international consensus meeting. BMJ Open 5:e006666

    Article  PubMed  PubMed Central  Google Scholar 

  51. Chan AW, Song F, Vickers A, et al. (2014) Increasing value and reducing waste: addressing inaccessible research. Lancet 383:257–266

    Article  PubMed  PubMed Central  Google Scholar 

  52. Taichman DB, Backus J, Baethge C, et al. (2016) Sharing clinical trial data: a proposal from the International Committee of Medical Journal Editors. PLoS Med 13:e1001950

    Article  PubMed  PubMed Central  Google Scholar 

  53. Shamseer L, Roberts J (2016) Disclosure of data and statistical commands should accompany completely reported studies. J Clin Epidemiol 70:272–274

    Article  PubMed  Google Scholar 

  54. Goldacre B (2016) Make journal report clinical trials properly. Nature 530:7

    Article  CAS  PubMed  Google Scholar 

  55. Allison DBR, Brown AW, George BJ, Kaiser KA (2016) A tragedy of errors. Nature 530:27–29

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Clinical Research, BUC (Biosciences UAM+CSIC) Program, International Campus of Excellence, Universidad Autónoma de Madrid, Ciudad Uniiversitaria de Cantoblanco, Einstein 3, Madrid, E-28049, Spain

    Rafael Dal-Ré

  2. Cochrane Croatia and Department of Research in Biomedicine and Health, School of Medicine, University of Split, Split, Croatia

    Ana Marušić

Authors
  1. Rafael Dal-Ré
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ana Marušić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rafael Dal-Ré.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflicts of interest.

Funding

This work was not supported by any external grants or funds. The authors assume full responsibility for the accuracy and completeness of the ideas presented.

Additional information

Authors’ contributions

RDR conceived the idea and wrote the first draft of the manuscript. AM made substantial revisions to the manuscript for important intellectual content. Both authors critically revised and approved the final version of the manuscript and are accountable for all aspects included in it.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dal-Ré, R., Marušić, A. Prevention of selective outcome reporting: let us start from the beginning. Eur J Clin Pharmacol 72, 1283–1288 (2016). https://doi.org/10.1007/s00228-016-2112-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00228-016-2112-3

Keywords

Search

Navigation