Missing trial results: analysis of the current publication rate of studies in pediatric dialysis from 2003 to 2020

Chronic kidney disease in childhood is associated with a significant morbidity and mortality [1]. Current data from the United States Renal Data System (USRDS) registry show that, depending on patient age at reaching chronic kidney disease stage 5, more than 60 to 90% of all affected children do not receive a preemptive kidney transplantation and require chronic dialysis treatment [2]. Yet, even on dialysis, problems related to kidney failure persist and complications related to the chosen dialysis mode itself reduce long-term survival of the affected children [3‐7].

Improvement of long-term outcome of children on dialysis depends largely on evidence derived from pediatric dialysis-specific clinical studies with a focus on patient management, dialysis technique and efficiency. However, currently by far not all completed studies in pediatric medicine get published. Recent literature shows that the publication rate in pediatric clinical research ranges between only 60 and 70% [8‐11]. Such an incomplete publication rate, if caused by selective non-publication [12], might give rise to a publication bias. This is a common concern in evidence-based medicine, because it might negatively impact clinical decision-making and thereby negatively affect patient outcome [13]. In addition to the AllTrials Campaign and the International Committee of Medical Journal Editors (ICMJE)-statement from 2004, which both require that all clinical trials shall be registered, the FDA Amendments Act (FDAAA) stated for the first time in 2007 that all clinical trials should not only be registered but also have available results published within 12 months after primary completion [14‐16]. This act has currently been enforced as US federal law in January 18, 2017 [17]. Despite these measures, the problem of non-publication and untimely publication of medical studies still remains, as has been shown recently [18].

We hypothesize that the specialized field of pediatric dialysis might also be affected by a publication bias. However, research on this specific question has not been performed yet. Thus, as a first measure, we designed this study to investigate the proportion of completed studies with unavailable results in the field of pediatric dialysis. Furthermore, we investigated the respective duration from primary study completion until publication and analyzed factors with a possible impact on time to publication or on the publication rate.

This is an observational study on completed clinical studies in pediatric dialysis.

In accordance with the requirements stated in the FDAAA, a study in our analysis was defined as published if we found a peer-reviewed publication primarily via Pubmed or if we found other sources of available study results. In the latter case, these were study results posted on ClinicalTrials.gov in most cases. The results of one study were directly posted by the industrial sponsor for public availability in a product description sheet. If we did not find available results of a study by the means described above, we directly contacted principal investigators or sponsors if named by ClinicalTrials.gov regarding data requests; hereby, we did not receive any answer. The investigators of five studies could not be contacted because of missing contact information. The closure of database was the end of our search for available study results on November 20, 2021. We identified the latest impact factors of the respective journals of peer-reviewed publications by accessing the respective journal homepages. We analyzed the time from primary completion until the earliest date of availability of study results. In this time-to-publication analysis (in accordance with the FDA regulations), we considered the date of first posting of results on ClinicalTrials.gov, even if there was a possible peer-reviewed publication later. This was the case regarding four studies. The FDAAA in principle only applies to so called “applicable clinical trials” [20], which we could not clearly identify as such based on the information given on ClinicalTrials.gov. However, as we felt that a timely publication of data should be the general goal of all clinical trials involving human subjects, we evaluated the time to publication referring to the timeline of 12 months as mandated by the FDA for all studies in our analysis. We specifically report the proportion of peer-reviewed publications, as we considered especially those to be scientifically valid. In addition to the proportion of currently available study results, we analyzed the percentage of publications within 12 months and the rate of publications within 24 months (“2-year publication rate”) and 60 months (“5-year publication rate”) after primary study completion.

The main finding of this analysis on the current publication rate of completed studies in pediatric dialysis is that half of the studies registered on ClinicalTrials.gov with a primary completion date between the years 2003 and 2020 currently remain unpublished. We consider this a significant problem. Children on maintenance dialysis represent a vulnerable patient group with high disease burden and comorbidity as well as a limited quality of life. A resulting publication bias arising from presumable non-submission of statistically insignificant or negative study results in the field of pediatric dialysis might negatively impact clinical decision-making regarding the treatment of these vulnerable children [21‐23].

Despite regulatory measures taken against non-submission of study results, for example the FDAAA from 2007 (which became final US law in 2017, “Final Rule”), or the ICMJE statement from 2004 and the AllTrials campaign, the reported publication rate of clinical studies in general is currently neither complete nor are the results published in a timely fashion. This was shown for different pediatric subspecialties and in adult medicine by several, partly high-ranking, publications [9, 10, 17, 18, 24, 25]. Our results show that an incomplete publication rate is also found in the specialized field of pediatric dialysis. Whereas recent pediatric analyses had reported the lowest percentage of publications in pediatric liver transplantation of only 58% until the respective close of database, we could show that the publication rate in pediatric dialysis is currently even lower [10].

The FDAAA does not only require the publication of all study results but also requires these results to be published within 12 months after primary study completion [18]. In our analysis on pediatric dialysis studies, we showed that respective studies were published faster since the first establishment of the FDAAA in 2007. However, only a small fraction of these studies had results available within 12 months after their primary completion and the median time to publication for studies completed after 2007 was still nearly 2 years. Similar findings were shown in adult and pediatric medicine recently. These also demonstrate an unacceptably long time to publication, but regulatory measures like the FDAAA were shown to inflict improvement [9, 10, 18]. Yet, even if the time to publication of pediatric dialysis studies improved after the FDAAA establishment, we could show no significant improvement regarding the overall publication rate in pediatric dialysis studies over time. Studies completed in the first 6 years after the FDAAA had a similar 5-year publication rate as compared to studies completed thereafter.

The reasons for non-submission or late submission of study results in medicine are not always known; however, some potentially influencing factors like the source of funding have been shown to be associated with the publication rate [18]. We also examined potential influencers on the publication rate and the time to publication in our study cohort. We found that pediatric dialysis studies did not seem to be preferably funded by industry. Half of all identified studies were funded by academia. Other researchers report higher rates of co-funding by industry in other areas of medicine [18]. However, pediatric dialysis studies co-funded by industry had a higher publication rate within 2 years, but not within 5 years, after completion than studies without an industrial sponsor. Other than academia, industrial sponsors usually can provide more resources to promote faster processing and analysis of study results which might explain these findings. Anderson et al. recently reported similar results in an analysis on the publication rate in adult medicine [18, 26]. One-fourth of all investigated studies had a randomized controlled design. However, this was not associated with a higher or lower publication rate within 2 or 5 years after completion. So, in the field of pediatric dialysis, the complexity of the studies does not necessarily seem to influence the publication rate. However, we observed that two-thirds of studies with currently available results and more than 80% of studies with peer-reviewed publications had an interventional design. This result indicates that among the currently published studies in pediatric dialysis, those studies with better evidence-grading clearly predominate, which is reassuring. Further, purely pediatric studies, which are likely more relevant and applicable to the field than studies enrolling both adult and pediatric patients, were also published more often (75%).

We further identified two major research topics in pediatric dialysis studies, (i) medication during dialysis and/or control of secondary complications of kidney failure and (ii) dialysis technique and efficiency. Nearly half of all studies on pediatric dialysis focused on medication and control of secondary complications, indicating a persistent need for further improvement in therapeutic chronic kidney disease management. We further found that studies which only focused on children were published faster and had a higher 5-year publication rate, compared to those studies which enrolled both children and adults. We could also show that studies which only focused on children had a significantly smaller cumulative enrollment, which might explain the faster publication rate. Lower patient numbers may limit the choice of statistical methods that can be reasonably performed, thereby require fewer resources and thus lead to a faster overall data analysis.

Our study has several limitations. First, we did not consult databases for trial registration other than ClinicalTrials.gov. ClinicalTrials.gov is currently the largest, most widely used and appreciated database for the registration of clinical studies, and it assesses study details completely as required by the FDA [20, 24]. In principle, the reference points for publication of results and time to publication as mandated by the FDAAA can only be applied to trials that are registered on ClinicalTrials.gov. Second, the requirements of the FDAAA in principle apply only to so-called “Applicable Trials” under the FDAAA [20]. ClinicalTrials.gov did not provide information about which studies were applicable and which were not. However, we believe that all clinical studies should be published in a timely manner after completion. Therefore, we measured the time to publication against the 12-month timeline as mandated by the FDAAA for all studies in our analysis. Third, we only included studies which were completed until November 20, 2020 and ended our search for publications on November 20, 2021 in order to consider all available publications within 12 months from primary study completion. However, most studies in our analysis had been published beyond the 12-month timeline mandated in the FDAAA. Thus, our analysis of the current percentage of studies with available results and of time to publication might miss results of more recently completed studies, which might still become available after database closure (Fig. 2a/b, Fig. 3b and Table 1). We tried to overcome this potential source of bias by analyzing the 5-year-publication rates based on the Kaplan–Meier method (Supplementary Fig. 1 and Fig. 5). Fourth, an analysis performed on studies with a primary completion date within 5 years and within 2 years until the closure of database on the one hand confirmed the trend toward more timely publication in more recent years. On the other hand, the publication rate in this analysis was comparably low (0–30%). We can only speculate whether this rate might improve within the next years after the closure of our database. Fifth, we had no information about possible legally acceptable exceptions, as stated in the FDAAA, regarding a delay of study results publication for studies sponsored by industry [18]. Sixth, pediatric dialysis is a highly specific subspecialty of pediatric nephrology and study results from adult dialysis medicine must not be extrapolated uncritically to children. However, it would have added to the information content of this study if the publication rate in pediatric dialysis had been compared with that from adult medicine. We found 915 completed adult dialysis studies on ClinicalTrials.gov at the time of database closure; however, an analysis of the publication rate of these studies was beyond our resources and the scope of this study.

Our analysis of the current publication rate in completed studies on pediatric dialysis revealed a low overall number of publications. Only half of all studies had currently available study results, and there was no significant improvement of the publication rate over time. Besides that, the time from primary completion to publication did not meet FDAAA requirement levels, even if an improvement with respect to faster publications became evident in recent years. It is likely that a publication bias results from the lack of published results, which could negatively impact clinical decision-making in pediatric dialysis. We were able to identify some relevant factors influencing the publication rate in pediatric dialysis, yet most of the assumable reasons for non-publication or late publication in the field remain purely speculative and thus unknown. Furthermore, we consider the high number of patients in currently unpublished pediatric dialysis studies problematic from an ethical point of view. The low publication rates entail a certain risk that accomplished but unpublished studies in the long run be repeated.