Introduction

ClinicalTrials.gov is an invaluable resource for researchers, clinicians and patients. Across-trial secondary analysis is made possible using the Database for Aggregate Analysis of ClinicalTrials.gov (AACT). [1] In theory, this resource allows for pooling of trial results around a specific disease or intervention, or studying trends in research over time. However, any such pooling of trial results requires not only that results are reported but also that the data types reported and their format are aligned.

The ClinicalTrials.gov registry holds over 224,000 studies, with 23,000 of them reporting summary results. [2] Made public in September 2016,[3] the final rule is a new regulation developed by the Health and Human Services (HHS) attempting to clarify the requirements for reporting of summary results in ClinicalTrials.gov registry in order to increase reporting. The National Institutes of Health (NIH) simultaneously issued a complementary final policy covering trials funded by the NIH. Under this policy, results are to be reported for all NIH-funded clinical trials, including those exempt from the Food and Drug Administration Amendments Act of 2007 (FDAAA requirements). [4]

Originally, mandatory registration of Interventional studies of drugs, biologics, or devices was covered under the FDAAA. Section 801of the FDAAA mandates that all covered studies report results within 1 year of completion. [5] At the minimum, FDAAA requires reporting of participants’ age and gender using a semi-structured format. Customized “Study-Specific Measures” may be also reported at the discretion of the trial sponsor or principal investigator.[6]

The final rule of 2016 [3] expands these reporting requirements. Reporting of either race or ethnicity is required, as is reporting of baseline measures analyzed with regard to the primary outcome measure of the clinical trial. Under the final rule, all trials completed after January 17, 2017 are subject to these mandatory reporting of results.

In this work, we evaluate the degree to which baseline measures reported for trials on ClinicalTrials.gov can be pooled for secondary analysis in their original format—first by analyzing counts and formats of mandatory baseline participant measures and second by comparing these to those reported in published journal articles (using a random sample) in literature.

Methods

Results

Clinical trials reporting baseline characteristics

There are 186,941 unique trials in the AACT version of database used for this study. Of these, 16,660 trials (8.9%) report baseline participant measures and 13,818 are HLACTs (S1 Table). There are 8,161 unique variables used in baseline measures reports and 1,206 unique “units of measurement” for baseline measures in the data dictionary. Fig 1 shows the frequency distribution for the top 100 reported unique baseline measures in the AACT database. Of note, 6,940 (85%) baseline measures are reported only once in the entire registry.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Frequency of reporting the top 100 unique baseline measures among trials with results listed on ClinicalTrials.gov.

https://doi.org/10.1371/journal.pone.0185886.g001

Table 1 lists the most common baseline measures aggregated on “dispersion” and “units of measurement”. Of note, since a trial may report a baseline measure in more than one format, the total count of reports in the Table could be higher than the number of reporting trials. We found “Number of Participants” reported 100% of the times in trials reporting results, followed by “Gender” and “Age” as most frequently reported measures (98% and 93% respectively).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Frequency of top aggregated baseline measures reported in >1 study on ClinicalTrials.gov (N = 16,660).

https://doi.org/10.1371/journal.pone.0185886.t001

Table 2 shows the most frequent baseline measures in records of trials reporting results. Here, 9 (47%) of the top 19 measures listed are formats of mandatory measures. However, many different formats here are used for reporting of Age and Gender. In fact 178 for representing Age and 49 for reporting Gender respectively (S2 Table). Furthermore, “Age” alone as the exact variable name is reported in 60 different formats (e.g. title: “Age”; units of Measurement:” weeks”; measure type: “median”; dispersion: “full range”).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. The most frequently reported baseline measures of participants in studies with results listed in ClinicalTrials.gov (N = 16,660) by units and dispersion.

https://doi.org/10.1371/journal.pone.0185886.t002

Highly likely applicable clinical trials (HLACTs)

Highly Likely Applicable Clinical Trials use 3,931 variables in baseline measures reports. The top baseline measures reported are listed in Table 3. There are 6 different formats for reporting Age, 4 for Race/ Ethnicity, and 3 for Gender. S3 Table lists the most frequently reported baseline measures for HLACTs (aggregated by “units of measurement” and “dispersion”). Interestingly, diversity in baseline reporting is found even in this highly selective set of studies. Although Age (after aggregation) is reported in majority of studies (62% of HLACTs), Number of Participants and Gender are less frequently (45%) reported overall. Only 4 baseline measures (Region of Enrollment, Age, Number of Participants, and Gender) are reported by at least 10% of HLACTs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. The most frequently reported baseline measures of participants in highly likely applicable clinical trials (HLACT)(N = 13,808) by units and dispersion.

https://doi.org/10.1371/journal.pone.0185886.t003

Discussion

In this work, we analyzed studies registered on ClinicalTrials.gov to assess reporting patterns of participant’s baseline measures. Overall, our findings suggest inconsistent reporting of these measures. As in other work[8–10] we found overall reporting rates of less than 10%. Even among HLACTs, which are trials highly likely to report results, less than half report Gender, a mandatory baseline measure. Among trials with results, Age and Gender are almost universally reported, however multiple reporting formats are used. Additional parameters are much less frequently reported. This may be owing to the use of “custom” measure definitions (e.g. “Fitzpatrick Skin Type”). The majority of these custom variables appear only once in the Clinical trials database. Indeed, we compare here a broad and heterogeneous collection of trials, which may have little in common, However, considerable variability in reporting pattern can be observed in published reports of trials around similar populations and research questions as also noted by [11–16].

Furthermore, our results show that ClinicalTrials.gov records and their corresponding journal articles differ substantially in the selection of reported measures and their format. On average, there are about twice as many measures included in journal articles than in the corresponding ClinicalTrials.gov record. Thus, we assert that ClinicalTrials.gov reports include only some of the parameters considered by researchers important enough to be published.

ClinicaTrials.gov is an invaluable resource for patients, their families and healthcare providers to navigate the complex and dynamic world of clinical trials. However, ClinicalTrials.gov targets researchers as well, providing a platform for conducting across-study analyses including meta-analyses, general [17] and domain-specific reviews. [18] To be generalizable, participant population reporting should reflect the population for investigated interventions [19], yet this is sometimes not the case. [20] as is also supported by our study. ClinicalTrials.gov can help researchers quantify the representation of populations of interest in trials. Tools to facilitate across-trial analysis have been designed.[21] Importantly, ClinicalTrials.gov is uniquely positioned to assess publication bias.[22] This is because unpublished studies are equally subject to mandatory reporting. Meta-analyses done on the ClinicalTrials.gov platform can thus potentially provide less biased results.

Yet, a pre-requisite for any across-trial analysis is that trial reports are comparable. As long as many different formats are used to report baseline measures, any attempt to pool this data from multiple trials would require manual alignment of variables. Thus, aligning the 178 different Age formats found in our work is labor intensive and error-prone. Furthermore, our results suggest that the discrepancies between format of baseline measures in published articles and ClinicalTrials.gov may hinder the use of ClinicalTirals.gov as an authoritative resource for secondary analyses of journal articles.

In this sense, the final rule is incomplete. While expanding mandatory reporting requirements, it does not mandate the use of uniformly coded, machine interpretable format in result reporting. Following the Notice of Proposed Rulemaking (NPRM) for Clinical Trials Registration and Results Submission, nearly 900 comments were received. The NPRM invited comments on the adequacy of the lists of proposed choices for Measure Type and Measure of Dispersion, however there were no specific comments on this topic. [3]

There is an inherent tension central to this matter. On one hand, secondary use of ClinicalTrials.gov requires standard formatting of trial reports. On the other, those conducting and sponsoring clinical trials want to have the freedom to collect, analyze and report their data as they see fit. These conflicting approaches are reflected in the comments made on the NPRM. [3] Authors of the final rule state that they generally agree with comments calling for maintaining flexibility in reporting results. This is in line with an objective of the final rule to improve the poor adherence rates to reporting of results on ClinicalTrials.gov. Admittedly, adopting stricter reporting requirements may have a negative effect on adherence, as it may require reporters to change the format of their results. However, reporters may arguably be more willing to invest and extra effort in this task if there is added value to it in the form of an increased capacity of ClinicalTrials.gov to support secondary use of trials.

Formulating the format of a structured results report is far from being straight forward. Considering the diversity of trials, aligning all components of their reports can also pose a real challenge. For example, when comparing trial-unique outcome definitions, such alignment may not even be possible. Yet when participant baseline measures are concerned, there can potentially be a core set defined for the most common and useful measures for machine-understandable, uniform reporting format. Experience from past shows, that when the National Library of Medicine (NLM) and the scientific community commit to the task, a standard nomenclature is generally definable. Successful terminology standardization including LOINC for lab results [23] and the Observational Medical Outcomes Partnership (OMOP) for medical records [24] are such example.

In the digital age, unlike in printed press, space is unlimited. The redundancy-tolerant digital medium can support a parallel, flexible reporting scheme. This means that researchers reporting trials results can use any format they deem right, as long as this is done in addition to-, and not instead of- the core uniform format.

Adherence to posting results of studies listed on ClinicalTrials.gov is poor, yet even among studies reporting results, inconsistent formats make automated across-trial analysis of these data impractical. The final rule aims to improve adherence to results reporting but does not solve the problem of multiple reporting formats. Our findings call for a discussion around implementing standards for unambiguous reporting of at least a core set of measures to support clinically and scientifically important secondary use of the ClinicalTrials.gov repository. Defining the set of measures to be reported and the format to be used should involve the entire research community, including funders and researchers.

Supporting information