Background
An improved understanding of the biology of cancer has led to remarkable progress in therapeutic approaches. Anticancer agents developed over the last 2 decades utilize multiple mechanisms of action including conventional cytotoxic agents as well as inhibition of oncogenic signalling pathways and angiogenesis. More recently, ‘immunotherapy’ agents that rely on immunomodulatory mechanisms to target and destroy cancer cells, most notably immune checkpoint inhibitors (ICPis), have been developed.
The first ICPi approved by the United States Food and Drug Administration (FDA) was ipilimumab, a fully humanized immunoglobulin G1 monoclonal antibody that blocks cytotoxic T-lymphocyte antigen [
1]. Pembrolizumab and nivolumab were the first ICPis that target programmed cell death protein 1; they showed high response rates with favourable toxicity profiles and were approved for treating metastatic melanoma in 2014 [
2,
3]. The notable successes of these pivotal trials may have led to unrealistically high expectations among patients and clinicians, as more recent studies have shown that only a subset of patients exhibit durable responses, and existing checkpoint-blocking monotherapies seldom lead to complete remission [
4‐
6]. These findings have prompted the search for next-generation ICPis as well as evaluations of their combinations with other biologic agents [
7].
Anticancer drugs are approved by the FDA based on substantial evidence of clinical benefit from adequate and well-controlled clinical trials. Their efficacies are demonstrated by prolonging patients’ survival and improving their quality of life by preventing or ameliorating cancer-related symptoms. Sponsors of a new drug are required to submit all data to the FDA, including complete protocols, protocol revisions, and data from successful and failed trials. Once the drug is approved, the FDA produces a ‘Summary Basis of Approval’ document that contains synopses and evaluations of clinical data and statistical analyses performed by FDA medical officers during the approval process. These documents contain detailed efficacy and safety data that are relevant to drug approval but are not necessarily intended to be shared with general evidence users such as clinicians, patients, and policymakers. In this context, the peer-reviewed medical literature has a powerful and important role in disseminating information relevant to both clinicians and the public. Nevertheless, the publication rates of sponsor-submitted trial results for drugs approved by the FDA have been low, and discrepancies exist between original trial data submitted to the FDA and data found in published trials [
8‐
10]. The lack of timely and complete dissemination of clinical trial data can lead to unnecessary duplication of research and impair evidence-based clinical decision-making, thus violating ethical obligations. Delayed and incomplete dissemination can have particularly deleterious effects on cancer patients.
Thus, we performed a comprehensive examination of the publication statuses of trials submitted by the sponsors of investigating the most recent FDA-approved anticancer drugs, with a focus on ICPis. As we hypothesized that the growing enthusiasm around ICPis may lead to expediting the publication of data involving these drugs, we further evaluated the role of the drug types in the time taken to publish their associated clinical trial results.
Methods
The protocol for this meta-epidemiological investigation was registered with the University Hospital Medical Information Network (
www.umin.ac.jp/ctr/index-j.htm; registration number UMIN000030475).
Drug analysis
We used the Drugs@FDA database to identify all ICPis that were newly approved for cancer treatment by the FDA between 2011 (the year the first ICPi was approved by the FDA) and 2014 (thus assuring a follow-up of at least 3 years post-approval). All other anticancer drugs approved by the FDA between 2011 and 2014 were also identified, 10 of which were randomly selected for comparison using the Excel software (Microsoft Corp, Redmond, WA, USA). We included only new drugs against novel molecular targets and excluded those that are preventative or palliative.
Identification of clinical trials
We retrieved the FDA Summary Basis for Approval of each drug and assessed medical review documents to identify clinical trials submitted by the sponsor. The medical reviews included an overview of safety and efficacy, an outline of the data sources, integrated summaries of safety and efficacy, and (where relevant) a description of individual clinical trials. We included trials that were or were not covered by the Food and Drug Administration Amendments Act of 2007 (FDAAA) mandate for submission of results (efficacy trials: phase 2–3) [
11], because the nonpublication of any clinical trial stage has potentially deleterious impacts on patients and clinicians, represents a waste of resources, and violates ethical imperatives to share results. Ethical board review and informed consent were not required for this survey of publicly available databases and articles in which aggregated data were inherently anonymized.
Search strategy and data extraction
First, we recorded the following characteristics for each submitted trial when available in FDA documents: the drug name (generic and trade), initial approval date, approval characteristics (FDA review process and approval pathway), drug target, delivery method, dosage and evaluation schedules, indication, number and location of study sites, sponsors’ and principal investigators’ names, authors’ industry affiliations, study phase, study type (superiority, non-inferiority, or equivalence trial), number of arms, control conditions, number of study participants, primary and secondary outcomes, sample size in the primary analysis, and effect size of each primary outcome. Second, using the above information as search terms, we electronically searched PubMed, Google/Google Scholar, and their sponsors’ websites to obtain study identifiers (
ClinicalTrials.gov registry [NCT] number and/or trial unique ID) for each trial identified in the FDA review documents.
Next, we searched
ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform with the study identifier to obtain the following detailed information for each trial: dosing schedules, number and location of study centres, principal investigators’ names, authors’ industry affiliations, study phase, study type (superiority, non-inferiority, or equivalence trial), number of arms, control conditions, planned sample sizes, compared parameters, number of study participants, primary and secondary outcomes, sample size in the primary analysis, effect size of the primary outcome, statistical significance of the primary outcome (
P < 0.05 or confidence interval [CI] excluding those with ‘no difference’; or if the study was a non-inferiority evaluation, the CI including ‘no difference’ and excluding the prespecified margin described in the protocol; or if the study was an equivalence evaluation, the CI between the no difference and prespecified margin). Nonsignificant or null results were defined as
P > 0.05 or a CI including ‘no difference’, or else a CI including the prespecified margin if the study investigated non-inferiority or its equivalent. We also noted whether the trial was randomized and/or double-blinded. Missing, unclear, or important additional data were requested from sponsors or primary study authors.
Publication matching
We searched PubMed, Google/Google Scholar, and their sponsors’ websites to match each identified trial to publications in the medical literature between June and August 2018. We also searched abstracts in the proceedings of relevant periodic meetings as well as reference lists. Studies in all languages were reviewed as abstracts or full texts. Trials identified in FDA documents were matched to publications based on the following characteristics: study identifier (NCT number and/or trial ID), drug name, sample size, dosing schedules, arm number, primary and secondary outcome measures, and statistical significance or estimated effect of the primary outcome results. The publication type of each trial was recorded as follows: (1) full publication, (2) full report, (3) partial publication, (4) conference abstract, (5) none (neither published nor reported, but verified), or (6) unclear (no information found). Only original research reports in full peer-reviewed journals were considered full publications and included all the primary outcomes predefined in the protocol (#1 above) or partial publications containing incomplete descriptions of the prespecified primary outcomes (#3 above). For trials that were terminated early because of perceived effectiveness, only original research reports were considered full publications (#1 above) including all findings and results. If all the predefined primary outcomes were available in
ClinicalTrials.gov or the sponsors’ websites, the trial was considered a full report (#2 above). If multiple publications were found for the same trial, we prioritized the category with the smaller number; for example, if a trial was fully reported (#2 above) and published (#1 above), then it was categorized as a full publication (#1 above). If trials remained unmatched to a publication, we contacted the sponsors or authors to clarify their publication statuses. Four reviewers (KO, YK, YT, and YT) screened all abstracts and full-text articles independently. Disagreements were resolved by discussion; otherwise, a fifth independent reviewer (TAF) arbitrated.
Statistical analysis
We performed descriptive statistics of the included trials stratified by drug type (ICPis vs. other anticancer drugs). The primary endpoint was the rate of ‘full publication’ within 2 years after FDA approval [
9]; we also analysed the publication statuses at 0 and 3 years. Moreover, we evaluated whether study identifiers were reported to determine the articles’ discoverability; for example, once a trial’s NCT number is published as part of the original journal article, it is automatically identified and indexed by
ClinicalTrials.gov.
Next, we examined the influence of study phase and drug type on the time from FDA approval to ‘full publication’ using log-rank tests. In time-to-event analyses, trials that were not published were censored, and time 0 was defined as the date of FDA approval per the Administration’s documents. Trials published before their FDA approval date were considered published at time 0.
We further performed multivariable analysis of the association between drug type/study phase and time to publication using a multilevel Cox regression model that was adjusted for clustering within drug indications and potential confounders, including sample size and ethnicity. We classified trials as ‘smaller’ if the sample size was smaller than the median value of all the studies combined; otherwise, they were deemed ‘larger’.
We conducted a limited number of prespecified subgroup and sensitivity analyses and examined the time to publication among all as well as randomized phase 2/3 trials. The sensitivity analyses employed a multilevel ordered logistic regression model to evaluate the association between drug type and publication status according to the abovementioned categories (categories 5 and 6 were combined) at 0, 2, and 3 years with adjustment for clustering within drug indications and individual covariates. Additionally, we performed a post-hoc analysis of the “full publication” rate at 2 years post-approval of trials that supported only the drug indications for which priority review was granted by the FDA; this was to determine the impact of such priority review on the time to publication. Statistical significance was set at P < 0.05 (2-tailed test). We used STATA version 14 (Stata Corp LP, College Station, TX, USA) for our analyses.
Discussion
The median time from FDA approval to full publication of the 120 trials supporting the 3 ICPis and 10 randomly selected non-ICPi drugs was 2.3 years, and one-third of the trials remained unpublished 2 years post-approval. Although we found no association between any drug type and time to publication overall, the publication of randomized phase 2 and 3 trials for ICPis took longer than for other anticancer drug types. Interestingly, the publication rates of all trials were very similar, including for those supporting drug indications to which priority review was granted by the FDA.
A previous study found that over half of the trials supporting new drugs approved between 1998 and 2000 remained unpublished ≥5 years after approval, and that statistically significant results were more likely to be reported [
9]. Another study found that nearly half of phase 2 and 3 trials for antidepressant agents approved between 1987 and 2004 were unpublished, and possible selective reporting biases were present [
12]. Additionally, 97% of clinical trials for cardiovascular disease and diabetes drugs were published in the peer-reviewed literature after the FDAAA was implemented [
13].
The publication rate revealed in our investigation was higher than those found in 2 earlier studies performed before the FDAAA implementation [
9,
12]. The statistical significance of the results was not associated with earlier trial publication, suggesting an improvement in the dissemination and transparency of trial results related to FDA approval. However, the overall publication rate of 66% remains insufficient to satisfy the responsibilities of medical and academic enterprises. Recent research on all pharmaceutical and biopharmaceutical trials registered with
clinicaltrials.gov demonstrated that publication rates varied substantially depending on the disease area, and that oncology-related trials had the lowest publication rates [
14]. Stakeholders, including researchers and sponsors as well as journals, ethical committees, and governments, ought to invest additional effort to promote the timely and complete dissemination of clinical trial findings, especially those related to oncology.
Including all the clinical trial that supported drug approval, as required by the Declaration of Helsinki [
15], enabled us to quantify the differences in the timing of trial publication across study phases. We also clarified the differences in the discoverability and accessibility of published articles according to study phases. Although previous investigators have described the underreporting of trial registration numbers in biomedical publications related to randomized clinical trials (RCTs) [
16,
17], the current study expanded the scope of research to all clinical trials (including RCTs and non-RCTs), and found that such study identifiers were less frequently included in articles describing earlier-phase trials. This suggests that systematically searching for trials (especially earlier ones) using study identifiers is unreliable and could result in undercounting publications and in incomplete data dissemination. Authors and sponsors are encouraged to include study identifiers in all their articles regardless of the study phase or statistical significance of study outcomes.
The results of randomized phase 2 and 3 trials are usually considered ‘gold standard’ evidence of drug efficacy, and thus directly affect both drug marketing approval as well as drug sales. In our study, subgroup analyses of randomized phase 2 and 3 trials showed that the drug type (ICPi vs. non-ICPi) was associated with time to publication; the difference remained significant after adjusting for trial-level confounders. We speculate that the novel ICPi mechanism of action may have influenced each step of the trials’ publication processes, especially as various stakeholders were involved. Recently disclosed details of sponsored trial publication histories indicated that some industry sponsors require the timely submission of all trial results for publication [
18,
19]. Evaluators of the dissemination and transparency of clinical trial results should consider such publication-related policies.
Our study had several limitations. First, it was restricted to trials supporting FDA approval of anticancer drugs; therefore, our results are not generalizable. Second, because we focused on recently approved drugs, follow-up times were limited; as such, longer follow-up may yield additional publications (although they may not qualify as timely). Third, our analysis may have been statistically underpowered to detect significant relationships or differences given the limited number of trials. Fourth, it remains possible that we missed some published studies. Lastly, as is inherent in all observational studies, causal inferences cannot be made, and additional unmeasured variables may explain the differences in times to publication.
However, our study also has several strengths, such as the inclusion of all trials irrespective of study phase as well as rigorous search algorithms and thorough statistical analyses.
In conclusion, our results showed that incomplete transparency and delays in disseminating sponsor-submitted clinical trials supporting FDA drug approval are still prevalent. Further efforts and continuous monitoring are necessary to improve the timely and complete publication of clinical trial results.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.