Benefits and harms of Risperidone and Paliperidone for treatment of patients with schizophrenia or bipolar disorder: a meta-analysis involving individual participant data and clinical study reports

Schizophrenia and bipolar disorder are highly prevalent and debilitating severe mental illnesses worldwide. Patients often experience both simultaneously because they share a similar causative process of diagnoses [1].

Risperidone is a leading second-generation antipsychotic drug approved for the treatment of schizophrenia in adults and adolescents and for the short-term treatment of manic or mixed episodes of bipolar disorder [2, 3]. It is indexed on the World Health Organization’s List of Essential Medicines as one of the most effective and safe medicines [4]. However, the manufacturers ‘Johnson & Johnson’ have been involved in over 13,500 legal (lawsuit) cases because of their failure to disclose that Risperidone may cause hormonal imbalances that could lead to breast tissue development (‘gynecomastia’) and increased blood prolactin levels (‘galactorrhoea hyperprolactinaemia’) in boys and girls [5, 6]. There have also been ‘black box warnings’ of misleading marketing of the drug for off label uses in children and adolescents with schizophrenia or bipolar disorder and in elderly patients with dementia [7, 8]. Moreover, the clinical benefit of Risperidone is questionable when compared to other antipsychotics and was recently found to be only the sixth best treatment option for overall change in symptoms of schizophrenia [9, 10].

Paliperidone is another second-generation antipsychotic drug also manufactured by Johnson & Johnson and used to treat schizophrenia. As both Risperidone and Paliperidone act via the same pathways in the body, research have suggested strong links of drug-induced hormonal imbalances in both antipsychotics [11], and like Risperidone, the efficacy of Paliperidone over other antipsychotics is also questionable [10]. Other serious adverse events experienced by patients with severe mental illnesses receiving either Risperidone or Paliperidone include muscle-related ‘extrapyramidal effects’; permanent movement disorders such as ‘tardive dyskinesia’; cerebrovascular events including stroke, transient ischaemic attack, vascular malformation and venous thromboembolism [12, 13]; and neuroleptic malignant syndrome, increased risk of suicide and weight gain [14, 15]. However, these harms are rarely reported exhaustively in journal publications.

The existing evidence that support policy on both antipsychotic drugs is solely based on aggregate data meta-analyses of published randomised controlled trials (RCTs) involving mostly adults [16]. Analyses of more exhaustive forms of unpublished data including individual participant data (IPD) and clinical study reports (CSRs) are increasingly recommended for evaluating the full evidence base for the effectiveness and safety of these drugs. Such analyses can create clear hierarchies of evidence about the benefit and harms of both interventions, taking into account reporting bias, and have the potential to better inform policy decisions.

The Yale University Open Data (YODA) project was setup to promote better transparency of clinical trial results. As of 2014, YODA have been enabling scientists across the world to gain access to Johnson & Johnsons clinical trial data assets including Risperidone and Paliperidone [17‐19]. Using more innovative methodologies on a hitherto unavailable and more complete database from IPD and CSRs contained in YODA, we aim to compare in a meta-analysis the benefit and harms of Risperidone and Paliperidone with data from corresponding trial register entries and journal publications.

The study followed a registered (PROSPERO CRD42019140556 [20]) protocol and findings are reported in accordance with the PRISMA-IPD statement [21].

A total of 1288 references were retrieved. Following full-text screening of 204 studies, 57 RCTs met our inclusion criteria and IPD/CSRs were accessible for 35 (61%) studies involving 12,316 patients (Fig. 1). The characteristics of the 35 studies are provided in Additional file 3: Table S3 [31, 35, 36, 66‐96]. The remaining 22 studies were excluded due to their low quality, differences in primary outcome selection with only a single subscale being used to measure PANSS and a general poor reporting of harms meaning data could not be include in the meta-analysis.

This meta-analysis presents the most in-depth investigation to date of the efficacy and safety of Risperidone and Paliperidone in patients with schizophrenia or bipolar disorder.

Our IPD meta-analysis showed a significant reduction in the PANSS total scores for the use of both Risperidone and Paliperidone. However, the effects were more conservative when compared with the results of a recent network meta-analysis involving both oral forms of interventions compared against placebo (i.e., Risperidone: SMD = − 0.55 vs. − 0.32; Paliperidone: SMD = − 0.49 vs. − 0.31) [10]. More importantly, the actual clinical reduction in the PANSS total score was smaller than the evidence reported in three separate Cochrane reviews comparing each intervention to placebo (Risperidone − 17.81 (− 18.17, − 17.45) [27] vs. − 5.83 (− 10.79 to − 0.87); Paliperidone − 8.99 (− 11.1, − 6.9) [28] vs. − 6.01 (− 8.7 to − 3.32); Paliperidone palmitate − 8.07 (− 9.75, − 6.39) [65] vs. − 7.89 (− 12.1 to − 3.69)). Given that our estimates are based on a similar number of patients, these differences could be attributed to methodological limitations of these previous studies compared to our more in-depth and robust analysis approach. Previous meta-analyses have relied upon data from published literature of these three interventions which make them more vulnerable to reporting biases [99] and are overly reliant on treatment effects that have been estimated using less robust methods.

Our most notable finding was the inconsistencies in the reporting of harms between CSRs and journal publications. CSRs provided over double the number of AEs, and almost eight times more SAEs showing a significant increase in the relative differences favouring CSRs. The effect estimates from CSRs which involved more patients and studies, revealed an increased risk in the number of AEs and elevated risks for other harms such as ‘extrapyramidal disorder’, ‘tardive dyskinesia’ and ‘weight increased’.

Limited outcome data were available for ‘gynecomastia’ and ‘cerebrovascular events’. As these are very rare events with considerable delayed onset in younger children, it’s likely that they are not well documented in these short-term randomised trials of only 63 days follow-up [11]. Therefore, we recommend that both antipsychotics are administered with caution to children and adolescents until more solid evidence about the risks of hormonal imbalances and infertility is available from trials with long term follow-up data.

The combination of IPD and the corresponding CSR are the most comprehensive and trustworthy source of Johnson & Johnson trial evidence available to date. However, a significant limitation of this study is that we were unable to access all company data and CSRs for 22 further eligible trials, which meant that this review was not entirely systematic and may be subject to selection bias. We made several requests for CSRs for Paliperidone trials at the European Medicines Agency, but one ongoing court case [38, 39] meant that they were unable to provide access to the CSR for seven further trials. Thus, to alleviate the potential for selection bias, we inspected the 22 studies and found that 15 of the studies did not appear to report any of the relevant adverse event’s data or provide the overall number of adverse events, and the other 7 studies did not provide amendable ‘arm-level’ data for inclusion in meta-analysis. Furthermore, the baseline scores for the primary outcome PANSS were measured using only one of the subscales in 13 of the studies and therefore the total PANSS score could not be estimated; the other 9 studies did not provide amenable PANSS data meaning they could not be meta-analysed with the IPD.

Another limitation of the study is that we did not compare the effects of all other efficacy outcomes using IPD and only a subset of AEs that were considered controversial as informed by earlier meta-analyses or previous ‘black box’ warnings were assessed using the patient safety listings data. Nevertheless, we did compare the PANSS total score using all available IPD provided, and CSRs conveniently supplemented where IPD for the outcome were unavailable. Moreover, comparing the differences of the effect-sizes between two different data sources by using the ratio of means/relative risk when there is clear overlap of studies and a lack of independence could well lead to unwieldly results. However, this was the only way to compare significance between the pooled effects of the data sources with any real confidence and precision.

The benefits of all three antipsychotics were more conservative than reported in the earlier studies restricted to only publicly available sources (journal publications and registry reports) [2, 10, 27, 28]. A less than 20% clinical improvement of Risperidone, Paliperidone and Paliperidone palmitate was observed whereas earlier studies showed between 20 and 50% clinical improvement.

The poor or miss reporting on harms in journal publications and registry reports was also concerning and disappointing, but it was in line with other findings looking at different pharmacological treatments [100, 101]. The pooled estimates from CSRs revealed a significant increase in risk across several AEs, which signifies the importance of CSRs and their use in supporting evidence-based practices. The thresholds for reporting harms across all sources of data could also be problematic, especially as rare events such as hyperprolactinaemia, gynecomastia and cerebrovascular events usually fall below the 5% or 2.5% incidence threshold which is often used in these populations and CSRs generally.

Platforms such as the YODA project and the Clinical Study Data Request [102] where the public can access IPD and CSRs from the manufacturers remain key and should be utilised more by systematic reviewers where possible. However, incorporating these data into systematic reviews requires better awareness of where these data can be sourced and greater resources [103‐105]. In terms of data sourcing, regulatory agencies such as the European Medicines Agency [37] and Health Canada [106] have committed to releasing CSRs for trials on products that are centrally licenced. This will provide opportunities for investigators to freely access CSRs easily by ‘click and download’ for newly licenced interventional products. The Food and Drug Administration in the USA also launched its pilot scheme back in 2018, which is similar of that of the EMAs clinical trials data sharing policy. But access still remains largely restricted with few known cases of anyone accessing trial data/documents to date [107].

Moreover, in the UK, there exists a current policy drive towards improving transparency and open access of clinical trial data [108]. In terms of resource and time involvement, the challenge for funders of research grant applications is deciding where such comprehensive evidence syntheses are likely to make a difference and prioritising their funding even though they require greater resource and time demands. A framework approach would help support groups grappling with how to respond to the increasing availability of these new sources of information. It would likely even increase the scientific return on the funder's investment in the trial and most importantly the benefits to the public and future patients.

Calls for improved reporting standards, especially for harm outcomes in journal publications, continues to be debated even after the introduction of reporting guidelines such as the CONSORT harms [100, 109‐111]. Many journals now allow for the publication of segments of de-identified IPD, redacted sections of CSRs including tables or narratives of harms alongside the main journal publication as supplementary online material. However, not all trials provide these levels of data, and they are still not as exhaustive as the full CSRs and/or IPD. The development of a central repository where all available redacted CSRs can be indexed and uploaded could help to alleviate some of the challenges associated with accessing and using these data. Such a development would require full co-operation and backing from all key stakeholders.

Using IPD from the drug manufacturer, we found that Risperidone and Paliperidone lead to small clinical improvements in patients with schizophrenia or bipolar disorder which are more conservative compared to those of previous studies based on publicly available data. CSRs also contained a significantly greater volume of major harms compared to journal articles and trial registry reports. Our findings clearly support the use of IPD and CSRs for accurately assessing the efficacy and safety of antipsychotic drug interventions. For this type of in-depth analysis, more informed guidance is needed to help encourage researchers to access and analyse these data sources in an efficient manner.