Signals of adverse drug reactions communicated by pharmacovigilance stakeholders: protocol for a scoping review of the global literature

A systematic review of definitions, later adopted by the Council for International Organizations of Medical Sciences, defined a signal in pharmacovigilance as “information that arises from one or multiple sources (including observations and experiments), which suggests a new potentially causal association, or a new aspect of a known association, between an intervention and an event or set of related events, either adverse or beneficial, which would command regulatory, societal or clinical attention, and is judged to be of sufficient likelihood to justify verificatory and, when necessary, remedial actions” [1, 2].

Based on this definition, a signal can be supported by diverse levels of evidence, ranging from one or more reports of suspected adverse drug reactions (ADRs) to observational studies and randomized clinical trials [3‐5]. On the other hand, signals of disproportionate reporting (SDRs) [1] can be highlighted solely by statistical associations resulting from ratios of observed versus expected numbers of reports (i.e. disproportionality) [6]. Different stakeholders can detect signals—including regulatory agencies, pharmaceutical companies, academics, independent research organizations, or patients and their carers—and avail themselves of several channels to disseminate them, such as peer-reviewed articles, abstracts, drug bulletins, websites, and restricted publications. Minutes of meetings of committees can also document discussions relating to signals [7].

Comparative assessments of signals discussed at the Pharmacovigilance Risk Assessment Committee have suggested that reports of ADRs often supported signals, but that support from multiple sources of evidence increased the likelihood of regulatory action (e.g. changes to Summaries of Product Characteristics) [8, 9]. Similar investigations carried out in the USA have highlighted the same finding [10, 11]. Regulatory actions, however, appear to be inconsistent across international settings; discrepancies in communications of the same risks of drug-induced harms in healthcare systems have been recorded, and it has been suggested that they may be better understood by characterizing the strength of the underpinning evidence [12, 13].

An important component of detection of signals of ADRs is clinical judgment of the data. The Bradford-Hill viewpoints or guidelines [14] and other structured methods of causality assessment [15] have typically guided clinical judgments in pharmacovigilance, but their intrinsic subjectivity has yielded low interrater agreement across independent assessors [16]. Though some methods have a higher interrater agreement than others, there appears to be no consensus on which ought to be the gold standard [17]. Consequently, regulators’ clinical judgments about features of reports of ADRs have occasionally been criticized by healthcare workers or their professional associations [18‐21]. It has been reported that difficulties in assessing causality result in delays in the withdrawal of medicinal products from the market when fatal adverse events are attributed to their use [22]. Furthermore, the time intervals before signals of ADRs appear and regulatory actions follow are determined by the availability of information about the ADRs at the time of product launch; in fact, these intervals were shorter when ADRs were present in a sample of the Food and Drug Administration’s “drug label” at launch (e.g. severe liver injury with telithromycin or suicidal behaviour with varenicline) and longer when ADRs were absent [23]. Other accounts report that it may take 2–12 years for drug-induced harms to be communicated to healthcare workers after marketing authorization of a medicinal product [24‐27]. These studies have computed the difference between dates of communication of signals and launch dates to measure the time-to-signal, while databases of reports of ADRs, such as VigiBase, could estimate this interval and account for unpublished records.

Over several decades of pharmacovigilance, many definitions of signals have been developed and applied, and different types of studies have been used to detect signals, resulting in a complex body of evidence whose breadth has been only partly characterized. Delays to withdrawals have been investigated systematically, while those relating to the time to signal have been limited in scope. Furthermore, while reports of ADRs remain a staple in signal detection, it is unclear to what extent their features (e.g. time to onset, dose-response relationship, positive de-/re-challenge) have been described or contributed to putative causal relationships. This is relevant when one considers that recent data-driven methods to prioritize signals arising from reports of ADRs are based on the quality of information of the reports themselves and a limited set of features, i.e. the presence of a narrative, the geographical distribution of the reports of ADRs, how recently reports of ADRs have been entered into a database, and a numerical measure of data completeness [28, 29]. Comprehensive overviews of the features of reports of ADRs that support signals may further inform improvements to such methods.

We therefore believe that it is worthwhile to conduct a scoping review, since such reviews are suitable for addressing broad exploratory questions pertaining to types of evidence in an area of research [30‐32]

The aims of this scoping review are as follows: (a) to provide an evidence synthesis for decision-making at a regulatory level; (b) to collate signals currently in different platforms; (c) to produce an overview of the types and levels of evidence of studies that support signals; (d) to highlight any differences or similarities in assessment of the evidence used to support signals, with a particular focus on reports of ADRs; and (e) to better understand the delays before signals are detected and to clarify whether they have changed over time.

The six-step framework introduced by Arksey and O’Malley [30] and refined by Levac et al. [31] and Colquhoun et al. [32] will set the structure of this protocol.

The review protocol has been registered in the Open Science Framework database (registration number: osf.​io/​a4xns) and is being reported in accordance with the reporting guidance provided in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) statement [33, 34] (see checklist in Additional file 1). The proposed scoping review will be reported in accordance with the reporting guidance provided in the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) extension for Scoping Reviews (PRISMA-ScR) [35].

To the best of the authors’ knowledge, this would be the first study to systematically chart the evidence in support of signals and draw international comparisons of its levels using a standardized framework. Previous characterizations of the evidence underpinning “initial safety signals” [11] have focussed, for example, on Drug Safety Communications published by the US Food and Drug Administration [49]. These postings concern “important drug safety issue(s)” with “potential to alter the benefit–risk analysis for a drug in such a way as to affect decisions about prescribing or taking the drug” [50]. The agency’s website, however, also lists “potential signals of serious risks” [51], that concerns “about an excess of adverse events compared to what would be expected to be associated with a product’s use” [52]. This indicates that resorting to a pre-established set of webpages may result in the omission of relevant records and may misrepresent the activities of those regulatory agencies who refer to “signal” using different expressions. The time from the first report of an ADR to the communication of a related signal will also be quantified and compared across different healthcare settings, accounting for unpublished reports of ADRs available in VigiBase—the largest database of this kind. The findings from this review will underscore research questions for future reviews.

We expect to retrieve publications that report large lists of SDRs. This will pose some challenges, particularly during data charting, as the electronic form we choose may be unsuitable to accommodate all of them. Should this be the case, we shall adopt a customized Microsoft Excel sheet.

We anticipate that when abstracts do not transparently report methods, we may discard relevant records. In fact, there are no guidelines for reporting signals or SDRs. Furthermore, our exclusion criteria may result in the omission of eligible records: considering as eligible only records whose findings are described as “signals” is an approach that is insensitive to the use of other expressions (e.g. “safety risk”). In principle, contact with original authors would disambiguate those cases in which other expressions have been used, but might render such a broad review infeasible. In the case of contact with regulatory agencies, however, we may ascertain whether communications of, for example, “safety concerns” or “important safety concerns” were considered signals, as long as they reply to our queries. It is possible that we may not receive replies.

Furthermore, some publications may use discontinued terminologies to describe ADRs, such as the Coding Symbols for a Thesaurus of Adverse Reaction Terms, which have not been officially reconciled with MedDRA. This may introduce discrepancies in coding terms across terminologies. This is less likely to occur for medicinal products, as WHO-DD are quite comprehensive. However, WHO-DD are insensitive to vaccines; for example, trivalent inactivated influenza vaccine would be coded to “influenza vaccine”. We therefore plan to make this clear in the limitations of the resulting publication. Finally, as concerns the time to signal, the date of entry in VigiBase may be later than the date of receipt of a report to a national pharmacovigilance centre, which may underestimate the time to signal. Between the occurrence of an ADR and the date of entry in VigiBase, there may be a gap ranging from months to several years, thus exaggerating the time to signal. In the interest of transparency, we shall report both dates for each retrieved signal. Also, ADRs may have never been reported to VigiBase in association with some medicinal products, which would impede calculations of the time to signal.