Mind the gaps - the epidemiology of poor-quality anti-malarials in the malarious world - analysis of the WorldWide Antimalarial Resistance Network database

A systematic review was conducted of scientific and lay reports on anti-malarial medicine quality, using Pubmed, Scielo, Embase, Embase-Classic, Medline, Google, Google Scholar, World Health Organization (WHO), United States Pharmacopeia (USP), and Medicines Regulatory Agencies (MRA) websites from 1946 to March 2013 in English, French and Spanish (Table 1). Search terms used were ‘anti-malarials’ or ‘artemisinin derivatives’ or ‘antimalarial agent’ and ‘counterfeit’ or ‘substandard’ or ‘fake’ or ‘spurious’ or ‘falsified’ or ‘quality’. Abstracts and full text of 404 studies were reviewed (See the “Inclusion and exclusion criteria of published reports included in WWARN database” subsection; Additional file 1). Data were extracted and entered in a database constructed using MS Access 2007. Descriptive analysis was conducted in Excel and STATA (v11.2, Stata Corp, College Station, TX, USA).

Inclusion criteria; Any of:

Exclusion criteria:

Anti-malarial quality failure rate is quoted from each report without additional analysis. If additional important information, other than packaging and amount of active pharmaceutical ingredient (API), such as disintegration, dissolution, and microbiology are given they are also included in the database. (For a more detailed description of each variable and methodology see [31]).

In view of the controversy over the terms used to describe medicine quality, author’s definitions were tabulated in and interpreted relation to the definitions used by WHO (Additional file 2), whilst a consensus is sought. ‘Falsified’ is used as a synonym for counterfeit or spurious, referring to a medical product produced with criminal intent to mislead, but without reference to intellectual property concerns. Samples that failed chemical assays, but without detection of wrong active ingredients and without packaging analysis, are classified as poor quality and not as falsified or substandard as this distinction cannot be reliably made without reference to the packaging [17]. However, samples that contained wrong API or no API but without packaging analysis were assumed to be falsified. There is a small risk of misclassification of such samples as falsified when they are actually substandard, due to gross manufacturing errors. However, such catastrophic errors of potential criminal negligence appear to be relatively rare [6, 7]. Samples that did not fail chemical and packaging tests (when these were done) are considered as good quality. As there is little information available to distinguish substandard (i.e. errors in factory production) from degraded (i.e. degradation due to post-production inappropriate storage) medicines, substandard medicines may also, in error, include degraded products that left the factory as good quality [17, 32].

The Antimalarial Quality (AQ) Surveyor is an open access, web-based visualization tool that tabulates and maps published reports on the quality of anti-malarial medicines [9]. The system was designed with medicine regulatory authorities (MRAs), national malaria control programmes (NMCPs) and medicine funding agencies as the target main users. A simple dashboard allows users to filter and explore the data and examine standardized summaries of anti-malarial quality reports. Key information that can be filtered includes the quality of anti-malarial medicines, where they were obtained and from what type of outlet and what sampling techniques and chemical assays were used.

The AQ Surveyor also includes a filterable tabular view, listing all the source reports plus other publications, such as reviews and descriptions of assay techniques, without primary data amenable to mapping, but relevant to medicine quality. The AQ Surveyor was reviewed by key audiences, including MRAs, NMCPs, pharmacists, and academics for feedback and adapted accordingly.

Of 104 malaria-endemic countries [1], some published information on anti-malarial quality is publicly available from 41.3% (43), with over half of these (58.1%, 25) only having one or two reports available. Of malarious countries in South and Central America, Africa and Asia, anti-malarial quality data are only available from 19, 61 and 50% countries, respectively. There are no reports from vast swathes of central Africa, such as Zambia. There is only one report each from DRC, Angola and Gabon (that represent 40% of the estimated global burden of malaria); and Sri Lanka, Nepal and Melanesia. There are very few from India, South and Central America and southern Africa. No report was found from the Eastern Mediterranean WHO Region or the six endemic-malaria countries in Europe (Additional file 3). Nigeria has highest number of reports followed by Tanzania, Ghana, Cambodia, Kenya, and the Lao PDR (Laos) (Figure 1). There are only 82 locations listed in the AQ Surveyor, with 36.7% (61) of publications not stating where within countries samples were collected.

The number of surveys of anti-malarial quality has increased over the last decade (Additional file 4). In the last five years the focus of anti-malarial quality publications appears to have shifted to medicine regulation and chemical analysis techniques, with fewer peer-reviewed primary research articles on anti-malarial quality surveys. Peer-reviewed articles accounted for 24.3% (61) of the 251 publications, with medicine regulation 16.7% (42), analysis techniques 11.2% (28) and lay press 10.4% (26) accounting for the remainder (Figure 2). The median (range) delay between collection and publication was one (zero to 11) years, with 26.7% (39) of publications not stating when the sampling was conducted.

Of the 130 published reports only 5% (six) included evidence for randomization of sample location selection (Additional file 5) [33‐38]. For papers published since the Medicine Quality Assessment Reporting Guidelines (MEDQUARG) [29] for conducting and reporting surveys of medicine quality were published in 2009, only 15.4% (six of 39), describe following them. Of 529 surveys, 32.3% (171) explicitly describe their definitions of counterfeit, falsified, and/or substandard and 20.6% reports (27) used contemporary WHO definitions. Another four reports mentioned the WHO definitions but called the medicines substandard, classifying them according to whether they complied with colour tests, had a different retention time (Rf) value in thin layer chromatography (TLC) or used the terms ‘poor quality’ and ‘substandard’ without conducting packaging analysis and based only on amount of active ingredient.

Of the 130 primary survey research papers, 47.7% (62) stated in which country the chemical analysis was performed, for 32.3% (20) of these, it was performed in high-income, non-malarious countries and in 17.7% (11) the analysis was conducted in WHO prequalified labs.

The majority of the surveys (61.4%, 323) examined the quality of non-artemisinin derivative anti-malarials, especially chloroquine (23.4%, 76) and sulphadoxine-pyrimethamine (SP) (23.1%, 75; see Additional file 6). To date only one report of falsified ACT has been reported in Asia (a seizure in China with medicines for sale in Africa, [5]) but there have been numerous reports from Africa. Figure 3 shows the number of failing samples classified by anti-malarial category and region.

Out of 9,348 samples included in the database, 30.1% (2,813) failed chemical/packaging quality tests. Oral artesunate was the medicine most commonly reported as falsified (with 61.9% of it failing) see Figure 4. Of the 2,813 samples that failed a chemical quality test, 39.3% (1,107) were classified as falsified, 2.3% (66) as substandard and 58.3% (1,640) were classified as poor quality, without evidence to categorize them as either substandard or falsified (Additional file 7).

An important minority (14.0%, 74) of the 529 surveys did not state the number of samples collected, and 45.7% (242) included less than ten samples per international non-proprietary name (INN) see Figure 5. The median (range) number of samples collected per study was ten (one to 258), excluding reports describing confiscations.

No surveys were found of a significant number of anti-malarials, including atovaquone, cycloguanil, clindamycin, dapsone-pyrimethamine, napthoquine, piperaquine, and parenteral artesunate. Surveys have mainly been conducted on oral artesunate, chloroquine and sulphadoxine-pyrimethamine with a mean number of different anti-malarial INNs collected per study of two, up to a maximum of nine (Additional file 8). As an example, the figure in Additional file 9 gives the different published failure rate results for all the anti-malarials sampled and classified as falsified in Cambodia, a country with many published reports. The numbers vary greatly depending on the number of samples analysed and methodology followed.

Of 130 publications, 45% (59) stated whether the 1,132 failing samples (including samples from one confiscation [39]) had high, low or a different API from that stated on the packaging. Of the failed samples with chemical data, 27.0% (306) had no API detected, 25.7% (291) had high API, 24.7% (280) had low API and 22.5% (255) had wrong API (Additional files 10 and 11). Of samples with different ingredients to those stated on the packaging, 20 different ingredients were found, ranging from aspirin, acetaminophen, mercaptobenzothiazole to soya flour (Table 2).

The types of outlets where medicines were sampled was not specified in 23.8% (126) of the 529 surveys; 2.5% (13) of the surveys stated that they sampled only public-sector outlets, 37.0% (196) included private outlets alone and 28.0% (148) did not specify which results referred to private or public outlets.

Thirty-three different techniques to determine the quality of the medicines was stated in one-hundred and seven publications, including high-performance liquid chromatography (HPLC) 37.4% (40), thin layer chromatography (TLC) 12.1% (13), direct mass spectrometry (MS) 4.6% (five), liquid chromatography–mass spectrometry (LC-MS) 2.8% (three), colorimetry 2.8% (three) and dissolution 24.3% (26). Examination of the packaging was reported in 21.5% (23) publications. For API content analysis technique and interpretation, the US Pharmacopoeia (31.1%, 41) was most commonly used, followed by the British Pharmacopoeia (9.9%, 13), but this vital information was not given in 41.7% (55) of publications.

In 60.2% (53) of reports, details of the manufacturers stated on the sample were not included. In a further 25 reports the country of manufacture was given without the stated manufacturer’s name. Samples labelled, correctly or incorrectly, as made by Guilin Pharmaceutical Co Ltd, China, were the most commonly collected but details of 90 anti-malarial manufacturers from 36 countries, as stated on the packaging, were found in the reports. The alleged country of manufacture was specified in 934 failing samples (of 2,813) and China was the alleged country of manufacture of 35.2% (329) failing samples followed by Pakistan 16.2% (151), India 13.7% (128) and Switzerland 11.8% (110); whether these were the countries of manufacture is unknown.

Fifteen of the 36 (50%) stated countries of manufacture, have stringent regulatory authorities (SRA) that are members/observers/associates of the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceutical for Human Use (ICH) [54].

Of the samples that failed chemical and/or packaging analysis 8.2% (209) were labelled as from a WHO prequalified manufacturer, 0.9% (24) labelled as from a manufacturer with an address within a SRA country and 4.1% (104) labelled both from a WHO prequalification programme (PQP) and from a SRA country. Of the eight WHO prequalified anti-malarial manufacturers [55], products from three manufacturers were found in the database.

No reports of anti-malarial quality were found for 60.6% (63) of the 104 malaria-endemic countries and 38.6% (17) of the 44 African malarious countries. Although there are clear foci of poor quality anti-malarials, the current global situation remains unclear, poorly documented; and their impact on public health uncertain. The data are insufficient to understand the distribution of ‘hotspots’ of poor anti-malarial quality and the reasons for heterogeneity.

The data suffer from important limitations that are vital to bear in mind in their interpretation. There is likely to be a large unpublished ‘grey literature’ held by MRAs and the pharmaceutical industry that would contain useful information. The pharmaceutical industry, while paying close attention to falsification of their products, are often reluctant to share figures or specific examples [56, 57]. The lack of standardization of reporting is also serious limitation. Results are frequently not broken down by country and/or medicine, making it very difficult for individual countries to plan their response unless they are sent data by the authors of reports.

Studies are not reported in a consistent manner, hampering comparison with other studies through time and space. The number of samples is commonly not representative and the small number of units tested also limits the interpretation of the results. One third of the reports (29.2%) did not state the number of samples they collected.

Estimates vary greatly depending on the sampling methodology and the technique used, probably greatly influencing reported failure rates. In 13 reports a portable technique was used to determine medicine quality and in eight of those, no confirmation analysis was conducted in a certified quality control laboratory whilst the accuracy of these portable techniques has not been properly evaluated in the field.

Most importantly, randomization, usually considered as the gold standard in estimation of disease prevalence was used very rarely. Without such an objective sampling strategy it will not be possible to obtain confidence intervals, objectively compare between regions or through time or test the effectiveness of interventions [58]. Convenience sampling can be useful for yielding alerts about poor quality medicines but cannot be used reliably to estimate their frequency.

Thirty-four (26%) studies did not report when the collection was conducted and there were long delays, up to seven years, between sample collection and publication.

Ambiguous definitions over what constitutes falsified and substandard medicines make data standardization difficult and 74% of the reports did not state what definition they used. A vital problem is the frequent lack of distinction between falsified and substandard medicines. This distinction requires additional time-consuming packaging analysis but is indispensable, as without these data MRAs do not know if they are dealing with fraud and criminal ‘pharmaceutical’ production or with poorly functioning manufacturing plants, or issues related to medicine storage. How an MRA will respond to reports of poor medicine quality depends on this distinction. This problem is exacerbated by the difficulty of accessing examples of genuine packaging to conduct packaging analysis. Out of the failing samples, 58.3% (1,640) were classified as poor quality. These poor quality anti-malarials tended to have either greater or lesser amounts of API in comparison to that stated on the packaging, suggesting that they are most likely substandard. Quinine and artesunate seem to be the most commonly falsified with wrong API, together with artemether-lumefantrine with no API. SP and chloroquine most commonly had incorrect amounts of API.

A further important issue is that dissolution testing was only included in 24.3% (26) of the chemical analyses, presumably because of the large investments in equipment, funds and laboratory time required. Anti-malarials may have the correct amount of API, but especially for SP, may have very poor dissolution [59]. Recent data suggest that similar problems may affect poor quality ACT [60].

More investigation needs to be done in the Americas and the central and southern African regions as there are very few reports from these malarious areas. Six countries: Nigeria, the Democratic Republic of Congo, Burkina Faso, Mozambique, Côte d'Ivoire, and Mali, account for 60%, or 390,000, of estimated global malaria deaths [1], but only one report of anti-malarial quality was found from Burkina Faso and Mali, and two reports for Côte d'Ivoire. There is little confidence in the generalizability of anti-malarial quality for large populations afflicted with malaria. More surveys are needed in populations with high malaria risk (Figure 6). Also, little information is available about the registration status of the manufacturers in endemic countries or about expiry date tampering. Only 13.1% (17) reports stated whether samples were registered in the country of collection and 16.9% stated whether expired samples were found.

Other reviews summarizing the prevalence of poor quality anti-malarials did not have access to a full database of a long period of data in multiple languages [11, 61] and gave aggregated estimates of the prevalence of poor quality anti-malarials through space and time without emphasizing the limitations of the data and dangers of over-interpretation. The aggregation of information from sources with variable credibility with equal weight may not be appropriate and this issue needs to be addressed.

Therefore, the data reviewed here cannot be summarized as stating that 30.1% of the world’s anti-malarial medicine supply is poor quality. All that can be concluded is, mindful of the above caveats, that 30.1% of anti-malarials tested over 67 years were poor quality and that this is an important public health problem. Enormous quantities of anti-malarials are consumed in comparison to the number of samples with published quality data. In Africa, an estimated 200-400 million courses of anti-malarial treatment are used per year with an additional 100 million courses elsewhere [62]. The number of ACT treatment courses delivered globally to the public and private sectors increased from 11 million in 2005 to 278 million in 2011 [1]. However, information of the quality of 9,348 samples and 1,034 ACT were only found since 1946.

Despite evidence that poor quality anti-malarials are focally common and are bad for patients, there have been few interventions and the subject has been dominated by discussions with little action. The efforts put into addressing the quality of medicines have had little tangible impact, in comparison to the size of the problem, because the programmes have mostly been slow, under-funded and fragmented. Many parallel interventions are needed [3, 5, 11, 16, 19, 63]. The controversy over definitions has disabled much that could be done and the use of the unwieldy term ‘substandard/spurious/falsely labelled/falsified/counterfeit medical products (SSFFCs)’ has not helped. The international community should act with much greater speed and focus to build interventions to safeguard global medicine supply (Newton et al. submitted).

The newly created Member State Mechanism on SSFFCs should rationalize this term into definitions that are accessible and accurate and aid public health [16, 17], help to focus on the development and maintenance of up-to-date international tools, guidelines and standards, identify major needs and challenges and make policy recommendations to strengthen national and regional capacities [64‐68]. The provision of free or subsidized ACT through the Global Fund, President’s Malaria Initiative (PMI), the World Bank and other major donors has probably had a large impact on improving the quality of anti-malarials that patients take in the public sector, although it was not designed with this primary aim in mind [62]. The WHO Rapid Alert System [69] should allow a better coordinated sharing of information and help fill in the many gaps. However, more data from countries with high malaria incidence but few data points on anti-malarial quality are urgently needed. Similarly, the pharmaceutical industry, both innovative and generic, should have a legal duty to report suspicions of poor quality medicines to key stakeholders such as MRAs and WHO [56]. A major problem is that there are very few laboratories in the malarious world able to accurately analyse the quality of anti-malarials – there are only three countries in sub-Saharan Africa and five in Southeast Asia with WHO prequalified laboratories [70]. Many of the current problems have arisen as a consequence of the lack of investment in MRAs in low- and middle-income countries, by national and international organizations, to allow them to effectively regulate and police the medicine supply. Of 100 primary survey research papers not performed by MRAs, only 26% (26) mentioned that they informed the MRA. Appropriate interventions to support MRAs are urgently needed.

An ethics statement was not required for this work.