Effectiveness of needle and syringe Programmes in people who inject drugs – An overview of systematic reviews

Our primary objective was to conduct an overview of systematic reviews that evaluated the evidence of the effectiveness of NSP for PWID across a range of different relevant outcomes, i.e. blood-borne infection transmission and injecting risk behaviours (IRB).

Our secondary objective was to assess how different aspects of NSP provision, including provider, setting, coverage and any related component delivered in parallel, such as harm reduction services and opiate substitution therapy, modified the effect of NSP, with a particular focus on pharmacy-driven NSP.

For inclusion in this overview, studies had to meet the following criteria:

When more than one review included exactly the same studies, the review that reported the most complete presentation of results was selected for inclusion in the overview.

Searches were undertaken on MEDLINE®In-Process & Other Non-Indexed Citations (from 1946 to 12th of May 2015), EMBASE (from 1974 to 12th of May 2015) and PsycINFO (from 1806 to 12th of May 2015) via the OVID SP interface. In addition, the following databases were searched: the NHS Economic Evaluation Database (NHS EED), the Database of Abstracts of Reviews of Effects (DARE), the Cochrane Database of Systematic Reviews (CDSR), the National Institute for Health and Clinical Excellence (NICE), the Campbell Library of Systematic Reviews and the Database of Promoting Health Effectiveness Reviews (DoPHER). The search strategies for each database are shown in a supplementary file (Additional file 1). No language or other types of restrictions were applied. In addition to the database searches, handsearching of the references of the included reviews was undertaken to identify further relevant studies.

Titles and abstracts of articles identified were screened independently by two authors (MC, JA), and classified as include, unclear or exclude. The full reports of all articles that classified as include or unclear were then obtained, and two authors (MC, GD) examined compliance of reviews with eligibility criteria, with a third author acting as an arbiter (RF).

Data from reports of all included systematic reviews were extracted by two authors (MC, GD) and validated by a third author (RF), using a data extraction form designed and pre-piloted for this overview. The following general characteristics were extracted from each systematic review: publication details; study objectives; eligibility criteria (population, intervention, comparators, outcomes, study designs); any reported protocol; and methods used for search, screening, data extraction and synthesis. For each systematic review, we listed all included studies and evaluated whether they matched the eligibility criteria for this overview regarding population, interventions and outcomes. We then extracted the following results from the eligible group of studies, whenever reported at the systematic review level: study design; countries involved; characteristics of included participants (demographics, prevalence of HIV/HCV); description of interventions and any co-intervention, duration of intervention and follow-up; effect estimates from meta-analysis (if available) or at individual study-level, for each relevant outcome; and any subgroup or sensitivity analyses. The authors’ conclusions for each relevant outcome were also collected. We contacted the authors of reviews for relevant missing data.

At a study level, we extracted data on any risk of bias assessments of primary studies when performed and reported by reviewers in each systematic review, including tools used and summarized results. We also collected data on any reported evaluations of the quality of evidence concerning our outcomes of interest in included reviews, particularly those using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) tool [18], as well as data on assessments of publication bias.

At a systematic review level, we assessed the methodological quality of each included systematic review using the Risk Of Bias In Systematic Reviews (ROBIS) tool [19]. Two review authors (MC, RF) performed quality assessments independently, using piloted decision rules. Disagreements regarding overall assessments were resolved through discussion, with a third reviewer serving as the final arbitrator (AVC). The rationale behind assessments was documented. We calculated measures of agreement and reliability between raters for each ROBIS domain.

We stratified the included systematic reviews by: (i) type of outcomes assessed (blood-borne infections, IRB), and (ii) type of analysis (with or without meta-analysis). We summarized data from the included reviews both in text and in summary tables and figures. When meta-analysis was performed, we report pooled estimates using the models and measures of effect reported by systematic review authors, with 95% confidence intervals (95% CI); we did not perform any additional statistical analysis. When reported, the accompanying I ² values, which describe the percentage of total variation across studies that is due to heterogeneity rather than chance, were collected [20].

Table 1 lists the key characteristics of the 13 included systematic reviews that evaluated the evidence of the effectiveness of NSP for PWID in the community setting. We classified reviews by type of outcomes reported and type of data analysis (Fig. 2).

The number of databases searched per review varied between two and 15, with the three most common sources being MEDLINE, EMBASE and PsycINFO. The dates of last search varied between 1995 [26] and 2012 [27]. Of 287 studies included in these 13 reviews, 200 studies matched the eligibility criteria for this overview, corresponding to 133 unique studies (a matrix with included studies by review is available in a supplementary file: Additional file 2). We included the complete set of studies from four reviews, as in the remaining reviews at least one study did not fulfill our eligibility criteria [27‐30]. The number of relevant studies included per review ranged from three [31, 32] to 43 [24]. While there was some overlap in the included studies between reviews, the majority (68%) was included in one review only.

We found substantial variability in the criteria used by review authors to define and classify study designs, as well as types of designs of included studies. Only four reviews specified the types of study design in their eligibility criteria [12, 26, 29, 33]. Three reviews included randomized controlled trials (four RCTs overall) [12, 26, 31], while the remaining reviews included cohorts and case-controls (101 overall), as well as other study designs (time series, before/after studies or ecological studies) (95 overall).

Three of the included reviews were restricted to studies conducted in specific countries or economic regions (China, UK, and low- and middle-income countries) [28, 31, 32]. Nine reviews reported data on HIV, eight on HCV, and six on IRB. Seven reviews evaluated more than one of these outcomes. Four reviews used meta-analysis [26, 27, 32, 34], one of which using individual-participant data [32]. The remaining nine reviews used narrative synthesis [12, 24, 25, 28‐31, 33, 35]. The reasons reported by the authors for conducting or not conducting meta-analysis varied between reviews, even when there was a degree of overlap between included studies.

Seven reviews reported having assessed the methodological quality of included studies [12, 24, 25, 27, 29, 33, 34]. In two reviews the authors developed ad hoc tools [33, 34], while the remaining reviews used previously existing instruments, often with modifications (tools used: Newcastle-Ottawa, GRADE, WHO-Johns Hopkins 9-Point Rigour Scale, Quality assessment tools developed by NICE Centre for Public Health Excellence, England and Wales and the Effective Public Health Practice Project, Canada) [12, 24, 25, 27, 29]. Only two instruments were used more than once, and all other instruments varied. Not all reviews reported the results of these assessments; when reported, most studies were considered to have low methodological quality, given methodological weaknesses and biases related to observational study designs. Only one quantitative review [27] incorporated quality in data synthesis, by conducting a sensitivity analysis. In reviews without meta-analysis, methodological quality and risk of bias was presented descriptively in the interpretation of study results. Two reviews [24, 27] reported using the GRADE tool to assess the quality of the body of evidence. One considered the overall quality of evidence as low [27], while the other, using an adaptation of GRADE, evaluated the quality of evidence as modest to moderate for different outcomes [24].

Using the ROBIS tool, only two of the included reviews were considered to have low risk of bias, whereas the remaining were considered to have either high (n = 8) or unclear risk of bias (n = 3) (summary of results available in a supplementary file: Additional file 3). The majority of reviews were rated as having high or unclear risk of bias across all ROBIS domains, i.e. study eligibility criteria, identification and selection of studies, data collection and study appraisal, synthesis and findings. Percentage of agreement between ROBIS raters varied between 77% and 92%, and weighted kappa (quadratic) between 0.82 and 0.95.

The results from the PRISMA-P statement applied to this overview are presented at Additional file 4.

In this overview of systematic reviews examining the effectiveness of NSP for PWID in reducing blood-borne infection transmission and injecting risk behaviours, we identified 13 systematic reviews contributing with 133 unique studies, which were mostly observational. Methods used in these reviews varied at all levels of review design and conduct. Only two reviews were considered to have low risk of bias by reviewers [12, 27] and most included studies were evaluated as having low methodological quality. The quality of evidence, when assessed, was considered low or modest to moderate. Nine reviews reported outcome data on HIV prevalence/incidence, eight on HCV, and six on IRB. Meta-analysis was performed in four of these reviews.

Our interpretation of the findings is that the overall results of the included systematic reviews are supportive of the effectiveness of NSP in reducing HIV transmission and IRB among PWID, as well as in reducing HCV infection, although the latter to lesser extent. The overall quality of the evidence is higher for HIV transmission and IRB than for HCV infection. However, for HCV infection, the strength of the evidence increases (because studies’ results are more consistent) if the intervention under consideration is not solely NSP, but includes other components such as opiate replacement treatment, in a strategy of full harm reduction intervention.

Furthermore it is well known that sharing other injecting equipment (e.g. swabs, cookers, water and filters) is an important route of transmission of blood-borne infections, particularly in the case of HCV. In addition, clean injecting equipment is also available through sources other than NSP (dilution bias) [44, 45]. This overview did not identify any studies evaluating the effects of paraphernalia distribution at reducing the incidence or prevalence of HCV. One further aspect is that individual NSP intervention studies are prone to selection (volunteer) bias, as these exchange programmes attract and retain higher-risk PWID. Taken together, these aspects may have contributed to some mixed results reported in the systematic reviews and individual studies addressing HCV infection.

To sum up, aspects of NSP provision may be relevant, including structural-level NSP (i.e. high-level coverage), and multi-component programmes including full harm reduction seem to benefit all outcomes more than individual NSP.

In this field of public health, interventions and results are highly influenced by economic, legal, ethical, social and cultural circumstances. In the past, results from some of the included studies led to major political decisions and fuelled controversial debates, particularly in the US. There are considerable differences in perspectives regarding the role and provision of harm reduction services, which impact study design and conduct and selection of participants, and thus affect the validity and generalizability of study results. This may at least partially explain why we found heterogeneity in study results and time trends, although there was consistency in many outcomes. Further, while the majority of the published research presented in different reviews originated from North America and Europe, one of the reviews [28] supported part of these results for both high and low/middle-income countries. Importantly, provision of NSP interventions is low at a global level, particularly in countries where the incidence and prevalence of both HIV and HCV is highest.

Aspects of NSP provision are also key to decision-making, and few included reviews examined how different types of NSP provision impact on effectiveness. NSP are extremely diverse in their design, staffing, characteristics of participants, operation and programme delivery policies. Further, PWID are diverse populations with different preferences, behaviours, and life circumstances, who often have difficulty in accessing formal healthcare services. As such, the potential impact of NSP in reducing injecting related-harms is limited by the extent to which the programmes provide effective access to sterile injecting equipment and other services, and are therefore able to attract their potential clients [46, 47].

Results from this overview support the importance of high-level coverage and comprehensive services provided by many NSP in reducing bloodborne infections and risk behaviours. We should note that outcomes which were not included in this review may be relevant to evaluate possible benefits of NSP. For example, pharmacy access to sterile needles and syringes has been found to provide specific benefits in addition to those available through specialist NSPs [12]. Importantly, the coexistence of different modes of injecting equipment delivery, as well as tailoring services offered at different venues, addresses several barriers encountered by PWID, as modalities for improving syringe availability are likely complementary and not competitive [48‐50].

The additional benefits provided through this complementary role of pharmacies may have prompted Governments of Australia, Ireland, Spain (Basque Country) and UK to remunerate pharmacies for the important public health role provided through NSP [8, 51‐55]. As pharmacy-based NSP gradually expand, it is important to pursue future research with a focus on this setting.

The results hereby presented have to be treated with considerable caution. The methodological quality of the included systematic reviews varied, but a majority of reviews were considered to be at high risk of bias. Importantly, we found considerable heterogeneity and inadequacies in methods used by different reviews, at all stages of the design, conduct and reporting of the review. Differences between eligibility criteria and search and screening methods likely explain a striking mismatch between included studies, despite some reviews having similar research questions and covering relatively close publication periods. Assessment of quality of included studies was seldom performed, with the use of different tools for assessment of risk of bias, some of which were developed ad hoc with no validation, and hardly any use of grading of quality of evidence. We also found variability in the decision as to whether or not to perform quantitative synthesis, even when the body of evidence was similar. Few review authors chose to present pooled estimates from meta-analyses, and when present, there was variation in models, choice of outcome measures and use of sensitivity and subgroup analyses. Other authors opted to present narrative summaries of results, often with vote counting, but there were discrepancies in the interpretation of trial results. Thus, there were obstacles in summarizing, comparing and synthesizing results from all included reviews.

These factors also reflect the challenges that review authors had in summarizing results from individual studies, most of which were judged to be of low methodological quality. Classification of study design was discrepant between reviews, but most studies were observational, at either individual or population-level. These designs are limited in establishing causality, and we found different approaches regarding adjustment for possible mediators and confounders. Further, a range of selection biases were identified and explored in some landmark studies, including volunteer bias of highest risk users into NSP. Less biased evaluations of NSP would require the use of randomization, for example at a cluster level, but there are many ethical, scientific and practical challenges. Other aspects to consider are the challenges in selecting, comparing and pooling different outcome measures and measurement tools when assessing HIV and HCV infection, as well as needle and syringe sharing and injecting frequency. A final limitation of this body of evidence is that studies generally were not designed to allow for separate examination of different aspects of NSP provision. As most public health interventions, NSP are complex health interventions with several interacting components, multilevel factors (users, providers, setting, health systems), and some degree of flexibility of interventions allowed in the real-world setting. Hence, challenges will likely persist in identifying the independent contribution of improving access to sterile needles and syringes, both at review and individual study level.

Two other overviews of reviews were published on this topic [12, 14]. The first overview, published in 2008 [12], was undertaken as a section of an extensive review of the effectiveness and cost-effectiveness of NSPs for PWID to inform guidance on the optimal provision of NSPs from the perspective of the UK National Health Service. The authors concluded that none of the included systematic reviews examined HCV in any depth and that there was insufficient evidence to support the effectiveness of NPS on HCV incidence/prevalence. On the other hand, there was tentative and sufficient evidence to support the effectiveness of NSP on HIV incidence/prevalence and on self-reported IRB, respectively. The second overview was an updated overview published in 2014 [13, 14] and focused on the effectiveness of different harm reduction interventions, including NSP, in preventing IRB, HIV and HCV transmission among PWID [14]. This review was related to guidance from the European Centre for Disease Prevention and Control and the European Monitoring Centre for Drugs and Drug Addiction. Taking into consideration the individual conclusions of the core included systematic reviews, as well as the number of individual studies showing positive findings (i.e. a reduction of IRB, HIV and/or HCV incidence/prevalence), the authors concluded that there was: 1) sufficient review-level evidence of effectiveness in relation to IRB; 2) tentative review-level evidence to support the effectiveness of NSP in reducing HIV transmission among PWID; and 3) insufficient review-level evidence to support the effectiveness of NSP in reducing HCV transmission.

The field of overviews is relatively new, and this type of study design has been gaining momentum as a valuable knowledge synthesis methodology that can collate extensive information to facilitate the uptake and application of knowledge by decision-makers [56]. However, published overviews show considerable variation in their methods and reporting due to the unique methodological challenges inherent in summarizing and synthesizing evidence from different heterogeneous sources [57, 58]. Reviews in public health pose additional challenges, as methods are likely to vary more, with less quantitative analysis and different approaches to synthesizing and interpreting evidence.

Previous overviews have used different approaches to review eligibility, selection, quality assessment and analysis. Also, there were differences in the framework used to classify included reviews (e.g. core/supplementary), the type of summary and synthesis used (e.g. vote counting according to study or review results), and the conclusions and evidence statements. In comparison, the strengths of our overview include the use of strict eligibility criteria, the evaluation of methodological quality of included systematic reviews, and the categorization of review by type of analysis and outcome. We used the novel ROBIS tool [19], which was developed and validated to overcome limitations seen in previous instruments to assess the conduct and reporting of systematic reviews. This tool allowed us to stratify review quality using clear and transparent criteria, thus providing support to our confidence in each review’s findings. Further, by presenting pooled estimates of the effect of NSP in different outcomes taken from reviews that performed meta-analysis, we provide the opportunity for readers to quantify the direction and magnitude of this effect. We also highlight the need to use this quantitative data to assess the likely relevance of findings.

There are a number of limitations to our overview. While we identified and listed all unique studies included in each review and assessed the degree of overlap between reviews, we did not assess these studies individually, as other overviews have [57]. We focused on results from different reviews, which had some degree of overlap, and thus have a risk of double counting results, both for qualitative and quantitative reviews. Further, the application of ROBIS was challenging, although measures of agreement between raters were relatively adequate, likely due to the use of piloted decision rules.