What lessons can be drawn from the NCPI which, to our knowledge, is the first and only effort to collect data systematically on country self-reported national laws and policies related to HIV?
The Purpose of the NCPI
The NCPI has evolved into a comprehensive questionnaire collecting information on laws, policies and regulations related to HIV, as well as other aspects of program implementation. Reporting rates remain high.
Initially it was envisioned that data collected through the NCPI would be analyzed through the construction of an index which could provide insight into a country’s AIDS-related policy environment. However, it was found challenging to construct a meaningful index that adequately reflected the breadth and heterogeneity of available data. Although indices have been recognized as providing an easy-to-understand measure of complex issues and effective tools to support advocacy efforts, they risk oversimplifying and overstretching available data [
19]. The NCPI has therefore transitioned into an instrument, with analysis performed on the basis of individual questions—each of which measures some aspect of whether a country provides an enabling environment for a robust AIDS response.
The responses captured through the NCPI sometimes differ from data reported through other sources, such as the Law Database or population-based surveys. For example, the Law Database only includes information about a specific set of legal obstacles for effective HIV prevention, treatment, care and support, e.g. criminalizing same-sex-relationships for MSM. The NCPI, by contrast, asks more generally if the country has laws, regulations or policies that present obstacles to effective HIV prevention, treatment, care and support for key populations and vulnerable groups. In other words, the NCPI recognizes that access may be restricted by measures and practices beyond criminal law. This can explain why, for example, the UNAIDS Law Database and the responses from civil society in Latin America differed from one another on reported obstacles (men who have sex with men in a third of the countries, and in more than half of all countries for sex workers, even where no criminal law is recorded) (Table
2). Conversely, in a country where laws exist which criminalize same sex relationships and government and/or non-governmental actors answer in the NCPI that there are ‘no obstacles’ –a lack of understanding or willingness to acknowledge that such obstacles exist is revealed. This can be seen in many regions, but most notably in West and Central Africa (Table
2). Discussions on legal obstacles for key populations are therefore strongest if findings from both the NCPI and complementary validated sources are consulted.
There are also differences in relation to service coverage as reported through surveys and the implementation score reported through the NCPI. Such variation may arise through the way the measures capture coverage, with one focusing predominantly on access and the other on use. The NCPI asks if “The majority of people in need have access to…” whereas most surveys seek to understand if people actually have received services, in this case an HIV test in the last 12 months.
The criteria which respondents take into consideration in responding to the NCPI raise another set of issues. Respondents rarely document the reason for the answer provided. This makes interpretation difficult and at times has led to suggestions of unreliability, as responses do not reflect data available through validated (e.g. quantitative) methods. The subjective interpretation of questions by respondents as well as the lack of clarity of some terminology and definitions [
20] may hinder the possibility of comparison over time for a country, as well as across countries and therefore global monitoring. In some cases, responses may reflect perceptions of implementation of laws or policies, for example, rather than their actual implementation which is in and of itself important, particularly as concerns key populations.
As a starting point for a discussion at country and global levels, the NCPI process has proved useful. Going forward, in relation to its programmatic components the Instrument’s greater utility may be achieved by focusing questions related to programmatic elements on areas that can provide further insight on the reasons for good or poor performance, or related qualitative aspects that quantitative indicators cannot capture.
The Value of the NCPI
High reporting rates suggest that the majority of countries continue to consider monitoring the policy and legal environment of the AIDS response relevant to improving access and use of services—or complying with international norms [
33]. Limited use of NCPI data once collected may be due to challenges in interpreting responses meaningfully [
21], as well as challenges in accessing the data in a format that facilitates broad dissemination, analysis and use. Making NCPI data publicly available in an easily analyzable format, rather than simply publishing reports on the UNAIDS webpage in PDF format, could increase its use. A database of responses is maintained by UNAIDS and used to produce analyses of annual responses and trends for consistently-reporting countries, along with extracts of narrative responses. Tabulations of NCPI data can be provided by UNAIDS upon request.
As described above, the differences between government and non-governmental responses have been extremely useful for triggering dialogue among country stakeholders [
22]. Variances in responses between Parts A and B suggest there is value in providing a space for government and non-governmental actors to report separately, and subsequently engage in dialogue around identified differences.
Perceptions of the existence of laws or policies, even if not accurate, may impact service delivery and uptake and is therefore important country level information. The broad and open questions of the NCPI on the existence of obstacles may also facilitate awareness of the legal environment more broadly. Probing the existence of specific laws and policies may be something captured through other existing tools (such as the above-mentioned Law Database), and may not be a good use of limited time and resources. However not including questions on the existence of laws and policies provides less scope for addressing specific harmful laws and policies, unless additional details from other sources are provided and discussed during stakeholder dialogues or narrative responses.
Additional data on the actual implementation of laws and policies, or related practices, would be an important addition to any future iteration of the tool [
21]. Modifications to the questionnaire content should also account for the evolution in data availability on HIV-related laws and policies from other sources..
Policy processes are notoriously complicated, the term itself is contested, and there is often a major disconnect between stated policy and what gets implemented in practice [
23]. As a participatory policy monitoring tool, the NCPI could be well placed to capture gaps between law and policy existence and implementation, which can have an important impact on access to services. Moreover, some of the concepts which the NCPI attempts to capture, while elusive and difficult to measure, are important and not well captured through other mechanisms. For example, the concepts of ‘political will’ and ‘leadership’—one knows when they are present, but to capture them through the number of times reference is made to an issue in leader’s speeches, or in party manifestos, may or may not provide an adequate measure of ‘will’. President Mbeki of South Africa exercised significant political will in relation to his ‘denialist’ position on AIDS—but was at odds with the need for evidence-informed prevention and treatment services [
24].
The space for narrative responses in the NCPI provides important insights for the interpretation of reported data; even as the human resource requirements for their analysis have been cited as challenges to the greater use of such information [
18]. By documenting progress in the development and implementation of national HIV policies, strategies and laws, which may not be captured elsewhere, these narratives provide important indications of progress, particularly for laws or policies that may take a long time to change, such as abolishing a law that criminalizes a behavior.
The NCPI process has created an important space for dialogue between governments, civil society and other stakeholders on a range of policy issues that play an important role in determining access to services and a human rights-sensitive HIV response
4 [
18]. The AIDS response is marked by the engagement of people living with HIV and those affected by the epidemic. The areas where progress has been greatest have been where mutisectoral, multistakeholder engagement has been fostered as the norm, particularly that of affected communities [
25]. Community participation and engagement have been found to improve access to services in various settings through the construction of citizenship, strengthening of participation practices, and strengthening the responsiveness and accountability of states, although the results of engagement have been found to also vary by context and form of participation [
26]. GARPR has provided a platform for such engagement and the NCPI provides the data to begin discussion of issues across the breadth of the response. Although a national consultation or dialogue on NCPI responses is encouraged in the recommended process for completing the NCPI, these have not taken place systematically across countries or over time. Respondents to the 2014 NCPI survey noted limited time to complete the tool, in particular given its length, and lack of human and financial resources to bring stakeholders together as challenges. Limited capacity of civil society organizations to provide answers and engage with the tool was also cited. Respondents recommended a simpler and shorter questionnaire that would allow for more meaningful responses, as well as to facilitate engagement and dialogue among stakeholders. [
18].
The extent to which the instrument can contribute to the reform of policies or legal frameworks which promote more inclusive, evidence-informed and human rights-based responses has not yet been assessed, and is an area ripe for investigation.
While not perfect, the NCPI has been recognized as one of the most comprehensive sources of information on HIV laws and policies, and as having contributed to the assessment of political commitment through standardized questions across countries [
27]. NCPI-based questions have been integrated and adapted in monitoring and evaluation guidance and tools by partners, including The United States President’s Emergency Plan for AIDS Relief (PEPFAR) [
28] and Measure Evaluation [
29]. Similar tools have been developed for monitoring and evaluation of specific program areas, including certification for the virtual elimination of mother-to-child HIV (and syphilis) transmission [
30], assessments of legal and policy environments related to reproductive, maternal and newborn health [
31], and the International Conference on Population and Development (ICPD) [
32] review process [
32].