Improving HIV outcomes in resource-limited countries: the importance of quality indicators

Since first reported in 1981, HIV/AIDS has claimed the lives of nearly 27 million people worldwide [1, 2]. The United Nations Development Program named HIV responsible for “the single greatest reversal in human development” in modern history [3]. In response, the international community has committed to reversing the toll of this pandemic through financial support, political advocacy, and civic engagement [4]. With these efforts, 6.7 million people living with HIV in low-and middle-income countries now have access to lifesaving treatment, representing a 16-fold increase in 7 years [5]. The introduction of antiretroviral therapy (ART) has been estimated to have averted 2.5 million deaths in those settings [5].

This progress has been inconsistent across communities and countries. HIV remains under-diagnosed, many patients present late to care or not at all, some do not receive therapy despite clinical eligibility, and others do not remain in care over time [1, 6‐12]. Addressing these challenges will require provision of new services in addition to improvement of the quality of existing services. The latter suggests an opportunity to apply the tools of quality improvement (QI) championed in well-resourced nations [4, 13].

Several major global HIV donor and advocacy agencies have worked to consolidate performance indicators for HIV programs [1]. Increasing attention has been paid to the quality and impact of HIV services, and to greater alignment with host-country goals. However, several weaknesses remain. The strategic framework for monitoring and evaluation utilized by donor and advocacy agencies may not always be consistent with the improvement model where the focus of measurement is on process and outcome measures, and change is implemented at the smallest replicable unit – the microsystem of healthcare delivery. National and global indicators provide a broad picture of the scope, size, and impact of HIV services, and are important for strategic planning and resource allocation. Therefore, national and global indicators should not be replaced by program-level indicators. However, program-level indicators remain incomplete, and a carefully selected set of measures collected at 6 to 12-month intervals provide important data that can be disseminated to staff to guide QI efforts [1].

Among the available quality indicators for HIV, several critical points including linkage to care, ART eligibility, and ART preparations as well as important patient reported and clinical outcomes are either not represented, or are poorly represented. In addition, current indicator and reporting requirements are often decentralized and dependent on donor agencies [14]. In a study of data quality from the international ART-LINC group, 67% of sites received funding from two sources, and 24% from three [14]. In the same study, only 61% of programs used the same software for data collection and management, creating a gap between measurement, clinical care and improvement, while increasing the cost and complexity of monitoring and evaluation [14]. This study suggests that competing priorities from multiple funding sources create barriers to data collection on additional indicators that may be crucial to local improvement and underscores the importance of identifying a set of measures that will meet reporting, stakeholder, and local quality improvement needs.

The aim of this manuscript is to help care delivery programs better utilize quality improvement methods by providing them with a better ability to assess their own performance. More specifically, we propose a framework for considering performance measures based on the continuum of HIV care. We then identify where gaps exist between available measures and those measure concepts identified by our framework.

To accomplish this aim, we first provide an overview of quality improvement (QI) science in healthcare, highlighting important QI concepts. We then summarize the global HIV-specific QI measurement efforts in resource-limited settings utilized by three pivotal advocacy and donor agencies with broad international reach (UNAIDS, the Global Fund to fight AIDS, Tuberculosis, and Malaria, and the President’s Emergency Plan for AIDS Relief [PEPFAR]). We describe performance along the HIV care continuum, identifying potential foci for quality improvement, and gaps in current measurement efforts. Finally, we propose a set of quality indicators for HIV service delivery programs to address current measurement gaps along the HIV care continuum.

Borrowing from industrial quality control methods developed in the 1950’s and 1960’s, healthcare in resource-rich countries has seen rapid growth in the understanding and development of methods that improve quality [16]. Quality improvement relies both on a philosophy, the pursuit of continuous performance improvement, and a diverse armamentarium of methods. These methods include operations research, assessment of healthcare delivery processes, participative management, and setting of performance benchmarks. They also include determining best practices, often codified in practice guidelines, and measuring adherence to those best practices [17].

Central to achieving high quality of care is the definition, observation and measurement of key system indicators – the fundamental metrics in quality improvement measurement. Many healthcare QI experts support the Donabedian model, which suggests that indicators should assess the structures (characteristics of physical deployment of resources such as physicians, nurses, buildings and supplies), processes (what health professionals do with people), and outcomes of care (what happens to people, especially with regard to their health) [18]. These attributes of a system of care are not of equal value. Outcomes matter most, but are also the most challenging to measure. Process measures are a better indicator of healthcare quality than structural measures because they allow assessment of timeliness and appropriateness of care [19]. While individual indicators may reflect specific measures of importance, they may not in isolation lead to an understanding of the broader landscape and affect change that leads to improvement. To address this, the “balanced scorecard” approach was designed to merge a set of measures (including structures, processes and outcomes) across several domains (e.g., timeliness, cost-effectiveness, appropriateness) in a format that would enable managers to translate the organization’s mission into a specific subset of measurable objectives [20].

The development and selection of indicators is linked to data collection, implementation of targeted activities, and evaluation of their impact on health outcomes [16, 21]. Once identified, quality indicators should be evaluated for their importance, scientific acceptability, usability, and feasibility [22]. These measures can then be used at the program level (the point of service delivery) to assess healthcare quality and identify areas for improvement. This process is most effectively executed at the level of the microsystem – a small group of people who work together to provide care to patients; who best understand the challenges to improvement; and are responsible for implementing the changes that drive improvement [23]. For HIV treatment programs, such a team may consist of clinicians (physicians, nurses, pharmacists, aides, etc.), counselors, home-based care/outreach workers, and clerical/data entry staff.

The Joint United Nations Program on HIV/AIDS (UNAIDS), along with major funding bodies, has provided guidance and leadership to improve the capacity and quality of HIV care globally. UN member states pledged to regularly report progress on their commitment to reversing the spread of HIV/AIDS; in 2003, the UNAIDS Secretariat developed a set of core indicators to monitor that progress [1, 13]. The largest multilateral donor agency for the HIV response, the Global Fund to fight AIDS, Tuberculosis and Malaria (GFATM) was established to increase financing for these infections in 2002 [24]. The US President’s Emergency Plan for AIDS Relief (PEPFAR), the largest bilateral funding organization for the global HIV/AIDS pandemic, was launched by the US Congress in 2003 [25‐27].

UNAIDS, the Global Fund, and PEPFAR have adopted an overarching framework for the monitoring and evaluation of national HIV/AIDS programs in host countries. This framework relies on a logic model that measures inputs, activities, outputs, outcomes, and impacts, which provide information on what goes into a program and the results achieved [28]. The measures are indicators collected at the program, provincial, and local levels, and are aggregated to reflect the progress of national HIV/AIDS programs in meeting country, donor, and advocacy goals. They are multifunctional and provide guidance on the strategic planning, coordination and implementation of programmatic efforts for the global HIV/AIDS response, accountability to donor agencies, and assessments of program effectiveness [28‐31].

Despite attempts to align monitoring and evaluation efforts, the UN and others, recognizing a lack of uniformity in key system measures, published a set of 25 core indicators in 2007 on which all countries were to report (Tables 1) [1]. GFATM-supported programs report on a minimum set of 40 national indicators; 25 of these are from the UN General Assembly requirements, and an additional 15 indicators span antiretroviral therapy treatment programs, development of HIV workplace programs, and reduction of stigma (Tables 1) [28]. While some indicators were appropriate for program-level monitoring, many were not, and the Global Fund recognized that additional program-level indicators were needed [28].

Most recently, PEPFAR changed its monitoring and evaluation framework, calling for increased host country ownership, greater alignment of indicators with host country reporting needs, and inclusion of indicators measuring coverage and quality of services [31]. In addition, since 2006, PEPFAR has implemented a QI initiative, HEALTHQUAL, which helps participants measure key indicators and develop QI programs to meet improvement goals [26]. HEALTHQUAL programs are currently utilized in several PEPFAR countries and many of the indicators endorsed are program-level measures defined to meet program and country needs [26, 32].

Despite improved care resulting from scale-up of the global HIV/AIDS response, substantial morbidity and mortality can still be traced to inadequacies in care delivery at distinct points along the HIV care continuum (Figure 1). These critical points include diagnosis, linkage to care, initiation of antiretroviral therapy, treatment or prophylaxis of opportunistic infections, and retention in care over time [8, 33]. Successful patient outcomes are contingent upon a high degree of success at each and every point on this continuum [34, 35]. While barriers to optimal care are not yet fully elucidated, quality indicators focused on essential points along the care continuum will be necessary to identify weaknesses and design interventions to maximize performance. In the descriptions below we highlight the seven key process steps in the HIV care continuum; to: 1) summarize the evidence regarding performance at each step, and 2) present our assessment of the opportunities for performance measure development.

The set of quality indicators currently used to assess the treatment and care of HIV programs in resource-limited countries lacks focus on several areas most likely to improve HIV outcomes (Table 1) [11, 77, 103]. The tension between funder needs for accountability and strategic planning, and evaluation and program needs for local improvement, may be difficult to harmonize with a single set of indicators. Therefore, using the model in Figure 1 as a framework, we assessed the literature describing the operational challenges facing HIV care in resource-poor settings. Utilization of this framework, along with the selection of indicators measuring both processes of care and key clinical and patient reported outcomes guided the development of a set of 24 program-level measure concepts to guide improvement for HIV treatment programs (Table 2). The measure concepts were designed to address the limitations of current indicators described above, as well as Utilization of this framework, along with Utilization of this framework, along with the NQF criteria of importance, usability, and feasibility.

To assess HIV testing and diagnosis, we propose five process measures to illustrate where patients who successfully present to clinic receive their testing (clinic-based vs. home or mobile testing), and to inform whether partners of known HIV-infected patients are receiving testing. These data will allow HIV treatment programs to provide critical feedback about which local HIV testing modalities are most successful and may benefit from scale-up, in addition to which are least successful and could benefit from improvement efforts.

To determine program performance around linkage to HIV care, we recommend five process measures measuring both stage 1 (from receipt of a positive HIV test to receipt of CD4 count or clinical staging result and referral to ART or pre-ART care) and stage 2 (from referral to pre-ART care to ART eligibility) linkage as defined by Rosen et al., [44]. The most effective measurements of linkage to care require the merging of data from HIV testing and treatment centers, which are rarely available in resource-limited settings [44, 46]. Our proposed measures can be captured at the point of HIV care alone.

We propose six process and one outcome measure to assess program performance on it’s efficiency in determining ART eligibility and effectively initiating patients on ART. These include screening for opportunistic infections and nutritional needs in addition to ensuring adequate drug supply, measuring the time it takes to identify and initiate ART-eligible patients on life-saving therapy, and concordance with local ART monitoring guidelines. The proposed outcome measure provides programs with important data on potentially preventable deaths of ART-eligible patients who die before starting therapy.

Finally, we outline five outcome and one process measure addressing retention in care, patient health status, and important clinical outcomes (virologic suppression, use of second-line therapy, and mortality). Our measures of care retention, in addition to assessing this as an outcome, also necessitate that HIV care delivery programs ask whether they have intervened to decrease LTFU and improve retention. This set of measures encompasses data on endpoints important to patients, clinicians, and administrators.