http://orcid.org/0000-0003-1853-728X
Figures
Abstract
Background
Results-based aid (RBA) is increasingly used to incentivize action in health. In Mesoamerica, the region consisting of southern Mexico and Central America, the RBA project known as the Salud Mesoamérica Initiative (SMI) was designed to target disparities in maternal and child health, focusing on the poorest 20% of the population across the region.
Methods and findings
Data were first collected in 365 intervention health facilities to establish a baseline of indicators. For the first follow-up measure, 18 to 24 months later, 368 facilities were evaluated in these same areas. At both stages, we measured a near-identical set of supply-side performance indicators in line with country-specific priorities in maternal and child health. All countries showed progress in performance indicators, although with different levels. El Salvador, Honduras, Nicaragua, and Panama reached their 18-month targets, while the State of Chiapas in Mexico, Guatemala, and Belize did not. A second follow-up measurement in Chiapas and Guatemala showed continued progress, as they achieved previously missed targets nine to 12 months later, after implementing a performance improvement plan.
Conclusions
Our findings show an initial success in the supply-side indicators of SMI. Our data suggest that the RBA approach can be a motivator to improve availability of drugs and services in poor areas. Moreover, our innovative monitoring and evaluation framework will allow health officials with limited resources to identify and target areas of greatest need.
Citation: Mokdad AH, Palmisano EB, Zúñiga-Brenes P, Ríos-Zertuche D, Johanns CK, Schaefer A, et al. (2018) Supply-side interventions to improve health: Findings from the Salud Mesoamérica Initiative. PLoS ONE 13(4): e0195292. https://doi.org/10.1371/journal.pone.0195292
Editor: Emily Vala-Haynes, Western Oregon University, UNITED STATES
Received: June 21, 2016; Accepted: March 20, 2018; Published: April 16, 2018
Copyright: © 2018 Mokdad et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are owned by the Inter-American Development Bank. SMI baseline survey data, which comprises the minimal dataset, are publicly available on the SMI website: http://www.saludmesoamerica2015.org/es/iniciativa-salud-mesoamerica/la-iniciativa/open-data,20379.html. SMI follow-up survey data are currently available upon written request with the Inter-American Development Bank whose contact information is listed below. Jennifer A. Nelson, MPH Learning, Innovation, and Technology Officer (jennifern@iadb.org) Salud Mesoamérica Initiative Calle 50, Edificio Tower Financial Center (Towerbank) – Piso 23 Panamá, Panamá.
Funding: This study was funded by the Bill & Melinda Gates Foundation, the Spanish Agency for International Development Cooperation, and the Carlos Slim Foundation, through the Inter-American Development Bank.
Competing interests: The authors declare that they have no competing interests. The Inter-American Development Bank is the largest source of development financing for Latin America and the Caribbean. It is not a commercial bank and all information about the Bank activities is available on www.iadb.org. SMI was funded by Bill and Melinda Gates Foundation, Carlos Slim Foundation, and the Spanish Government.
Introduction
Mesoamerica, the region consisting of southern Mexico and Central America, has made significant progress in population health over the past decade, though substantial inequalities in health outcomes, access, and quality of medical care remain between and within countries [1–5]. In these areas, vulnerable groups such as poor, indigenous, and rural populations have considerably worse health outcomes than national or regional averages [6–16].
The Salud Mesoamérica Initiative (SMI) was established to address the health issues faced by the poorest quintile of the population in El Salvador, Guatemala, Honduras, Nicaragua, Belize, Costa Rica, Panama, and the State of Chiapas, Mexico. Administered by the Inter-American Development Bank (IDB), SMI uses a results-based aid (RBA) approach to improve maternal and child health across the region. The initiative is a public-private partnership including the Bill & Melinda Gates Foundation, the Carlos Slim Foundation, Spain’s Cooperation Agency for International Development, and ministries of health in each of the eight participating countries.
RBA and results-based financing (RBF) are financial assistance models involving contractual arrangements between a principal and an agent and involve the transfer of funds in exchange for delivery of pre-specified performance targets or results [17–20]. The approaches differ by the defined source of funding and contractual partners involved. In typical RBF for health, contracts are executed between a government and an implementing partner at the sub-national level and financial incentives are often applied at the facility level. The RBA model includes a financial assistance contract between a donor and a national government, where either a portion of or all donor funds are disbursed based upon performance or predetermined targets [17–21].
Very few RBA models for health exist and due to its novelty, success has not yet been confirmed. The evaluation of the Gavi Immunization Services Support (ISS) grants in Cambodia, Laos, Tanzania, Zambia, Democratic Republic of the Congo, and Guinea cites an association between ISS funding and immunization coverage rates but provides limited data to assess the effectiveness of the RBA component of the program [22]. In this case, it is not possible to ascertain the direct effect of the RBA given the lack of specific controls in the evaluation design. A literature review of major results-based financial assistance schemes suggest that they generally appear to deliver results, but that there is extremely weak evidence to attribute results specifically to the RBA/RBF component [17].
Several studies have evaluated RBF programs in the health sector and have provided mixed results [23–34]. Systematic reviews cite some RBF evaluations which present no effect or suggest potential for negative outcomes, such as perverse incentives, gaming, and technical difficulties [17,23]. At the same time, many evaluations suggest promising results, implying that RBF can be an important tool for achieving health targets as well as health sector reform more broadly, if designed carefully [23–25].
Key factors have been identified for achieving success in results-based programs, including political commitment and a participatory approach, clearly defined rules, indicators, and targets, strong verification systems, and robust measuring and evaluation [23,24,34]. Improvements in health service utilization and perceived quality of care indicators were observed from RBF programs in Burundi, the Democratic Republic of the Congo, Tanzania, and Zambia [29]. Engagement with local and national health management from the onset of a program is cited as a crucial component for RBF; programs that did not deploy such an approach were less successful in strengthening health systems [29]. Furthermore, the evaluation of an RBF program to increase the use and quality of child and maternal care services in Rwanda found that within 23 months, the program led to increased use and quality of health services [30].
The design of SMI as an RBA is different from previous studies. Participating countries receive 50% of the cost of program intervention from funders and contribute the remaining 50% themselves. At the end of each operation, pre-defined performance indicators are measured independently, and if 80% of these indicators are met, the country is awarded half of its contribution share back. Funds awarded to recipient countries for meeting targets can be spent on other health programs at the country’s discretion. SMI’s design enables IDB to dispense these pre-agreed funds to countries based on their compliance with country-specific health targets in selected geographic areas. SMI uses this RBA approach and delivers integrated evidenced-based supply- and demand-side interventions to improve maternal and child health for the poorest quintile of the population across Mesoamerica.
In this manuscript, we describe the improvement from the baseline to the first follow-up supply-side measurements of SMI.
Methods
The SMI methodology was previously published [35,36]. Briefly, the IDB worked with countries to select the poorest regions for the intervention. The selection was based on national poverty measures, roughly equivalent to the bottom 20% of the income or wealth distribution. A scale was created to order health jurisdictions and municipalities according to the country-defined poverty metrics and to select municipalities eligible for intervention from the poorest quintile. Selection from these groupings of municipalities also considered additional criteria, such as the presence of gaps in health service coverage, the existence of ongoing health reforms, and the presence of complementary financing by other donors and the government. In each country, SMI is implementing two to three consecutive projects. Each project includes a set of performance (payment) indicators and targets that were set with governments, in line with country-specific priorities in maternal and child health. Key indicators include coverage of contraceptives, antenatal and postnatal care for women and newborns, in-facility delivery and skilled birth attendance, management of maternal and neonatal complications, complete vaccination coverage for age, prevalence of anemia in children, and quality of care for antenatal, delivery, postnatal, and child health care visits. As part of establishing the baseline performance for SMI indicators, surveys were conducted in households and health facilities in each country.
Written informed consent was obtained from all participants. The study received approval from the institutional review board (IRB) from the University of Washington, partnering data collection agencies, and the Ministry of Health in each country.
Interventions included implementing the Essential Obstetric and Neonatal Care (EONC) strategy, strengthening referral networks, improving the supply chain, encouraging the cultural adaptation of services for indigenous populations, supporting new service delivery platforms and community platforms, and the design and approval of updated country norms and protocols, among other activities. SMI design calls for follow-up measurements at the close of each of three intervention periods in each country to capture the impact of the interventions. Depending on whether targets for improvement are met at each of these critical junctures, countries will be reimbursed with funds corresponding to half of the counterpart investment that they had to put forward up front, to be used freely within the health sector.
SMI dispensed about $23 million for the first operation, with similar matching funds coming from the participating countries (S1 Table). A tranche equal to half of the government contribution was awarded if a country met the 18-month targets.
Indicator targets were set through negotiation with the countries’ health officials and IDB based on literature reviews of intervention effectiveness from previous country-level studies, trend analysis using data from the Global Burden of Disease 2010 study [2, 37–41], expert advice, and a cost-benefit model developed by IDB. Several health facility indicators were used by SMI and measured at the baseline and 18-month junctures (S2 Table). Performance indicators for payment varied in their requirements by country (S3 Table shows the most comparable ones). For example, the indicator related to the supply of antenatal and postnatal care, “health facilities with permanent availability of inputs and equipment necessary for prenatal and postpartum care” had five components in El Salvador compared to 29 in Belize due to negotiation and agreement between each country and IDB. In order for a country to “pass,” 80% of facilities must meet the indicator targets. For a facility to be declared as having met an indicator, every single component of that indicator must be met. For the 18-month measurement, countries committed to a predetermined target for each supply-side performance indicator. Targets were set specific to each country and were quantified as a percentage of facilities complying with all indicator components.
For each round, we selected a random sample of the facilities to be representative. All facilities were eligible for inclusion in each round. We conducted a third round of measurement for Chiapas and Guatemala, for the performance indicators only, after both implemented a performance improvement plan, given that they failed to achieve their targets at 18 months but they showed improvement. We only measured the performance indicators during the repeat to reduce the cost of the measurement. Though Belize also did not meet 18-month targets, a third round measurement was not conducted due to the limited SMI investment funds in Belize, as detailed in S1 Table.
The health facility surveys at baseline and 18 months collected data on facility conditions, service provision and utilization, and quality of care. The survey involved three main components: an interview questionnaire, an observation checklist, and medical record reviews (MRRs). Health facilities were grouped according to three EONC levels—ambulatory, basic, and complete—as provided by SMI. Mainly, ambulatory level facilities have the capacity to provide outpatient care only; basic-level facilities are able to attend uncomplicated vaginal deliveries and provide immediate emergency obstetric and neonatal care; and complete-level facilities have a surgery room and the capacity to attend most obstetric and neonatal complications (not including intensive care). Different health facility indicator criteria were assessed depending on the EONC classification level. In the interview questionnaire, the facility director, manager, or other person in charge of the health facility was interviewed to capture information on general facility characteristics, infrastructure, human resource composition and trainings, supply logistics, infection control, child health care provision, vaccine availability, family planning service provision, availability of contraceptives, and antenatal, delivery, and postpartum care. Once these were completed, surveyors used an observation checklist to record direct observations of the availability and functionality, as applicable, of essential equipment and supplies for maternal and child health care, including pharmaceuticals. Surveyors also reviewed administrative records of pharmaceutical stocks in this module, capturing drug and contraceptive stock-outs occurring in the three months prior to the date of the survey.
We used MRRs to capture retrospective data on record-keeping and treatment practices of surveyed facilities. MRRs were selected at random from each facility using the records of encounters and applying systematic sampling to cover the entire study period. The MRRs covered various medical complications that mothers and infants experienced during delivery and how each sampled case was treated at a given health facility. The MRRs also captured the facilities’ medical practices before, during, and after uncomplicated births. Depending on the country, other MRRs on diarrhea, pneumonia, low birth weight, child registration and growth charting, deworming, and family planning services offered were also implemented. Indicators from MRRs were collected for monitoring purposes during the 18-month measurement but will be included as performance- or payment-based indicators in the subsequent evaluation.
The SMI surveys were conducted using a computer-assisted personal interview (CAPI), and incoming data were continuously monitored by the Institute for Health Metrics and Evaluation (IHME). We conducted training sessions and pilots in each country before implementation. We also conducted supervisory visits during health facility data collection. We used Stata 13.1 for the analyses and to account for the complex study design. Baseline surveys were conducted from March 1, 2011, to August 31, 2013, while the 18-month follow-up surveys were conducted between January 20, 2014, and October 24, 2014. Dates of the surveys were designed to ensure an 18-month period between the baseline and the follow-up for each country. Performance improvement plan measurements (PIPM) in Chiapas and Guatemala were conducted between May 11, 2015, and June 25, 2015. Additional detail on SMI methodology and implementation is available elsewhere [35].
Results
In total, 365 health facilities were interviewed for the baseline measurement, and 6,332 medical records were reviewed, compared to 368 and 11,143, respectively, for the 18-month follow-up (Table 1). Moreover, 30 facilities were measured as part of the performance improvement plan nine to 12 months after the 18-month follow-up in Guatemala and Chiapas.
Table 2 shows the progress in selected payment indicators from baseline to 18 months and related indicator targets. All countries showed progress in the indicators, although with different levels. El Salvador, Honduras, Nicaragua, and Panama reached their 18-month targets, while Chiapas, Guatemala, and Belize did not. The third measurement showed that both Guatemala and Chiapas reached their 18-month targets nine to 12 months later.
All countries showed improvement in SMI indicators from the baseline to the 18-month measurements (S1 Appendix). In total, 448 individual components were measured for the 18-month performance indicators for the seven countries, and only 47 showed a decline, 72 showed no change, and 279 improved from the baseline (Table 3). Fig 1 shows a heat map of the improvement in selected components of the SMI indicator measuring the availability of equipment, pharmaceuticals, and lab inputs necessary for antenatal and postpartum care. Fig 1 details, by country and measurement round, the overall percentage of facilities that had specific components of the antenatal and postnatal care indicator. A clear improvement was observed in the availability of supplies, in all countries.
When we examined gains in equipment and supplies by health facilities, progress had been achieved in 18 months (Figs 2 and 3). Most facilities that did not meet the goal and qualify for disbursement of the performance tranche were missing only one or two components of complex indicators.
Discussion
To our knowledge, this is the largest RBA project to improve supplies and services at health facilities in poor areas of Mesoamerica. Our data show improvement in the availability of supplies and medicines at health facilities in a short time. SMI data suggest that the RBA approach is a motivator to improve availability of drugs and services in poor areas. Our results are encouraging for public health as this improvement happened in poor and remote areas, indicating that lessons learned from SMI could be applied in other regions of these countries and globally.
SMI is a very ambitious project, and the targets set for each country were strict. For a facility to pass, it had to have all components of an indicator observed or achieved. The rule for disbursing the performance tranche was designed as a pass or fail, irrespective of the level of improvement in each facility, and was designed to offer a reward for extraordinary effort in a short time. Belize, Guatemala, and Chiapas did not pass the 18-month measurement, while Panama, Nicaragua, Honduras, and El Salvador did. The design of SMI ensures that the intervention continues in all countries, even though some countries did not receive a performance tranche at 18 months. However, improvement in supplies and equipment was observed in the three countries that did not meet their targets at the 18-month juncture. Our data show that most facilities in these countries did not pass because they were missing only one or two components of complex indicators. When both Guatemala and Chiapas were given an extra nine to 12 months to implement a performance improvement plan, they were able to meet all targets.
Although RBA/RBF is increasingly used to incentivize action in health, most results-based evaluations provide evidence for schemes that apply financial incentives at the health facility or health provider level. Less evidence is available on the impact of RBA approaches that provide incentives at the national level, such as the Gavi ISS program [22].
Our supply-side results add to the limited available evidence of the effect of RBA schemes that apply financial incentives at the national level. Our study shows an improvement in the availability of supplies and equipment at facilities in 18 months, though we are not yet able to show an increase in uptake of services by the population, since the SMI measurement calls for another household survey in 2017–2018. However, increasing availability of supplies and equipment has been associated with increases in uptake previously [42]. Subsequent rounds of evaluation of SMI will provide more robust evidence to this area of study. Continued rigorous evaluation is needed to assess successes and failures in RBA schemes where financial incentives are applied at the national level.
SMI raised the bar on the requirement of supplies and equipment in poor and remote areas. These demanding indicators should be maintained and used in future SMI follow-up measurements. For example, the presence of stethoscopes was a requirement, though many facilities that reported not owning a stethoscope were relying on facility staff’s personal stethoscopes. A strong argument could be made that stethoscopes should be available at the facilities since doctors or other facility staff may be absent on certain days (vacation or illness). Moreover, the improvement in such a strict and ambitious intervention in poor and remote areas is encouraging and suggests that such an approach could easily be adopted in other parts of each country. Showing decision-makers and health authorities that success can be achieved through proper strategies can ensure improvement in health elsewhere.
Our findings showed that countries had more success with certain components of an indicator. This was not surprising given the number of components and indicators we have in our evaluations. Our ability to provide each country with a detailed report on what are the missed targets and in what facilities will enable them to address these shortages. Moreover, our approach for evaluation could be used by countries to track their own progress in future years.
SMI provided a platform for action to improve health, giving all countries technical assistance to help remove bottlenecks, improve processes, analyze failures, and adjust strategies. Challenges and solutions for overcoming them should be documented, as should failures, to ensure that they are not repeated. After Chiapas and Guatemala did not meet targets and were given the external measurement results, they proved through the PIPM to be able to mobilize quickly in order to supply and equip facilities that were lacking. There is a need to include a qualitative component to this evaluation to complement the quantitative findings in order to better understand the drivers of success. Indeed, finding a way for people in remote areas to communicate their experience and provide the lessons learned is essential. Our findings call for creating a network of local health officials and residents to exchange ideas through the implementation of SMI.
Several factors have to be considered in SMI. First, as supplies increase in one area of the community (intervention areas), it is possible that the demand would increase from the community and neighboring ones, especially if they are not part of the intervention. Previous studies have shown that individuals may travel a longer distance to seek care at a better-equipped facility. Hence, these facilities could face a challenge in supplies and time going forward. Second, it is possible that the attention to comply with targets in the intervention areas has led to diverting resources to these areas at the expense of others, although we have no evidence that this occurred. Still, both of these scenarios have to be examined to ensure continued success.
Finally, a major concern for SMI is sustainability. Although the governments have agreed to provide funds to sustain the programs and intervention, it is possible that when donor investment and reimbursement end and the measurements stop things may change. A challenge for SMI is how to maintain these improvements once the funds from IDB are dispensed and IHME monitoring is complete. The sustainability of programs has been a major challenge in global health. However, we have already observed that governments have started their own monitoring of supplies and equipment. In fact, our interviewers in Chiapas and Guatemala documented how each component of country-specific indicators was numbered and labeled at some health facilities. Moreover, we hope that supply-side improvements will generate health-seeking behaviors and public and official awareness of the need to continue and maintain these achievements. Indeed, the population’s expectations will result in increased demands for these supplies and availability at their health facilities.
Our study has several limitations. First, we did not conduct a household survey with the health facility survey this measurement round, and thus were not able to examine the change in demand. However, this is a planned activity for the next round of the evaluation. Second, we did not conduct a qualitative assessment in all countries to explain our quantitative results and identify the drivers or obstacles for change. Indeed, our qualitative work in Chiapas showed that the RBA model created a culture of accountability, which was strengthened following the failure to meet targets during the first measurement round. With the improvement plan, accountability trickled down to the level of health workers, who developed the habit of routinely checking supplies and ensuring shortages were avoided [36]. Third, it is possible that we have some self-reporting bias in our survey. However, this does not affect most of our indicators, which are based on observation and MRRs. Finally, we did not examine indicators in control areas during this round of health facility surveys to see if the same improvements in supplies occurred outside our study area. However, subsequent rounds of measurement include data collection activities in both our comparison and study areas.
Our study showed that when local staff, central health, and federal officials set a priority and are given a monetary and a professional incentive, positive changes could occur in a short time and in a challenging environment. We found that this type of RBF is the right approach to improve health in Mesoamerica. Our findings have great implications on health funding for reducing the burden of diseases.
Supporting information
S1 Table. SMI first and second operation resources*.
https://doi.org/10.1371/journal.pone.0195292.s001
(DOCX)
S2 Table. SMI 18-month performance indicators by country.
https://doi.org/10.1371/journal.pone.0195292.s002
(DOCX)
S3 Table. SMI selected 18-month performance indicators and requirements by country.
https://doi.org/10.1371/journal.pone.0195292.s003
(DOCX)
S1 Appendix. Improvements in supplies and equipment from baseline to 18-month follow-up, by country.
https://doi.org/10.1371/journal.pone.0195292.s004
(DOCX)
Acknowledgments
We thank central and local governments for the support they extended to the study teams and their facilitation of access to communities and health facilities. We thank all the agencies, teams, and supervisors who conducted data collection, including El Colegio de la Frontera Sur-Mexico, UNIMER, and the University of Belize. We also thank Adrienne Chew at the Institute for Health Metrics and Evaluation for editing this manuscript.
References
- 1. Wang H, Liddell CA, Coates MM, Mooney MD, Levitz CE, Schumacher AE, et al. Global, regional, and national levels of neonatal, infant, and under-5 mortality during 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. The Lancet. 2014 Sep; 384(9947):957–979.
- 2. Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. The Lancet. 2012 Dec; 380(9859):2095–2128.
- 3. Kassebaum NJ, Bertozzi-Villa A, Coggeshall MS, Shackelford KA, Steiner C, Heuton KR, et al. Global, regional, and national levels and causes of maternal mortality during 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. The Lancet. 2014 Sep; 384(9947):980–1004.
- 4. Braveman P, Tarimo E. Social inequalities in health within countries: not only an issue for affluent nations. Soc Sci Med. 2002 Jun; 54(11):1621–1635 pmid:12113445
- 5. Victora CG, Wagstaff A, Schellenberg JA, Gwatkin D, Claeson M, Habicht J-P. Applying an equity lens to child health and mortality: more of the same is not enough. The Lancet. 2003 Jul; 362(9379):233–241.
- 6.
Asociación Demográfica Salvadoreña (ADS), Division of Reproductive Health-Centers for Disease Control and Prevention (CDC). El Salvador Reproductive Health Survey 2008. San Salvador, El Salvador: ADS; 2009.
- 7.
Costa Rica Ministry of Health: Informe de Los Resultados de La Encuesta de Salud Sexual Y Reproductiva 2010. San José, Costa Rica: Ministry of Health (Costa Rica); 2011.
- 8.
Costa Rican Institute for Research and Education on Nutrition and Health, Costa Rican Social Security Fund (CCCS), Institute of Nutrition of Central America and Panama, Ministry of Health (Costa Rica), National Institute of Statistics and Censuses (Costa Rica): Costa Rica National Nutrition Survey 2008–2009; 2009.
- 9.
Directorate of Statistics and Census (Panama), Ministry of Economy and Finance (Panama), World Bank: Panama Living Standard Measurement Survey 2008. Washington DC, United States: World Bank; 2008.
- 10.
Guatemala Ministry of Health and Social Assistance, University of Valle and Division of Reproductive Health-Centers for Disease Control and Prevention (CDC): Guatemala Ministry of Health and Social Assistance, University of Valle and Division of Reproductive Health-Centers for Disease Control and Prevention (CDC). Atlanta, United States: Centers for Disease Control and Prevention (CDC); 2009.
- 11.
ICF International, National Institute of Statistics (Honduras). Honduras Demographic and Health Survey 2011–2012. Tegucigalpa, Honduras: ICF Macro; 2013.
- 12.
Instituto Conmemorativo Gorgas de Estudios de la Salud (Panama). Panama National Survey of Sexual and Reproductive Health 2009. Panama City, Panama: Instituto Conmemorativo Gorgas de Estudios de la Salud; 2010.
- 13.
Ministry of Health (Nicaragua), National Institute for Development Information (Nicaragua): Nicaragua National Demographic and Health Survey 2011–2012. Managua, Nicaragua: National Institute for Development Information (Nicaragua); 2013.
- 14.
National Institute of Public Health (Mexico): Mexico National Survey of Health and Nutrition 2011–2012. Cuernavaca, Mexico: National Institute of Public Health (Mexico); 2012.
- 15.
National Institute of Statistics and Censuses (INEC) (Costa Rica): Costa Rica Multipurpose Household Survey 2009. San José, Costa Rica: National Institute of Statistics and Censuses (INEC) (Costa Rica); 2009.
- 16.
Statistical Institute of Belize, United Nations Children’s Fund (UNICEF): Belize Multiple Indicator Cluster Survey 2006. New York, United States: United Nations Children’s Fund (UNICEF); 2006.
- 17.
Pearson M, Johnson M, Ellison R. Review of major Results Based Aid (RBA) and Results Based Financing (RBF) schemes [Internet]. London: HLSP Institute; 2010 [cited 2018 Jan 2]. https://www.oecd.org/dac/peer-reviews/Review-of-Major-RBA-and-RBF-Schemes.pdf
- 18.
Klingebiel S. Results-based aid (RBA) and its application to promote good governance [Internet]. (German Development Institute; 2012 [cited 2018 Jan 2]. https://www.cgdev.org/doc/Initiatives/RBA_Publication_2012_04_11.pdf
- 19.
Klingebiel S. Results-Based Aid (RBA): new aid approaches, limitations and the application to promote good governance [Internet]. Bonn: German Development Institute; 2012 [cited 2018 Jan 2]. https://www.die-gdi.de/uploads/media/DP_14.2012.pdf
- 20.
Grittner AM. Results-based financing: evidence from performance-based financing in the health sector [Internet]. Bonn: German Development Institute; 2013 [cited 2018 Jan 2]. https://www.oecd.org/dac/peer-reviews/Results-based-financing.pdf
- 21.
Perakis R, Savedoff W. Does Results-Based Aid Change Anything? Pecuniary Interests, Attention, Accountability and Discretion in Four Case Studies: Policy paper 052 [Internet]. Center for Global Development, Washington D.C.; 2015 Feb [cited 2018 Jan 2]. https://www.cgdev.org/publication/does-results-based-aid-change-anything-pecuniary-interests-attention-accountability-and
- 22.
Grace Chee NH. GAVI Immunization Services Support Funding: Evaluation of the First Five Years. 2007.
- 23. Oxman AD, Fretheim A. Can paying for results help to achieve the Millennium Development Goals? Overview of the effectiveness of results-based financing. J Evid Based Med. 2009 May; 2(2):70–83. pmid:21348993
- 24. Beane CR, Hobbs SH, Thirumurthy H. Exploring the potential for using results-based financing to address non-communicable diseases in low- and middle-income countries. BMC Public Health. 2013 Feb 1; 13(1):92.
- 25. Meessen B, Soucat A, Sekabaraga C. Performance-based financing: just a donor fad or a catalyst towards comprehensive health-care reform? Bull World Health Organ. 2011 Feb 1; 89(2):153–156. pmid:21346927
- 26. Fox S, Witter S, Wylde E, Mafuta E, Lievens T. Paying health workers for performance in a fragmented, fragile state: reflections from Katanga Province, Democratic Republic of Congo. Health Policy Plan. 2014 Jan; 29(1):96–105. pmid:23325585
- 27. Freedman LP, Graham WJ, Brazier E, Smith JM, Ensor T, Fauveau V, et al. Practical lessons from global safe motherhood initiatives: time for a new focus on implementation. The Lancet. 2007 Oct; 370(9595):1383–1391.
- 28. Honda A. 10 best resources on … pay for performance in low- and middle-income countries. Health Policy Plan. 2012 Sep 7; czs078.
- 29.
WHO | Learning lessons on implementing performance-based financing, from a multi-country evaluation [Internet]. WHO. [cited 2015 Nov 3]. http://www.who.int/health_financing/documents/performance_based_financing_synthesis_report/en/.
- 30. Basinga P, Gertler PJ, Binagwaho A, Soucat AL, Sturdy J, Vermeersch CM. Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. The Lancet. 2011 Apr; 377(9775):1421–1428.
- 31. Meessen B, Shroff ZC, Ir P, Bigdeli M. From Scheme to System (Part 1): Notes on Conceptual and Methodological Innovations in the Multicountry Research Program on Scaling Up Results-Based Financing in Health Systems. Health Syst Reform. 2017 Apr 3;3(2):129–36.
- 32. Das A, Gopalan SS, Chandramohan D. Effect of pay for performance to improve quality of maternal and child care in low- and middle-income countries: a systematic review. BMC Public Health. 2016 Apr 14; 16:321. pmid:27074711
- 33. Blacklock C, MacPepple E, Kunutsor S, Witter S. Paying for Performance to Improve the Delivery and Uptake of Family Planning in Low and Middle Income Countries: A Systematic Review. Studies in Family Planning. 2016 Nov 17; 47: 309–324. pmid:27859313
- 34. Renmans D, Holvoet N, Criel B, Meessen B. Performance-based financing: the same is different. Health Policy Plan. 2017 Jul 1;32(6):860–8. pmid:28369426
- 35. Mokdad AH, Colson KE, Zúñiga-Brenes P, Ríos-Zertuche D, Palmisano EB, Alfaro-Porras E, et al. Salud Mesoamérica 2015 Initiative: design, implementation, and baseline findings. Population Health Metrics [Internet]. 2015 Dec [cited 2015 Feb 20];13(1). Available from: http://www.pophealthmetrics.com/content/13/1/3.
- 36. El Bcheraoui C, Palmisano EB, Dansereau E, Schaefer A, Woldeab A, Moradi-Lakeh M, et al. Healthy competition drives success in results-based aid: Lessons from the Salud Mesoamérica Initiative. PLoS ONE. 2017 Oct 27; 12(10).
- 37. Lim SS, Vos T, Flaxman AD, Danaei G, Shibuya K, Adair-Rohani H, et al. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. The Lancet. 2012 Dec; 380(9859):2224–2260.
- 38. Murray CJL, Vos T, Lozano R, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. The Lancet 2012; 380: 2197–2223.
- 39. Salomon JA, Wang H, Freeman MK, et al. Healthy life expectancy for 187 countries, 1990–2010: a systematic analysis for the Global Burden Disease Study 2010. The Lancet 2012; 380: 2144–2162.
- 40. Vos T, Flaxman AD, Naghavi M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. The Lancet 2012; 380: 2163–2196.
- 41. Wang H, Dwyer-Lindgren L, Lofgren KT, Rajaratnam JK, Marcus JR, Levin-Rector A, Levitz CE, Lopez AD, Murray CJ: Age-specific and sex-specific mortality in 187 countries, 1970–2010: a systematic analysis for the Global Burden of Disease Study 2010. The Lancet 2012, 380:2071–2094
- 42. Jacobs B, Ir P, Bigdeli M, Annear PL, Damme WV. Addressing access barriers to health services: an analytical framework for selecting appropriate interventions in low-income Asian countries. Health Policy Plan. 2012 Jul 1; 27(4):288–300.43. pmid:21565939