A common monitoring framework for ending preventable maternal mortality, 2015–2030: phase I of a multi-step process

In September 2015, the era of the Millennium Development Goals (MDGs) came to an end, and while the worldwide maternal mortality ratio (MMR) declined by 44 % from 1990 to 2015, the global target of a 75 % reduction was not reached. In fact, the majority of countries fell short of attaining MDG5 targets for maternal mortality reduction. Maternal mortality remains unacceptably high with approximately 303,000 maternal deaths occurring each year, with the largest burden in Sub-Saharan Africa and Asia [1].

Beginning in January 2013, the Ending Preventable Maternal Mortality (EPMM) Working Group, led by the World Health Organization (WHO) with support from partner organizations, achieved multi-stakeholder consensus on goals for maternal health and survival from 2015 to 2030.

The EPMM targets for maternal mortality reduction at the global and country levels are:

A comprehensive paper outlining the Strategies toward Ending Preventable Maternal Mortality (EPMM) was released by WHO in February 2015 [2]. This was followed by a synthesis of priorities for ending preventable maternal and newborn deaths and stillbirths in a series of technical papers in support of the updated United Nations Global Strategy for Women’s, Children’s, and Adolescent’s Health (2016–2030) [3].

The EPMM global MMR target was incorporated into the Sustainable Development Goals (SDGs) adopted by member states and launched in late 2015 [4]. The SDGs form the basis of a new global development agenda, which is broad and comprehensive, covering a wide range of social, economic and environmental goals. The EPMM global target was also included in the updated UN Global Strategy.

To achieve the ambitious SDG and Global Strategy MMR targets, it will be imperative to accelerate coverage of quality essential services and to address the underlying social, political and economic determinants of maternal health across all settings. To support implementation of EPMM strategies and to track progress toward global and country MMR targets, consensus on priority, methodologically robust maternal health (MH) indicators is urgently needed. To this end, in May 2015, the EPMM Working Group specified a plan to develop a comprehensive maternal health global monitoring and reporting framework in two phases:

This paper describes the process and results of Phase I – to develop a core set of maternal health indicators for global monitoring and reporting by all countries. This core set of maternal health indicators will be harmonized with Every Newborn Action Plan (ENAP) core metrics to create a joint global maternal and newborn monitoring framework [5].

The World Health Organization, Maternal Health Taskforce (MHTF), and the US Agency for International Development (USAID) along with its flagship Maternal and Child Survival Program (MCSP), facilitated an iterative consultative process to reach consensus on a core set of maternal health indicators for global monitoring and reporting under Phase I on behalf of the EPMM Working Group. Key phases of the process led by an EPMM-designated steering committee included: (1) structured mapping exercise to identify indicators for monitoring and evaluation of maternal health; (2) development of key principles, criteria, and consultative process for selection of EPMM phase I indicators; (3) selection of experts/stakeholders to participate in consultative process in line with agreed criteria; (4) facilitation of a series of virtual consultations via webinar, phone and in writing to rank and prioritize indicators based on a set of pre-determined questions and criteria; and (5) an in-person one-day technical meeting with identified experts/stakeholders to reach consensus on a core set of MH indicators for global monitoring (Fig. 1).

The EPMM-designated steering committee included six persons selected to represent a range of expertise, including technical, policy and measurement experts, who were tasked with overseeing the process of defining a core set of global maternal health indicators (Table 1). The Steering Committee identified four principles to guide development and prioritization of a set of core MH indicators for global monitoring and reporting. First, the scope was limited to priority indicators appropriate for global monitoring and reporting by all countries. It was agreed that the primary purpose of the indicators would be to track progress toward maternal mortality reduction at the global level, with no more than 10 to 12 indicators in total. It was acknowledged that complementary (and sometimes overlapping) indicators would be needed to support effective program management at sub-national and service delivery levels but that these indicators were beyond the scope of the EPMM global indicator exercise. During the process, indicators for other levels of monitoring and reporting were systematically captured and recorded for future research and development, building on numerous ongoing indicator development efforts. A second guiding principle was to limit Phase I indicators to measures directly linked to the proximal causes of maternal death to mirror the core set of newborn health metrics outlined in the Every Newborn Action Plan. More distal determinants, such as social, economic, political, and health systems factors, will be developed under Phase II of the EPMM process.

The third agreed principle was to use the same organizational categories of these ENAP metrics, to facilitate a harmonized global maternal and newborn monitoring framework. Thus it was agreed to retain the ENAP metrics organizational categories of impact, coverage and input indicators. Finally, it was assumed that indicators would be reported at least every 5 years to track progress toward global targets.

In parallel, the steering committee finalized indicator selection criteria, drawing from criteria used in other indicator prioritization activities (e.g. WHO 100 core indicators [6]). Selection criteria included: (1) relevance for directly reducing preventable mortality; (2) validity (measures what it is supposed to measure); (3) feasibility (feasible to measure routinely through household surveys, health facility assessments, routine information systems); (4) degree of data availability across countries; and (5) strengthens or complements existing monitoring frameworks (MDGs, SDGs; ENAP; and other global efforts (e.g. Global Strategy for Women's, Children's, and Adolescent's Health; WHO Quality of Care Initiative, etc.)). The WHO indicator mapping exercise (phase one above) included searching online databases, reviewing program documents and other global monitoring frameworks, and discussions with implementing partners and donor agencies. An initial expert consultation was conducted in May 2015. Applying these principles and selection criteria, the Steering Committee compiled the indicators prioritized during this consultation, as well as priority indicators for tracking SDGs, and other indicator mapping efforts. This list was refined and updated, which resulted in a final list of 20 to 25 indicators.

In the second phase a series of virtual consultations was conducted with selected expert participants. Key stakeholders and experts were identified through review of participant lists from other indicator/measurement consultations, discussions with the broader EPMM Working Group, and other relevant stakeholders. The focus was on identifying participants who work directly in measurement of maternal health including clinicians, researchers, program implementers, global policy makers and representatives from Ministries of Health, UN agencies, and donors. (Table 2). Based on the structured indicator review and input from measurement experts, the Steering Committee developed key questions related to the underlying principles and indicator selection criteria to guide discussion and feedback during a series of virtual consultations. In each virtual technical review session, the indicators were discussed, guided by the key questions. Feedback from previous sessions was also shared, so learnings were incorporated and refined during successive phases. Written feedback was solicited from those unable to participate in planned consultations.

To culminate the consultation process, an in-person meeting was held in Washington, DC in mid-September 2015, bringing together a broader group of experts. The meeting was convened by the Maternal and Child Survival Program with financial support from the US Agency for International Development (USAID). Some participants were supported through funding from other institutions and development partners. This in-person consultation was guided by the same key questions addressed in previous virtual session. During the in-person meeting, participants considered the results from the first rounds of feedback to deliberate and reach consensus on a final core set of maternal health indicators for global monitoring and reporting. The in-person one-day consultation also addressed commonly perceived gaps in maternal health indicators, proposals for additional indicators and other issues raised by participants. Inputs from all steps of the consultation (virtual, written and in-person) were compiled in a matrix summarizing feedback and main discussion points (Additional file 1). A report of the in-person consultation highlighting the main discussion points was also developed.

In total, 45 experts from 11 countries participated in one or more stages of the structured consultative process to reach consensus on a core set of maternal health indicators for global monitoring and reporting. The experts agreed on 12 maternal health indicators - three impact, seven coverage, and two input indicators (Table 3). Five indicators overlap with the Every Newborn Action Plan core metrics, six indicators overlap with the WHO list of 100 core indicators, and one indicator overlaps with the WHO 2013 Quality of MNCH care indicators [5‐7] (Table 3; Additional file 2). The definitions, disaggregators, and data sources to accompany the Core Maternal Health Indicators for Global Monitoring and Reporting have been finalized (Table 4). Four areas were prioritized for further research and testing, namely content of antenatal care, content of postpartum care, respectful maternity care, and Met Need for Emergency Obstetric Care (EmOC).

The group was unanimous in its consensus to deliver this core set of maternal health indicators to WHO for further member state consultation and deliberation through global processes, including harmonization with core metrics from the ENAP and consideration as part of a combined monitoring framework for maternal and newborn health within the Indicator Framework of the Global Strategy for Women’s, Children’s and Adolescents’ Health (2016–2030).

Consensus was reached through extensive discussion of all proposed indicators (see Table 5). The final core list of indicators focuses primarily on routine care for all women; however, ending preventable maternal death requires appropriate management of maternal complications. The group discussed two potential composite indicators related to essential services for obstetric complications- Unmet Obstetric Need (UON: the proportion of women who receive surgery/major obstetric intervention among all women who experience a severe complication that is an indication for surgery/major obstetric intervention) and Met Need for Emergency Obstetric Care (the proportion of women who are expected to experience a complication who receive treatment) [8, 9]. Neither indicator met all the selection criteria for inclusion, due to issues related to feasibility and validity. For example, Met Need for EmOC assumes 15 % of women will experience a complication; this denominator has not been validated and may vary by setting. UON assumes 1 to 2 % of women will experience a severe complication, which has been validated in several settings. However the UON numerator is based on obstetric complications requiring surgical interventions which can be variable even for a single complication (e.g. pre-eclampsia/eclampsia (PEE) or postpartum hemorrhage (PPH) can often be managed without surgical intervention.) [9]. The group agreed that Met Need for EmOC was more appropriate for global monitoring, but that further refinement of the denominator would be needed for inclusion in a global monitoring framework. It was agreed that countries that are currently collecting Met Need for EmOC should continue to monitor this indicator at national and sub-national levels until the indicator has been updated.

There was widespread agreement on the need to refine the skilled birth attendant (SBA) indicator to increase its validity. WHO and UNICEF are working in this area to review and revise the current metadata and enhance measurement (personal communication, Chou and Amouzou), and the indicator met other inclusion criteria. Therefore, SBA was included in the core MH indicator list, with institutional delivery added as a supplemental indicator. There were also important discussions about how to interpret caesarean section rates, given the global ongoing debates about the appropriate population-based proportion and rising rates of voluntary caesarean sections. It was agreed that it will be particularly important for this indicator to be disaggregated by residence and socio-economic quintile, and increasingly to capture indications [10]. The group discussed a measure to indicate the availability of functional EmOC facilities. Some participants felt this indicator needed further refinement prior to inclusion, especially to re-define the denominator to be based on expected pregnancies/births instead of population. Ultimately, the group agreed that this indicator met the criteria for global monitoring and that the definition would be updated moving forward.

Particular areas of contention included whether measures of facility-level quality of care and coverage are useful and feasible for global monitoring and reporting (e.g. measures of specific content of antenatal, labor and delivery, and postpartum care). While many participants agreed that such measures are critically needed for program improvement and effectiveness at country level, there was a lack of consensus on whether or not quality measures should be included in a global monitoring framework as well as active debate about which indicator would adequately capture all elements of care. A single indicator for routine labor and delivery care—provision of uterotonic immediately after birth—was selected. In addition, the group strongly agreed on the importance of a high-level indicator of provision of respectful maternity care (RMC) and elimination of mistreatment in childbirth [11]. There was agreement that such metrics need further development and testing prior to inclusion for global monitoring and reporting. The development of indicators related to quality of antenatal care, postnatal care, RMC and Met Need for EmOC were flagged as an important priority for further development and testing. The group acknowledged ongoing work by WHO to define quality of care indicators for maternal and newborn care in health facilities as part of the WHO Quality of Care framework.

“What gets measured gets done.” This simple phrase captures the synergistic relationship between measurement and action [12], which is critical for achieving ambitious targets outlined in EPMM Strategy documents, the SDGs, and the UN Global Strategy for Women’s, Children’s, and Adolescent’s Health. This paper describes the results and process of defining core maternal health indicators for global monitoring and reporting as part of the EPMM strategy. Agreement on a core set of global maternal health indicators is important for galvanizing action and accountability to reduce preventable maternal mortality as well as to highlight technical areas needing further development and indicator testing. WHO will propose the agreed indicators for further member state consultation through global processes, including harmonization with Every Newborn Action Plan core metrics into a combined global monitoring framework for maternal and newborn health. All participants pledged their support for these processes.

The MDG5 goals and targets raised the profile of global maternal mortality reduction, galvanizing political will and resource allocation, which contributed to accelerated progress in mortality reduction. Between 1990 and 2000, MMR declined at an estimated average annual rate of 1.2 % globally, compared with an average of 3.0 % between 2005 and 2015 after the launch of the Millennium Development Goals in 2000 [1]. But accelerating further declines and ensuring equity will now require different strategies and concerted efforts to reach vulnerable and at-risk populations. A common global monitoring framework that includes key coverage and outcome indicators will be crucial to monitor whether strategies are working as intended to track progress toward global targets.

The core maternal health indicators are the result of a systematic sequential process to prioritize a small number of impact, coverage and input measures for global monitoring. These global indicators are not intended to address the measurement needs and priorities for improvement at all levels of the system across all contexts. Other complementary efforts are underway to address maternal and newborn health measurement needs related to processes and outcomes at national, subnational and service delivery levels. The core global level indicators complement these other measurement efforts and contribute to policy decision-making, resource mobilization, program planning, and aim to accountability for achieving preventable maternal deaths across the globe.

There has been promising progress in defining, testing and using indicators to monitor maternal health programs at subnational and service-delivery levels [13‐15]. The global indicator on availability of functional EmOC facilities will build on this existing body of work to incorporate elements of basic maternity care, newborn care, human resources, travel time, and others. In addition, WHO is leading an initiative to define standards and metrics for quality of facility-based care around the time of birth. The WHO quality of care framework and associated metrics prioritize measures for use by managers and providers to drive improvements in maternal and newborn care at the point of service [16]. In collaboration with the ENAP metrics improvement plan, work is underway to validate several facility-based indicators, including the Phase I EPMM indicator uterotonic use immediately after birth. This validation work will include assessing whether the woman was aware that a uterotonic was given after birth to prevent postpartum hemorrhage.

There are also many efforts underway to develop and test innovative methods to generate higher quality data to more accurately track facility-level quality of care metrics routinely. Innovative approaches around combining population-based surveys and health facility surveys are in progress [17, 18] as well as unique methods for routine monitoring of signal functions for obstetric and newborn care through quarterly monitoring visits (personal communication, UNFPA). Furthermore, there are efforts underway to use geographic information systems (GIS) to capture access to care and visualize data at sub-national levels to improve decision making [19]. As these methods are tested and refined, they will greatly contribute to more robust and accurate data to track program implementation and outcomes, which can be utilized at sub-national and service delivery levels to improve quality of care, and ultimately feed into global level monitoring and reporting. To allow for evolving improvements in measurement strategies and data quality, the global monitoring framework must be dynamic and flexible to incorporate new developments and indicators.

There were some limitations. First, because the goal was to reach expert consensus on technically sound indicators for monitoring progress toward global maternal health goals, the timeframe was dictated by time-bound global political processes, including the launch of the SDGs and the Global Strategy. As a result, it was not possible to conduct a lengthy process utilized in other indicator prioritization exercises or research studies. However, every attempt was made to follow a systematic and transparent process, including a series of virtual consultations with experts from diverse backgrounds based on pre-determined principles, indicator selection criteria, and standard questions, which culminated with an in-person meeting. This process provided a transparent mechanism for consensus-building, within the time constraints. In total, over 45 stakeholders participated in the process; however, there were limited participants from the country level. WHO will conduct further consultations with member states through global processes, and the core MH indicators from this process have been introduced into consultations regarding indicator selection for the UN Global Strategy for Women’s, Children’s and Adolescent’s Health.

Second, to meet the aim of a core set of indicators acceptable for global monitoring and reporting by all countries, the issues of feasibility, such as current data availability and quality, were by necessity balanced against the desire to “push the envelope” and drive accelerated health information system improvements. The group attempted to deal with this tension openly in the discussions and some indicators that will require further refinement and testing were included for this reason while others were not. For example, maternal cause of death is included as a core indicator, although further work is needed to standardize diagnosis, coding, and promote recording of all deaths.

Finally, monitoring the core set of indicators for maternal health cannot be implemented without attention to data systems and efforts to ensure adequate data quality. We must support countries in developing and implementing plans to routinely collect these indicators through existing data systems and surveys, such as national health management information systems, household survey programs such as Demographic and Health Surveys (DHS) or Multiple Indicator Cluster Surveys (MICS), and health facility assessments such as the Service Availability and Readiness Assessment (SARA), the Service Provision Assessment (SPA), and not create parallel structures. The recently launched Health Data Collaborative is working to coordinate investments and build capacity at country level to collect indicators required for SDG monitoring.

The global Strategies toward Ending Preventable Maternal Mortality [2] highlights an overall call to improve metrics, measurement systems and data quality. Representing one among several essential components of that goal, a core set of global indicators is key to galvanizing action and prioritizing attention to maternal health and survival, driving investment in the improvement of services, delivery systems and other proximal determinants, and providing a mechanism for accountability to stakeholders. The next phase of this process aims to develop consensus on measures to address the more distal causes of maternal survival and health, drawing from the key objectives in EPMM strategic framework that focus on the underlying social, political, and economic determinants of maternal health and survival, providing a pathway for countries to operationalize the EPMM strategic objectives and measure change over time.

The SDGs are intended to realize the rights of all and to achieve gender equality and the empowerment of women and girls, balancing the dimensions of sustainable development: the economic, social, and environmental [20]. The UN Secretary General’s Global Strategy for Women’s, Children’s, Adolescent’s Health addresses health across sectors and SDGs to move beyond the focus on survival to thriving and transformation. The Indicator and Monitoring Framework for the Global Strategy describes 16 key indicators considered the minimal subset of a total of 60 indicators to provide assessment of progress towards the implementation of the Global Strategy objectives [21]. The EPMM Phase I indicators maternal mortality ratio and adolescent birth rate are included within this minimum dataset, however expanded monitoring of outcomes related to the key themes of EPMM will help to fully realize and address priority issues in the SDGs, including equity, human resources, gender, and human rights. Taken together, the development of the EPMM Phase I and II indicators will track progress and address a comprehensive set of issues related to not only “survival” but also “thrive and transform”; together they track progress.

Within the global maternal health community, there has been a lack of consensus on core metrics for monitoring maternal health status, health system performance and program implementation necessary to achieve EPMM targets. This is largely due to the complex nature of maternal health programs that must address a wide range of health determinants and deliver a package of services across a continuum within the context of differing political priorities, health system structures, and in coordination with non-health sector actors. However, the lack of robust measures, functioning measurement systems, and data of reliable quality have stymied efforts to accurately track whether countries are meeting global health targets. It is important that members of the global and maternal health communities work together through participatory processes to track progress at all levels (global, regional, national, subnational, and facility) to end preventable maternal mortality. This paper describes a core set of maternal health indicators for global monitoring and reporting for all countries, developed through an inclusive, iterative, and consultative process led by WHO on behalf of the EPMM Working Group. Monitoring these core indicators will be critical to support achievement of the global targets outlined in the SDGs and ultimately to end preventable maternal mortality.